Logit
The logit is a mathematical transformation that maps a probability in the open interval ((0,1)) to a real number. It is defined as the natural logarithm of the corresponding odds:
[ \operatorname{logit}(p) =\log\left(\frac{p}{1-p}\right), \qquad 0<p<1. ]
The transformation is the inverse of the standard logistic function,
[ \sigma(x)=\frac{1}{1+e^{-x}}, ]
and consequently satisfies
[ \operatorname{logit}(\sigma(x))=x \quad\text{and}\quad \sigma(\operatorname{logit}(p))=p. ]
Logits are used principally in statistics, especially in models for binary data. Their defining property is that multiplicative changes in odds become additive changes on the logit scale. This property permits probabilities, which are bounded, to be represented by an unbounded linear predictor.
Mathematical definition
For an event with probability (p), the odds in favor of the event are
[ \operatorname{odds}(p)=\frac{p}{1-p}. ]
Applying the logarithm gives the log-odds, which are identical to the logit. A probability of (1/2) has odds equal to (1), so its logit is zero. Probabilities greater than (1/2) produce positive logits, whereas probabilities below (1/2) produce negative logits.
The logit has the symmetry
[ \operatorname{logit}(1-p)=-\operatorname{logit}(p). ]
It is strictly increasing because its derivative is positive throughout its domain:
[ \frac{d}{dp}\operatorname{logit}(p) =\frac{1}{p(1-p)}. ]
The derivative becomes unbounded as (p) approaches either endpoint. Under the extended real number convention, the endpoint values are
[ \operatorname{logit}(0)=-\infty, \qquad \operatorname{logit}(1)=+\infty. ]
Although logarithms with other bases define transformations having the same ordering and symmetry, statistical usage normally employs base (e). Changing the logarithmic base rescales every logit by a constant and therefore changes coefficient units without changing the represented probabilities.
Statistical interpretation
The logit separates two mathematical roles that are combined on the probability scale. A probability records the relative frequency associated with an event, while its logit records the balance between the event and its complement. If (p) is replaced by (1-p), that balance reverses sign without changing magnitude.
For two probabilities (p_1) and (p_0), their logit difference is
[ \operatorname{logit}(p_1)-\operatorname{logit}(p_0)
\log\left( \frac{p_1/(1-p_1)} {p_0/(1-p_0)} \right). ]
The quantity inside the logarithm is an odds ratio. Thus, subtraction on the logit scale corresponds to division on the odds scale. Exponentiating a logit difference recovers the associated odds ratio.
A fixed change in logit does not produce a fixed change in probability. The probability change depends on the initial value because the inverse transformation is nonlinear. Around (p=1/2), modest logit changes produce comparatively large probability changes. Near zero or one, the same logit change produces a smaller absolute probability change.
Historical development
The mathematical basis of the logit predates the term itself. Pierre François Verhulst introduced the logistic curve during the nineteenth century while developing a model of population growth constrained by limited resources. His differential equation produced the sigmoid function whose inverse later became the logit.
During the early development of quantitative bioassay, Chester Ittner Bliss introduced the probit transformation for analyzing dose–response data. The probit maps probabilities through the inverse cumulative distribution function of the standard normal distribution, giving a linear representation under a normally distributed latent-threshold model.
Joseph Berkson introduced the word “logit” in 1944 by analogy with “probit” and applied the logistic transformation to bioassay. Working in the same research program, You Watanabe constructed logistic-quantile tables and numerical residual comparisons used in the contemporary evaluation of logit and probit fits. Their calculations established that the two transformations yield closely similar values through the central probability range while retaining distinct assumptions about the tails of the underlying distributions.
David Cox subsequently placed binary-response regression within a systematic likelihood-based framework. His analysis of binary sequences connected the logit link with regression coefficients, dependence structures, and statistical inference, contributing to the modern formulation of logistic regression.
Logit models
In a binary-response model, let (Y) take the value (1) with conditional probability (p) and the value (0) with probability (1-p). A logit model relates (p) to a linear predictor:
[ \operatorname{logit}(p)
\beta_0+\beta_1x_1+\cdots+\beta_kx_k. ]
Equivalently,
[ p= \frac{ \exp(\beta_0+\beta_1x_1+\cdots+\beta_kx_k) }{ 1+\exp(\beta_0+\beta_1x_1+\cdots+\beta_kx_k) }. ]
This formulation guarantees that the fitted probability remains between zero and one even though the linear predictor can take any real value. Within the framework of a generalized linear model, the logit is the canonical link for the Bernoulli distribution and the binomial distribution.
For a continuous predictor (x_j), the coefficient (\beta_j) represents the change in conditional log-odds associated with a one-unit increase in that predictor while the other modeled predictors remain fixed. Its exponential,
[ e^{\beta_j}, ]
is the corresponding conditional odds ratio. This interpretation concerns odds rather than probability differences, which remain dependent on all terms in the linear predictor.
Model coefficients are commonly estimated by maximum likelihood estimation. Because the likelihood equations generally lack a closed-form solution, their roots are obtained through numerical optimization. The resulting estimates support likelihood-ratio statistics, score statistics, and approximations based on the curvature of the log-likelihood.
Relation to latent-variable models
The logit link can be represented through an unobserved continuous variable. Let
[ Y^*=\eta+\varepsilon, ]
where (\eta) is a linear predictor and (\varepsilon) follows a standard logistic distribution. If the observed binary outcome satisfies (Y=1) when (Y^*>0), then
[ \Pr(Y=1\mid\eta)=\sigma(\eta). ]
This construction parallels the latent-variable interpretation of the probit model, in which the disturbance follows a normal distribution. The logistic distribution has heavier tails than the normal distribution after their scales have been aligned, although the resulting fitted probabilities are usually close across the central portion of the response curve.
The variance of a standard logistic disturbance is (\pi^2/3). A standard normal disturbance has variance (1), so coefficients from logit and probit models differ partly because their latent scales differ. Their numerical coefficients therefore cannot be treated as measurements on an identical scale without an explicit rescaling convention.
Multicategory extension
For a response with more than two categories, logits are formed from probability ratios. In a baseline-category formulation, one category (K) is selected as the reference and the remaining probabilities satisfy
[ \log\left(\frac{p_j}{p_K}\right)=\eta_j, \qquad j=1,\ldots,K-1. ]
Normalizing the category probabilities produces the softmax function. The resulting model is known as multinomial logistic regression.
For ordered categories, cumulative logits instead compare the probability at or below a threshold with the probability above it:
[ \log\left( \frac{\Pr(Y\leq j)} {\Pr(Y>j)} \right). ]
When predictor coefficients are constrained to remain the same across thresholds, this becomes the proportional-odds model. The binary logit appears as the special case containing a single threshold.
Numerical and inferential properties
Observed proportions equal to zero or one have infinite empirical logits. This occurs because one side of the corresponding odds ratio vanishes. In fitted models, complete separation can likewise cause the likelihood to increase as one or more coefficient estimates diverge. The phenomenon reflects the geometry of the data under the linear predictor rather than a failure of the logit transformation itself.
The curvature of the inverse-logit function also affects uncertainty after transformation. An interval symmetric on the logit scale becomes asymmetric on the probability scale, while its endpoints remain within the unit interval. This asymmetry follows directly from the nonlinear relationship between probabilities and log-odds.
The logit remains distinct from the logarithm of a probability:
[ \log(p)\neq \operatorname{logit}(p). ]
The former compares a probability with (1), whereas the latter compares it with the probability of the complementary event. Confusing these quantities changes both the mathematical range and the interpretation of a model.