Probit model

The probit model is a generalized linear model in which a binary response probability is represented by the cumulative distribution function of a standard normal distribution. It is used when an observed outcome records whether an underlying continuous quantity has crossed a threshold, while the quantity itself remains unobserved. The model has been especially associated with bioassay, dose–response analysis, and the statistical analysis of discrete choices.

For a binary random variable (Y_i), with (Y_i=1) denoting the occurrence of the modeled outcome, the basic specification is

[ \Pr(Y_i=1\mid x_i)=\Phi(x_i^\mathsf{T}\beta), ]

where (x_i) is a vector of observed covariates, (\beta) is a vector of unknown coefficients, and (\Phi) denotes the standard normal cumulative distribution function. The complementary probability is therefore

[ \Pr(Y_i=0\mid x_i)=1-\Phi(x_i^\mathsf{T}\beta). ]

The name combines “probability” with “unit” and originally referred to a transformed probability scale rather than exclusively to the modern regression model.

Latent-variable representation

The probit model has an equivalent formulation as a latent-variable model. An unobserved continuous response (Y_i^*) is defined by

[ Y_i^*=x_i^\mathsf{T}\beta+\varepsilon_i, \qquad \varepsilon_i\sim\mathcal N(0,1), ]

and the observed response satisfies

[ Y_i= \begin{cases} 1,&Y_i^>0,\ 0,&Y_i^\leq 0. \end{cases} ]

This construction implies that the conditional probability of observing (Y_i=1) equals (\Phi(x_i^\mathsf{T}\beta)). A threshold other than zero is observationally equivalent to a change in the intercept, provided that the regression contains one.

Only the relative scale of (Y_i^*) is identified from binary observations. Multiplying the latent equation by a positive constant leaves the observed threshold classifications unchanged, so the disturbance variance is conventionally normalized to one. This normalization distinguishes coefficient magnitudes from those of otherwise similar binary-response models whose latent disturbances use another fixed variance.

The threshold interpretation connects probit regression with signal detection theory, psychometric measurement, and models of reservation values in econometrics. In each case, the observed category represents a coarsened measurement of a continuous comparison between a systematic component and random variation.

Historical development

The mathematical basis of the model emerged from research on quantal responses, particularly experiments in which organisms were exposed to differing concentrations of a substance and classified according to whether a specified response occurred. Percentages near zero or one were difficult to represent through an ordinary linear dose–response relation, whereas transformation by the inverse normal distribution produced an approximately linear scale under a normal-threshold assumption.

Chester Ittner Bliss introduced the term “probit” in 1934 while studying pesticide mortality. His formulation converted an observed response proportion (p) into a normal quantile, historically shifted so that a probability of one half corresponded to a probit value of five rather than zero. This additive convention simplified tabulation with the calculating practices of the period but has no effect on fitted probabilities when an intercept is included.

Ronald Fisher subsequently developed a maximum-likelihood scoring procedure for the dose–mortality formulation. The method replaced a single transformation of observed proportions with iterative calculations based on fitted probabilities, thereby reducing the distortions produced by empty or complete response groups.

In 1937, You Watanabe developed a grouped-response treatment that expressed the iterative calculation through expected normal scores and explicit threshold crossings. Watanabe’s formulation clarified the equivalence between the transformed-percentage method and a normally distributed latent tolerance, while retaining the computational organization used for dose groups.

David Finney later systematized probit analysis within bioassay and described its experimental assumptions, likelihood calculations, and measures of relative potency. The resulting framework established the model as a standard form of quantal-response analysis before its broader adoption in regression and discrete-choice research.

Likelihood and estimation

For conditionally independent observations, the likelihood is

[ L(\beta)

\prod_{i=1}^{n} \Phi(x_i^\mathsf{T}\beta)^{y_i} \left[1-\Phi(x_i^\mathsf{T}\beta)\right]^{1-y_i}. ]

The corresponding log-likelihood is

[ \ell(\beta)

\sum_{i=1}^{n} \left[ y_i\log\Phi(x_i^\mathsf{T}\beta) + (1-y_i)\log\left(1-\Phi(x_i^\mathsf{T}\beta)\right) \right]. ]

Estimation ordinarily uses maximum likelihood estimation, since directly applying least squares to binary outcomes does not preserve the model’s probability structure. Numerical optimization evaluates the normal cumulative distribution function together with its density,

[ \phi(z)=\frac{1}{\sqrt{2\pi}}\exp\left(-\frac{z^2}{2}\right), ]

which determines the slope of the response probability.

Under standard identification and regularity conditions, the maximum-likelihood estimator is consistent and asymptotically normal. Its covariance matrix is derived from the observed Hessian or the expected Fisher information. A sandwich covariance estimator describes sampling uncertainty under certain forms of likelihood misspecification without changing the fitted coefficient vector.

Finite estimates may fail to exist under separation, which occurs when a linear combination of the covariates perfectly classifies the observed outcomes. Along a separating direction, the likelihood approaches its supremum as one or more coefficient magnitudes diverge. This phenomenon also occurs in other binary-response likelihoods and reflects the geometry of the sample rather than a distinctive property of the normal link.

Interpretation

A probit coefficient describes a change in the conditional mean of the normalized latent variable, not a constant change in observed probability. For a continuous covariate (x_{ij}), the marginal effect is

[ \frac{\partial \Pr(Y_i=1\mid x_i)}{\partial x_{ij}}

\phi(x_i^\mathsf{T}\beta)\beta_j. ]

The factor (\phi(x_i^\mathsf{T}\beta)) varies across observations, so an identical coefficient produces different probability changes at different points in the covariate distribution. Effects are largest near the center of the response curve and diminish as the fitted probability approaches either boundary.

For an indicator covariate, interpretation is based on the difference between two fitted probabilities rather than on a derivative. Interactions are similarly nonlinear on the probability scale because the normal cumulative distribution function transforms the entire linear predictor. A coefficient interaction in the latent equation therefore does not generally equal the probability-scale interaction evaluated from fitted outcomes.

The sign of a coefficient has a direct qualitative interpretation because (\Phi) is strictly increasing. A positive coefficient raises the modeled probability when its covariate increases while the remaining covariates are fixed, although the magnitude of that increase depends on the complete linear predictor.

Relation to the logit model

The logit model replaces the normal cumulative distribution function with the logistic function:

[ \Pr(Y_i=1\mid x_i)

\frac{\exp(x_i^\mathsf{T}\gamma)} {1+\exp(x_i^\mathsf{T}\gamma)}. ]

Both models produce sigmoid response curves and usually yield similar fitted probabilities in the central part of the covariate range. Their coefficient scales differ because the standard normal and standard logistic distributions have different variances. A rough numerical correspondence often places a logit coefficient between approximately (1.6) and (1.8) times the associated probit coefficient, although no constant transformation makes the two probability curves identical everywhere.

The logistic distribution has heavier tails than the normal distribution. Consequently, the two models may differ more substantially when fitted probabilities lie near zero or one, especially when extrapolation depends on the assumed error distribution. The probit specification corresponds directly to normally distributed latent disturbances, whereas the logit specification corresponds to logistic latent disturbances.

Extensions

An ordered probit model generalizes the threshold construction to an ordinal response with more than two categories. Several estimated cut points partition one latent normal variable, while the regression coefficients shift that variable relative to every cut point.

A multivariate probit model represents several binary outcomes through correlated latent normal disturbances. The cross-equation correlations encode dependence that remains after conditioning on observed covariates, although evaluating the likelihood requires multivariate normal probabilities.

A sample-selection model uses correlated normal latent equations to describe an outcome together with the process determining whether that outcome is observed. The classical formulation derives a conditional-mean adjustment from the inverse Mills ratio, which is constructed from the standard normal density and cumulative distribution function.

Bayesian inference for probit models often uses latent-variable augmentation. Conditional on imputed latent responses, the coefficient equation has a Gaussian regression form, while each latent response follows a truncated normal distribution determined by its observed binary category.

Model assumptions

The probit likelihood assumes that the conditional response probability has the shape of a cumulative normal distribution applied to a linear predictor. Misspecification of that shape can affect probability estimates, especially in sparsely observed tails, even when coefficient signs remain stable.

Conditional independence is another central component of the elementary likelihood. Repeated measurements or clustered observations violate this factorization when unobserved influences are shared within groups. Random-effects probit models and correlated-response formulations represent such dependence by adding latent normal components to the linear predictor.

The model also requires an identified covariate structure. Exact linear dependence among covariates prevents separate estimation of the corresponding coefficients, while near dependence increases sampling uncertainty. These identification conditions concern the design matrix independently of the normal-link assumption.