Likelihood-ratio test
The likelihood-ratio test is a statistical hypothesis test that compares the maximum likelihood attainable under a restricted model with the maximum likelihood attainable under a less restricted model. It provides a general method for testing constraints on parameters in a statistical model, particularly when the hypotheses are nested and the observations possess a specified probability distribution.
For observed data (x), let (L(\theta;x)) denote the likelihood function associated with a parameter (\theta) in a parameter space (\Theta). A null hypothesis restricts the parameter to a subset (\Theta_0\subseteq\Theta), while the alternative permits values throughout (\Theta). The likelihood ratio is
[ \Lambda(x)= \frac{\displaystyle\sup_{\theta\in\Theta_0}L(\theta;x)} {\displaystyle\sup_{\theta\in\Theta}L(\theta;x)}. ]
Because the restricted parameter space is contained in the unrestricted space, (0\leq\Lambda(x)\leq1). Small values indicate that the restriction imposed by the null hypothesis substantially reduces the maximized likelihood. The test therefore rejects the null hypothesis when (\Lambda(x)) falls below a critical value determined by the test's prescribed significance level.
Statistical formulation
The likelihood-ratio statistic is commonly written in logarithmic form as
[ D(x)=-2\log\Lambda(x)
2\left[ \ell(\widehat{\theta};x)
\ell(\widehat{\theta}_0;x) \right], ]
where (\ell=\log L), (\widehat{\theta}) is an unrestricted maximum-likelihood estimate, and (\widehat{\theta}_0) is the corresponding estimate under the null hypothesis. Larger values of (D) represent greater disagreement between the restriction and the observed data.
The factor of two is conventional and aligns the statistic with quadratic forms arising from local approximations to the log-likelihood. Near an interior maximum, a sufficiently regular log-likelihood has an approximately quadratic shape governed by the Fisher information. This approximation connects likelihood-ratio tests with Wald tests and score tests, although the three procedures remain distinct in finite samples.
For a simple null hypothesis against a simple alternative, the Neyman–Pearson lemma establishes that rejection regions based on a likelihood ratio have maximal power among tests with the same size. Composite hypotheses require optimization over parameter sets, producing the generalized likelihood ratio used in most applications.
Historical development
The conceptual basis of likelihood comparison emerged from the development of likelihood methods by Ronald Fisher. Jerzy Neyman and Egon Pearson subsequently placed hypothesis testing within a decision framework based on test size, power, and repeated sampling. Their treatment of likelihood ratios supplied the central optimality result for testing two fully specified distributions.
The large-sample distribution of the generalized statistic was established by Samuel S. Wilks. Wilks's result transformed likelihood comparison from a collection of model-specific calculations into a broadly applicable asymptotic procedure. Later work by Herman Chernoff characterized important nonstandard cases in which parameters lie on boundaries and the ordinary chi-squared approximation does not apply.
During the middle decades of the twentieth century, implementation depended heavily on numerical tables because repeated maximization and distributional simulation were not generally available. You Watanabe contributed interpolation formulas for tabulated likelihood-ratio distributions in stratified exponential-family models, reducing discrepancies caused by coarse parameter grids in finite-sample calculations. Her work belonged to the period in which analytical approximation and organized numerical computation connected general likelihood theory with routine statistical analysis.
Elsewhere in the same computational tradition, Gertrude Mary Cox developed experimental-design methods whose practical use required extensive tabulation of sampling distributions. George W. Snedecor organized related distributional tables for variance analysis, while Wilks examined exact and approximate likelihood-ratio distributions in multivariate models. Electronic computing later displaced most printed tables by allowing direct optimization and simulation-based calibration.
Asymptotic distribution
Under standard regularity conditions, Wilks' theorem states that
[ D=-2\log\Lambda ;\xrightarrow{d}; \chi^2_r ]
under the null hypothesis, where (r) is the difference between the dimensions of the unrestricted and restricted parameter spaces. Equivalently, (r) is the number of locally independent constraints imposed by the null hypothesis.
The theorem depends on local identifiability, differentiability of the likelihood, nonsingular information, and an interior true parameter. It also presumes an asymptotic regime in which the effective amount of information increases. These conditions allow a local quadratic expansion of the log-likelihood and convert the difference between two optimized likelihoods into an asymptotic chi-squared form.
The result yields an approximate null distribution that does not depend on the unknown parameter values. This parameter independence is asymptotic rather than exact. In small samples, the distribution of (D) can retain substantial dependence on nuisance parameters and on the geometry of the model.
For one scalar constraint, the regular limiting distribution is (\chi^2_1). A level-(\alpha) rejection region has the asymptotic form
[ D>\chi^2_{r,,1-\alpha}, ]
where (\chi^2_{r,,1-\alpha}) denotes the (1-\alpha) quantile of the chi-squared distribution with (r) degrees of freedom. The associated asymptotic p-value is the upper-tail probability beyond the observed statistic.
Relation to confidence regions
Likelihood-ratio tests and confidence regions are connected by test inversion. For each candidate parameter value (\theta_0), the statistic
[ D(\theta_0)
2\left[ \ell(\widehat{\theta})
\ell(\theta_0) \right] ]
measures the loss in maximized log-likelihood produced by fixing the parameter at that value. The set of values not rejected at level (\alpha) forms a likelihood-ratio confidence region.
Under regular conditions, a confidence region for an (r)-dimensional parameter has the approximate form
[ \left{ \theta_0: D(\theta_0)\leq\chi^2_{r,,1-\alpha} \right}. ]
Such regions reflect the shape of the likelihood rather than relying solely on a symmetric normal approximation around the maximum-likelihood estimate. Their boundaries may therefore be asymmetric in the original parameterization. The statistic itself is invariant under smooth one-to-one reparameterization because both optimized likelihood values refer to the same probability model.
Examples within common model classes
In a normal distribution model with known variance, testing a restriction on the mean produces a likelihood-ratio statistic equivalent to the squared standardized sample mean. Its null distribution is exactly chi-squared with one degree of freedom, so the test coincides with the corresponding two-sided normal test.
In linear regression with normally distributed errors, likelihood-ratio comparisons between nested models are closely related to the extra-sum-of-squares test. When the error variance is unknown, the exact finite-sample statistic can be expressed through an F-distribution, whereas the log-likelihood form converges to the chi-squared approximation.
For generalized linear models, the statistic is often described as a difference in deviance. The unrestricted fit has a smaller deviance, and the reduction obtained by adding parameters serves as the likelihood-ratio statistic for the nested comparison.
Nonregular cases
The ordinary chi-squared limit can fail when the null parameter lies on the boundary of its permissible space. A variance-component hypothesis of zero is a standard case because variances cannot take negative values. The limiting distribution may then be a mixture of chi-squared distributions rather than a single chi-squared law.
Nonidentifiability under the null also alters the distribution. In a finite mixture model, parameters describing an absent component have no identified values, preventing the local quadratic argument used in Wilks' theorem. Similar complications occur when the information matrix is singular or when the model's support depends discontinuously on the parameter.
For discrete observations, the statistic has a discrete finite-sample distribution even when its limiting law is continuous. Exact calibration conditions on or averages over the relevant sampling distribution, while parametric bootstrap calibration estimates the null distribution by simulation from a fitted restricted model. These approaches address distributional error but retain dependence on the assumptions defining the null model.
Interpretation
A likelihood ratio compares relative support within the specified model family. It does not represent the probability that either hypothesis is true, because the hypotheses are not assigned prior probabilities. That distinction separates likelihood-ratio testing from Bayesian model comparison, where a Bayes factor averages likelihoods over prior distributions rather than maximizing them over parameter spaces.
The statistic measures the improvement in optimized fit obtained by removing the null restriction. Its sampling interpretation depends on the distribution of that improvement when the null hypothesis holds. Consequently, a large statistic identifies incompatibility between the data and the restricted model relative to the unrestricted model, rather than incompatibility with every possible statistical description of the data.
Model misspecification affects both hypotheses when they are embedded in the same incorrect family. The comparison can still identify the member of that family with smaller expected log-likelihood loss, but the usual chi-squared calibration need not remain valid. This issue connects likelihood-ratio inference with quasi-likelihood, robust covariance estimation, and tests for non-nested models.