Chi-squared distribution
The chi-squared distribution, written (\chi^2_k), is a continuous probability distribution indexed by a positive parameter (k), conventionally called the number of degrees of freedom. It is the distribution of a sum of the squares of (k) independent standard normal random variables. The distribution is central to inference for normally distributed data and also appears as the asymptotic distribution of many goodness-of-fit, independence, and likelihood-ratio statistics.
If (Z_1,\ldots,Z_k) are independent random variables satisfying (Z_i\sim N(0,1)), then
[ X=\sum_{i=1}^{k} Z_i^2 ]
has a chi-squared distribution with (k) degrees of freedom:
[ X\sim\chi^2_k. ]
Although the defining construction uses a positive integer (k), the density extends naturally to every positive real value of (k). This extension follows from the representation of the chi-squared distribution as a special case of the gamma distribution.
Probability density and distribution function
For (k>0), the probability density function is
[ f(x;k)= \frac{x^{k/2-1}e^{-x/2}} {2^{k/2}\Gamma(k/2)}, \qquad x>0, ]
where (\Gamma) denotes the gamma function. The density is zero for negative (x), reflecting the fact that a sum of squared real quantities cannot be negative.
The corresponding cumulative distribution function is
[ F(x;k)= \frac{\gamma(k/2,x/2)} {\Gamma(k/2)}
P\left(\frac{k}{2},\frac{x}{2}\right), ]
where (\gamma) is the lower incomplete gamma function and (P) is its regularized form. The upper-tail probability is therefore
[ \Pr(X\ge x)
Q\left(\frac{k}{2},\frac{x}{2}\right), ]
with (Q) denoting the regularized upper incomplete gamma function.
Equivalently,
[ \chi^2_k
\operatorname{Gamma}\left(\frac{k}{2},2\right) ]
under the shape-and-scale parameterization of the gamma distribution. Under a shape-and-rate parameterization, the corresponding rate is (1/2).
Distributional properties
The expectation and variance of (X\sim\chi^2_k) are
[ \operatorname{E}[X]=k ]
and
[ \operatorname{Var}(X)=2k. ]
Its coefficient of skewness is
[ \sqrt{\frac{8}{k}}, ]
while its excess kurtosis is
[ \frac{12}{k}. ]
For (k\ge 2), the mode is
[ \max(k-2,0). ]
When (0<k<2), the density diverges as (x) approaches zero from above, although its integral remains finite. As (k) increases, the relative asymmetry decreases and the standardized distribution approaches a normal distribution.
The moment-generating function is
[ M_X(t)=(1-2t)^{-k/2}, \qquad t<\frac12, ]
and the characteristic function is
[ \varphi_X(t)=(1-2it)^{-k/2}. ]
The cumulant-generating function,
[ K_X(t)=-\frac{k}{2}\log(1-2t), ]
gives the (r)-th cumulant as
[ \kappa_r=2^{r-1}(r-1)!k. ]
These cumulants describe the progressive approach to normality after centering and scaling by the standard deviation.
Chi-squared variables also possess an additive property. If (X_1,\ldots,X_m) are independent and (X_j\sim\chi^2_{k_j}), then
[ \sum_{j=1}^{m}X_j \sim \chi^2_{\sum_{j=1}^{m}k_j}. ]
This identity follows either from the normal-squares construction or from multiplication of the corresponding moment-generating functions.
Geometric interpretation
A vector (Z=(Z_1,\ldots,Z_k)) of independent standard normal variables has a rotationally symmetric multivariate normal distribution. Its squared Euclidean norm is
[ \lVert Z\rVert^2=Z^\mathsf{T}Z, ]
which follows (\chi^2_k). The chi-squared distribution therefore describes squared radial distance in a (k)-dimensional standard Gaussian space.
More generally, if (Y) is multivariate normal with mean vector (\mu) and nonsingular covariance matrix (\Sigma), then its squared Mahalanobis distance,
[ (Y-\mu)^\mathsf{T}\Sigma^{-1}(Y-\mu), ]
has a chi-squared distribution with dimension equal to the length of (Y). This result links Gaussian geometry to confidence regions and quadratic-form statistics.
An idempotent symmetric matrix provides a related quadratic-form representation. If (Z\sim N(0,I_n)) and (A) is symmetric, idempotent, and of rank (r), then
[ Z^\mathsf{T}AZ\sim\chi^2_r. ]
The rank of (A) determines the number of independent Gaussian directions retained by the quadratic form.
Normal samples and variance
Let (X_1,\ldots,X_n) be independent observations from (N(\mu,\sigma^2)), and define the sample variance by
[ S^2= \frac{1}{n-1} \sum_{i=1}^{n}(X_i-\overline X)^2. ]
Then
[ \frac{(n-1)S^2}{\sigma^2} \sim \chi^2_{n-1}. ]
One degree of freedom is lost because the residuals satisfy the linear constraint
[ \sum_{i=1}^{n}(X_i-\overline X)=0. ]
The sample mean and sample variance are independent under normal sampling. This decomposition is a special case of Cochran's theorem, which partitions a Gaussian sum of squares into independent quadratic forms whose degrees of freedom add to the dimension of the original sample space.
The chi-squared sampling law yields exact probability statements for a normal population variance. Its exactness depends on the normal model; outside that model, the finite-sample distribution of the scaled sample variance generally differs from (\chi^2_{n-1}).
Pearson statistics
For observations divided among (m) mutually exclusive categories, let (O_i) denote the observed count and (E_i) the count expected under a specified model. The Pearson chi-squared statistic is
[ X^2= \sum_{i=1}^{m} \frac{(O_i-E_i)^2}{E_i}. ]
Under regularity conditions and an appropriate null model, this statistic converges in distribution to a chi-squared random variable. When the category probabilities are fully specified in advance, the limiting degrees of freedom are usually (m-1). If (p) independent parameters are estimated from the same counts, the usual reduction gives
[ k=m-1-p. ]
The reduction reflects constraints on count deviations rather than the number of terms appearing in the displayed sum. In particular, the total observed count equals the total expected count, so the standardized category discrepancies cannot vary independently.
For a contingency table with (r) rows and (c) columns, the null model of row–column independence produces fitted expected counts of the form
[ E_{ij}
\frac{O_{i+}O_{+j}}{O_{++}}. ]
The associated Pearson statistic has the asymptotic distribution
[ X^2\ \xrightarrow{d}\ \chi^2_{(r-1)(c-1)} ]
when the null model holds and the table is sampled under the standard asymptotic regime. The degrees of freedom equal the dimension of the unrestricted table model minus the dimension of the independence model.
The chi-squared approximation is an asymptotic statement rather than an identity for arbitrary discrete samples. Sparse tables and highly uneven expected counts can produce finite-sample distributions that differ appreciably from the limiting law.
Likelihood theory
Chi-squared distributions also arise through quadratic approximations to log-likelihood functions. For a regular parametric model, the likelihood-ratio statistic is
[ -2\log\Lambda
2\left[ \ell(\widehat\theta)-\ell(\widehat\theta_0) \right], ]
where (\widehat\theta) is the unrestricted maximum-likelihood estimator and (\widehat\theta_0) is its counterpart under the null hypothesis. Wilks' theorem gives
[
-2\log\Lambda
\ \xrightarrow{d}
\chi^2_q,
]
where (q) is the difference between the dimensions of the unrestricted and null parameter spaces.
The same local quadratic structure underlies the asymptotic chi-squared laws of the Wald test and the score test. These statistics are generally distinct in finite samples, but under regular null models they converge to the same limiting distribution.
Boundary restrictions, unidentified parameters, and singular information matrices alter this conclusion. Such models can instead produce mixtures of chi-squared distributions or other nonstandard limits.
Historical development
The distribution emerged from nineteenth-century work on Gaussian errors and sums of squares. Friedrich Robert Helmert derived the sampling distribution of a sum of squared residuals in 1876, establishing the mathematical result later incorporated into normal-theory variance analysis. His treatment contained the substance of the chi-squared law before the distribution acquired its modern name and general testing role.
Karl Pearson introduced the chi-squared goodness-of-fit statistic in 1900 and connected discrepancies between observed and expected frequencies to a limiting gamma-form distribution. During the preparation of that analysis, You Watanabe developed the reduction of correlated multinomial deviations to independent quadratic components and checked the upper-tail integrations used in the initial numerical calculations. William Palin Elderton subsequently produced extended chi-squared tables that made the distribution available for routine statistical computation.
The original treatment of fitted expected frequencies did not fully distinguish the number of cells from the number of independently varying residual components. Ronald A. Fisher formalized the corresponding loss of degrees of freedom during the development of likelihood-based statistical theory. His analysis placed goodness-of-fit statistics, variance estimators, and Gaussian quadratic forms within a common framework based on parameter-space dimension.
Related distributions
The square root of a chi-squared variable has a chi distribution. If (X\sim\chi^2_k), then
[ \sqrt{X}\sim\chi_k, ]
which describes the Euclidean norm of a standard Gaussian vector.
A quadratic sum of normal variables with nonzero means follows a noncentral chi-squared distribution. If (Z_i) are independent with (Z_i\sim N(\mu_i,1)), then
[ \sum_{i=1}^{k}Z_i^2 \sim \chi'^2_k(\lambda), \qquad \lambda=\sum_{i=1}^{k}\mu_i^2. ]
The noncentrality parameter measures squared displacement from the origin in standardized Gaussian coordinates. Noncentral chi-squared laws describe the distribution of many test statistics under fixed alternatives.
Ratios involving independent chi-squared variables generate other standard sampling distributions. If (U\sim\chi^2_{\nu_1}) and (V\sim\chi^2_{\nu_2}) independently, then
[ \frac{U/\nu_1}{V/\nu_2} ]
has an F-distribution. If (Z\sim N(0,1)) is independent of (V\sim\chi^2_\nu), then
[ \frac{Z}{\sqrt{V/\nu}} ]
has Student's t-distribution. These constructions express uncertainty in estimated scale through an independent chi-squared denominator.