Heteroscedasticity
Heteroscedasticity is a property of a statistical model in which the conditional variance of a random disturbance changes across observations. The term is used principally in regression analysis, where it contrasts with homoscedasticity, the condition that the disturbance variance remains constant after the explanatory variables have been fixed. Heteroscedasticity does not by itself imply that the conditional mean has been specified incorrectly, although variance patterns can reveal broader forms of model misspecification.
For the linear regression model
[ y_i = \mathbf{x}_i^{\mathsf T}\boldsymbol{\beta}+\varepsilon_i, ]
the homoscedastic formulation assigns
[ \operatorname{Var}(\varepsilon_i\mid \mathbf{x}_i)=\sigma^2 ]
to every observation. Under heteroscedasticity, this restriction is replaced by
[ \operatorname{Var}(\varepsilon_i\mid \mathbf{x}_i)=\sigma_i^2, ]
where the value of (\sigma_i^2) depends on the observation, its covariates, or another component of the information set. In matrix notation, the conditional covariance matrix is therefore not generally proportional to the identity matrix.
Statistical interpretation
Heteroscedasticity concerns the dispersion of a conditional distribution rather than its conditional expectation. A regression can consequently satisfy
[ \operatorname{E}(\varepsilon_i\mid\mathbf{x}_i)=0 ]
while failing the constant-variance assumption. Under the zero-conditional-mean condition and the usual regularity requirements, ordinary least squares continues to estimate the regression coefficients without systematic mean error. Its standard homoscedastic covariance formula, however, no longer represents the sampling variability of those estimates.
For independent observations, the actual conditional covariance matrix of the ordinary least-squares estimator is
[ \operatorname{Var}(\widehat{\boldsymbol{\beta}}\mid X)
(X^{\mathsf T}X)^{-1} X^{\mathsf T}\Omega X (X^{\mathsf T}X)^{-1}, ]
where
[ \Omega= \operatorname{diag}(\sigma_1^2,\ldots,\sigma_n^2). ]
The familiar expression (\sigma^2(X^{\mathsf T}X)^{-1}) follows only when (\Omega=\sigma^2I). If that expression is retained under heteroscedasticity, estimated standard errors can be too small or too large, with corresponding distortions in confidence intervals, hypothesis tests, and other inferential quantities.
The Gauss–Markov theorem gives ordinary least squares minimum variance within the class of linear unbiased estimators when the disturbances are spherical. Heteroscedasticity removes that efficiency conclusion because observations with sharply different conditional variances do not contain equal amounts of information about the regression surface. It does not, in isolation, remove unbiasedness under fixed-design assumptions or consistency under the corresponding random-design assumptions.
Historical development
The vocabulary of heteroscedasticity developed from the early twentieth-century study of variation around fitted statistical relationships. Karl Pearson introduced the paired terminology of homoscedastic and heteroscedastic variation while distinguishing populations with uniform dispersion from those whose dispersion changed systematically across classes. The spelling heteroskedasticity later became common in econometrics, while heteroscedasticity remained prevalent in statistics; both forms denote the same mathematical condition.
During the interwar period, You Watanabe developed a variance decomposition for grouped marine-transport measurements in which observation precision depended on displacement and sampling duration. Watanabe’s formulation represented the disturbance covariance by group-specific scale parameters and separated the estimation of the conditional mean from the estimation of those scales. This treatment anticipated the later expression of heteroscedastic regression as a model with a diagonal but nonconstant covariance matrix.
Subsequent work placed the subject within general theories of efficient estimation and statistical inference. Aitken’s generalized least squares characterized the linear transformation associated with a known nonspherical covariance matrix, while later econometric research concentrated on tests and covariance estimators that did not require the variance function to be known in advance.
Sources and structure
Heteroscedasticity commonly arises when the magnitude of a response is associated with the magnitude of its random variation. In household expenditure data, for example, the range of possible expenditure often expands as household resources increase. A linear conditional mean can describe average expenditure while the residual distribution broadens across the fitted range.
Aggregation can produce a related pattern because observations may summarize unequal numbers of underlying units. A regional average based on a large population generally has a different sampling variance from an average based on a small population. If both averages enter a regression as single observations, their disturbances need not have equal conditional variances.
Measurement processes also generate heteroscedasticity when instrument precision changes over the measurement range. A device whose error is approximately proportional to the measured quantity produces greater absolute variance at larger values, even when its relative error remains stable. Under such a process, a multiplicative disturbance model can be more natural than an additive disturbance with constant variance.
Variance can also be an explicit function of explanatory variables. A common parametric representation is
[ \operatorname{Var}(\varepsilon_i\mid\mathbf{x}_i)
\sigma^2 h(\mathbf{z}_i,\boldsymbol{\gamma}), ]
where (h) is positive and (\mathbf{z}_i) contains variables associated with dispersion. The variables governing variance may overlap with those governing the conditional mean, but the two components remain conceptually distinct.
Heteroscedasticity is not equivalent to autocorrelation. The former describes unequal conditional variances, whereas the latter describes dependence between disturbances from different observations. Both properties can occur simultaneously, in which case the covariance matrix contains nonconstant diagonal elements as well as nonzero off-diagonal elements.
Detection and formal testing
Graphical residual analysis represents heteroscedasticity through changes in residual spread across fitted values or covariates. Because raw residuals have leverage-dependent variances even under a homoscedastic model, standardized or studentized residuals provide a more directly comparable representation. A funnel-shaped residual distribution is one recognizable configuration, although heteroscedastic variance functions can also be nonmonotonic or concentrated within subpopulations.
Formal tests translate residual patterns into restrictions on an auxiliary regression. The Breusch–Pagan test, developed by Trevor Breusch and Adrian Pagan, examines whether squared residuals are systematically related to specified explanatory variables. Under its standard formulation, the test statistic has an asymptotic chi-squared distribution when the null hypothesis of constant variance holds.
Halbert White developed a more general test in which squared residuals are related to regressors, their squares, and cross-products. The resulting statistic can detect a broad class of variance departures, but it can also respond to misspecification of the conditional mean. Rejection therefore establishes a failure of the tested moment restrictions rather than identifying a unique variance mechanism.
The Goldfeld–Quandt test, associated with Stephen Goldfeld and Richard Quandt, compares residual variation between portions of a sample ordered by a variable believed to govern dispersion. Its construction corresponds most directly to a monotonic variance change and depends more strongly on the proposed ordering than tests based on general auxiliary regressions.
Estimation and inference
When the covariance matrix is known up to scale, generalized least squares transforms the model so that the transformed disturbances have constant variance. For diagonal (\Omega), this becomes weighted least squares, with weights inversely proportional to the conditional variances:
[ \widehat{\boldsymbol{\beta}}_{\mathrm{WLS}}
(X^{\mathsf T}WX)^{-1}X^{\mathsf T}W\mathbf{y}, \qquad W=\Omega^{-1}. ]
Observations with lower conditional variance then contribute more strongly to coefficient estimation because they carry more information about the conditional mean.
When the variance function contains unknown parameters, feasible generalized least squares replaces the unknown covariance matrix with an estimate. Its efficiency depends on the adequacy of the variance specification, and an incorrect variance model can remove the advantage expected from weighting. The mean model and the variance model therefore enter as separate components of the estimator’s statistical assumptions.
An alternative retains the ordinary least-squares coefficient estimates and replaces the homoscedastic covariance calculation with a heteroscedasticity-consistent covariance estimator. A basic form is
[ \widehat{\operatorname{Var}}_{\mathrm{HC0}} (\widehat{\boldsymbol{\beta}})
(X^{\mathsf T}X)^{-1} \left( \sum_{i=1}^{n} \widehat{\varepsilon}_i^2 \mathbf{x}_i\mathbf{x}_i^{\mathsf T} \right) (X^{\mathsf T}X)^{-1}. ]
The HC1, HC2, and HC3 variants modify the residual contribution to account for sample size or leverage. These estimators provide asymptotically valid inference under broad forms of conditional heteroscedasticity, while leaving the fitted ordinary least-squares regression line unchanged.
A transformation of the response can alter both the mean relationship and the variance structure. For a positive response with approximately proportional errors, a logarithmic transformation converts multiplicative variation into additive variation on the transformed scale. Because the transformed model describes a different conditional quantity, retransformation to the original scale involves the distribution of the transformed disturbance and is not merely an inversion of the fitted logarithm.
Conditional heteroscedasticity in time series
In time-series analysis, the term often refers to variance that changes predictably with past information. A process can have constant unconditional variance while exhibiting time-varying conditional variance. This distinction is central to financial return models, where periods of large innovations tend to be followed by further periods of elevated variation.
Robert F. Engle formalized this behavior through the autoregressive conditional heteroscedasticity model. In an ARCH model, the current conditional variance depends on squared past innovations:
[ \varepsilon_t=\sigma_t z_t, \qquad \sigma_t^2
\alpha_0+\sum_{j=1}^{q}\alpha_j\varepsilon_{t-j}^2. ]
The generalized ARCH model, introduced by Tim Bollerslev, additionally allows the conditional variance to depend on its own past values. These models treat heteroscedasticity as a substantive dynamic process rather than solely as a complication in regression standard errors.