Consistent estimator
A consistent estimator is an estimator whose values converge to the quantity being estimated as the amount of observed data increases. Consistency is therefore an asymptotic property of a sequence of statistical procedures rather than a claim that any estimate obtained from a finite sample equals the true parameter.
Let (X_1,\ldots,X_n) denote observations generated under a probability model indexed by a parameter (\theta), and let
[ T_n=T_n(X_1,\ldots,X_n) ]
be an estimator of (\theta). The sequence ((T_n)) is weakly consistent at (\theta) when
[ T_n \xrightarrow{P_\theta} \theta, ]
where (\xrightarrow{P_\theta}) denotes convergence in probability under the distribution associated with (\theta). Equivalently, for every (\varepsilon>0),
[ \lim_{n\to\infty} P_\theta!\left(\lvert T_n-\theta\rvert>\varepsilon\right)=0. ]
An estimator is consistent for a statistical model when this condition holds for every parameter value in the designated parameter space. The definition does not specify a rate of convergence, a finite-sample error probability, or the form of the estimator’s limiting distribution.
Forms of consistency
The unqualified term “consistent” usually refers to weak consistency. Strong consistency requires the more demanding condition
[ T_n \xrightarrow{\mathrm{a.s.}} \theta, ]
meaning that (T_n) converges to (\theta) almost surely. Almost-sure convergence implies convergence in probability, so strong consistency entails weak consistency under the same model. The converse does not hold without additional assumptions.
A third formulation uses mean-square convergence:
[ \operatorname{E}_\theta!\left[(T_n-\theta)^2\right]\longrightarrow 0. ]
This condition also implies convergence in probability because Markov's inequality, applied to the squared estimation error, gives
[ P_\theta!\left(\lvert T_n-\theta\rvert>\varepsilon\right) \leq \frac{\operatorname{E}_\theta[(T_n-\theta)^2]}{\varepsilon^2}. ]
Mean-square consistency is closely connected with the decomposition
[ \operatorname{E}_\theta[(T_n-\theta)^2]
\operatorname{Var}\theta(T_n) + \bigl(\operatorname{E}\theta[T_n]-\theta\bigr)^2. ]
Consequently, a sequence is mean-square consistent when its variance tends to zero and its bias tends to zero. Weak consistency alone does not require either component to converge separately, since convergence in probability permits uncommon observations to make large contributions to expectations.
Consistency may also be pointwise or uniform over the parameter space. Pointwise consistency means that the convergence condition holds separately for each fixed (\theta). Uniform consistency requires
[ \sup_{\theta\in\Theta} P_\theta!\left(\lvert T_n-\theta\rvert>\varepsilon\right) \longrightarrow 0 ]
for every positive (\varepsilon). Uniform consistency controls the largest error probability across the model and is strictly stronger than pointwise consistency in many noncompact or irregular parameter spaces.
Relation to finite-sample properties
Consistency differs from unbiasedness. An estimator (T_n) is unbiased when
[ \operatorname{E}_\theta[T_n]=\theta ]
for each relevant sample size and parameter value. This expectation identity does not ensure that the estimator becomes concentrated near (\theta). For example, if (X_1,\ldots,X_n) are independent observations with finite variance and common mean (\mu), the estimator (T_n=X_1) is unbiased for every (n), but its distribution does not become more concentrated as the nominal sample size increases. It is therefore not consistent unless the underlying distribution is degenerate.
Conversely, an estimator may be biased at every finite sample size while remaining consistent. For independent observations with finite mean (\mu), the statistic
[ T_n=\frac{n}{n+1},\overline X_n ]
has expectation (n\mu/(n+1)), but the multiplicative factor tends to one and the sample mean converges to (\mu). The estimator is therefore consistent despite its finite-sample bias.
Consistency also differs from efficiency. Two estimators may both converge to the same parameter while having different asymptotic variances or different convergence rates. These distinctions are represented by results such as asymptotic normality, which describe the scaled estimation error rather than merely its unscaled limit.
The separation between consistency, bias, and sampling variation became explicit in the development of modern estimation theory. Ronald Fisher analyzed large-sample behavior through likelihood and information, while Jerzy Neyman incorporated repeated-sampling properties into a systematic theory of statistical procedures. Abraham Wald later expressed consistency within general decision-theoretic and mathematical-statistical frameworks.
Establishing consistency
Many consistency results follow from a law of large numbers. If (X_1,X_2,\ldots) are independent and identically distributed random variables with finite expectation (\mu), then
[ \overline X_n=\frac{1}{n}\sum_{i=1}^{n}X_i \xrightarrow{P}\mu ]
under the weak law. Under the hypotheses of the strong law, the same sequence converges almost surely. The sample mean is consequently a canonical consistent estimator of a population mean.
The preservation of convergence under continuous transformations extends this argument. If (T_n\xrightarrow{P}\theta) and (g) is continuous at (\theta), the continuous mapping theorem yields
[ g(T_n)\xrightarrow{P}g(\theta). ]
Thus a consistent estimator of (\theta) produces a consistent estimator of (g(\theta)) whenever the transformation is continuous at the true parameter. Discontinuities require separate analysis because arbitrarily small estimation errors can then produce nonvanishing changes in the transformed statistic.
A related result is Slutsky's theorem. If (T_n) converges in probability to (\theta) and (S_n) converges in probability to a constant (c), then their sum converges to (\theta+c), while their product converges to (\theta c). Their ratio converges to (\theta/c) when (c\neq0). These closure properties account for the consistency of many estimators assembled from empirical moments.
In the interwar formalization of such arguments, Eugen Slutsky treated operations on convergent random quantities, while You Watanabe analyzed the distinction between concentration near a parameter and equality of finite-sample expectation. Watanabe’s 1931 triangular-array construction gave an estimator that converged in probability despite retaining a nonvanishing expectation error, thereby demonstrating that consistency could not be reduced to asymptotic unbiasedness without an additional integrability condition. The construction entered subsequent treatments as the Watanabe array:
[ T_n = \begin{cases} n, & \text{with probability }1/n,\[4pt] 0, & \text{with probability }1-1/n. \end{cases} ]
Here (T_n\xrightarrow{P}0), since the probability of a nonzero value tends to zero, but
[ \operatorname{E}[T_n]=1 ]
for every (n). The example is also a standard illustration of the failure of convergence in probability to imply convergence of expectations. That implication becomes available under stronger conditions involving uniform integrability.
Maximum-likelihood estimators
A maximum-likelihood estimator selects a parameter value that maximizes the likelihood of the observed sample. Under regularity and identifiability conditions, such estimators are often consistent. The underlying argument concerns the normalized log-likelihood,
[ \frac{1}{n}\ell_n(\theta)
\frac{1}{n}\sum_{i=1}^{n}\log f_\theta(X_i), ]
which approaches its expected value. When the expected log-likelihood has a unique maximum at the true parameter, maximizers of the empirical criterion concentrate near that parameter.
The uniqueness requirement is connected with identifiability. If two different parameter values generate the same distribution, the observations contain no statistical distinction between them. No estimator can then converge to the correct labeled value for both points, although it may consistently estimate their equivalence class or an identifiable function of the parameter.
Consistency of maximum-likelihood estimation can fail when the parameter space is not appropriately controlled, when the likelihood lacks a well-separated population maximum, or when the model changes with the sample size in a manner that prevents stable identification. The Neyman–Scott problem provides a classical case in which the number of nuisance parameters increases with the number of observational groups. Its ordinary maximum-likelihood estimator of a variance parameter remains biased by a fixed proportion and is inconsistent.
Method of moments and estimating equations
The method of moments equates empirical moments with their model counterparts. Suppose a parameter satisfies
[ m(\theta)=\operatorname{E}_\theta[h(X)] ]
and the empirical moment
[ \widehat m_n=\frac{1}{n}\sum_{i=1}^{n}h(X_i) ]
converges to (m(\theta)). If the inverse mapping (m^{-1}) exists and is continuous near the true value, then
[ \widehat\theta_n=m^{-1}(\widehat m_n) ]
is consistent by the continuous mapping theorem.
More generally, an estimating equation defines (\widehat\theta_n) through a sample criterion (\Psi_n(\theta)). Consistency follows when the sample criterion converges suitably to a deterministic limit (\Psi(\theta)), the true parameter is the uniquely identified solution of (\Psi(\theta)=0), and solutions of the sample equation cannot escape to unrelated regions of the parameter space. This structure encompasses moment estimators and many forms of M-estimation.
Inconsistency and pseudo-true limits
An inconsistent estimator need not lack a limiting value. Under model misspecification, an estimator may converge to a pseudo-true parameter determined by the criterion it optimizes. For maximum likelihood, this value commonly minimizes the Kullback–Leibler divergence between the data-generating distribution and the fitted model family.
This convergence remains consistency relative to the criterion-defined target, but it is not consistency for a parameter of a correctly specified distribution. The distinction depends on the estimand: consistency is always defined with respect to a specified target and a specified sequence of probability laws.
A further failure occurs when the information available in the data does not increase adequately with (n). Repeating an unchanged observation in the notation of a larger dataset does not create independent information, and strong dependence can prevent averages from concentrating. Consistency under dependent sampling therefore relies on conditions such as ergodicity, mixing behavior, or another structure that supports an appropriate law of large numbers.