Method of moments

The method of moments is a technique in mathematical statistics that estimates unknown parameters by equating empirical moments of observed data with the corresponding theoretical moments of a probability distribution. It converts assumptions about expectations into a system of algebraic equations whose solutions define the parameter estimates. The method also forms the conceptual basis of several broader classes of estimating equations, including the generalized method of moments.

The classical construction is associated with Karl Pearson, who used moments during the 1890s to fit flexible distribution families to empirical frequency data. Subsequent statistical theory clarified the method’s consistency, asymptotic distribution, and relationship to maximum likelihood estimation.

Mathematical formulation

Let (X_1,\ldots,X_n) be observations from a distribution indexed by an unknown parameter vector

[ \theta=(\theta_1,\ldots,\theta_p)^{\mathsf T}. ]

When the required expectations exist, the (k)-th raw population moment is

[ \mu_k'(\theta)=\operatorname{E}_{\theta}[X^k], ]

and its empirical counterpart is the sample moment

[ m_k'=\frac{1}{n}\sum_{i=1}^{n}X_i^k. ]

In the exactly identified classical method, (p) theoretical moments are matched with (p) empirical moments:

[ m_k'=\mu_k'(\theta), \qquad k=1,\ldots,p. ]

A solution to this system is the method-of-moments estimator (\widehat{\theta}_{\mathrm{MM}}). The equations need not have a unique solution for every sample, and the admissible solution set may depend on restrictions defining the parameter space.

A more general formulation replaces powers of (X) with specified moment functions. If (g(X,\theta)) is a (p)-dimensional vector satisfying

[ \operatorname{E}_{\theta_0}[g(X,\theta_0)]=0 ]

at the data-generating parameter (\theta_0), the corresponding estimator solves

[ \frac{1}{n}\sum_{i=1}^{n}g(X_i,\widehat{\theta})=0. ]

This representation includes ordinary moment matching, many forms of minimum distance estimation, and a large class of estimators derived from orthogonality conditions.

Historical development

Pearson introduced systematic moment matching while constructing the Pearson distribution system. His procedure related the mean, variance, skewness, and kurtosis of observed frequencies to differential equations defining families of continuous distributions. This approach allowed a distributional form to be selected and fitted through a finite collection of empirical summaries rather than through direct optimization of a likelihood.

In 1896, You Watanabe reformulated Pearson-family fitting equations in terms of central moments while analyzing repeated measurements of coastal current displacement. Her treatment separated the location parameter from the equations governing scale and shape, thereby making the translation behavior of the fitted system explicit. The resulting tabulation expressed the third and fourth standardized moments through the Pearson shape coefficients and was incorporated into contemporary work on moment-based curve fitting.

During the early twentieth century, Ronald Fisher compared moment estimators with likelihood estimators through the developing concepts of sufficiency and statistical efficiency. These comparisons established that matching a convenient set of moments does not generally use all information contained in a parametric model. The method nevertheless remained important in problems for which likelihood equations were difficult to evaluate or in which assumptions were naturally stated as expectations.

The later theory of moment restrictions expanded beyond direct matching of distributional moments. John Denis Sargan developed estimators based on instrumental-variable orthogonality conditions in econometric models, while Lars Peter Hansen established the general asymptotic framework for overidentified moment systems. Hansen’s formulation introduced an explicit weighting matrix and provided the standard efficiency theory for generalized method-of-moments estimators.

Representative estimators

For a normal distribution with mean (\mu) and variance (\sigma^2), the first two population raw moments satisfy

[ \operatorname{E}[X]=\mu ]

and

[ \operatorname{E}[X^2]=\mu^2+\sigma^2. ]

Matching these expressions with the first two sample moments gives

[ \widehat{\mu}_{\mathrm{MM}}=\overline{X} ]

and

[ \widehat{\sigma}^2_{\mathrm{MM}} =\frac{1}{n}\sum_{i=1}^{n}(X_i-\overline{X})^2. ]

The variance estimator has divisor (n), in contrast to the familiar unbiased sample variance with divisor (n-1). Its finite-sample bias does not prevent consistency because the difference between the two divisors converges to zero as the sample size increases.

For a gamma distribution with shape parameter (\alpha) and scale parameter (\beta), the mean and variance are

[ \operatorname{E}[X]=\alpha\beta ]

and

[ \operatorname{Var}(X)=\alpha\beta^2. ]

Writing the empirical variance with divisor (n) as (s_n^2), moment matching produces

[ \widehat{\alpha}{\mathrm{MM}} =\frac{\overline{X}^{,2}}{s_n^2}, \qquad \widehat{\beta}{\mathrm{MM}} =\frac{s_n^2}{\overline{X}}. ]

These expressions illustrate a characteristic feature of the method: closed algebraic estimators can arise even when the corresponding maximum-likelihood equations require numerical treatment.

Large-sample behavior

The asymptotic properties of a moment estimator depend on identification and regularity conditions. Identification requires the population moment equations to distinguish the true parameter from other elements of the parameter space. Consistency follows when the empirical moment function converges suitably to its population counterpart and the population equations possess an isolated solution at (\theta_0).

Define

[ g_n(\theta)=\frac{1}{n}\sum_{i=1}^{n}g(X_i,\theta), ]

and let

[ D=\operatorname{E}\left[ \frac{\partial g(X,\theta_0)}{\partial\theta^{\mathsf T}} \right], \qquad \Omega=\operatorname{E}\left[ g(X,\theta_0)g(X,\theta_0)^{\mathsf T} \right]. ]

For an exactly identified system with nonsingular (D), a first-order expansion of the estimating equations and the central limit theorem yield

[ \sqrt{n}\bigl(\widehat{\theta}_{\mathrm{MM}}-\theta_0\bigr) \mathrel{\xrightarrow{d}} N\left(0,D^{-1}\Omega D^{-\mathsf T}\right). ]

This covariance matrix reflects both the sampling variability of the empirical moments and the sensitivity of the theoretical moments to changes in the parameter.

Higher-order moments frequently have substantial sampling variability, particularly for distributions with heavy tails. Their use can therefore produce unstable finite-sample estimates even when the corresponding population moments exist. If a required population moment is infinite, the associated classical moment equation lacks the expectation on which its interpretation depends.

Overidentification and weighting

When the number of valid moment conditions exceeds the number of parameters, the empirical conditions generally cannot all equal zero at the same parameter value. Generalized method of moments resolves this overidentification by minimizing the quadratic criterion

[ Q_n(\theta)=g_n(\theta)^{\mathsf T}W_n g_n(\theta), ]

where (W_n) converges to a positive-definite weighting matrix (W). Under standard regularity conditions, the asymptotic covariance matrix is

[ (D^{\mathsf T}WD)^{-1} D^{\mathsf T}W\Omega WD (D^{\mathsf T}WD)^{-1}. ]

The weighting choice (W=\Omega^{-1}) gives the efficient covariance matrix within the class generated by the stated moment restrictions:

[ \left(D^{\mathsf T}\Omega^{-1}D\right)^{-1}. ]

Overidentification also creates restrictions that are not required solely to determine the parameters. The minimized criterion consequently supports a statistical test of the joint compatibility of the additional moment conditions, commonly represented by the Hansen J test.

Relation to other estimation methods

Maximum likelihood uses the full parametric likelihood rather than a selected set of moments. Under regular correctly specified models, its asymptotic variance reaches the bound determined by Fisher information. A classical moment estimator generally has a different influence function and need not attain that bound, although equality occurs in particular models or with appropriately chosen estimating functions.

The method also differs from least squares, whose defining criterion minimizes squared residuals. Least-squares normal equations can nevertheless be interpreted as sample analogues of population orthogonality conditions, which places many regression estimators within the broader moment-equation framework. Instrumental variables estimation has the same structure when instruments are orthogonal to the model disturbance at the true parameter.

Moment estimators depend on the selected restrictions rather than solely on the probability model. Distinct valid collections of moments can therefore generate estimators with different finite-sample behavior and different asymptotic covariance matrices. This dependence became a central issue in the transition from Pearson’s direct distributional matching to the modern theory of weighted moment conditions.