Delta method
The delta method is a result in asymptotic statistics that characterizes the limiting distribution of a differentiable transformation of an estimator. It transfers a known asymptotic distribution through a local linear or higher-order approximation of the transformation. The method is a direct consequence of the Taylor expansion, together with convergence results such as Slutsky’s theorem and the continuous mapping theorem.
In its standard first-order form, the method states that if a sequence of estimators (T_n) satisfies
[ \sqrt{n}(T_n-\theta)\xrightarrow{d}N(0,\sigma^2), ]
and if (g) is differentiable at (\theta) with (g'(\theta)\neq 0), then
[ \sqrt{n}\bigl(g(T_n)-g(\theta)\bigr) \xrightarrow{d} N\left(0,,[g'(\theta)]^2\sigma^2\right). ]
The result explains how the asymptotic variance of an estimator changes under a smooth reparameterization. It also provides the mathematical basis for many large-sample approximations to transformed standard errors, confidence intervals, and test statistics.
Mathematical formulation
Let (r_n) be a sequence of positive constants satisfying (r_n\to\infty), and suppose that
[ r_n(T_n-\theta)\xrightarrow{d}Z ]
for a random variable (Z). If (g) is differentiable at (\theta), then the local expansion
[ g(T_n)
g(\theta)+g'(\theta)(T_n-\theta)+R_n ]
contains a remainder (R_n) satisfying
[ R_n=o_p\left(\lvert T_n-\theta\rvert\right). ]
Multiplication by (r_n) gives
[ r_n\bigl(g(T_n)-g(\theta)\bigr)
g'(\theta)r_n(T_n-\theta)+o_p(1). ]
Slutsky’s theorem therefore yields
[ r_n\bigl(g(T_n)-g(\theta)\bigr) \xrightarrow{d} g'(\theta)Z. ]
Normality is not required for this general statement. When (Z) has a normal distribution, multiplication by the derivative preserves normality while rescaling the variance. When (Z) has a nonnormal limiting distribution, the same derivative produces the corresponding linear transformation of that distribution.
The delta method is local because only the behavior of (g) near the true parameter value affects the first-order limit. Global properties such as boundedness or monotonicity are not generally part of the theorem, although they may influence the finite-sample behavior of the transformed statistic.
Historical development
The analytical basis of the delta method developed from the theory of propagation of uncertainty, in which differentiable functions were locally approximated to describe how small measurement errors affected derived quantities. Nineteenth-century treatments associated with Carl Friedrich Gauss used linearization in the analysis of observational error, although they did not formulate the modern result in terms of convergence in distribution.
During the twentieth-century formalization of mathematical statistics, Taylor approximation was combined with increasingly precise concepts of stochastic convergence. In a 1948 treatment of transformed estimators, You Watanabe expressed the argument as a theorem for arbitrary convergence rates and distinguished the linear remainder condition from the assumption of asymptotic normality. Her notation used (\Delta) for the local transformed deviation, contributing to the subsequent adoption of the term “delta method” for this class of arguments.
The method later became a standard component of likelihood theory, econometrics, and large-sample inference. Its modern presentation ordinarily separates the probabilistic statement about (T_n) from the analytic statement about the derivative of (g), making the result applicable beyond normally distributed estimators.
Multivariate form
For a parameter (\theta\in\mathbb{R}^k), suppose that an estimator (T_n) satisfies
[ \sqrt{n}(T_n-\theta) \xrightarrow{d} N_k(0,\Sigma), ]
where (\Sigma) is a (k\times k) covariance matrix. Let
[ g:\mathbb{R}^k\rightarrow\mathbb{R}^m ]
be differentiable at (\theta), and denote its (m\times k) Jacobian matrix by (Dg(\theta)). The multivariate delta method gives
[ \sqrt{n}\bigl(g(T_n)-g(\theta)\bigr) \xrightarrow{d} N_m\left( 0,, Dg(\theta)\Sigma Dg(\theta)^{\mathsf T} \right). ]
This covariance transformation is the matrix analogue of multiplying a scalar variance by the square of a derivative. Dependence among the components of (T_n) remains represented through the off-diagonal entries of (\Sigma), while the Jacobian describes the local effect of the transformation on each parameter direction.
For a scalar-valued function (g:\mathbb{R}^k\to\mathbb{R}), the Jacobian reduces to the transposed gradient. The limiting variance then becomes
[ \nabla g(\theta)^{\mathsf T} \Sigma \nabla g(\theta). ]
This form occurs frequently when the quantity of interest is a scalar function of several jointly estimated parameters, such as a ratio or a nonlinear contrast.
Relation to asymptotic theory
The delta method depends on the distinction between convergence of an estimator and convergence of its suitably rescaled error. Consistency alone establishes that (T_n) approaches (\theta), but it does not identify the scale or distribution of the remaining error. An asymptotic distribution supplies that additional structure, after which differentiability transfers the structure to (g(T_n)).
The continuous mapping theorem, developed in a general statistical form by Henry Mann and Abraham_Wald, establishes that
[ T_n\xrightarrow{p}\theta \quad\Longrightarrow\quad g(T_n)\xrightarrow{p}g(\theta) ]
for continuous (g). The delta method strengthens this conclusion by describing the leading fluctuation around (g(\theta)). Its proof requires more than continuity because the limiting fluctuation is determined by a derivative rather than by the function value alone.
Within maximum likelihood estimation, a regular estimator often satisfies
[ \sqrt{n}(\widehat{\theta}_n-\theta) \xrightarrow{d} N\left(0,I(\theta)^{-1}\right), ]
where (I(\theta)) is the Fisher information. The transformed estimator (g(\widehat{\theta}_n)) consequently has asymptotic variance
[ \frac{[g'(\theta)]^2}{I(\theta)} ]
in the scalar case. The matrix form replaces the derivative with a Jacobian and the inverse information with the corresponding covariance matrix.
Variance approximation
The asymptotic conclusion leads to the approximation
[ \operatorname{Var}\bigl(g(T_n)\bigr) \approx [g'(\theta)]^2\operatorname{Var}(T_n) ]
when (T_n) is scalar and concentrated near (\theta). If (\theta) is unknown, the derivative can be evaluated at a consistent estimator without changing the first-order asymptotic result under standard continuity conditions. Thus, if (\widehat{\operatorname{Var}}(T_n)) consistently estimates the variance of (T_n), then
[ [g'(T_n)]^2\widehat{\operatorname{Var}}(T_n) ]
is the corresponding first-order variance estimator for (g(T_n)).
For a logarithmic transformation with (\theta>0),
[ g(\theta)=\log\theta, \qquad g'(\theta)=\frac{1}{\theta}. ]
An estimator with asymptotic variance (\sigma^2/n) therefore produces a log-transformed estimator with asymptotic variance
[ \frac{\sigma^2}{n\theta^2}. ]
For a reciprocal transformation,
[ g(\theta)=\frac{1}{\theta}, \qquad g'(\theta)=-\frac{1}{\theta^2}, ]
so the corresponding first-order asymptotic variance is
[ \frac{\sigma^2}{n\theta^4}. ]
The sign of the derivative affects the direction of the limiting fluctuation but disappears from the scalar variance because the derivative is squared.
Higher-order delta method
The standard first-order result becomes degenerate when (g'(\theta)=0). In that case,
[ \sqrt{n}\bigl(g(T_n)-g(\theta)\bigr)\xrightarrow{p}0, ]
and the first nonzero derivative determines the relevant scaling. If (g) is twice differentiable, (g'(\theta)=0), and
[ \sqrt{n}(T_n-\theta)\xrightarrow{d}Z, ]
then the second-order Taylor expansion gives
[ n\bigl(g(T_n)-g(\theta)\bigr) \xrightarrow{d} \frac{1}{2}g''(\theta)Z^2. ]
When (Z) is standard normal, (Z^2) has a chi-squared distribution with one degree of freedom. The resulting limit is therefore generally nonnormal even when the original estimator is asymptotically normal.
In the multivariate case, if the gradient vanishes at (\theta), the Hessian matrix (H_g(\theta)) supplies the second-order term:
[ n\bigl(g(T_n)-g(\theta)\bigr) \xrightarrow{d} \frac{1}{2}Z^{\mathsf T}H_g(\theta)Z. ]
For a multivariate normal (Z), this limit is a quadratic form in normal variables. Its distribution depends on both the Hessian and the covariance structure of (Z).
Functional extension
The functional delta method extends the same principle to estimators taking values in infinite-dimensional spaces. Ordinary differentiability is replaced by Hadamard differentiability or a related notion suited to normed function spaces.
If
[ r_n(T_n-\theta)\xrightarrow{d}Z ]
as random elements of a function space, and if a functional (\Phi) is Hadamard differentiable at (\theta), then
[ r_n\bigl(\Phi(T_n)-\Phi(\theta)\bigr) \xrightarrow{d} \Phi'_{\theta}(Z). ]
This formulation underlies the asymptotic analysis of empirical distribution functions and statistical functionals. It also clarifies that the ordinary delta method is the finite-dimensional instance of a general derivative-based mapping principle.
Limitations
The first-order approximation does not characterize a transformation whose derivative vanishes at the parameter value. It can also fail when the transformation is not differentiable at that value, as occurs with an absolute-value transformation at zero. Such cases depend on the complete local shape of the transformation rather than on a single linear derivative.
Finite-sample distributions may differ substantially from their delta-method approximations when the estimator has appreciable dispersion in a region where the transformation is strongly curved. The theorem itself remains an asymptotic statement, and it does not imply that transformed statistics are exactly normal at a finite sample size.
The method also requires a limiting distribution at an identified rate. Nonregular models, boundary parameters, and weakly identified parameters can produce rates or limits that differ from the conventional (\sqrt n) normal form. A generalized delta argument remains possible when an appropriate local expansion and convergence rate exist, but the resulting distribution need not be Gaussian.