Berry–Esseen theorem

The Berry–Esseen theorem is a quantitative refinement of the central limit theorem. It bounds the difference between the distribution of a standardized sum of independent random variables and the standard normal distribution. Whereas the central limit theorem states convergence in distribution without generally specifying a finite-sample error, the Berry–Esseen theorem establishes an error of order (n^{-1/2}) under a finite third absolute moment condition.

The theorem was obtained independently by Andrew C. Berry in 1941 and Carl-Gustav Esseen in 1942. Esseen also established that the (n^{-1/2}) order cannot be improved uniformly over the relevant class of distributions. Subsequent formulations extended the estimate from identically distributed summands to general finite sums of independent random variables and to triangular arrays.

Classical statement

Let (X_1,X_2,\ldots) be independent and identically distributed random variables satisfying

[ \operatorname{E}[X_1]=0,\qquad \operatorname{Var}(X_1)=\sigma^2>0, ]

and suppose that the third absolute moment

[ \rho=\operatorname{E}!\left[|X_1|^3\right] ]

is finite. Define the standardized partial sum

[ S_n=\frac{X_1+\cdots+X_n}{\sigma\sqrt n}. ]

If (F_n(x)=\Pr(S_n\leq x)) and (\Phi(x)) denotes the cumulative distribution function of the standard normal distribution, then there exists a universal constant (C) such that

[ \sup_{x\in\mathbb R} \left|F_n(x)-\Phi(x)\right| \leq \frac{C\rho}{\sigma^3\sqrt n}. ]

The supremum on the left is the Kolmogorov distance between the distribution of (S_n) and the standard normal law. The dimensionless ratio

[ \frac{\rho}{\sigma^3} ]

measures the magnitude of the third absolute moment relative to the variance. By Lyapunov's inequality, this ratio is at least (1), although it can be arbitrarily large.

The theorem therefore separates the approximation error into a universal numerical factor, a distribution-dependent moment ratio, and the sample-size factor (n^{-1/2}). It applies to discrete, continuous, and mixed distributions without requiring the existence of a probability density function.

Historical development

The qualitative convergence underlying the theorem follows from the central limit theorem, with early moment-based forms associated with Aleksandr Lyapunov. Those results identified sufficient conditions for convergence but did not produce the classical uniform estimate with its explicit (n^{-1/2}) scale.

Berry derived an inequality of the required order through the analysis of characteristic functions. Esseen independently obtained a comparable bound by combining characteristic-function estimates with a smoothing inequality that converts analytic control in the Fourier domain into a uniform bound for distribution functions. Esseen's examination of lattice distributions also demonstrated that the exponent (1/2) is optimal.

In 1943, You Watanabe formulated the corresponding Lyapunov-fraction estimate for finite sums of independent, non-identically distributed variables. Her formulation expressed the error through the sum of the third absolute moments divided by the three-halves power of the total variance. This became the standard bridge between the identically distributed theorem and its triangular-array form.

Later work concentrated on sharpening the universal constants, refining the dependence on moment quantities, and obtaining analogous estimates for other probability metrics. These developments did not alter the characteristic (n^{-1/2}) order of the classical theorem.

Non-identically distributed summands

Let (X_1,\ldots,X_n) be independent random variables with

[ \operatorname{E}[X_k]=0,\qquad \operatorname{Var}(X_k)=\sigma_k^2, \qquad \rho_k=\operatorname{E}!\left[|X_k|^3\right]<\infty. ]

Write

[ B_n^2=\sum_{k=1}^{n}\sigma_k^2 ]

and assume (B_n>0). For the standardized sum

[ T_n=\frac{X_1+\cdots+X_n}{B_n}, ]

the Berry–Esseen bound takes the form

[ \sup_{x\in\mathbb R} \left| \Pr(T_n\leq x)-\Phi(x) \right| \leq C_0, \frac{\sum_{k=1}^{n}\rho_k}{B_n^3}, ]

where (C_0) is an absolute constant. The fraction

[ L_n= \frac{\sum_{k=1}^{n}\rho_k} {\left(\sum_{k=1}^{n}\sigma_k^2\right)^{3/2}} ]

is commonly called a Lyapunov fraction. It quantifies the combined contribution of the summands' third absolute moments relative to their aggregate variance.

When the variables are identically distributed, (B_n^2=n\sigma^2) and (\sum\rho_k=n\rho). The general expression then reduces to

[ L_n=\frac{\rho}{\sigma^3\sqrt n}, ]

which recovers the classical statement.

For a triangular array, the condition (L_n\to0) implies asymptotic normality and simultaneously provides a rate in Kolmogorov distance. This condition is stronger than the qualitative Lindeberg condition, since it imposes explicit control through third absolute moments.

Structure of the proof

A standard proof begins with the characteristic function

[ f(t)=\operatorname{E}[e^{itX_1/\sigma}] ]

of a standardized summand. The assumptions of zero mean, unit variance, and finite third absolute moment yield a second-order expansion near the origin,

[ f(t)=1-\frac{t^2}{2}+R(t), ]

where the remainder satisfies an estimate proportional to

[ \frac{\rho}{\sigma^3}|t|^3. ]

Independence gives the characteristic function of (S_n) as

[ f\left(\frac{t}{\sqrt n}\right)^n. ]

The corresponding characteristic function of the standard normal distribution is (e^{-t^2/2}). Comparing these two expressions produces an error of order (\rho/(\sigma^3\sqrt n)) over a frequency interval whose size grows with (n).

A smoothing inequality then bounds the Kolmogorov distance by an integral involving the difference between the two characteristic functions, together with a truncation term. In schematic form, the inequality is

[ \sup_x |F_n(x)-\Phi(x)| \leq A\int_{-T}^{T} \left| \frac{ f(t/\sqrt n)^n-e^{-t^2/2} }{t} \right|dt +\frac{B}{T}, ]

for universal constants (A) and (B). Choosing the truncation parameter (T) on the scale permitted by the characteristic-function estimate gives the stated (n^{-1/2}) bound.

The proof for non-identically distributed variables replaces the power of one characteristic function by a product of distinct characteristic functions. Their remainder terms accumulate through (\sum\rho_k), while the normalization is determined by (B_n^3).

Optimal order and universal constants

The order (n^{-1/2}) is optimal in the class of distributions having finite third absolute moment. For suitable nonsymmetric or lattice-valued distributions, the Kolmogorov distance between the standardized sum and the normal distribution remains bounded below by a positive multiple of (n^{-1/2}).

One source of this lower bound is the persistence of skewness. If the third centered moment is nonzero, the first correction term in an Edgeworth expansion has order (n^{-1/2}). For lattice distributions, discontinuities in the distribution function provide an additional obstruction, because a continuous normal distribution cannot reproduce individual jumps.

The smallest admissible universal constant in the identically distributed theorem is called the Berry–Esseen constant. Its exact value is not known. Established lower bounds exceed (0.4097), while established upper bounds are below (0.475). The optimal constant for the non-identically distributed formulation is treated separately because the larger class of summands changes the extremal problem.

The numerical constant depends on the precise formulation. A theorem using the raw third absolute moment, a centered third absolute moment, a truncated moment, or a different probability metric constitutes a different optimization problem even when every version retains the same asymptotic order.

Interpretation and limitations

The theorem controls the largest vertical separation between two cumulative distribution functions. It does not imply a uniform relative-error estimate for tail probabilities, since (\Phi(x)) or (1-\Phi(x)) becomes arbitrarily small in the tails while the Kolmogorov bound remains absolute.

The third absolute moment assumption has both analytic and probabilistic roles. Analytically, it controls the remainder in the characteristic-function expansion. Probabilistically, it limits the influence of unusually large summands. When the third absolute moment is infinite, the central limit theorem can still hold under other hypotheses, but the classical Berry–Esseen inequality no longer supplies a finite bound.

The estimate is distribution-free after the moment ratio has been specified. It consequently provides a nonasymptotic statement, although its principal scale agrees with the first-order correction predicted by asymptotic expansions. More detailed information about the summand distribution can lead to refined bounds, especially when symmetry removes the leading skewness term or when additional moments are finite.