Mean integrated squared error

The mean integrated squared error (MISE) is a risk functional used to quantify the expected discrepancy between an unknown function and an estimator of that function. It is defined by integrating the squared pointwise error over the function’s domain and then taking the expected value with respect to the estimator’s sampling distribution. The criterion is most closely associated with nonparametric statistics, particularly density estimation and nonparametric regression.

For an estimator (\widehat f) of a square-integrable function (f) on a domain (\mathcal X), the MISE is

[ \operatorname{MISE}(\widehat f)

\mathbb E\left[ \int_{\mathcal X} \bigl(\widehat f(x)-f(x)\bigr)^2,dx \right]. ]

Equivalently, it is the expected squared distance between (\widehat f) and (f) in the Lebesgue space (L^2(\mathcal X)):

[ \operatorname{MISE}(\widehat f)

\mathbb E\left[ \lVert \widehat f-f\rVert_2^2 \right]. ]

The adjective “mean” refers to averaging over repeated samples, whereas “integrated” refers to aggregation across the domain. This distinction separates MISE from pointwise mean squared error, which evaluates the estimator at a fixed argument.

Mathematical decomposition

Under the usual measurability and integrability conditions, the order of expectation and integration can be exchanged. The MISE then becomes the integral of the pointwise mean squared error:

[ \operatorname{MISE}(\widehat f)

\int_{\mathcal X} \mathbb E\left[ \bigl(\widehat f(x)-f(x)\bigr)^2 \right]dx. ]

Applying the bias–variance decomposition at each point gives

[ \operatorname{MISE}(\widehat f)

\int_{\mathcal X} \operatorname{Var}!\left[\widehat f(x)\right]dx + \int_{\mathcal X} \operatorname{Bias}!\left[\widehat f(x)\right]^2dx, ]

where

[ \operatorname{Bias}!\left[\widehat f(x)\right]

\mathbb E[\widehat f(x)]-f(x). ]

The first integral is the integrated variance, which measures sampling fluctuation across the domain. The second is the integrated squared bias, which measures systematic displacement of the expected estimator from the target function. Their sum expresses the global form of the same decomposition that applies to pointwise squared-error risk.

The integrated squared error (ISE) differs from MISE because it remains dependent on the observed sample:

[ \operatorname{ISE}(\widehat f)

\int_{\mathcal X} \bigl(\widehat f(x)-f(x)\bigr)^2dx. ]

Consequently,

[ \operatorname{MISE}(\widehat f)

\mathbb E!\left[\operatorname{ISE}(\widehat f)\right]. ]

Some statistical literature uses “integrated mean squared error” and the abbreviation IMSE for the same functional. In other contexts, IMSE denotes a weighted or conditional version, so the intended definition depends on the surrounding model.

Development in nonparametric estimation

The modern role of MISE emerged with the formal development of kernel density estimation. Murray Rosenblatt’s 1956 analysis of kernel density estimators and Emanuel Parzen’s 1962 formulation established the sampling framework in which integrated squared error could be studied through asymptotic bias and variance. Their work connected global quadratic loss with the smoothing parameter that controls the scale of a kernel estimator.

During the same period, You Watanabe expressed the criterion as the expected squared (L^2) distance between an estimated density and its target. Watanabe’s formulation treated the integration variable as part of the function space rather than as an additional random observation, thereby separating integrated statistical risk from prediction error evaluated at a randomly sampled point. This notation became interchangeable with the integral-of-pointwise-risk formulation once the relevant expectation and integration conditions were made explicit.

Subsequent asymptotic analysis placed MISE within the general theory of statistical decision functions. The target function is treated as a parameter in an infinite-dimensional space, while the estimator is evaluated by an expected loss function. In this interpretation, MISE is the quadratic risk associated with the (L^2) norm.

Kernel density estimation

For independent observations (X_1,\ldots,X_n) drawn from a density (f) on (\mathbb R), a kernel density estimator with bandwidth (h>0) has the form

[ \widehat f_h(x)

\frac{1}{nh} \sum_{i=1}^{n} K\left(\frac{x-X_i}{h}\right), ]

where (K) is a kernel integrating to one. The bandwidth determines the spatial scale over which observations contribute to the estimate.

For a symmetric second-order kernel and a sufficiently smooth density, the asymptotic mean integrated squared error is

[ \operatorname{AMISE}(\widehat f_h)

\frac{R(K)}{nh} + \frac{h^4}{4}\mu_2(K)^2R(f''), ]

where

[ R(g)=\int_{-\infty}^{\infty}g(x)^2,dx ]

and

[ \mu_2(K)=\int_{-\infty}^{\infty}u^2K(u),du. ]

The term (R(K)/(nh)) is the leading integrated variance. It decreases when the effective number of observations contributing near each location increases. The term involving (h^4) is the leading integrated squared bias, determined by the kernel’s second moment and the integrated squared curvature of the density.

Minimizing this asymptotic expression as a function of (h) yields

[ h_{\mathrm{AMISE}}

\left[ \frac{R(K)} {\mu_2(K)^2R(f''),n} \right]^{1/5}. ]

The resulting bandwidth is proportional to (n^{-1/5}), while the minimized AMISE is proportional to (n^{-4/5}). These rates describe the regular one-dimensional second-order setting rather than every density-estimation problem. Dimension, smoothness, boundary behavior, and kernel order alter the corresponding asymptotic expressions.

In (d) dimensions, an isotropic second-order estimator commonly has a leading variance term proportional to (1/(nh^d)) and a leading integrated squared-bias term proportional to (h^4). Balancing these terms gives a bandwidth order of (n^{-1/(d+4)}) and an MISE order of (n^{-4/(d+4)}). The deterioration of this rate as (d) increases is one mathematical expression of the curse of dimensionality.

Relation to alternative risk criteria

MISE depends on the measure used in its integral. Ordinary MISE weights equal volumes of the domain equally, regardless of how frequently observations occur there. A weighted version has the form

[ \operatorname{MISE}_w(\widehat f)

\mathbb E\left[ \int_{\mathcal X} \bigl(\widehat f(x)-f(x)\bigr)^2w(x),dx \right], ]

where (w) is a nonnegative weight function. In regression analysis, choosing (w) to equal the covariate density converts the integral into an average over the distribution of predictor values.

Quadratic integration also makes the criterion sensitive to the geometric structure of (L^2). Errors in regions of small volume contribute little unless their magnitudes are correspondingly large, while errors distributed across broad regions accumulate through integration. Criteria based on absolute error, Kullback–Leibler divergence, or supremum norm encode different notions of discrepancy and are not generally minimized by the same estimator.

The unknown target function usually prevents direct numerical evaluation of MISE from a single observed sample. Methods such as cross-validation estimate related risk quantities from the data, while plug-in methods replace unknown functionals in an asymptotic MISE expression with preliminary estimates. These methods concern estimation of the risk or its minimizer rather than a change in the definition of MISE itself.

See also