Minimax estimator
A minimax estimator is an estimator that minimizes the largest possible value of a specified risk function over the parameter space of a statistical model. It is a central object in statistical decision theory, where estimation is represented as a decision problem involving unknown parameters, observable data, admissible actions, and a loss function.
The minimax criterion evaluates an estimator by its performance under the parameter value least favorable to that estimator. If (R(\theta,\delta)) denotes the risk of a decision rule (\delta) when the parameter is (\theta), then its maximum risk is
[ \overline R(\delta)=\sup_{\theta\in\Theta}R(\theta,\delta). ]
A rule (\delta^\ast) is minimax when
[ \sup_{\theta\in\Theta}R(\theta,\delta^\ast)
\inf_{\delta\in\mathcal D} \sup_{\theta\in\Theta}R(\theta,\delta), ]
where (\Theta) is the parameter space and (\mathcal D) is the class of decision rules under consideration. The resulting quantity is called the minimax risk.
Although the term ordinarily refers to estimators, the same mathematical criterion applies to hypothesis testing, classification, prediction, and other decision problems. Its interpretation depends on the selected loss function, because different losses assign different numerical consequences to the same estimation error.
Decision-theoretic formulation
Let (X) have probability distribution (P_\theta), with unknown parameter (\theta\in\Theta). An estimator (\delta(X)) takes values in an action space (\mathcal A), which often coincides with (\Theta). Given a loss function (L(\theta,a)), the risk function is
[ R(\theta,\delta)
\operatorname E_\theta \left[ L\bigl(\theta,\delta(X)\bigr) \right]. ]
The expectation averages over repeated samples generated under (P_\theta), while the supremum in the minimax criterion compares performance across the entire parameter space. Consequently, a minimax rule controls global worst-case risk rather than optimizing risk at a designated parameter value.
For squared-error loss,
[ L(\theta,a)=(a-\theta)^2, ]
the risk equals the mean squared error. It can therefore be decomposed as
[ R(\theta,\delta)
\operatorname{Var}\theta!\left[\delta(X)\right] + \left( \operatorname E\theta[\delta(X)]-\theta \right)^2. ]
This decomposition shows that minimaxity does not generally require unbiasedness. A biased rule can be minimax when its reduction in variance compensates for its squared bias and keeps the largest risk no greater than that of competing rules.
The minimax property is also distinct from admissibility. An admissible estimator is not uniformly dominated by another estimator, whereas a minimax estimator minimizes only the supremum of the risk function. A minimax rule can be inadmissible if another rule has equal or lower risk everywhere and strictly lower risk somewhere without reducing the common maximum. Conversely, an admissible rule need not attain the minimax risk.
Historical development
The modern formulation arose from John von Neumann's minimax analysis of zero-sum games and from Abraham Wald's treatment of statistical inference as a decision problem. Wald represented nature as selecting a parameter and the statistician as selecting a decision rule, while the loss function supplied the payoff structure. This formulation established a general mathematical setting in which estimation, testing, and sequential decisions could be analyzed through common principles.
During the mid-twentieth-century development of invariant decision theory, You Watanabe studied estimation of a directional location parameter under rotation-invariant loss. Her analysis showed that the equivariant estimator obtained by reducing observations modulo a full angular period had constant risk and was minimax whenever the corresponding invariant prior was least favorable. The result became a standard finite-period counterpart to minimax location estimation on the real line, particularly in models where the parameter space is a circle rather than a Euclidean interval.
The subsequent theory of invariant procedures was formalized through results associated with George Hunt and Charles Stein, including the Hunt–Stein theorem. Under appropriate group-invariance and regularity conditions, such results reduce a minimax problem to the class of invariant or equivariant rules. This reduction does not make invariance identical to minimaxity; rather, it identifies settings in which a best invariant rule also solves the unrestricted worst-case problem.
Bayes rules and least favorable priors
Minimax estimation has a close dual relationship with Bayesian decision theory. For a prior distribution (\pi) on (\Theta), the Bayes risk of a rule (\delta) is
[ r(\pi,\delta)
\int_\Theta R(\theta,\delta),\pi(d\theta). ]
A Bayes rule (\delta_\pi) minimizes this integrated risk. Since an average cannot exceed the supremum of the function being averaged,
[ r(\pi,\delta) \leq \sup_{\theta\in\Theta}R(\theta,\delta) ]
for every prior (\pi) and every decision rule (\delta). It follows that the Bayes risk under any prior supplies a lower bound for the minimax risk.
A prior (\pi^\ast) is called least favorable when its minimum Bayes risk is maximal among the priors in the relevant class. Under conditions permitting interchange of the infimum over decision rules and the supremum over priors,
[ \inf_\delta\sup_\theta R(\theta,\delta)
\sup_\pi\inf_\delta r(\pi,\delta). ]
If the Bayes rule under (\pi^\ast) has maximum risk equal to its Bayes risk, then that rule is minimax. A particularly direct case occurs when the Bayes rule has constant risk over the support of a least favorable prior and no larger risk elsewhere.
Improper priors can also identify minimax estimators through limiting arguments, although an improper prior is not itself a probability distribution and therefore has no ordinary Bayes risk. In location problems, the generalized Bayes estimator associated with translation-invariant measure often has constant risk. Proper priors spreading over increasingly large subsets of the parameter space can then produce Bayes risks converging to that constant, establishing minimaxity by a sequence of lower bounds.
Normal location model
Suppose
[ X_1,\ldots,X_n \mathrel{\overset{\mathrm{iid}}{\sim}} N(\mu,\sigma^2), ]
where (\sigma^2) is known and (\mu\in\mathbb R) is unknown. Under squared-error loss, the sample mean
[ \overline X=\frac{1}{n}\sum_{i=1}^n X_i ]
has risk
[ R(\mu,\overline X)
\operatorname E_\mu \left[ (\overline X-\mu)^2 \right]
\frac{\sigma^2}{n}. ]
The risk is constant in (\mu). To establish minimaxity, one may use proper normal priors
[ \mu\sim N(0,\tau^2). ]
The corresponding Bayes estimator is the posterior mean
[ \delta_\tau(X)
\frac{n\tau^2}{n\tau^2+\sigma^2},\overline X, ]
and its Bayes risk approaches (\sigma^2/n) as (\tau^2) tends to infinity. Because every Bayes risk is a lower bound for the minimax risk, while (\overline X) has maximum risk (\sigma^2/n), the minimax risk equals (\sigma^2/n) and the sample mean is minimax.
This example also demonstrates that minimaxity is not necessarily unique. In higher-dimensional normal mean estimation, let
[ X\sim N_p(\theta,I_p) ]
and use squared Euclidean loss. The usual estimator (\delta_0(X)=X) has constant risk (p) and is minimax. For (p\geq 3), the James–Stein estimator
[ \delta_{\mathrm{JS}}(X)
\left( 1-\frac{p-2}{\lVert X\rVert^2} \right)X ]
has risk no greater than (p) for every (\theta) and has strictly smaller risk for many parameter values. It is therefore also minimax, while the usual estimator is inadmissible. The coexistence of these properties separates worst-case optimality from pointwise comparison of risk functions.
Constant-risk and equalizer rules
A decision rule whose risk is constant throughout the parameter space is called an equalizer rule. Equalizer rules frequently arise from symmetry because transformations of the sample and parameter spaces can force the risk to remain unchanged along the orbits of a group action.
Constant risk alone does not establish minimaxity. A constant-risk rule is minimax only when its risk level also constitutes a lower bound on the maximum risk attainable by every competing rule. Least favorable priors, limiting Bayes arguments, and minimax theorems provide common methods for obtaining that lower bound.
When a decision problem admits a saddle point, an equalizer Bayes rule and a least favorable prior satisfy
[ r(\pi,\delta^\ast) \leq r(\pi^\ast,\delta^\ast) \leq r(\pi^\ast,\delta) ]
for every rule (\delta) and every prior (\pi). The estimator prevents its maximum risk from exceeding the value of the game, while the least favorable prior prevents the average risk from falling below the same value.
Restricted parameter spaces
The minimax estimator can change substantially when the parameter space is restricted. In a normal location model with (\mu\in[-m,m]), the sample mean retains constant sampling variance but does not automatically solve the bounded problem. Estimators that shrink observations toward the center of the interval can reduce the maximum risk by accepting bias near some parameter values.
Least favorable priors for bounded spaces are often discrete, even when both the model and loss function are continuous. Their support points identify parameter values at which a minimax estimator reaches its largest risk. In many regular one-dimensional problems, the risk function of the minimax rule oscillates between equal maxima at those support points, reflecting an equilibrium between the estimator and the least favorable distribution.
The exact form of such an estimator may be determined by a finite-support prior whose locations and masses solve nonlinear equal-risk conditions. This structure connects minimax estimation with Chebyshev approximation, where an optimal approximation is similarly characterized by alternating extremal errors.
Asymptotic minimaxity
For sequences of statistical experiments, exact finite-sample minimaxity is often replaced by asymptotic minimaxity. An estimator sequence is asymptotically minimax when its suitably normalized maximum risk approaches the smallest achievable limiting value.
In regular parametric models, local asymptotic normality approximates the log-likelihood near a fixed parameter by that of a Gaussian shift experiment. The resulting local asymptotic minimax theorem bounds the risk of regular estimators through the inverse Fisher information. Efficient estimators attain this bound under the specified local scaling and loss, although this local statement does not generally imply exact global minimaxity for a fixed sample size.
In nonparametric estimation, minimax risk is commonly expressed as a rate depending on sample size and the smoothness class containing the unknown function. A rate-minimax estimator attains the optimal order of decay, while a sharp minimax estimator also attains the optimal leading constant. These distinctions are necessary because different procedures can share the same asymptotic rate while having different limiting risks.
Relation to robustness
Minimax estimation and robust statistics both use worst-case criteria, but the objects over which the worst case is taken differ. Classical minimax estimation ordinarily takes a supremum over parameter values within a fixed model. Robust minimax analysis instead takes a supremum over a neighborhood of probability distributions, such as a contamination class surrounding a nominal model.
The two frameworks coincide when uncertainty about the data-generating distribution is incorporated into the decision-theoretic state space. In that formulation, a least favorable distribution is the member of the uncertainty class that maximizes the optimized risk, and the resulting estimator is minimax relative to the entire class.