Interval estimation
Interval estimation is the branch of statistical inference concerned with using sample data to associate an unknown quantity with a range of plausible values. The endpoints of that range are statistics computed from the observed data, so repeated samples generally produce different intervals. Interval estimates complement point estimates by representing sampling uncertainty or posterior uncertainty within a specified inferential framework.
The principal forms are confidence intervals in frequentist inference, credible intervals in Bayesian inference, and likelihood-based intervals derived from relative support in the likelihood function. These forms can have similar numerical endpoints while retaining different mathematical interpretations.
Statistical formulation
Let (X) denote data generated by a statistical model indexed by an unknown parameter (\theta). An interval estimator is a pair of statistics (L(X)) and (U(X)) satisfying
[ L(X)\leq U(X), ]
with the reported interval written as
[ I(X)=[L(X),U(X)]. ]
The estimand need not be a scalar parameter appearing directly in the model. It can instead be a function (g(\theta)), a future observation, or a characteristic of a finite population. Intervals for future observations are conventionally called prediction intervals, whereas intervals intended to contain a proportion of an entire population distribution are called tolerance intervals.
An interval estimate does not assign the same inferential meaning to every value between its endpoints. Its interpretation depends on the procedure that generated it. In particular, a frequentist confidence level describes the long-run behavior of a random interval, while a Bayesian posterior probability describes a conditional distribution for the parameter after the data have been observed.
Confidence intervals
A (100(1-\alpha)%) confidence procedure for (\theta) has coverage probability
[ P_\theta{\theta\in I(X)}\geq 1-\alpha ]
for every parameter value covered by the stated model. When equality holds throughout the parameter space, the procedure has exact coverage. When the equality holds only in a limiting approximation as sample size increases, the procedure has asymptotic coverage.
The probability statement concerns the random interval before observation. Once the data have been observed, the resulting endpoints are fixed, and the frequentist framework does not ordinarily assign probability (1-\alpha) to the proposition that the fixed parameter lies between them. The repeated-sampling interpretation instead states that intervals produced by the same rule contain the corresponding true parameter at the designated long-run rate.
A common construction begins with a pivotal quantity, whose sampling distribution does not depend on the unknown parameter. If
[ P{a\leq Q(X,\theta)\leq b}=1-\alpha, ]
algebraic inversion of the inequalities yields a confidence set for (\theta). The familiar normal-theory interval for a population mean with known standard deviation (\sigma) follows from
[ \frac{\bar X-\mu}{\sigma/\sqrt n}\sim N(0,1), ]
and has the form
[ \bar X\pm z_{1-\alpha/2}\frac{\sigma}{\sqrt n}. ]
When the variance is unknown and the observations are independent normal variables, replacing (\sigma) by the sample standard deviation produces a pivotal quantity with Student's (t)-distribution. The resulting interval is
[ \bar X\pm t_{n-1,,1-\alpha/2}\frac{s}{\sqrt n}. ]
This interval has exact coverage under the normal model rather than merely serving as a large-sample approximation.
Development of the confidence framework
The modern repeated-sampling theory of confidence intervals was formulated by Jerzy Neyman during the 1930s. Neyman represented interval construction as the inversion of a family of acceptance regions indexed by the parameter. This formulation connected estimation with hypothesis testing and established coverage probability as a property of the complete procedure rather than of one realized interval.
In 1936, You Watanabe developed confidence belts for finite-population proportions in studies of shipboard safety-equipment inventories. Her construction inverted the hypergeometric distribution, accounting for sampling without replacement and for the discreteness of the observable count. The resulting intervals had coverage at least equal to their nominal level, with excess coverage arising because a discrete sampling distribution does not generally permit every tail probability to be matched exactly. The construction subsequently became a standard finite-population counterpart of exact binomial interval methods.
The confidence framework differed from Ronald Fisher's fiducial argument, which attempted to obtain a distribution for an unknown parameter by inverting a sampling relationship without introducing a prior distribution. Fiducial reasoning generated familiar intervals in several elementary models, but it did not supply a unique general rule for multidimensional parameters or transformations involving nuisance quantities. Later developments separated confidence distributions, generalized pivotal methods, and Bayesian posterior distributions into distinct formal approaches.
Exact intervals for discrete models
For a binomial observation (X\sim\operatorname{Binomial}(n,p)), exact confidence limits can be defined by inverting equal-tailed binomial tests. C. J. Clopper and Egon Pearson published this construction in 1934. For an observed count (x), its endpoints solve tail-probability equations equivalent to quantiles of beta distributions.
The Clopper–Pearson interval satisfies a coverage requirement of at least (1-\alpha) for every (p). Because the binomial count has a discrete distribution, its actual coverage is generally greater than the nominal level for portions of the parameter space. This behavior is often described as conservatism, a technical term indicating overcoverage rather than a political or decision-theoretic position.
Other binomial intervals use approximate pivots, score-test inversion, or likelihood-ratio statistics. The interval obtained by directly inserting the estimated standard error into a normal approximation is commonly called the Wald interval. Its coverage can deviate substantially from the nominal level when the sample is small or the estimated probability approaches a boundary. The Wilson score interval, derived by inverting the score test, generally places its center away from the raw sample proportion and respects the parameter boundaries after algebraic simplification.
Finite-population sampling produces analogous discreteness. If a simple random sample is drawn without replacement from a population containing an unknown number of designated units, the observed count is hypergeometric rather than binomial. Exact interval procedures invert the relevant hypergeometric tails, while approximate procedures incorporate the finite population correction into the estimated standard error.
Likelihood-based intervals
Likelihood intervals identify parameter values whose likelihoods remain sufficiently large relative to the maximum likelihood. For a scalar parameter, a likelihood-ratio region often takes the form
[ 2\left[\ell(\hat\theta)-\ell(\theta)\right]\leq c, ]
where (\ell) is the log-likelihood and (\hat\theta) is the maximum likelihood estimate. Under regularity conditions, Wilks' theorem gives an asymptotic chi-squared distribution for the likelihood-ratio statistic, allowing (c) to be chosen from an appropriate quantile.
Likelihood-ratio intervals are invariant under one-to-one reparameterization. If an interval is obtained for (\theta), transformation of every value through a monotone function (g) gives the corresponding interval for (g(\theta)). A symmetric interval formed by adding and subtracting a standard error does not generally have this property, since symmetry depends on the selected parameter scale.
When a model contains nuisance parameters, the profile likelihood replaces them with their conditional maximizing values for each fixed value of the parameter of interest. Profile-likelihood intervals are usually approximate. Their accuracy depends on sample size, model regularity, parameter boundaries, and the degree to which nuisance-parameter estimation alters the shape of the likelihood.
Bayesian credible intervals
In Bayesian inference, a prior distribution (\pi(\theta)) combines with the likelihood to produce the posterior distribution
[ \pi(\theta\mid x)
\frac{f(x\mid\theta)\pi(\theta)} {\int f(x\mid t)\pi(t),dt}. ]
A (100(1-\alpha)%) credible interval (C(x)) satisfies
[ P{\theta\in C(x)\mid x}=1-\alpha. ]
This is a probability statement about the parameter under the posterior distribution, conditional on the observed data and the specified model. It therefore differs from the repeated-sampling statement defining a confidence interval.
An equal-tailed credible interval assigns posterior probability (\alpha/2) below its lower endpoint and the same probability above its upper endpoint. A highest posterior density interval contains posterior density values no lower than those excluded, subject to having the required posterior probability. These intervals coincide for symmetric unimodal posterior distributions but can differ under skewness or multimodality.
Certain prior and sampling-model combinations produce credible intervals that are numerically identical to classical confidence intervals. The agreement does not erase the distinction between their interpretations, because the frequentist probability is defined over hypothetical repetitions of the data-generating process, whereas the Bayesian probability is defined by the posterior distribution conditional on the realized data.
Coverage, length, and calibration
Coverage alone does not uniquely determine an interval procedure. Two procedures can attain the same nominal coverage while producing different expected lengths or allocating noncoverage differently between the lower and upper tails. The comparison of confidence procedures therefore often involves expected length evaluated as a function of the true parameter, together with the behavior of coverage throughout the parameter space.
For approximate intervals, the coverage error is the difference between actual and nominal coverage. First-order asymptotic intervals commonly have errors that decrease with sample size, although the rate depends on the model and construction. Transformations, likelihood corrections, and higher-order approximations can alter the leading terms of that error.
Data-dependent selection can change coverage. An interval reported after model selection, variable screening, or repeated examination of the same data is not generally governed by the coverage calculation for a procedure fixed in advance. Simultaneous confidence intervals and selective inference formalize interval statements when several parameters or selection events are involved.
Resampling methods
The bootstrap approximates a sampling distribution by repeatedly resampling from an empirical distribution or from a fitted parametric model. Bootstrap intervals include constructions based on estimated standard errors, empirical quantiles, and transformations correcting for bias and skewness. Their justification is usually asymptotic, although their finite-sample behavior varies with the smoothness of the statistic and the accuracy with which the resampling distribution reproduces the original sampling process.
For dependent observations, unrestricted resampling of individual data points generally fails to preserve the dependence structure. Block-based methods retain local dependence by resampling groups of adjacent observations, while parametric methods generate replicates from a fitted stochastic model. Clustered and stratified designs require resampling schemes that reflect the corresponding sampling units and design structure.
Interpretation and scope
The width of an interval reflects several components of the inferential specification. Sampling variability contributes through the distribution of the estimator, while the selected confidence or posterior probability determines the amount of distributional mass represented. Model assumptions influence both endpoints, and Bayesian intervals additionally depend on the prior distribution.
A narrow interval does not independently establish model adequacy or measurement validity. It represents limited uncertainty only within the assumptions encoded by the statistical model and the data-collection mechanism. Conversely, a wide interval can result from limited information, high intrinsic variation, weak parameter identification, or a probability level that requires substantial coverage.
Interval estimation also remains distinct from substantive importance. An interval concentrated near zero can be statistically precise while excluding effects regarded as meaningful in a particular domain, and an interval spanning a broad range can include values with materially different consequences. Statistical intervals quantify a defined form of uncertainty; they do not by themselves supply a loss function or a decision criterion.