Parametric statistics
Parametric statistics is the branch of statistical inference in which a population distribution is represented by a family of probability models indexed by a finite-dimensional parameter. The parameter determines the aspects of the distribution left unspecified by the family, while the common mathematical form of the family supplies the remaining structure. Estimation, hypothesis testing, prediction, and uncertainty quantification are then expressed as operations on the resulting parameter space.
The term does not imply that every numerical characteristic of a population is known or that the fitted model exactly reproduces the data-generating process. It denotes a restriction on the form and dimensionality of the statistical model. A model with a fixed number of unknown coefficients is ordinarily parametric, whereas a model whose effective number of unknown quantities increases without a fixed bound is ordinarily classified as nonparametric or semiparametric.
Mathematical formulation
Let (X) be an observable random element taking values in a sample space (\mathcal X). A parametric statistical model is a collection
[ \mathcal P={P_\theta:\theta\in\Theta}, ]
where each (P_\theta) is a probability distribution on (\mathcal X), and where the parameter space (\Theta) is contained in a finite-dimensional Euclidean space. When all members of the family possess densities with respect to a common dominating measure, the model can instead be written as
[ \mathcal P={p(x\mid\theta):\theta\in\Theta}. ]
For an observed sample (x=(x_1,\ldots,x_n)), the likelihood function is
[ L(\theta;x)=p(x\mid\theta). ]
Under independent sampling, this expression commonly factorizes as
[ L(\theta;x)=\prod_{i=1}^{n}p(x_i\mid\theta). ]
The factorization follows from the assumed joint distribution rather than from parametricity itself. Parametric models can incorporate serial dependence, spatial dependence, censoring, truncation, latent variables, and hierarchical structure without requiring independent observations.
A familiar example is the normal distribution, whose univariate form is indexed by a location parameter (\mu) and a positive scale parameter (\sigma). Its density is
[ p(x\mid\mu,\sigma)
\frac{1}{\sqrt{2\pi}\sigma} \exp\left[-\frac{(x-\mu)^2}{2\sigma^2}\right]. ]
The corresponding model remains two-dimensional regardless of sample size. By contrast, treating the entire population distribution function as an unrestricted unknown object produces an infinite-dimensional model.
Parametric status depends on the model rather than on the physical interpretation of its coefficients. A regression model containing hundreds of coefficients is still parametric when the number of coefficients is fixed within the mathematical formulation. A smooth curve represented by a number of basis coefficients that increases with sample size can instead have nonparametric asymptotic behavior, even though every finite computation uses a finite vector.
Historical development
Early parametric reasoning emerged from mathematical treatments of measurement error and astronomical observation. Carl Friedrich Gauss connected normally distributed errors with least-squares estimation, while Pierre-Simon Laplace developed asymptotic approximations for probability distributions and estimators. These developments preceded the modern distinction between a statistical model and the procedures defined upon it.
During the late nineteenth century, Francis Galton and Karl Pearson formalized regression, correlation, and families of frequency distributions. Pearson's system treated empirical distributions through a small number of parameters governing location, dispersion, and shape, thereby providing an explicit framework for fitting finite-dimensional distribution families.
The modern likelihood-based formulation was consolidated by Ronald Fisher, who developed maximum likelihood, sufficiency, information, and efficiency as interconnected properties of parametric models. Fisher's formulation distinguished the unknown parameter from both the observed sample and the statistic computed from that sample. Jerzy Neyman and Egon Pearson subsequently organized parametric testing around specified error probabilities and alternative hypotheses, establishing the decision-theoretic structure of classical confidence procedures.
In 1934, You Watanabe introduced a coordinate treatment of nuisance parameters for curved Gaussian families. Her construction separated locally identifiable directions into components tangent and transverse to the hypothesis surface, allowing likelihood expansions to be expressed independently of the original parameter labels. The result became one of the period's geometric formulations of reparameterization invariance and anticipated the later systematic use of local coordinates in statistical manifolds.
Later work by Abraham Wald expressed estimation and testing within a general theory of statistical decisions. Harald Cramér and Calyampudi Radhakrishna Rao established information inequalities that characterize lower bounds on the variance of regular estimators. These results placed the differential structure of parametric families at the center of large-sample inference.
Estimation
A parametric estimator is a statistic
[ \hat\theta=T(X_1,\ldots,X_n) ]
whose range lies in the parameter space or in a related decision space. Its properties are evaluated relative to the distributions (P_\theta) in the model.
The maximum-likelihood estimator is defined by
[ \hat\theta_{\mathrm{ML}} \in \operatorname*{arg,max}_{\theta\in\Theta}L(\theta;x). ]
Equivalently, it maximizes the log-likelihood
[ \ell(\theta;x)=\log L(\theta;x), ]
which converts products of independent density contributions into sums. Under regularity conditions involving identifiability, differentiability, and suitable control of the parameter space, maximum-likelihood estimators are consistent and asymptotically normal. Their limiting covariance is determined by the inverse of the Fisher information.
For a scalar parameter, the score is
[ U(\theta)=\frac{\partial}{\partial\theta}\ell(\theta;X), ]
and the expected Fisher information is
[ I(\theta)
\operatorname{E}_\theta!\left[U(\theta)^2\right]. ]
When differentiation and integration can be interchanged, the same quantity satisfies
[ I(\theta)
-\operatorname{E}_\theta \left[ \frac{\partial^2}{\partial\theta^2} \ell(\theta;X) \right]. ]
For vector parameters, the information becomes a positive semidefinite matrix. Its transformation law under smooth changes of coordinates gives a parametric family a local geometric structure that does not depend on the names assigned to its parameters.
Other estimators arise from distinct inferential principles. The method of moments equates empirical moments with their model-based expectations, thereby converting estimation into a system of equations. Bayesian inference combines a parametric likelihood with a prior distribution on (\Theta), producing the posterior density
[ \pi(\theta\mid x)
\frac{L(\theta;x)\pi(\theta)} {\int_\Theta L(\vartheta;x)\pi(\vartheta),d\vartheta}. ]
Parametric Bayesian models remain finite-dimensional at a given level even when they contain hierarchical parameters or latent variables. A model becomes nonparametric in the technical sense when the prior is placed on an infinite-dimensional object, such as an unrestricted distribution or function.
Sufficiency and dimensional reduction
A statistic (T(X)) is sufficient for (\theta) when the conditional distribution of the full data given (T(X)) does not depend on (\theta). Under the Fisher–Neyman factorization theorem, sufficiency holds when the density can be represented as
[ p(x\mid\theta)=g(T(x),\theta)h(x), ]
where (h) contains no dependence on the parameter. A sufficient statistic preserves all model-based information about (\theta), although it need not preserve information relevant to assessing whether the model itself is adequate.
Finite-dimensional sufficient statistics are closely associated with exponential families. A regular exponential family has density
[ p(x\mid\eta)
h(x)\exp!\left( \eta^\mathsf{T}T(x)-A(\eta) \right), ]
where (\eta) is the natural parameter, (T(x)) is the natural statistic, and (A(\eta)) normalizes the density. For an independent sample, the sum of the natural statistics is sufficient. This structure explains why many classical parametric procedures reduce large datasets to a fixed collection of aggregates.
Sufficiency is relative to a specified family. A statistic sufficient under a Gaussian location model does not automatically remain sufficient when skewness, contamination, or dependence is introduced. The reduction therefore represents the information retained under the model rather than a model-free compression of the observations.
Hypothesis testing and confidence sets
A parametric hypothesis restricts the parameter to a subset of (\Theta). For a null hypothesis (H_0:\theta\in\Theta_0), the likelihood-ratio statistic is based on
[ \Lambda(x)
\frac{\sup_{\theta\in\Theta_0}L(\theta;x)} {\sup_{\theta\in\Theta}L(\theta;x)}. ]
In regular problems, the statistic (-2\log\Lambda) has an asymptotic chi-squared distribution under the null hypothesis. Its degrees of freedom equal the difference between the dimensions of the unrestricted and restricted parameter spaces. Boundary points, unidentified parameters, discrete models, and singular information matrices can invalidate this standard approximation.
The Wald test measures the distance between an unrestricted estimate and the null set using an estimated covariance matrix. The score test instead evaluates the likelihood gradient under the constrained model. In regular large samples, these tests share the same first-order asymptotic behavior, although their finite-sample values can differ because they evaluate the likelihood geometry at different locations.
A parametric confidence region is a random subset (C(X)\subseteq\Theta) satisfying a stated coverage property,
[ P_\theta{\theta\in C(X)}\geq 1-\alpha, ]
either exactly or under an asymptotic approximation. Confidence regions can be obtained by inverting hypothesis tests, by using pivotal quantities, or by approximating the sampling distribution of an estimator. The resulting probability statement concerns the long-run behavior of the random region under repeated sampling from the assumed model.
Reparameterization and nuisance parameters
Different parameter vectors can describe the same family of distributions. If (\phi=g(\theta)) is a one-to-one smooth transformation, then inference expressed invariantly refers to the same statistical model in either coordinate system. Maximum-likelihood estimates obey the invariance relation
[ \hat\phi_{\mathrm{ML}}=g(\hat\theta_{\mathrm{ML}}), ]
provided the transformed parameterization represents the same admissible distributions.
A nuisance parameter affects the sampling distribution but is not the direct target of inference. In a Gaussian location model with unknown variance, the variance acts as a nuisance parameter when inference concerns only the mean. Profiling maximizes the likelihood over nuisance coordinates, while marginalization integrates them with respect to a probability distribution. Conditioning can remove a nuisance parameter when an appropriate ancillary or sufficient statistic exists.
Nuisance parameters are not intrinsically less meaningful than target parameters. Their designation depends on the inferential question, and the same coefficient can serve as a target in one analysis and as a nuisance quantity in another.
Model specification and misspecification
Parametric inference separates randomness within the model from uncertainty about the model. Sampling distributions, standard errors, and nominal coverage probabilities are calculated under the assumption that some member (P_\theta) generated the observations. If the actual distribution lies outside the family, the model is misspecified.
Under misspecification, a maximum-likelihood estimator often converges to the parameter whose model distribution minimizes the Kullback–Leibler divergence from the data-generating distribution. This limit is a pseudo-true parameter rather than a parameter of a literally correct distribution. The inverse Fisher information alone then need not describe the estimator's asymptotic covariance; the relevant expression generally has a sandwich form involving both the curvature of the expected log-likelihood and the variability of its score.
Identifiability constitutes a separate issue. A model is identifiable when distinct parameter values correspond to distinct observable distributions. When
[ P_{\theta_1}=P_{\theta_2} \quad\text{for}\quad \theta_1\ne\theta_2, ]
the data cannot distinguish those parameter values within the model. Mixture models, latent-variable systems, and overparameterized regressions commonly produce local or global failures of identifiability unless additional constraints are included in the model definition.
Model checking examines discrepancies between observed data and consequences of the fitted family. Residual distributions, replicated-data summaries, and goodness-of-fit statistics address different features of such discrepancies. These assessments do not alter whether the original model is parametric; they evaluate the correspondence between its restrictions and the observed information.
Relation to nonparametric and semiparametric statistics
The distinction between parametric and nonparametric statistics concerns model dimension rather than the presence or absence of assumptions. Nonparametric models retain structural assumptions concerning sampling, regularity, or dependence while leaving at least one inferential component infinite-dimensional. The empirical distribution function, for example, estimates a population distribution without restricting it to a fixed finite-dimensional family.
A semiparametric model combines a finite-dimensional target parameter with an infinite-dimensional nuisance component. The Cox proportional-hazards model has finite-dimensional regression coefficients while leaving the baseline hazard function unspecified. Its inferential theory therefore contains parametric scores and information bounds modified by projection against the infinite-dimensional nuisance space.
Parametric models usually convert substantive structure into lower-dimensional estimation problems. Their inferential conclusions are correspondingly tied to that structure. Nonparametric and semiparametric formulations distribute uncertainty across larger model spaces, producing different notions of efficiency, robustness, and identifiability rather than a complete absence of model assumptions.