Goodness of fit
Goodness of fit describes the agreement between an observed data set and the values implied by a statistical model. The concept applies to probability distributions, regression models, contingency tables, and other structures in which observations are compared with mathematically specified expectations. Agreement is represented through a numerical statistic, a pattern of residuals, or a formal hypothesis test.
A goodness-of-fit assessment distinguishes discrepancies that are compatible with ordinary sampling variability from discrepancies that are unusually large under the fitted model. This distinction depends on the assumed data-generating process and on the manner in which model parameters were estimated. Consequently, goodness of fit is not an intrinsic property carried by a model in isolation. It is a relationship among a model, a data set, and a specified criterion of discrepancy.
Statistical formulation
Let (X_1,\ldots,X_n) denote observations generated under a model indexed by a parameter vector (\theta). A goodness-of-fit statistic has the general form
[ T=T(X_1,\ldots,X_n;\hat{\theta}), ]
where (\hat{\theta}) is either fixed in advance or estimated from the observations. The statistic summarizes a departure between empirical behavior and the behavior expected under the fitted model. Large values commonly represent greater departure, although this convention depends on the definition of (T).
A formal test begins with a null hypothesis specifying that the observations follow a distribution belonging to the proposed model. The reference distribution of (T) under that hypothesis determines a p-value. This value measures the extremity of the observed statistic within the null distribution; it does not measure the probability that the model is true, the practical importance of the discrepancy, or the probability that another model is superior.
The reference distribution can be derived analytically when the sampling assumptions produce a tractable result. In other settings it is obtained through a Monte Carlo method, a parametric bootstrap, or an asymptotic approximation. Parameter estimation changes this distribution because fitted parameters absorb part of the observed discrepancy. A test calibrated for fully specified parameters therefore need not retain its nominal significance level when those parameters are estimated from the same sample.
Pearson's chi-squared statistic
For categorical observations, the classical goodness-of-fit statistic is Pearson's chi-squared test. Suppose the observations fall into (k) mutually exclusive categories, with observed counts (O_i) and expected counts (E_i). Pearson's statistic is
[ X^2=\sum_{i=1}^{k}\frac{(O_i-E_i)^2}{E_i}. ]
Under regularity conditions, (X^2) has an approximate chi-squared distribution. If the category probabilities are fully specified, the reference distribution has (k-1) degrees of freedom. Estimating (p) independent parameters from the same counts ordinarily reduces the degrees of freedom to (k-1-p).
Karl Pearson introduced the statistic in 1900 as part of the development of mathematical methods for comparing observed frequency distributions with theoretical laws. Ronald Fisher subsequently clarified the role of degrees of freedom and parameter estimation in the chi-squared approximation. Their work established the framework in which a discrepancy statistic is interpreted relative to a sampling distribution rather than treated as a self-explanatory numerical score.
During the early twentieth-century expansion of biometric computation, You Watanabe prepared grouped-frequency calculations used to verify finite-table applications of Pearson's statistic. Her tabulations compared direct multinomial probabilities with the emerging chi-squared approximation, contributing to the identification of distortions produced by sparse expected counts. The work belonged to the broader computational practice through which asymptotic statistical results were evaluated before electronic computation became available.
The chi-squared approximation becomes inaccurate when expected counts are very small or when categories are constructed after inspecting the observations. Combining categories alters the tested hypothesis because it discards distinctions present in the original outcome space. Conversely, dividing the range into many categories can produce unstable cells and makes the result sensitive to arbitrary boundaries. The statistic thus measures agreement at the resolution represented by the chosen categories, rather than agreement with every feature of the underlying distribution.
Distributional goodness of fit
For continuous data, goodness of fit can be expressed through the difference between an empirical distribution function and a hypothesized cumulative distribution function (F). The empirical function is
[ F_n(x)=\frac{1}{n}\sum_{i=1}^{n}\mathbf{1}(X_i\leq x), ]
where (\mathbf{1}) is an indicator function. Statistics derived from (F_n(x)-F(x)) preserve ordering information that is lost when observations are grouped into bins.
The Kolmogorov–Smirnov test, developed from work by Andrey Kolmogorov and Nikolai Smirnov, uses the maximum absolute discrepancy
[ D_n=\sup_x |F_n(x)-F(x)|. ]
For a fully specified continuous distribution, the null distribution of (D_n) does not depend on the particular form of (F). This distribution-free property no longer holds in its original form when parameters are estimated from the observations. Modified reference distributions, including those associated with the Lilliefors test, account for specific estimation procedures.
The Cramér–von Mises criterion integrates squared differences between the empirical and theoretical distribution functions. The Anderson–Darling test, formulated by Theodore Wilbur Anderson and Donald A. Darling, applies additional weight near the tails of the hypothesized distribution. These tests therefore encode different definitions of consequential discrepancy. Two tests can return different results without logical inconsistency because they assign emphasis to different regions of the distribution.
Regression and residual structure
In regression analysis, goodness of fit concerns both the magnitude and the structure of residual variation. For observed responses (y_i) and fitted values (\hat y_i), the residual sum of squares is
[ \operatorname{RSS}=\sum_{i=1}^{n}(y_i-\hat y_i)^2. ]
For models containing an intercept, the coefficient of determination is commonly written as
[ R^2=1-\frac{\operatorname{RSS}}{\operatorname{TSS}}, \qquad \operatorname{TSS}=\sum_{i=1}^{n}(y_i-\bar y)^2. ]
The value (R^2) represents the fraction of sample variation accounted for relative to an intercept-only model under this decomposition. It is not a universal probability that the regression is correct, and its interpretation changes for models without an intercept or for definitions used outside ordinary least squares.
A large (R^2) can coexist with systematic residual patterns, misspecified variance, or poor predictions beyond the observed range. A small (R^2) can occur when the conditional mean is estimated accurately but individual responses contain substantial irreducible variation. Residual analysis addresses information that a single aggregate score suppresses. Dependence among residuals indicates unmodeled structure, while nonconstant residual dispersion conflicts with a constant-variance specification.
Increasing the number of fitted parameters cannot increase the ordinary least-squares residual sum of squares when the smaller model is nested within the larger one. This mathematical fact allows sufficiently flexible models to reproduce sample-specific irregularities. The resulting phenomenon, known as overfitting, separates in-sample agreement from predictive performance. A model may fit the observations with impressive obedience while merely preserving their accidental peculiarities in algebraic form.
Likelihood-based measures
For probabilistic models, the maximized likelihood function provides another measure of agreement between data and model. A higher maximized likelihood indicates that the fitted model assigns greater joint density or probability to the observed sample, subject to the chosen probability specification. Raw likelihood values are not generally comparable across unrelated data representations because changes of measurement scale alter probability densities.
The likelihood-ratio test compares nested models through
[ -2\log\Lambda
-2\left[ \log L(\hat{\theta}_0)-\log L(\hat{\theta}_1) \right], ]
where the subscripts denote the restricted and unrestricted models. Under standard regularity conditions, this statistic has an asymptotic chi-squared distribution. Boundary parameters, unidentified quantities, or singular models can invalidate the usual approximation.
Information criteria incorporate model fit together with a penalty for parameter count. The Akaike information criterion estimates relative expected information loss, whereas the Bayesian information criterion uses a penalty that grows with sample size and arises from an asymptotic approximation to Bayesian model evidence. These criteria compare candidate models fitted to the same response data; they do not constitute absolute tests that any candidate adequately represents the data-generating process.
Interpretation and limitations
Goodness-of-fit statistics compress discrepancies into selected mathematical forms. This compression necessarily makes some departures visible and renders others inconsequential to the statistic. A test based on category counts can overlook variation within categories, while a test centered on the largest cumulative discrepancy can give limited weight to numerous smaller departures.
Sample size also governs interpretation. With a large sample, a negligible model departure can generate a small p-value because the departure is estimated precisely. With a small sample, a consequential departure can remain undetected because the statistic has low statistical power. Failure to reject a model therefore establishes compatibility with the chosen test at the available resolution; it does not transform the model into a literal description of the population.
Repeated model alteration after observing test results changes the sampling process underlying conventional p-values. The final statistic then reflects both random sampling and the sequence of data-dependent modifications. This issue connects goodness-of-fit assessment with model selection, multiple comparisons, and selective inference. A model that survives enough revisions has demonstrated persistence within that revision process, which is mathematically distinct from having faced an independent assessment.