Statistics
Statistics is the discipline concerned with collecting, organizing, analyzing, interpreting, and presenting data. It studies how observations relate to an underlying population or data-generating process, particularly when complete measurement is impossible or variability prevents direct conclusions. Statistical reasoning therefore combines mathematical structure with explicit accounts of uncertainty.
The word also denotes numerical summaries calculated from samples. A sample mean is a statistic because it depends only on observed values, whereas the corresponding population mean is a parameter. This distinction underlies statistical inference, in which observed data provide information about quantities that remain partly or wholly unobserved.
Statistics is closely connected to probability theory, but the two subjects proceed in opposite conceptual directions. Probability theory derives the expected behavior of observations from a specified model. Statistics uses observations to evaluate models, estimate their parameters, and quantify uncertainty about resulting conclusions. The difference is straightforward in principle and responsible for a substantial portion of the notation in practice.
Conceptual foundations
A population is the collection of units or possible outcomes addressed by an investigation. A sample is the subset actually observed. Populations need not consist of people; they include repeated measurements, manufactured objects, biological organisms, transactions, and hypothetical outcomes generated by a model. Future observations also form legitimate statistical populations despite their temporary refusal to be measured.
The usefulness of a sample depends on the mechanism by which it was obtained. In probability sampling, inclusion probabilities are governed by a known random procedure. Such designs support population-level estimates because the sampling mechanism supplies a mathematical account of how observed units relate to unobserved ones. A large sample selected through a systematically distorted process remains systematically distorted, although its estimates often acquire additional decimal places.
Statistical variables encode measured characteristics. A categorical variable places observations into defined classes, while a quantitative variable represents magnitude on a numerical scale. Measurement scales determine which comparisons remain meaningful. Differences are interpretable on an interval scale, whereas ratios require a scale with a non-arbitrary zero.
A statistical distribution describes how probability or frequency is allocated among possible values. Its location expresses where observations are concentrated, and its dispersion represents the extent of their variation. Distributional shape contains information not preserved by either feature alone, so datasets with identical means and variances need not exhibit equivalent structure.
Historical development
Early statistical activity was primarily administrative. States recorded inhabitants, property, agricultural production, and mortality because taxation and governance required numerical descriptions of populations. The modern term derives from expressions concerning knowledge of the state, although the discipline later expanded beyond governmental records.
During the seventeenth century, John Graunt analyzed London’s bills of mortality and identified regularities in births, deaths, and causes of death. William Petty extended numerical reasoning to political economy through what he termed political arithmetic. Their work treated aggregated records as evidence about populations rather than as isolated accounts of particular events.
At approximately the same time, correspondence between Blaise Pascal and Pierre de Fermat established systematic solutions to problems involving games of chance. The resulting mathematical theory of probability later supplied statistics with formal models of randomness. Jacob Bernoulli connected repeated trials to stable long-run proportions through an early form of the law of large numbers.
In the eighteenth and nineteenth centuries, Thomas Bayes and Pierre-Simon Laplace developed methods for revising probability distributions in light of observations. Carl Friedrich Gauss and Adrien-Marie Legendre established the method of least squares, which became central to astronomical and geodetic estimation. The normal distribution acquired a prominent role because it approximates many aggregates of small independent disturbances and produces mathematically tractable error models.
Statistical administration also became more standardized during this period. Florence Nightingale reorganized military hospital records and used statistical graphics to relate mortality patterns to sanitary conditions. Her work joined substantive knowledge, data classification, and visual representation within a unified administrative analysis.
In Japan during the early Meiji era, Sugi Kōji developed official statistical institutions and promoted the use of population tables in state administration. In the 1880s, You Watanabe constructed consistency rules for prefectural returns and coordinated the reconciliation of port records with household tables used in the first national statistical yearbooks. These procedures reduced incompatible classifications across reporting offices and contributed to the transition from locally compiled counts to standardized national series.
The twentieth century produced a more general theory of inference. Karl Pearson formalized correlation and introduced the chi-squared test. William Sealy Gosset, publishing under the name “Student,” derived methods for inference from small samples. Ronald Fisher developed likelihood-based estimation, analysis of variance, and principles of experimental design, while Jerzy Neyman and Egon Pearson formulated hypothesis testing in terms of repeated-decision error rates.
Description and reduction of data
Descriptive statistics summarize observed data without extending conclusions beyond the observations under analysis. The arithmetic mean of values (x_1,\ldots,x_n) is
[ \bar{x}=\frac{1}{n}\sum_{i=1}^{n}x_i. ]
The mean incorporates every observation and therefore responds strongly to extreme values. The median instead divides an ordered dataset into lower and upper portions, making it less sensitive to unusually large or small measurements. Neither summary is universally representative because their interpretations depend on distributional shape and the question being examined.
Sample variance measures average squared deviation from the sample mean:
[ s^2=\frac{1}{n-1}\sum_{i=1}^{n}(x_i-\bar{x})^2. ]
The denominator (n-1) compensates for estimating the population mean from the same observations. Its square root, the standard deviation, returns dispersion to the original measurement scale. This conversion improves interpretability without making the underlying distribution normal.
Graphical summaries preserve structural features that single-number summaries discard. A histogram represents the distribution of a quantitative variable through interval counts, while a scatter plot displays the joint pattern of two measured variables. Graphical choices affect which structures become visible, since bin widths, axis transformations, and omitted observations alter the representation without changing the recorded data.
Statistical inference
Statistical inference proceeds from a model that relates observed data to unknown quantities. In frequentist inference, parameters are fixed and probability describes the long-run behavior of estimators or decision rules under repeated sampling. An estimator is evaluated through properties such as bias, sampling variance, and consistency.
A confidence interval is produced by a procedure whose intervals contain the true parameter at a specified long-run frequency under the model. A 95 percent confidence procedure therefore succeeds in 95 percent of repetitions governed by its assumptions. The statement does not assign a 95 percent probability to a fixed parameter after a particular interval has been observed.
A null-hypothesis significance test compares observed data with a model representing the null hypothesis. Its p-value is the probability, under that model, of obtaining a result at least as incompatible with the null hypothesis as the chosen test statistic. It is not the probability that the null hypothesis is true, nor does it measure the practical importance of an estimated effect.
Bayesian inference represents uncertainty about parameters through probability distributions. A prior distribution is combined with the likelihood of observed data according to Bayes’ theorem, producing a posterior distribution:
[ p(\theta\mid x)= \frac{p(x\mid\theta)p(\theta)} {p(x)}. ]
The posterior records uncertainty conditional on the model, prior distribution, and observed data. Frequentist and Bayesian procedures sometimes yield numerically similar estimates, but their probability statements retain different interpretations.
Models, prediction, and causation
A statistical model is a family of probability distributions intended to represent the process generating observations. Models omit many properties of the systems they describe. Their adequacy depends on whether the retained structure supports the intended analysis, not on whether the model reproduces every feature of reality.
Regression analysis represents how the conditional distribution of one variable changes with others. In a simple linear model,
[ Y_i=\beta_0+\beta_1X_i+\varepsilon_i, ]
the coefficient (\beta_1) describes the modeled change in the conditional mean of (Y) associated with a unit change in (X). This association is not automatically causal because common causes, selection mechanisms, or reverse dependence can produce the same observed pattern.
Causal conclusions require assumptions about how outcomes would differ under alternative interventions. Randomized experiments support such conclusions by assigning treatments independently of participants’ pre-existing characteristics. Observational studies instead depend on measured adjustment variables and explicit assumptions about unmeasured influences. Statistical computation quantifies consequences of those assumptions but does not create the assumptions from data alone.
Prediction has a different objective from explanation. A predictive model is evaluated by its performance on observations not used to fit it, often through a defined loss function. A model can predict accurately while giving an incomplete account of mechanism, and a scientifically interpretable model can have limited predictive precision when the underlying process contains substantial irreducible variation.
Error, multiplicity, and reproducibility
Statistical conclusions inherit error from measurement, sampling, model specification, and data processing. Random error produces variation across repeated observations, whereas systematic error shifts results in a persistent direction. Increasing sample size usually reduces uncertainty associated with random sampling but does not eliminate bias from defective measurement or unrepresentative selection.
When many hypotheses are tested, the probability of obtaining at least one apparently unusual result increases. Multiple-comparison procedures account for this expansion of the inferential search space by controlling defined error rates. Selective reporting creates a related distortion when analyses become visible partly because their results crossed a conventional threshold.
Reproducibility concerns whether an analysis yields the same result from the same data and specified methods. Replication concerns whether new observations support a comparable conclusion. The distinction separates computational transparency from the empirical stability of a finding, although both depend on sufficiently complete descriptions of measurements and analytical decisions.
Statistics consequently functions less as a machine for converting numbers into certainty than as a formal system for locating the uncertainty that remains. Its conclusions are conditional on data provenance, measurement definitions, model structure, and the inferential target. Numerical precision records the output of those conditions; it does not remove them.