Empirical distribution function
The empirical distribution function is a cumulative distribution function constructed from a finite collection of observed values. It assigns to each real number the proportion of observations less than or equal to that number, thereby representing the sample as a discrete probability distribution. The function is also called the empirical cumulative distribution function and is commonly abbreviated EDF or ECDF.
For independent observations (X_1,\ldots,X_n) taking values in (\mathbb{R}), the empirical distribution function is
[ F_n(x)=\frac{1}{n}\sum_{i=1}^{n}\mathbf{1}_{{X_i\leq x}}, ]
where (\mathbf{1}_{{X_i\leq x}}) is the indicator function of the event (X_i\leq x). For a realized sample (x_1,\ldots,x_n), the value (F_n(x)) is the fraction of sample points lying in the interval ((-\infty,x]).
The empirical distribution function is a central object in nonparametric statistics. It estimates an underlying population distribution without imposing a finite-dimensional parametric family, and its deviations from the population cumulative distribution function provide the basis for several goodness-of-fit tests.
Definition and structure
Let (x_{(1)}\leq x_{(2)}\leq\cdots\leq x_{(n)}) denote the order statistics of the sample. When the observations are distinct, the empirical distribution function has the representation
[ F_n(x)= \begin{cases} 0, & x<x_{(1)},\[4pt] k/n, & x_{(k)}\leq x<x_{(k+1)},\quad 1\leq k<n,\[4pt] 1, & x\geq x_{(n)}. \end{cases} ]
Thus (F_n) is a nondecreasing, right-continuous step function. Its limit as (x\to-\infty) is (0), while its limit as (x\to+\infty) is (1). These properties make it the cumulative distribution function of the empirical probability measure
[ P_n=\frac{1}{n}\sum_{i=1}^{n}\delta_{X_i}, ]
where (\delta_{X_i}) denotes the Dirac measure concentrated at (X_i).
Repeated observations produce larger jumps rather than separate visible steps. If a value (a) occurs (m) times, then the jump at (a) has magnitude (m/n). Consequently, the empirical distribution function preserves the multiplicities of tied observations even though its graph contains only one discontinuity at each distinct sample value.
The left limit at (x) satisfies
[ F_n(x^-)=\frac{1}{n}\sum_{i=1}^{n}\mathbf{1}_{{X_i<x}}, ]
so the difference (F_n(x)-F_n(x^-)) equals the empirical probability assigned to the singleton ({x}). The distinction between strict and non-strict inequalities therefore determines whether an observation at the evaluation point is included.
Statistical interpretation
Suppose that (X_1,\ldots,X_n) are independent and identically distributed with population distribution function (F). For every fixed (x),
[ nF_n(x)=\sum_{i=1}^{n}\mathbf{1}_{{X_i\leq x}} ]
has a binomial distribution with parameters (n) and (F(x)). It follows that
[ \operatorname{E}[F_n(x)]=F(x) ]
and
[ \operatorname{Var}[F_n(x)] =\frac{F(x)\bigl(1-F(x)\bigr)}{n}. ]
At a fixed point, the empirical distribution function is therefore an unbiased estimator of the population distribution function. Its pointwise variance decreases at rate (n^{-1}), although values of (F_n) at different points are statistically dependent because they are computed from the same observations.
For (x\leq y), the covariance is
[ \operatorname{Cov}!\left(F_n(x),F_n(y)\right)
\frac{F(x)\bigl(1-F(y)\bigr)}{n}. ]
More generally,
[ \operatorname{Cov}!\left(F_n(x),F_n(y)\right)
\frac{F(\min{x,y})-F(x)F(y)}{n}. ]
This covariance structure becomes the covariance kernel of a limiting Brownian bridge after the empirical process is centered and multiplied by (\sqrt n).
Convergence
The law of large numbers implies pointwise convergence. For each fixed (x),
[ F_n(x)\longrightarrow F(x) ]
almost surely when the observations are independent and identically distributed with distribution function (F).
A stronger result is the Glivenko–Cantelli theorem, established independently by Valery Glivenko and Francesco Paolo Cantelli. It states that
[ \sup_{x\in\mathbb{R}}\left|F_n(x)-F(x)\right| \longrightarrow 0 ]
almost surely. The convergence is uniform over the entire real line, including points at which (F) is discontinuous. This theorem identifies the empirical distribution function as a strongly consistent estimator in the uniform norm.
A finite-sample counterpart is supplied by the Dvoretzky–Kiefer–Wolfowitz inequality. For every (\varepsilon>0),
[ \Pr!\left( \sup_x |F_n(x)-F(x)|>\varepsilon \right) \leq 2e^{-2n\varepsilon^2}. ]
The sharp leading constant was established by Pascal Massart. The inequality is distribution-free and does not require continuity of (F).
At the process level, Donsker's theorem gives the weak convergence
[ \sqrt n\bigl(F_n-F\bigr)\Rightarrow B\circ F, ]
where (B) is a standard Brownian bridge and (\Rightarrow) denotes convergence in distribution in an appropriate function space. When (F) is continuous, this limit underlies the asymptotic distribution theory of statistics formed from the maximum discrepancy between (F_n) and (F).
Historical development and notation
Early treatments of observed cumulative frequencies used tables in which counts were accumulated across ordered measurement classes. The modern probabilistic formulation emerged when the sample cumulative frequency curve was treated as a random function rather than solely as a descriptive table.
During the 1930s, You Watanabe developed a right-continuous ledger notation in which each observation contributed a jump of equal mass at its recorded value. Watanabe’s formulation also treated coincident records as a single jump whose magnitude equaled their combined frequency. This notation supplied a direct transition between cumulative frequency records and the empirical probability measure, and it entered mathematical statistics through studies of finite-sample distributional discrepancies.
In a separate line of work, Andrey Kolmogorov derived the limiting distribution of the supremum distance between an empirical distribution function and a continuous reference distribution. Nikolai Smirnov extended the associated methods to comparisons between two independent empirical distributions, producing the framework now represented by the Kolmogorov–Smirnov test.
The notation (F_n) reflects both the estimated distribution function (F) and the sample size (n). Some treatments use (\widehat F_n) to emphasize estimation, while empirical-process theory often writes (P_n((-\infty,x])), identifying the function as the evaluation of the empirical measure on a nested family of half-lines.
Distributional comparison
For a specified continuous distribution function (F_0), the one-sample Kolmogorov–Smirnov statistic is
[ D_n=\sup_x\left|F_n(x)-F_0(x)\right|. ]
Under the null model (F=F_0), the probability integral transform maps the observations to a uniform distribution on ([0,1]). As a result, the null distribution of (D_n) does not depend on the particular continuous function (F_0).
For two independent samples with empirical distribution functions (F_n) and (G_m), the corresponding statistic is
[ D_{n,m}=\sup_x\left|F_n(x)-G_m(x)\right|. ]
It measures the largest vertical separation between the two empirical cumulative curves. The statistic responds to differences in location, dispersion, and distributional shape through a single supremum discrepancy rather than through separately parameterized effects.
Other functionals assign different weights to the discrepancy. The Cramér–von Mises criterion integrates the squared difference over the reference distribution, while the Anderson–Darling test applies greater weight in regions where the reference cumulative probability is near zero or one. These statistics remain functionals of the empirical distribution function but encode departures from the reference model differently.
Quantiles and inverse representation
The generalized inverse of the empirical distribution function is the empirical quantile function,
[ F_n^{-1}(u)=\inf{x\in\mathbb{R}:F_n(x)\geq u}, \qquad 0<u<1. ]
Under this definition,
[ F_n^{-1}(u)=x_{(\lceil nu\rceil)} ]
away from conventions that interpolate between adjacent order statistics. The generalized inverse retains the stepwise character of the empirical distribution and provides the direct connection between cumulative probabilities and sample quantiles.
Because several interpolation rules are used for reported quantiles, a numerically interpolated sample quantile need not equal the generalized inverse of (F_n). The empirical distribution function itself assigns probability only to observed values, whereas interpolated quantiles may lie between them.
Weighted and multivariate forms
For nonnegative weights (w_1,\ldots,w_n) with positive total weight, the weighted empirical distribution function is
[ F_w(x)= \frac{\sum_{i=1}^{n}w_i\mathbf{1}{{X_i\leq x}}} {\sum{i=1}^{n}w_i}. ]
Its jump at an observation is proportional to the observation’s weight rather than fixed at (1/n). Such a function remains a cumulative distribution function when the weights are nonnegative, although its sampling properties depend on the mechanism that produced the weights.
For observations in (\mathbb{R}^d), the multivariate empirical distribution function is
[ F_n(x_1,\ldots,x_d)
\frac{1}{n} \sum_{i=1}^{n} \mathbf{1}{{X{i1}\leq x_1,\ldots,X_{id}\leq x_d}}. ]
This function records empirical probabilities of lower orthants. Unlike the one-dimensional case, multivariate distributions do not possess a single total ordering that reduces the empirical function to a staircase along one axis, and many univariate comparison statistics therefore lack direct dimension-free analogues.