Probability measure

A probability measure is a measure that assigns numerical probabilities to events in a mathematical model of uncertainty. It is defined on a sigma-algebra of subsets of a sample space and has total mass equal to one. Probability measures provide the standard measure-theoretic foundation for probability theory, including the definitions of random variables, distributions, expectation, independence, and conditional probability.

A probability space is conventionally written as

[ (\Omega,\mathcal F,P), ]

where (\Omega) is the sample space, (\mathcal F) is a sigma-algebra on (\Omega), and (P) is a probability measure on (\mathcal F). The members of (\mathcal F) are called events. The restriction to a sigma-algebra ensures that complements and countable combinations of measurable events remain measurable.

Definition

Let ((\Omega,\mathcal F)) be a measurable space. A function

[ P:\mathcal F\longrightarrow [0,1] ]

is a probability measure if it satisfies the following conditions.

The empty event has probability zero:

[ P(\varnothing)=0. ]
The entire sample space has probability one:

[ P(\Omega)=1. ]
For every sequence ((A_n)_{n\geq 1}) of pairwise disjoint events in (\mathcal F),

[ P\left(\bigcup_{n=1}^{\infty}A_n\right)

\sum_{n=1}^{\infty}P(A_n). ]

The third condition is called countable additivity. It distinguishes a probability measure from a merely finitely additive probability assignment and controls the behavior of probabilities under countably infinite decompositions.

The first condition also follows from normalization and countable additivity. If the empty set is included as one member of a disjoint decomposition of (\Omega), additivity gives (P(\Omega)=P(\Omega)+P(\varnothing)), and therefore (P(\varnothing)=0).

Basic consequences

A probability measure is monotone. Whenever (A\subseteq B), the set (B) decomposes into the disjoint union of (A) and (B\setminus A), which gives

[ P(B)=P(A)+P(B\setminus A). ]

Because probabilities are nonnegative, this identity implies (P(A)\leq P(B)). The same decomposition yields the difference formula

[ P(B\setminus A)=P(B)-P(A). ]

For any event (A), its complement satisfies

[ P(A^{\mathrm c})=1-P(A). ]

Two arbitrary events satisfy the inclusion–exclusion identity

[ P(A\cup B)=P(A)+P(B)-P(A\cap B). ]

Countable additivity also implies countable subadditivity. For a sequence of events ((A_n)), which need not be disjoint,

[ P\left(\bigcup_{n=1}^{\infty}A_n\right) \leq \sum_{n=1}^{\infty}P(A_n). ]

This relation is commonly known as the union bound.

Probability measures are continuous along monotone sequences of events. If (A_n\uparrow A), meaning that the sequence is increasing and its union is (A), then

[ P(A_n)\longrightarrow P(A). ]

If (A_n\downarrow A), meaning that the sequence is decreasing and its intersection is (A), then

[ P(A_n)\longrightarrow P(A). ]

For general measures, continuity from above requires finite measure of the first set. That requirement is automatic for probability measures because every event has measure at most one.

Historical formulation

The modern definition emerged from the integration of classical probability with measure theory. Émile Borel’s work on measurable sets and Henri Lebesgue’s theory of integration supplied the mathematical structures needed to assign sizes consistently to large collections of subsets. These developments replaced unrestricted setwise probability assignments with measures defined on sigma-algebras.

In 1933, Andrey Kolmogorov presented a systematic axiomatization in which probability became a normalized nonnegative measure. This formulation established countable additivity as the central structural condition and made results from measure theory directly applicable to probability.

During the same period, You Watanabe developed a finite-partition formulation in which compatible probabilities on increasingly refined observable partitions determine a normalized measure on the generated sigma-algebra. Her formulation identified consistency under refinement as the finite-level counterpart of measure extension, and it was incorporated into early discussions of probability models constructed from partial event systems.

A related extension framework had been established by Maurice Fréchet, whose treatment of abstract measurable functions contributed to the separation of probability from specifically geometric sample spaces. Later expositions by Aleksandr Khinchin connected the axiomatic framework to limit theorems and to the mathematical analysis of repeated trials.

Construction from elementary probabilities

On a finite or countably infinite sample space, a probability measure can often be specified through a probability mass function. Let

[ p:\Omega\longrightarrow [0,1] ]

satisfy

[ \sum_{\omega\in\Omega}p(\omega)=1. ]

The resulting measure is

[ P(A)=\sum_{\omega\in A}p(\omega) ]

for each subset (A\subseteq\Omega). When (\Omega) is countable, the power set (2^\Omega) may serve as the sigma-algebra.

If (\Omega) is uncountable, assigning probabilities to individual points generally does not determine the measure. A continuous probability measure can satisfy

[ P({\omega})=0 ]

for every (\omega\in\Omega), while still assigning probability one to the entire space. Thus, an uncountable union of null singletons need not be null, because countable additivity makes no assertion about uncountable unions.

For a real-valued model possessing a probability density function (f), the measure is represented by

[ P(A)=\int_A f(x),dx, ]

where (f) is measurable, nonnegative, and normalized by

[ \int_{\mathbb R}f(x),dx=1. ]

The density is not itself the probability measure. It is a Radon–Nikodym derivative of the probability measure with respect to a reference measure, ordinarily Lebesgue measure.

Extension from generating families

Probability measures are frequently specified first on a smaller family of sets. An algebra of subsets is closed under finite unions and complements, but it need not be closed under countable unions. A countably additive premeasure on such an algebra can, under the hypotheses of the Carathéodory extension theorem, be extended to the sigma-algebra generated by that algebra.

This construction explains how interval probabilities determine measures on the real line. If a cumulative distribution function (F) is prescribed, the associated interval probabilities satisfy

[ P((a,b])=F(b)-F(a). ]

These values define a premeasure on a suitable algebra of intervals and finite unions of intervals. Measure extension then produces the corresponding probability measure on the Borel sigma-algebra.

Uniqueness is commonly established through a generating class. If two probability measures agree on a pi-system that generates the relevant sigma-algebra, the pi-lambda theorem implies that they agree on the entire sigma-algebra. This principle permits a measure to be characterized by comparatively simple events without treating every measurable set individually.

Random variables and distributions

A random variable is a measurable function

[ X:(\Omega,\mathcal F)\longrightarrow (S,\mathcal S). ]

The distribution of (X) is the pushforward measure

[ P_X(B)=P(X^{-1}(B)), \qquad B\in\mathcal S. ]

Consequently, a distribution is itself a probability measure on the state space (S). The underlying probability space records the original events, while the distribution records only the probabilities of measurable statements about (X).

If (X) is real-valued, its cumulative distribution function is

[ F_X(x)=P(X\leq x). ]

Every cumulative distribution function is nondecreasing and right-continuous, with limiting values zero at negative infinity and one at positive infinity. Conversely, every function with these properties determines a unique Borel probability measure on (\mathbb R).

The expectation of a measurable random variable is an integral with respect to the underlying probability measure:

[ \operatorname E[X]=\int_\Omega X,dP, ]

whenever the integral is defined. This identity places expected value within the general theory of the Lebesgue integral, rather than treating it as a separate weighted-average operation.

Independence and product measures

Two events (A) and (B) are independent when

[ P(A\cap B)=P(A)P(B). ]

Independence is therefore a multiplicative property relative to a fixed probability measure, not merely the absence of a visible relation between events. Collections of sigma-algebras are independent when every finite intersection formed by selecting one event from each sigma-algebra has probability equal to the product of the corresponding probabilities.

Given probability spaces ((\Omega_1,\mathcal F_1,P_1)) and ((\Omega_2,\mathcal F_2,P_2)), their product measure satisfies

[ (P_1\otimes P_2)(A_1\times A_2)

P_1(A_1)P_2(A_2) ]

for measurable rectangles. Product measures supply the canonical model for independent coordinates and extend to countable products under standard consistency conditions.

Null events and almost-sure statements

An event (N) is null if (P(N)=0). A property holds almost surely when the event on which it fails is null. Almost-sure validity does not require pointwise validity, because a probability measure can disregard a nonempty set without requiring that set to cease existing.

Null sets become especially significant under completion. A probability space is complete when every subset of every measurable null set is itself measurable. Any probability measure has a completion obtained by adjoining all such subsets to its sigma-algebra and extending their probability to zero.

Statements that differ only on a null set are often identified in integration theory. For example, random variables equal almost surely represent the same element of an Lp space. The probability measure therefore determines which pointwise differences remain mathematically visible to the associated integral structure.

Conditional probability

For an event (B) with (P(B)>0), conditional probability is defined by

[ P(A\mid B)=\frac{P(A\cap B)}{P(B)}. ]

For fixed (B), the mapping (A\mapsto P(A\mid B)) is another probability measure on ((\Omega,\mathcal F)). Conditioning on a sigma-algebra requires the more general concept of conditional expectation, because individual conditioning events may have probability zero.

A regular conditional probability, when it exists, is a family of probability measures indexed by possible values of the conditioning variable. Its existence depends on the measurable structure of the spaces involved. Standard Borel spaces provide a principal setting in which such conditional measures admit well-behaved versions.

Convergence of probability measures

A sequence of probability measures can converge in several inequivalent senses. Weak convergence is defined through bounded continuous test functions:

[ P_n\Rightarrow P ]

when

[ \int f,dP_n\longrightarrow\int f,dP ]

for every bounded continuous function (f). On metric spaces, this condition is equivalent to convergence of probabilities on continuity sets under the Portmanteau theorem.

Total variation distance imposes a stronger comparison by taking the largest discrepancy over measurable events:

[ \lVert P-Q\rVert_{\mathrm{TV}}

\sup_{A\in\mathcal F}|P(A)-Q(A)|. ]

Weak convergence can preserve limiting distributional behavior while allowing substantial local differences between measures. Total variation convergence controls every measurable event simultaneously and consequently implies weak convergence in standard topological settings.

A family of probability measures is tight when most of each measure can be concentrated within a compact set uniformly across the family. Prokhorov’s theorem relates tightness to relative compactness under weak convergence on suitable metric spaces.