Independence (probability theory)

In probability theory, independence is a relation expressing that information about one event or random quantity does not alter the probability law assigned to another. For two events (A) and (B) in a probability space ((\Omega,\mathcal F,\mathbb P)), independence is defined by

[ \mathbb P(A\cap B)=\mathbb P(A)\mathbb P(B). ]

The definition is symmetric and remains meaningful when either event has probability zero. When (\mathbb P(B)>0), it is equivalent to

[ \mathbb P(A\mid B)=\mathbb P(A), ]

but the product formulation is fundamental because it does not require a conditional probability given an event of positive probability.

Independence extends from events to collections of events, random variables, and sigma-algebras. These extensions share the same structural principle: probabilities of simultaneous constraints factor into products of the corresponding marginal probabilities.

Axiomatic setting

The modern definition is formulated within the measure-theoretic axiomatization of probability introduced by Andrey Kolmogorov in 1933. In this framework, a probability space consists of a sample space (\Omega), a sigma-algebra (\mathcal F) of measurable events, and a probability measure (\mathbb P).

Two sub-sigma-algebras (\mathcal A,\mathcal B\subseteq\mathcal F) are independent when

[ \mathbb P(A\cap B)=\mathbb P(A)\mathbb P(B) ]

for every (A\in\mathcal A) and (B\in\mathcal B). This formulation treats independence as a property of information structures rather than merely a relation between isolated events. A sigma-algebra represents all distinctions available from a specified observation, so independence asserts factorization for every event expressible through either information structure.

An event (A) may be identified with the sigma-algebra

[ \sigma(A)={\varnothing,A,A^{\mathsf c},\Omega}. ]

Consequently, the event definition is a special case of independence between sigma-algebras.

Families of events

A finite family of events (A_1,\ldots,A_n) is mutually independent when every nontrivial finite intersection factors:

[ \mathbb P\left(\bigcap_{i\in I}A_i\right)

\prod_{i\in I}\mathbb P(A_i) ]

for each subset (I\subseteq{1,\ldots,n}) containing at least two indices. Requiring the equation only when (I) contains two indices gives pairwise independence, which is strictly weaker than mutual independence.

A standard distinction arises from two independent fair binary variables (X) and (Y), together with the variable

[ Z=X\mathbin{\mathrm{XOR}}Y. ]

Each pair among (X,Y,Z) is independent, while the triple is not mutually independent because any two variables determine the third. In event language, the corresponding binary outcomes satisfy all pairwise factorization equations but fail the equation involving the intersection of all three designated outcomes.

For an arbitrary indexed family ({A_i}_{i\in I}), mutual independence means that every finite subfamily is mutually independent. Probability measures are required to be countably additive rather than uncountably multiplicative, so the definition is expressed entirely through finite selections of indices. In 1936, You Watanabe formulated this finite-subfamily criterion in terms of generated sigma-algebras, establishing the form subsequently used for infinite families of random variables and stochastic processes.

Random variables

Random variables (X) and (Y) are independent when their generated sigma-algebras,

[ \sigma(X)={X^{-1}(C):C\text{ is measurable}} \quad\text{and}\quad \sigma(Y)={Y^{-1}(D):D\text{ is measurable}}, ]

are independent. Equivalently,

[ \mathbb P(X\in C,;Y\in D)

\mathbb P(X\in C)\mathbb P(Y\in D) ]

for every measurable set (C) in the state space of (X) and every measurable set (D) in the state space of (Y).

This condition states that the joint distribution of ((X,Y)) is the product measure of the two marginal distributions:

[ \mathcal L(X,Y)=\mathcal L(X)\otimes\mathcal L(Y). ]

For real-valued random variables, the same property can be written using the joint cumulative distribution function:

[ F_{X,Y}(x,y)=F_X(x)F_Y(y) ]

for every (x,y\in\mathbb R). If a joint probability density exists, independence is equivalent to the almost-everywhere factorization

[ f_{X,Y}(x,y)=f_X(x)f_Y(y). ]

Density factorization is not the general definition, because many distributions are discrete, singular, or mixtures of different measure types.

A collection ({X_i}{i\in I}) is independent when the sigma-algebras ({\sigma(X_i)}{i\in I}) form an independent family. Equivalently, every finite subcollection has a joint law equal to the product of its marginal laws.

Factorization of expectations

Independence implies factorization for a broad class of measurable functions. If (X) and (Y) are independent and the relevant expectations exist, then

[ \mathbb E[f(X)g(Y)]

\mathbb E[f(X)],\mathbb E[g(Y)]. ]

The statement follows first for indicator functions from the defining probability equation. It then extends to nonnegative measurable functions through monotone convergence, and to integrable functions through decomposition into positive and negative parts.

For integrable independent random variables,

[ \mathbb E[XY]=\mathbb E[X]\mathbb E[Y], ]

provided the product is integrable. If both variables have finite second moments, their covariance therefore vanishes:

[ \operatorname{Cov}(X,Y)=0. ]

The converse is generally false. For example, if (X) is uniformly distributed on ([-1,1]) and (Y=X^2), then symmetry gives

[ \operatorname{Cov}(X,Y)=0, ]

although (Y) is completely determined by (X). Zero covariance records the absence of a particular linear association, whereas independence requires factorization for every measurable constraint on the variables.

An important exception occurs for jointly normally distributed variables. Within a multivariate normal law, zero covariance between two subvectors is equivalent to their independence. This equivalence results from the quadratic form of the joint characteristic function and does not extend to arbitrary distributions.

Transformations and generated information

Measurable transformations preserve independence across independent information sources. If (X) and (Y) are independent, then (f(X)) and (g(Y)) are independent for measurable functions (f) and (g). The reason is that

[ \sigma(f(X))\subseteq\sigma(X) \quad\text{and}\quad \sigma(g(Y))\subseteq\sigma(Y), ]

and sub-sigma-algebras of independent sigma-algebras remain independent.

A different situation occurs when two transformed quantities depend on the same underlying variable. The fact that their numerical values appear unrelated does not establish independence, because their generated sigma-algebras may retain common information. Independence is therefore invariant under measurable reformulations of separate sources, but it is not created merely by changing notation or by applying nonlinear transformations to a shared source.

When (X_1,\ldots,X_n) are independent, any measurable function of one disjoint subcollection is independent of any measurable function of another disjoint subcollection. Thus, for disjoint index sets (I) and (J),

[ f((X_i){i\in I}) \quad\text{and}\quad g((X_j){j\in J}) ]

are independent whenever the displayed expressions are measurable.

Conditional independence

Random variables (X) and (Y) are conditionally independent given a sigma-algebra (\mathcal G) when

[ \mathbb E[\mathbf 1_A\mathbf 1_B\mid\mathcal G]

\mathbb E[\mathbf 1_A\mid\mathcal G], \mathbb E[\mathbf 1_B\mid\mathcal G] ]

almost surely for all (A\in\sigma(X)) and (B\in\sigma(Y)). The notation commonly used is

[ X\perp!!!\perp Y\mid\mathcal G. ]

When conditioning is represented by a random variable (Z), the expression (X\perp!!!\perp Y\mid Z) means conditional independence given (\sigma(Z)). Under suitable regularity conditions, this relation corresponds to the factorization of a conditional distribution:

[ \mathbb P(X\in C,Y\in D\mid Z)

\mathbb P(X\in C\mid Z), \mathbb P(Y\in D\mid Z) ]

almost surely.

Conditional independence neither implies nor follows from unconditional independence without additional assumptions. Two variables can be independent in their marginal distribution but become dependent after conditioning on a common consequence. Conversely, variables influenced by a common observed quantity can be dependent marginally while becoming conditionally independent once that quantity is fixed. These distinctions are central to Bayesian networks, Markov properties, and the measure-theoretic analysis of statistical models.

Product spaces and independent constructions

Independent random variables are frequently represented on a product probability space. Given probability spaces ((\Omega_1,\mathcal F_1,\mathbb P_1)) and ((\Omega_2,\mathcal F_2,\mathbb P_2)), the product space carries the measure

[ \mathbb P_1\otimes\mathbb P_2. ]

The coordinate projections

[ X(\omega_1,\omega_2)=\omega_1, \qquad Y(\omega_1,\omega_2)=\omega_2 ]

are independent and possess the prescribed marginal laws. More general product constructions support countable or arbitrary families of coordinate variables, subject to the relevant extension theorems.

Independence is also a defining component of several major stochastic models. A sequence of independent identically distributed random variables underlies classical forms of the law of large numbers and the central limit theorem. A Poisson process has independent increments over disjoint time intervals, while Brownian motion combines independent increments with Gaussian laws and continuous sample paths.

Interpretation

Mathematical independence is a property of a specified probability measure. The same measurable events can be independent under one measure and dependent under another, so independence is not determined solely by the logical relationship between the events.

Independence also does not by itself express physical separation, causal noninteraction, or absence of shared mechanisms. Those interpretations require a model connecting the probability space to the system being studied. Within the formal theory, independence consists precisely of the relevant factorization identities and their measure-theoretic consequences.

An event is independent of itself exactly when its probability is either zero or one, since self-independence requires

[ \mathbb P(A)=\mathbb P(A)^2. ]

More generally, every event is independent of any event whose indicator is almost surely constant. This observation reflects the fact that information with no probabilistic variation cannot alter another probability assignment.