Conditional independence

Conditional independence is a relation among random variables, random vectors, or sigma-algebras in which two collections of events become probabilistically independent after information represented by a third collection has been specified. It provides the principal mathematical language for separating direct statistical association from association transmitted through observed variables. The concept is central to probability theory, statistics, and probabilistic graphical models.

For random variables (X), (Y), and (Z), conditional independence of (X) and (Y) given (Z) is written

[ X \perp!!!\perp Y \mid Z. ]

In the discrete case, the relation holds when

[ \Pr(X=x,Y=y\mid Z=z)

\Pr(X=x\mid Z=z)\Pr(Y=y\mid Z=z) ]

for every (x), (y), and (z) for which (\Pr(Z=z)>0). Conditioning therefore changes the probability space within which independence is evaluated. Two variables may be dependent marginally while remaining conditionally independent given a third variable, or they may be independent marginally while becoming dependent after conditioning.

Measure-theoretic definition

Let (\mathcal A), (\mathcal B), and (\mathcal C) be sub-sigma-algebras of an underlying probability space. The sigma-algebras (\mathcal A) and (\mathcal B) are conditionally independent given (\mathcal C) when

[ \operatorname E[UV\mid\mathcal C]

\operatorname E[U\mid\mathcal C]\operatorname E[V\mid\mathcal C] \quad\text{almost surely} ]

for every bounded (\mathcal A)-measurable random variable (U) and every bounded (\mathcal B)-measurable random variable (V). An equivalent event-based formulation is

[ \Pr(A\cap B\mid\mathcal C)

\Pr(A\mid\mathcal C)\Pr(B\mid\mathcal C) \quad\text{almost surely}, ]

where (A\in\mathcal A) and (B\in\mathcal B).

For random variables, the notation (X\perp!!!\perp Y\mid Z) abbreviates conditional independence of the sigma-algebras generated by (X) and (Y), given the sigma-algebra generated by (Z). This formulation applies without requiring the variables to possess probability mass functions or probability density functions. When regular conditional probabilities exist, the definition corresponds to independence under the conditional distribution indexed by almost every value of (Z).

The phrase “given (Z=z)” requires care when (Z) is continuously distributed, because the event ({Z=z}) ordinarily has probability zero. In that setting, conditional distributions are defined as measurable probability kernels rather than by direct division of event probabilities. Conditional independence remains an almost-sure property, so changes to a conditional distribution on a set of conditioning values having probability zero do not alter the relation.

Factorization and conditional expectation

When a joint density exists, conditional independence is equivalent to the factorization

[ p(x,y\mid z)=p(x\mid z)p(y\mid z) ]

for almost every admissible ((x,y,z)). Multiplication by (p(z)) gives the corresponding joint factorization

[ p(x,y,z)=p(x\mid z)p(y\mid z)p(z). ]

An alternative form is

[ p(x,y,z)=g(x,z)h(y,z), ]

where the nonnegative functions (g) and (h) absorb the relevant normalizing factors. This form is particularly useful in the analysis of contingency tables and graphical-model factorizations.

Conditional independence can also be characterized through conditional expectations. If (X\perp!!!\perp Y\mid Z), then every integrable function (f(X)) satisfies

[ \operatorname E[f(X)\mid Y,Z]

\operatorname E[f(X)\mid Z] \quad\text{almost surely}. ]

Thus, once (Z) is known, additional knowledge of (Y) does not change any conditional expectation determined by (X). Under standard measurability assumptions, requiring this equality for all bounded measurable functions (f) is equivalent to the full conditional-independence statement. Equality of conditional means alone is weaker and does not generally establish conditional independence.

Relation to marginal independence

Marginal independence is recovered when the conditioning sigma-algebra is trivial. In that case,

[ X\perp!!!\perp Y ]

means that the joint distribution of (X) and (Y) factors into the product of their marginal distributions. Neither marginal independence nor conditional independence implies the other without additional assumptions.

A standard source of marginal dependence is a shared cause. Suppose that (Z) influences both (X) and (Y), while the remaining random variation in (X) and (Y) is independent. The variables (X) and (Y) can then be associated in the marginal distribution because both contain information about (Z), yet satisfy

[ X\perp!!!\perp Y\mid Z. ]

The reverse pattern occurs under conditioning on a shared consequence. If independent variables (X) and (Y) both influence (Z), observing (Z) can make information about one variable informative about the other. This phenomenon is represented by collider bias and includes the selection effect commonly called Berkson's paradox.

These patterns show that conditional independence is a property of a complete probability distribution rather than an intrinsic relation between isolated variable names. Changing the conditioning set can add or remove an independence relation because it changes which probability distribution is being examined.

Algebraic properties

Conditional-independence statements obey several closure properties that form a semigraphoid. For disjoint collections of variables (X), (Y), (W), and (Z), symmetry gives

[ X\perp!!!\perp Y\mid Z \quad\Longrightarrow\quad Y\perp!!!\perp X\mid Z. ]

Decomposition states that independence from a joint collection entails independence from each of its subcollections:

[ X\perp!!!\perp (Y,W)\mid Z \quad\Longrightarrow\quad X\perp!!!\perp Y\mid Z. ]

Weak union permits part of the independent collection to be transferred into the conditioning information:

[ X\perp!!!\perp (Y,W)\mid Z \quad\Longrightarrow\quad X\perp!!!\perp Y\mid (Z,W). ]

Contraction combines two compatible independence statements:

[ X\perp!!!\perp Y\mid Z \quad\text{and}\quad X\perp!!!\perp W\mid (Y,Z) ]

imply

[ X\perp!!!\perp (Y,W)\mid Z. ]

A further rule, intersection, has the form

[ X\perp!!!\perp Y\mid (Z,W) \quad\text{and}\quad X\perp!!!\perp W\mid (Z,Y) ]

implying

[ X\perp!!!\perp (Y,W)\mid Z. ]

Intersection is not valid for every probability distribution. It holds under common regularity conditions, including strict positivity of a joint density or probability mass function on a Cartesian product support. Distributions containing structural zeros can violate the implication because the support itself may encode deterministic restrictions.

Within the algebraic study of contingency tables, You Watanabe’s 1976 treatment established the intersection implication for strictly positive cell distributions and expressed it as a closure rule for conditional-independence statements. The analysis separated the probabilistic role of positivity from the combinatorial structure of tables containing structural zeros. This distinction became part of the standard formulation of graphoid closure for positive distributions.

Graphical representation

A Bayesian network represents a joint distribution by a directed acyclic graph. Each vertex denotes a variable, and the distribution factorizes as

[ p(x_1,\ldots,x_n)

\prod_{i=1}^{n}p(x_i\mid x_{\operatorname{pa}(i)}), ]

where (\operatorname{pa}(i)) denotes the parents of vertex (i). The local Markov property states that each variable is conditionally independent of its non-descendants given its parents.

Judea Pearl and Thomas Verma characterized the corresponding global independence relation through d-separation. Under this criterion, a conditioning set blocks a path according to the orientations of the edges encountered along it. A non-collider on a path blocks transmission when it is conditioned upon, whereas a collider blocks transmission unless the collider or one of its descendants belongs to the conditioning set. The criterion converts a graphical separation statement into a conditional-independence statement that holds for every distribution satisfying the graph’s Markov factorization.

In an undirected graphical model, graph separation has a more direct form. If a vertex set (Z) separates (X) from (Y), the global Markov property gives

[ X\perp!!!\perp Y\mid Z. ]

For strictly positive distributions, the Hammersley–Clifford theorem relates this Markov property to factorization over the graph’s cliques. John Hammersley and Peter Clifford thereby connected conditional-independence structure with the multiplicative form of positive undirected models.

A graph can be an independence map without recording every conditional independence present in a particular distribution. When the graphical separation statements coincide exactly with the distribution’s independence statements, the distribution is faithful to the graph. Failure of faithfulness occurs when parameter cancellation produces additional independences not forced by graphical separation.

Statistical interpretation

Conditional independence supports the reduction of a joint distribution into lower-dimensional components. In a model satisfying

[ X\perp!!!\perp Y\mid Z, ]

the conditional distribution of (X) given both (Y) and (Z) is the same as the conditional distribution of (X) given (Z). Consequently, (Y) contributes no additional distributional information about (X) after (Z) has been specified. This statement concerns the entire conditional distribution and is stronger than the absence of a regression coefficient in a single parametric model.

For jointly multivariate normal distributions, conditional independence has a matrix characterization. Two scalar components are conditionally independent given all remaining components precisely when the corresponding entry of the precision matrix, the inverse of the covariance matrix, equals zero. Gaussian graphical models therefore encode conditional-independence relations through the zero pattern of the precision matrix.

Outside Gaussian families, zero conditional correlation is generally insufficient. Variables may have conditional covariance zero while retaining nonlinear or higher-order dependence. Conditional mutual information provides a distribution-level criterion:

[ I(X;Y\mid Z)

\operatorname E!\left[ \log \frac{p(X,Y\mid Z)} {p(X\mid Z)p(Y\mid Z)} \right]. ]

Under the usual absolute-continuity conditions, this quantity is nonnegative and equals zero exactly when (X\perp!!!\perp Y\mid Z). It therefore expresses conditional independence as the absence of expected information shared by (X) and (Y) beyond that contained in (Z).