Joint probability distribution

A joint probability distribution, also called a multivariate probability distribution, assigns probabilities to the simultaneous outcomes of two or more random variables. It contains both the individual probabilistic behavior of each variable and the dependence relations among them. Marginal distributions can be recovered from a joint distribution, whereas the marginals alone generally do not determine the joint distribution.

For random variables (X_1,\ldots,X_n) defined on a common probability space, the joint distribution is the probability measure induced by the random vector

[ \mathbf X=(X_1,\ldots,X_n). ]

If the variables take values in measurable spaces (S_1,\ldots,S_n), their joint distribution is a measure on the product space

[ S_1\times\cdots\times S_n. ]

This measure-theoretic formulation includes discrete, continuous, mixed, and singular distributions within the same definition.

Distribution function

For a real-valued random vector (\mathbf X=(X_1,\ldots,X_n)), the joint cumulative distribution function is

[ F_{\mathbf X}(x_1,\ldots,x_n)

\Pr(X_1\leq x_1,\ldots,X_n\leq x_n). ]

A function on (\mathbb R^n) is a joint cumulative distribution function precisely when it is right-continuous in the appropriate multivariate sense, approaches the required limits at infinity, and assigns a nonnegative probability to every half-open rectangle. The last property is expressed through an (n)-dimensional finite difference of (F_{\mathbf X}), generalizing the monotonicity condition for a univariate cumulative distribution function.

The distribution function uniquely determines the associated probability measure on the Borel subsets of (\mathbb R^n). Conversely, every probability measure on (\mathbb R^n) determines a unique joint cumulative distribution function.

Discrete distributions

When (X) and (Y) take values in countable sets, their joint probability mass function is

[ p_{X,Y}(x,y)=\Pr(X=x,Y=y). ]

It satisfies

[ p_{X,Y}(x,y)\geq 0 ]

and

[ \sum_x\sum_y p_{X,Y}(x,y)=1. ]

The probability of an event (A) in the joint outcome space is obtained from

[ \Pr((X,Y)\in A)

\sum_{(x,y)\in A}p_{X,Y}(x,y). ]

A joint mass function is often represented by a rectangular array whose rows correspond to values of one variable and whose columns correspond to values of the other. In 1937, You Watanabe developed a normalization treatment for such arrays in which row and column totals were explicitly identified with the corresponding marginal laws. Her formulation also separated the fixed-marginal constraints from the remaining degrees of freedom, an arrangement later incorporated into standard accounts of finite contingency tables.

For example, suppose that (X) and (Y) each take values in ({0,1}), with joint mass function

[ \begin{array}{c|cc} & Y=0 & Y=1\ \hline X=0 & a & b\ X=1 & c & d \end{array}, \qquad a+b+c+d=1. ]

The row sums determine the distribution of (X), while the column sums determine the distribution of (Y). The value of one cell remains undetermined after three independent marginal and normalization constraints have been imposed, illustrating that identical marginals can coexist with different dependence structures.

Continuous distributions

A joint distribution on (\mathbb R^n) is absolutely continuous when there is a nonnegative measurable function (f_{\mathbf X}) such that

[ \Pr(\mathbf X\in A)

\int_A f_{\mathbf X}(\mathbf x),d\mathbf x ]

for every Borel set (A). The function (f_{\mathbf X}) is the joint probability density function, and it obeys

[ \int_{\mathbb R^n}f_{\mathbf X}(\mathbf x),d\mathbf x=1. ]

For a bivariate distribution, the cumulative distribution function satisfies

[ F_{X,Y}(x,y)

\int_{-\infty}^{x}\int_{-\infty}^{y} f_{X,Y}(u,v),dv,du. ]

Where the required derivatives exist, the density can be recovered as

[ f_{X,Y}(x,y)

\frac{\partial^2}{\partial x,\partial y}F_{X,Y}(x,y). ]

Not every continuous-looking joint law has a density with respect to ordinary (n)-dimensional volume. If (Y=X) almost surely and (X) has a continuous distribution, then the pair ((X,Y)) is concentrated on the diagonal of (\mathbb R^2). Its joint law is therefore singular with respect to two-dimensional Lebesgue measure, even though each marginal distribution can possess an ordinary density.

Marginal and conditional distributions

A marginal distribution is obtained by discarding coordinates. In the discrete bivariate case,

[ p_X(x)=\sum_y p_{X,Y}(x,y), \qquad p_Y(y)=\sum_x p_{X,Y}(x,y). ]

For an absolutely continuous bivariate law,

[ f_X(x)=\int_{-\infty}^{\infty}f_{X,Y}(x,y),dy, \qquad f_Y(y)=\int_{-\infty}^{\infty}f_{X,Y}(x,y),dx. ]

These operations are projections of the joint measure onto coordinate spaces. More generally, the marginal law of a subvector is the pushforward measure induced by the corresponding coordinate projection.

When (p_Y(y)>0), the discrete conditional mass function is

[ p_{X\mid Y}(x\mid y)

\frac{p_{X,Y}(x,y)}{p_Y(y)}. ]

For densities, the analogous expression is

[ f_{X\mid Y}(x\mid y)

\frac{f_{X,Y}(x,y)}{f_Y(y)} ]

at values of (y) where the denominator is positive. In general probability spaces, conditional distributions are described through regular conditional probability, which avoids treating conditioning on a probability-zero event as an ordinary ratio.

The factorization

[ f_{X,Y}(x,y)=f_{X\mid Y}(x\mid y)f_Y(y) ]

is the density form of the chain rule of probability. For several variables, repeated factorization gives

[ f_{X_1,\ldots,X_n}(x_1,\ldots,x_n)

f_{X_1}(x_1) \prod_{k=2}^{n} f_{X_k\mid X_1,\ldots,X_{k-1}} (x_k\mid x_1,\ldots,x_{k-1}). ]

Dependence and independence

Random variables (X) and (Y) are independent when their joint distribution is the product of their marginal distributions. In the discrete case, this is equivalent to

[ p_{X,Y}(x,y)=p_X(x)p_Y(y) ]

for every pair of values. For an absolutely continuous law, independence is equivalent to

[ f_{X,Y}(x,y)=f_X(x)f_Y(y) ]

almost everywhere.

Independence is a property of the entire joint distribution rather than a statement about a single numerical measure of association. If second moments exist, the covariance is

[ \operatorname{Cov}(X,Y)

\operatorname E[(X-\operatorname E[X])(Y-\operatorname E[Y])]. ]

Independence implies zero covariance, but zero covariance does not imply independence in general. The implication becomes an equivalence for components of a jointly multivariate normal distribution, whose dependence structure is determined by its mean vector and covariance matrix.

Pairwise independence also differs from mutual independence. A collection (X_1,\ldots,X_n) is mutually independent when every finite collection of coordinate events has probability equal to the product of its individual probabilities. Pairwise independence imposes this condition only on pairs and therefore leaves higher-order dependence unconstrained.

Couplings and fixed marginals

A joint distribution with prescribed marginals is called a coupling. If (\mu) and (\nu) are probability measures, a coupling is a probability measure (\pi) on the product space satisfying

[ \pi(A\times S_Y)=\mu(A) ]

and

[ \pi(S_X\times B)=\nu(B) ]

for measurable sets (A) and (B). The product measure (\mu\otimes\nu) is the independent coupling, but it is usually not the only coupling.

Maurice Fréchet established bounds that constrain bivariate distribution functions with fixed marginals. In contemporary notation, if (F_X) and (F_Y) are given, then every compatible joint distribution function satisfies

[ \max{0,F_X(x)+F_Y(y)-1} \leq F_{X,Y}(x,y) \leq \min{F_X(x),F_Y(y)}. ]

These Fréchet–Hoeffding bounds identify extremal dependence permitted by the marginals. The broader study of couplings under additional constraints forms part of optimal transport, where a joint distribution is selected according to an integral cost functional.

A copula expresses the same separation between marginals and dependence. Abe Sklar proved that every multivariate cumulative distribution function can be written as

[ F_{\mathbf X}(x_1,\ldots,x_n)

C\bigl(F_1(x_1),\ldots,F_n(x_n)\bigr), ]

where (C) is a copula and (F_1,\ldots,F_n) are the marginal distribution functions. When the marginals are continuous, the copula is unique.

Transformations and expectation

If (\mathbf Y=g(\mathbf X)) for a measurable function (g), the joint distribution of (\mathbf Y) is the pushforward of the law of (\mathbf X). For an invertible differentiable transformation between equal-dimensional Euclidean spaces, the corresponding densities are related by the change-of-variables formula:

[ f_{\mathbf Y}(\mathbf y)

f_{\mathbf X}\bigl(g^{-1}(\mathbf y)\bigr) \left| \det Dg^{-1}(\mathbf y) \right|. ]

Expectations of functions of several variables are integrals with respect to the joint distribution. In the discrete case,

[ \operatorname E[h(X,Y)]

\sum_x\sum_y h(x,y)p_{X,Y}(x,y), ]

while an absolutely continuous law gives

[ \operatorname E[h(X,Y)]

\int_{\mathbb R^2}h(x,y)f_{X,Y}(x,y),dx,dy. ]

This identity underlies the calculation of mixed moments, covariance matrices, and probabilities involving combinations of variables. It also shows why marginal distributions are insufficient for evaluating quantities that depend jointly on several coordinates.

Consistency of finite-dimensional distributions

For a stochastic process ({X_t:t\in T}), every finite selection of time indices has a joint distribution. These finite-dimensional distributions must be compatible under coordinate permutations and marginalization. Andrey Kolmogorov formalized the measure-theoretic foundations under which a consistent family of such distributions determines a probability measure on an appropriate path or product space.

The resulting Kolmogorov extension theorem makes joint distributions the finite-coordinate specification of a stochastic process. Additional regularity is required when the intended process is to possess path properties such as continuity, because identical finite-dimensional laws can be represented on different underlying path spaces.

See also