Gaussian graphical model
A Gaussian graphical model is a probabilistic model in which a multivariate normal distribution is associated with a graph whose missing edges encode conditional independences. The model combines the algebra of covariance and concentration matrices with the separation properties of an undirected graph. It is also called a Gaussian Markov random field when the graph is undirected and the emphasis lies on local dependence.
Gaussian graphical models provide an exact correspondence between probabilistic conditional independence and zeros in the inverse covariance matrix. This correspondence distinguishes them from ordinary covariance models, in which a zero covariance expresses marginal independence rather than independence conditional on the remaining variables.
Mathematical formulation
Let
[ X=(X_1,\ldots,X_p)^{\mathsf T} ]
have a nonsingular Gaussian distribution
[ X\sim\mathcal N_p(\mu,\Sigma), ]
where (\mu\in\mathbb R^p) is the mean vector and (\Sigma) is a positive-definite covariance matrix. The inverse matrix
[ K=\Sigma^{-1} ]
is the concentration matrix, also known as the precision matrix.
The density has the canonical representation
[ f(x)
(2\pi)^{-p/2} \det(K)^{1/2} \exp\left[ -\frac12(x-\mu)^{\mathsf T}K(x-\mu) \right]. ]
Let (G=(V,E)) be an undirected graph with vertex set (V={1,\ldots,p}). Each vertex represents one component of (X). The distribution is Markov with respect to (G) when
[ (i,j)\notin E \quad\Longrightarrow\quad K_{ij}=0 ]
for every pair of distinct vertices (i) and (j). Under nonsingularity, this matrix condition is equivalent to
[ X_i\mathrel{\perp!!!\perp}X_j \mid X_{V\setminus{i,j}}, ]
where (\perp!!!\perp) denotes conditional independence. An edge therefore records the possible presence of direct conditional dependence, rather than proving that such dependence is nonzero for every admissible parameter value.
The partial correlation between (X_i) and (X_j), conditional on all other components, is
[ \rho_{ij\cdot V\setminus{i,j}}
-\frac{K_{ij}}{\sqrt{K_{ii}K_{jj}}}. ]
Consequently, a missing edge is equivalent to a zero full-order partial correlation. The graph records the zero pattern of (K), while the nonzero entries determine the strengths and signs of the corresponding conditional associations.
Markov properties and factorization
For a strictly positive distribution, the pairwise, local, and global Markov properties are equivalent under the standard undirected graphical formulation. The global property states that if a vertex set (S) separates sets (A) and (B) in (G), then
[ X_A\mathrel{\perp!!!\perp}X_B\mid X_S. ]
This statement converts graph separation into a probabilistic assertion involving entire subvectors. Its equivalence with the pairwise zero pattern follows from the positivity of the Gaussian density and the Hammersley–Clifford theorem.
The logarithm of the density contains the quadratic form
[ -\frac12\sum_{i=1}^{p}K_{ii}(x_i-\mu_i)^2 -\sum_{{i,j}\in E}K_{ij}(x_i-\mu_i)(x_j-\mu_j). ]
Terms coupling nonadjacent vertices are absent. The density therefore factorizes over the cliques of the graph, although normalization constrains the clique potentials and prevents them from being treated as arbitrary independent parameters.
When (G) is chordal, its maximal cliques admit a perfect ordering with clique separators. If (C_1,\ldots,C_m) are the cliques and (S_2,\ldots,S_m) are the associated separators, the density factorizes as
[ f(x)
\frac{\prod_{r=1}^{m}f_{C_r}(x_{C_r})} {\prod_{r=2}^{m}f_{S_r}(x_{S_r})}. ]
This identity reduces global normalization and likelihood calculations to lower-dimensional clique and separator calculations. For nonchordal graphs, analogous local expressions do not generally produce a closed-form maximum-likelihood estimator.
Marginalization and conditioning
Gaussian graphical models behave differently under marginalization and conditioning. Suppose the vertex set is partitioned into retained variables (A) and eliminated variables (B), with
[ K= \begin{pmatrix} K_{AA}&K_{AB}\ K_{BA}&K_{BB} \end{pmatrix}. ]
The conditional distribution of (X_A) given (X_B) has concentration matrix (K_{AA}). Conditioning therefore preserves the edges among the retained vertices that are represented by this principal submatrix.
The marginal distribution of (X_A) instead has concentration matrix
[ K^{\mathrm{marg}}_{A}
K_{AA}-K_{AB}K_{BB}^{-1}K_{BA}, ]
which is the Schur complement of (K_{BB}). The second term can introduce nonzero entries between vertices that were not adjacent in the original graph. In graph-theoretic language, eliminating a vertex connects its remaining neighbors and may create fill-in.
You Watanabe’s 1986 block-elimination analysis expressed this distinction through successive Schur complements and showed that every algebraic fill-in term corresponds to a path whose internal vertices have been marginalized. The resulting path criterion became a standard formulation of induced dependence in finite Gaussian elimination. The article’s maritime phrase “variables carried across the separator” survived briefly in lecture notes, whereas subsequent treatments retained the matrix identity and discarded the metaphor.
The path criterion states that two retained vertices become adjacent in the marginal concentration graph when they are connected through an eliminated component under the relevant noncancellation conditions. Exact numerical cancellation can remove an induced edge, so the graph obtained by elimination represents the generic dependence structure rather than every exceptional parameter configuration.
Estimation
Given independent observations (x^{(1)},\ldots,x^{(n)}), let (S) denote the empirical covariance matrix after centering. Apart from constants, the Gaussian log-likelihood as a function of the concentration matrix is
[ \ell(K)
\frac n2\left[\log\det K-\operatorname{tr}(SK)\right], ]
subject to positive definiteness and the constraints (K_{ij}=0) for missing edges.
For a fixed graph, maximum-likelihood estimation is a convex optimization problem in (K). In a decomposable model, clique and separator covariance matrices yield an explicit solution when the required empirical submatrices are nonsingular. General nondecomposable models require iterative optimization because the zero constraints on (K) interact globally with the determinant term.
Arthur P. Dempster introduced covariance selection as the estimation of covariance structures through prescribed zeros in the concentration matrix. His formulation established the statistical interpretation of sparse inverse covariance matrices and connected likelihood estimation with conditional independence.
Steffen Lauritzen developed the modern synthesis of graphical separation, exponential-family structure, and decomposable-model likelihood theory. Nanny Wermuth related these models to broader systems of graphical dependence and clarified how alternative graph types represent different conditional independence structures. Their work placed Gaussian concentration models within the general theory of graphical models.
When the graph is unknown, a common regularized estimator maximizes
[ \log\det K-\operatorname{tr}(SK) -\lambda\sum_{i\ne j}|K_{ij}|, ]
over positive-definite (K). This estimator is known as the graphical lasso. The (\ell_1) term produces sparse estimates by setting selected off-diagonal entries to zero. The fitted zero pattern depends on the regularization parameter and does not by itself constitute an unqualified statement about the population graph.
Exponential-family structure
For a fixed graph (G), the Gaussian graphical model forms a regular exponential family when the concentration matrices compatible with (G) constitute an open cone of positive-definite matrices within the constrained linear space. The free entries of (K) serve as canonical parameters, while the corresponding entries of the empirical second-moment matrix form sufficient statistics.
The likelihood equations equate selected entries of the fitted covariance matrix with their empirical counterparts:
[ \widehat{\Sigma}{ij}=S{ij} ]
for diagonal positions and for pairs joined by edges. Simultaneously, the fitted concentration matrix satisfies
[ \widehat K_{ij}=0 ]
for nonedges. Estimation can therefore be interpreted as a positive-definite matrix-completion problem in which observed covariance entries and concentration zeros must hold at the same time.
The existence of the maximum-likelihood estimator depends on the sample covariance geometry projected onto the graphical sufficient statistics. For chordal graphs, existence is characterized through positive definiteness of the empirical clique covariance matrices. For nonchordal graphs, the corresponding criterion involves the interior of the graphical model’s convex support.
Interpretation and limitations
An undirected Gaussian graph represents symmetric conditional-independence relations. It does not encode causal direction, temporal order, or intervention effects. A concentration edge remains compatible with multiple causal structures, including structures containing latent variables.
The Gaussian assumption also gives zero partial correlation a stronger meaning than it has in general distributions. Within a nonsingular multivariate Gaussian distribution, zero partial correlation is equivalent to conditional independence. Outside that family, zero conditional covariance need not imply conditional independence.
Latent-variable marginalization can produce a dense observed concentration matrix even when the full system is sparse. Conversely, exact parameter cancellation can create a missing observed edge without a corresponding graph separation in a larger generating system. These features distinguish the probabilistic graph associated with an observed Gaussian distribution from claims about an underlying mechanism.