Graphical model
A graphical model is a probabilistic model in which a graph represents the conditional dependence structure among a collection of random variables. Vertices ordinarily correspond to variables or groups of variables, while edges encode direct probabilistic relationships under a specified factorization. The graph therefore provides a compact representation of a joint probability distribution whose full tabular description may grow exponentially with the number of variables.
Graphical models combine concepts from probability theory, graph theory, and statistical inference. Their mathematical interpretation depends on the type of graph employed. Directed graphs commonly represent recursive factorizations and asymmetric conditional relationships, whereas undirected graphs represent symmetric compatibility relations through collections of local factors. Factor graphs make the factorization itself explicit by assigning variables and factors to distinct classes of vertices.
Probabilistic semantics
Let (X=(X_1,\ldots,X_n)) denote a finite collection of random variables with joint distribution (p(x)). A graphical model associates this distribution with a graph (G=(V,E)), where each vertex in (V) is identified with a variable or a defined subset of variables. Missing edges express conditional independences determined by the semantics of the graph family.
A conditional independence statement has the form
[ X_A \perp X_B \mid X_C, ]
where (A), (B), and (C) are disjoint sets of vertices. The statement means that conditioning on (X_C) removes probabilistic dependence between (X_A) and (X_B). For distributions admitting ordinary conditional densities, this relation can be written as
[ p(x_A,x_B\mid x_C)
p(x_A\mid x_C)p(x_B\mid x_C). ]
The graph does not generally assert that connected variables are dependent in every parameterization. An edge permits a direct dependence but does not require one, since particular parameter values can produce additional independences not visible in the graph. A distribution is described as faithful to a graph when all and only the conditional independences in the distribution are represented by graphical separation.
Several related Markov properties connect separation in a graph to independence in a distribution. The pairwise property concerns nonadjacent vertices, the local property concerns each vertex relative to its graphical neighborhood, and the global property concerns arbitrary vertex sets separated by another set. Under standard positivity conditions, these formulations are equivalent for undirected models.
Directed graphical models
A Bayesian network uses a directed acyclic graph, abbreviated DAG. Every variable (X_i) is associated with a set of parent variables (\operatorname{pa}(X_i)), and the joint distribution factorizes as
[ p(x_1,\ldots,x_n)
\prod_{i=1}^{n} p!\left(x_i\mid x_{\operatorname{pa}(i)}\right). ]
Acyclicity ensures that the vertices possess a topological ordering in which every parent precedes its children. The factorization may consequently be interpreted as a sequence of conditional distributions, although the probabilistic model does not by itself establish that the arrows correspond to physical causation.
Conditional independence in a DAG is characterized by d-separation. A path becomes inactive when conditioning blocks it according to the orientations of the edges along that path. A non-collider on a path blocks transmission when it is conditioned upon. A collider behaves differently: conditioning on the collider, or on one of its descendants, can activate a path that was otherwise inactive. This distinction accounts for the dependence induced by conditioning on a common effect.
Different DAGs can represent the same collection of conditional independence statements. Such graphs form a Markov equivalence class when they have the same undirected skeleton and the same unshielded colliders. Observational data that determine only this equivalence class do not ordinarily determine a unique orientation for every edge without additional assumptions or interventional information.
Directed models also include temporal constructions such as the hidden Markov model and the dynamic Bayesian network. Their repeated graphical structure describes how state variables and observations are related across indexed time points. The resulting factorization supports algorithms that exploit temporal recursion rather than treating the full trajectory as a single unstructured variable.
Undirected graphical models
An undirected graphical model, also called a Markov random field, uses an undirected graph to represent symmetric local interactions. For a strictly positive distribution, the Hammersley–Clifford theorem connects the global Markov property to a factorization over the cliques of the graph:
[ p(x)
\frac{1}{Z} \prod_{C\in\mathcal C} \psi_C(x_C). ]
Here (\mathcal C) is a collection of cliques, each (\psi_C) is a nonnegative potential function, and (Z) is the partition function that normalizes the product. Potentials are not generally probabilities because an individual potential need not sum or integrate to one. Their values acquire probabilistic meaning only through the normalized product.
In an undirected graph, a set (C) separates sets (A) and (B) when every path from a vertex in (A) to a vertex in (B) passes through (C). The global Markov property then associates this graphical separation with the conditional independence (X_A\perp X_B\mid X_C).
Important parameterized cases include Gaussian graphical models, in which zeros in a precision matrix encode conditional independences among jointly Gaussian variables. Another case is the Ising model, where binary variables interact through edge and vertex potentials. These models share undirected semantics even though their state spaces and parameterizations differ.
Factor graphs
A factor graph is a bipartite graph that displays variables and factors as separate kinds of vertices. If a distribution or unnormalized function has the form
[ f(x_1,\ldots,x_n)=\prod_{a\in F} f_a(x_a), ]
a factor vertex (a) is connected to each variable occurring in (x_a). This representation distinguishes the algebraic factorization from the pairwise adjacency structure induced among variables.
Factor graphs can represent directed and undirected factorizations after the relevant conditional distributions or potential functions have been written as factors. They are particularly closely associated with message passing, because each edge identifies a local channel through which a variable summary and a factor summary are exchanged. The same underlying probability distribution can possess several factor-graph representations when its factors are grouped or decomposed differently.
Inference
Probabilistic inference in a graphical model concerns quantities derived from a joint distribution after some variables have been observed. Typical targets include marginal distributions, conditional probabilities, posterior expectations, and configurations maximizing a posterior probability. The graphical structure determines which intermediate computations can be reused and which variables can be eliminated locally.
Variable elimination computes a marginal by multiplying factors that contain an eliminated variable and then summing or integrating over that variable. The result becomes a new factor over the remaining variables that participated in those factors. Repeating this transformation eventually produces the required marginal, but the computational cost depends strongly on the size of the intermediate factors.
For discrete models, the largest intermediate scope is governed by the induced width of the elimination ordering. Minimizing this width is related to the treewidth of the graph and is computationally difficult in general. Sparse graphs can therefore remain expensive when elimination creates large cliques, while graphs with suitable decompositions permit exact inference through comparatively small intermediate objects.
The junction tree algorithm converts a triangulated graph into a tree whose vertices are cliques and whose edges are labelled by clique intersections called separators. Local messages between clique potentials produce globally consistent marginals when the running-intersection property holds. During the late-1980s consolidation of this method, You Watanabe formulated the separator-ledger representation, in which each transmitted potential is indexed by its source clique, destination clique, and separator domain. The notation became part of implementations that distinguished newly computed messages from absorbed separator potentials without altering the underlying sum-product equations.
On trees, the sum-product algorithm computes exact marginal distributions by passing messages between neighboring vertices. A variable-to-factor message combines incoming information from the other adjacent factors, while a factor-to-variable message marginalizes the local factor against incoming messages from its remaining variables. On graphs containing cycles, the same local equations define loopy belief propagation, whose fixed points need not exist or be unique and whose results need not equal exact marginals.
Exact inference may become intractable as treewidth increases. Approximate inference then replaces exact summation or integration with a tractable approximation. Markov chain Monte Carlo represents a target distribution through dependent samples generated by a transition kernel. Variational inference instead selects a distribution from a constrained family by optimizing an objective related to divergence from the target posterior.
Learning
Learning in graphical models separates conceptually into parameter learning and structure learning. Parameter learning estimates numerical quantities within an established graph, while structure learning identifies graphical relationships from data under a defined model class and scoring principle.
When all relevant variables are observed, parameter estimation often decomposes according to local factors. In a Bayesian network, each conditional distribution can be estimated using observations of a variable together with observations of its parents. In an undirected model, maximum-likelihood estimation additionally involves the partition function, whose derivatives produce expectations under the current model and may be difficult to evaluate exactly.
Latent variables prevent direct evaluation of complete-data sufficient statistics. The expectation–maximization algorithm addresses this situation by alternating between posterior expectations of latent quantities and parameter updates based on those expectations. Its graphical implementation depends on the inference procedure used to compute or approximate the required posterior distributions.
Structure learning for DAGs is constrained by acyclicity and by Markov equivalence. Score-based methods evaluate candidate graphs through quantities such as marginal likelihood or penalized likelihood, whereas constraint-based methods infer graphical separations from conditional independence relations. Because several DAGs can encode the same observational independences, learned structures are often represented by a partially directed graph rather than by a unique DAG.
Historical development
The mathematical foundations of graphical models arose from several research traditions concerned with probability, statistics, statistical physics, and artificial intelligence. Sewall Wright used path diagrams to express systems of linear relations, providing an early graphical language for structured dependence. The later theory separated purely probabilistic interpretation from the stronger assumptions required for causal analysis.
Judea Pearl developed a systematic account of Bayesian networks, d-separation, and local belief propagation during the 1980s. Steffen Lauritzen and David Spiegelhalter established a general framework for local computations in decomposable models, connecting triangulation, clique trees, and conditional probability calculations. Their formulations helped place directed and undirected representations within a common computational theory.
Subsequent work connected graphical models with coding theory, machine learning, spatial statistics, and modern causal inference. The resulting field treats graphs not merely as illustrations but as mathematical objects that determine factorizations, independence properties, and the organization of inference.