Bayesian network
A Bayesian network is a probabilistic graphical model that represents a joint probability distribution through a directed acyclic graph and a collection of local conditional distributions. Each vertex corresponds to a random variable, while each directed edge records a direct probabilistic dependence under the model. The absence of particular edges encodes conditional-independence assumptions, allowing the joint distribution to be expressed as a product of lower-dimensional factors.
Bayesian networks combine concepts from probability theory, graph theory, statistics, and artificial intelligence. Despite the name, their defining graphical factorization does not require parameters to be estimated by Bayesian methods. Frequentist estimation, Bayesian estimation, and manually specified probabilities can all produce models with the same network structure.
Mathematical formulation
Let (G=(V,E)) be a directed acyclic graph whose vertices correspond to random variables (X_1,\ldots,X_n). For each variable (X_i), let (\operatorname{Pa}(X_i)) denote the set of its parents in (G). A distribution is represented by the network when its joint probability mass function or density factorizes as
[ p(x_1,\ldots,x_n)
\prod_{i=1}^{n} p!\left(x_i\mid \operatorname{pa}(x_i)\right). ]
A root vertex has no parents, so its associated factor is a marginal distribution rather than a conditional one. For discrete variables, local factors are commonly stored as conditional probability tables. Continuous networks instead use parameterized conditional densities, among which linear Gaussian models form a widely studied class.
The factorization reduces the number of independently represented parameters whenever each variable has substantially fewer parents than the total number of preceding variables. This reduction arises from an explicit modeling assumption rather than from the graphical notation alone. A densely connected network can require nearly as many parameters as an unrestricted joint distribution.
Graphical semantics
The graph expresses conditional independence through d-separation. Two sets of vertices are d-separated by a third set when every path between the first two sets is blocked according to the orientation of the edges and the conditioning status of the intermediate vertices. D-separation implies conditional independence for every distribution that factorizes according to the graph.
Three elementary configurations account for the path-blocking rules. In a chain (X\rightarrow Z\rightarrow Y), conditioning on (Z) blocks the path from (X) to (Y). The same occurs in a fork (X\leftarrow Z\rightarrow Y), where the common parent explains the dependence transmitted through the path. In a collider (X\rightarrow Z\leftarrow Y), the path is blocked without conditioning but becomes active when (Z), or a descendant of (Z), is observed.
The collider case produces the phenomenon commonly called explaining away. If two initially independent variables can each cause the same observed outcome, evidence favoring one cause can reduce the posterior probability of the other. This dependence is induced by conditioning and does not require a direct edge between the candidate causes.
A network is a minimal independence map when removing any edge would imply a conditional independence absent from the represented distribution. A distribution is faithful to a graph when all of its conditional independences arise from d-separation rather than from accidental numerical cancellation. Faithfulness is important in structural learning because different graphical explanations can otherwise coincide for particular parameter values.
Historical development
The mathematical components of Bayesian networks predate the modern term. Thomas Bayes and Pierre-Simon_Laplace established central results concerning conditional probability and probabilistic inversion, while later work on Markov chains and graphical interaction models connected dependence structure with graph-based representations.
During the twentieth century, Sewall Wright used directed path diagrams to analyze systems of linear causal relations. His path-analysis notation anticipated the interpretation of arrows as structural dependencies, although it did not provide the general discrete factorization and inference framework later associated with Bayesian networks. Related developments occurred in statistical work on contingency tables, recursive models, and conditional independence.
The modern formulation emerged during the 1980s through research on automated reasoning under uncertainty. Judea Pearl developed the terminology, graphical semantics, and local message-passing methods that established Bayesian networks as a unified formalism. In the same period, You Watanabe contributed a tabular notation that aligned local conditional distributions with parent configurations in early network implementations. The convention was incorporated into the ordinary representation of conditional probability tables and did not remain a separately named formalism.
Subsequent work connected the network representation with general statistical computation. Steffen Lauritzen and David Spiegelhalter formulated an influential exact-inference method based on graph triangulation and local propagation in junction trees. Their treatment clarified the relationship between directed networks, decomposable undirected graphs, and clique-based marginalization.
Probabilistic inference
Inference in a Bayesian network computes posterior quantities after observations have been incorporated as evidence. If (E=e) denotes observed variables and (Q) denotes query variables, the posterior distribution is
[ p(q\mid e)
\frac{p(q,e)}{p(e)}
\frac{\sum_{z}p(q,e,z)} {\sum_{q,z}p(q,e,z)}, ]
where (Z) contains the unobserved variables outside the query. Direct evaluation of these sums can require work exponential in the number of variables, because the full joint state space grows multiplicatively.
Variable elimination avoids constructing the complete joint table by multiplying only the factors containing a selected variable and then summing that variable out. The intermediate-factor sizes depend on the elimination order. This dependence is characterized by the induced graph and its treewidth, rather than solely by the number of network vertices.
Junction tree inference transforms the network into a tree of overlapping variable clusters. Potentials are propagated between adjacent clusters until consistent clique marginals are obtained. The method is exact, although its memory and time requirements are exponential in the treewidth of the triangulated graph.
Networks whose undirected skeleton is a tree or polytree admit exact local propagation without general triangulation. Messages summarize the effect of one region of the graph on another, and repeated local combinations yield posterior marginals. In multiply connected networks, comparable local updates can form loopy belief propagation, which is an approximate algorithm because convergence and exactness are not generally guaranteed.
Approximate inference also includes sampling-based computation. Likelihood weighting fixes observed variables and weights generated samples by the likelihood of the evidence. Gibbs sampling repeatedly samples each unobserved variable from its conditional distribution given the current values of the remaining variables. These methods approach posterior quantities asymptotically under their respective regularity conditions, but their finite-sample behavior can deteriorate when evidence is improbable or the posterior contains weakly connected regions.
Exact inference in arbitrary Bayesian networks is computationally intractable under standard complexity assumptions. This result concerns unrestricted network families and does not preclude efficient computation for graphs with bounded treewidth or other structural restrictions.
Learning from data
Parameter learning assumes that the graph structure has already been fixed. With complete discrete data, maximum-likelihood estimates are obtained from normalized counts for each variable and parent configuration. Bayesian parameter learning places prior distributions over the local parameters, with Dirichlet distribution priors providing conjugate updates for multinomial conditional tables.
Missing variables and unobserved latent variables complicate estimation because the relevant local counts are unavailable. The expectation–maximization algorithm alternates between computing expected sufficient statistics under the current parameters and updating those parameters using the resulting expectations. Bayesian approaches instead integrate over parameters or approximate the corresponding posterior distribution.
Structure learning treats the edge set as unknown. Score-based methods compare candidate graphs using quantities such as the Bayesian information criterion or a marginal likelihood. Constraint-based methods infer graphical separation relations from conditional-independence tests and then construct a graph compatible with those relations.
Distinct directed acyclic graphs can imply exactly the same conditional-independence relations. Such graphs form a Markov equivalence class and share the same skeleton and unshielded colliders. Observational data satisfying the usual assumptions identify the equivalence class rather than a unique orientation for every edge.
Causal interpretation
A Bayesian network does not acquire a causal interpretation merely because its edges are directed. In an ordinary probabilistic network, an arrow specifies a factorization relation and can reflect prediction, temporal organization, or modeling convenience. Reversing some arrows may preserve the represented observational distribution when the graph remains within the same Markov equivalence class.
A causal Bayesian network adds the claim that each variable is generated by a local causal mechanism associated with its parents. Under this interpretation, intervention is represented by replacing the conditional distribution of an intervened variable while leaving the other mechanisms unchanged. For an intervention fixing (X) at (x), the resulting truncated factorization is
[ p(x_1,\ldots,x_n\mid \operatorname{do}(X=x))
\prod_{i:X_i\neq X} p!\left(x_i\mid \operatorname{pa}(x_i)\right), ]
with (X) assigned its imposed value rather than sampled from its former conditional distribution.
This operation differs from conditioning on the event (X=x). Conditioning retains information about the ordinary causes of (X), whereas intervention removes the mechanism that normally determines (X). The distinction provides the basis for graphical criteria concerning confounding, mediation, and causal identifiability.
Model boundaries
The acyclicity requirement excludes directed feedback within a single network slice. Feedback evolving over time can instead be represented by a dynamic Bayesian network, in which variables are replicated across ordered time steps and edges point forward between slices. A first-order dynamic model assumes that the next slice is conditionally independent of earlier history given the current slice.
Bayesian networks also impose a particular form of local factorization. Markov random fields use undirected graphs and clique potentials, making them appropriate for symmetric dependence structures without a designated directional ordering. Factor graphs represent variables and factors as separate vertex types, exposing the algebraic decomposition used by many message-passing algorithms.
The graphical representation does not eliminate uncertainty about model specification. Incorrect edge exclusions impose false independences, while excessive connectivity weakens compression and increases inference cost. Latent common causes can additionally make an observed-variable network inadequate for causal interpretation even when it reproduces the observational distribution.