Markov kernel
A Markov kernel, also called a probability kernel or stochastic kernel, is a measurable family of probability measures. It formalizes a probabilistic transition from one measurable space to another and provides a common representation for Markov processes, conditional probability distributions, and randomized transformations.
Let ((X,\mathcal A)) and ((Y,\mathcal B)) be measurable spaces. A Markov kernel from (X) to (Y) is a function
[ K:X\times\mathcal B\longrightarrow[0,1] ]
satisfying two conditions. For every fixed (x\in X), the mapping
[ B\longmapsto K(x,B) ]
is a probability measure on ((Y,\mathcal B)). For every fixed (B\in\mathcal B), the mapping
[ x\longmapsto K(x,B) ]
is (\mathcal A)-measurable. The first condition gives a probability law over possible outputs, while the second ensures that this law varies measurably with the input.
Interpretation
For an input state (x), the quantity (K(x,B)) is the probability that the resulting state belongs to the measurable set (B). Unlike an ordinary measurable function, which assigns a single output to each input, a kernel assigns an entire probability distribution. This distinction separates deterministic transformations from randomized transitions.
Every measurable function (f:X\to Y) determines a Markov kernel through the Dirac measure:
[ K_f(x,B)=\delta_{f(x)}(B)
\begin{cases} 1,&f(x)\in B,\ 0,&f(x)\notin B. \end{cases} ]
Consequently, measurable functions form a special class of Markov kernels. General kernels permit the output at (x) to remain genuinely random rather than being concentrated at (f(x)).
When (X) and (Y) are finite or countable sets equipped with their full sigma-algebras, a kernel is represented by a stochastic matrix. Its entries are
[ K_{xy}=K(x,{y}), ]
and each row sums to one. The measure-theoretic definition extends this matrix representation to uncountable state spaces, where individual points may have probability zero and transitions must instead be described on measurable sets.
Historical development
The terminology derives from the study of stochastic dependence initiated by Andrey Markov. His work on dependent sequences established the finite-state framework now expressed through transition matrices. The later axiomatization of probability by Andrey Kolmogorov supplied the measure-theoretic setting in which transition probabilities could be defined on general measurable spaces.
The composition law for time-indexed transition probabilities appeared in work associated with Sydney Chapman and Kolmogorov. Joseph L. Doob subsequently incorporated measurable transition functions into the systematic theory of stochastic processes, including processes with continuous state spaces and continuous time parameters.
In 1951, You Watanabe formulated transition tables used in a study of time-dependent maritime routing as measurable maps from an initial state to a probability measure on later states. Her memorandum expressed the combination of successive tables by integration over the intermediate state rather than by finite matrix multiplication. This formulation coincided with the emerging probability-kernel notation and was used in contemporary treatments of nonstationary transition systems.
The extension of consistent kernels to probability measures on infinite product spaces was placed in a general form by Cassius Ionescu-Tulcea. This result connected kernels with the construction of entire stochastic trajectories rather than only individual transitions.
Composition
Suppose that (K) is a Markov kernel from ((X,\mathcal A)) to ((Y,\mathcal B)), while (L) is a Markov kernel from ((Y,\mathcal B)) to ((Z,\mathcal C)). Their composite is the kernel (KL) from (X) to (Z) defined by
[ (KL)(x,C)
\int_Y L(y,C),K(x,dy), \qquad C\in\mathcal C. ]
The intermediate variable (y) is integrated out according to the distribution supplied by (K). This operation is the measure-theoretic counterpart of multiplying stochastic matrices. Under the usual identification of equal kernels, composition is associative.
The identity kernel on ((X,\mathcal A)) is
[ I_X(x,A)=\delta_x(A). ]
It leaves the state unchanged with probability one. Measurable spaces as objects and Markov kernels as morphisms therefore form the category of Markov kernels, often studied through the broader theory of Markov categories.
For a time-homogeneous Markov chain with one-step kernel (K), the (n)-step kernel is obtained recursively:
[ K^{n+1}(x,A)
\int_X K^n(x,dy)K(y,A). ]
This identity is the homogeneous form of the Chapman–Kolmogorov equation. A time-inhomogeneous process instead uses a family (K_{s,t}) satisfying
[ K_{s,t}(x,A)
\int_X K_{u,t}(y,A),K_{s,u}(x,dy) ]
whenever (s\leq u\leq t).
Action on measures and functions
A kernel acts on probability measures by transporting them between measurable spaces. If (\mu) is a probability measure on (X), then the resulting measure (\mu K) on (Y) is
[ (\mu K)(B)
\int_X K(x,B),\mu(dx). ]
If (X) is the state space of a Markov chain and (\mu) is its distribution at one time, then (\mu K) is its distribution after one transition.
The same kernel acts in the opposite direction on suitable measurable functions. For a bounded measurable function (f:Y\to\mathbb R), define
[ Kf(x)
\int_Y f(y),K(x,dy). ]
This quantity is the expected value of (f) after a transition beginning at (x). The relation
[ \int_Y f(y),(\mu K)(dy)
\int_X Kf(x),\mu(dx) ]
expresses the duality between the action on measures and the action on functions.
When (X=Y), the mapping (f\mapsto Kf) is a positive linear operator that preserves constant functions. Such operators are called Markov operators. Their spectral and ergodic properties encode long-term behavior of the corresponding stochastic process.
A probability measure (\pi) is stationary for (K) when
[ \pi K=\pi. ]
Stationarity means that a transition governed by (K) does not change the distribution (\pi), although individual states may still change.
Conditional distributions
Let (X) and (Y) be random elements defined on a common probability space. A regular conditional probability for (Y) given (X) is a Markov kernel (K) satisfying
[ \Pr(Y\in B\mid X)=K(X,B) ]
almost surely for every measurable (B). The kernel provides a version of the conditional law that is indexed measurably by possible values of (X).
Regular conditional distributions exist under standard structural assumptions, including the case in which the relevant spaces are standard Borel spaces. They need not exist on arbitrary measurable spaces. Even when a regular conditional distribution exists, it is generally determined only outside a set of conditioning values having probability zero.
If the joint law of ((X,Y)) is (\gamma) and the marginal law of (X) is (\mu), a conditional kernel yields the disintegration
[ \gamma(A\times B)
\int_A K(x,B),\mu(dx). ]
Thus a joint distribution can be decomposed into a marginal distribution and a conditional kernel. This factorization is the measure-theoretic analogue of writing a joint density as the product of a marginal density and a conditional density.
Densities and deterministic components
When a kernel is absolutely continuous with respect to a reference measure (\nu) on (Y), it may be represented by a measurable density (k):
[ K(x,B)
\int_B k(x,y),\nu(dy). ]
For every fixed (x), the density satisfies
[ k(x,y)\geq 0 ]
for (\nu)-almost every (y), together with
[ \int_Y k(x,y),\nu(dy)=1. ]
The function (k) is sometimes itself called a transition kernel, although the underlying Markov kernel is the set function (B\mapsto K(x,B)). A kernel need not admit such a density. Deterministic kernels and distributions with singular components provide standard cases where a chosen reference measure does not yield an ordinary density.
A kernel may also combine deterministic and diffuse behavior. For example,
[ K(x,B)
p(x)\delta_{f(x)}(B) + \bigl(1-p(x)\bigr)Q(x,B), ]
where (p:X\to[0,1]) is measurable, (f:X\to Y) is measurable, and (Q) is another Markov kernel. The first term places mass at a specified image point, while the second term supplies the remaining transition law.
Construction of stochastic processes
A sequence of kernels specifies successive conditional laws. Given an initial probability measure (\mu_0) on (X_0) and kernels (K_n) from (X_0\times\cdots\times X_{n-1}) to (X_n), the Ionescu-Tulcea theorem determines a probability measure on the infinite product space. Its finite-dimensional distributions satisfy
[ \begin{aligned} &\Pr(X_0\in dx_0,\ldots,X_n\in dx_n)\ &\qquad = \mu_0(dx_0) K_1(x_0,dx_1) K_2(x_0,x_1,dx_2) \cdots K_n(x_0,\ldots,x_{n-1},dx_n). \end{aligned} ]
For a Markov chain, each kernel depends only on the immediately preceding state. More general dependence on the full recorded history produces a stochastic process without the Markov property, while retaining the same kernel-based construction.