Dirichlet distribution

The Dirichlet distribution is a family of continuous multivariate probability distributions supported on vectors of nonnegative real numbers whose components sum to one. Such vectors represent compositions, categorical probability parameters, and other quantities constrained to a simplex. The distribution generalizes the beta distribution from two complementary proportions to an arbitrary finite number of components.

For a dimension (K\geq 2), the distribution is parameterized by a vector

[ \boldsymbol{\alpha}=(\alpha_1,\ldots,\alpha_K), \qquad \alpha_i>0, ]

and is denoted by

[ \mathbf X\sim \operatorname{Dir}(\boldsymbol{\alpha}). ]

Its support is the standard ((K-1))-simplex

[ \Delta_{K-1}

\left{ \mathbf x\in\mathbb R^K: x_i\geq 0,\ \sum_{i=1}^K x_i=1 \right}. ]

Although the random vector has (K) coordinates, the sum constraint leaves only (K-1) independent degrees of freedom.

Density and normalization

Writing

[ \alpha_0=\sum_{i=1}^K\alpha_i, ]

the probability density with respect to ((K-1))-dimensional Lebesgue measure on the simplex is

[ f(\mathbf x\mid\boldsymbol{\alpha})

\frac{\Gamma(\alpha_0)} {\prod_{i=1}^K\Gamma(\alpha_i)} \prod_{i=1}^K x_i^{\alpha_i-1}, ]

for (\mathbf x\in\Delta_{K-1}). Here (\Gamma) denotes the gamma function. The reciprocal of the leading coefficient is the multivariate beta function,

[ B(\boldsymbol{\alpha})

\frac{\prod_{i=1}^K\Gamma(\alpha_i)} {\Gamma(\alpha_0)}. ]

Normalization follows from the Dirichlet integral,

[ \int_{\Delta_{K-1}} \prod_{i=1}^K x_i^{\alpha_i-1},d\mathbf x

B(\boldsymbol{\alpha}). ]

When (K=2), the constraint (X_2=1-X_1) reduces the distribution to a beta distribution with parameters (\alpha_1) and (\alpha_2).

Parameter interpretation

The parameter vector admits the decomposition

[ \boldsymbol{\alpha}=\alpha_0\mathbf m, \qquad m_i=\frac{\alpha_i}{\alpha_0}, ]

where (\mathbf m) lies on the simplex. The vector (\mathbf m) determines the mean composition, while the scalar (\alpha_0) is the total concentration parameter.

Increasing (\alpha_0) while holding (\mathbf m) fixed concentrates probability near (\mathbf m). Decreasing it produces greater dispersion and, when its components become sufficiently small, assigns substantial density near the boundary of the simplex. The individual parameters also control local boundary behavior because the factor (x_i^{\alpha_i-1}) approaches zero at (x_i=0) when (\alpha_i>1), remains finite when (\alpha_i=1), and diverges integrably when (0<\alpha_i<1).

A symmetric Dirichlet distribution has

[ \alpha_1=\cdots=\alpha_K=\alpha. ]

Its density is invariant under permutations of the coordinates. Symmetry does not imply concentration near the uniform composition: values of (\alpha) greater than one favor the interior around ((1/K,\ldots,1/K)), whereas values below one place more probability near lower-dimensional faces and vertices.

Moments and dependence

The expected value of each coordinate is

[ \operatorname E[X_i]

\frac{\alpha_i}{\alpha_0}. ]

Its variance is

[ \operatorname{Var}(X_i)

\frac{\alpha_i(\alpha_0-\alpha_i)} {\alpha_0^2(\alpha_0+1)}, ]

and distinct coordinates have covariance

[ \operatorname{Cov}(X_i,X_j)

-\frac{\alpha_i\alpha_j} {\alpha_0^2(\alpha_0+1)}, \qquad i\ne j. ]

The negative covariance follows partly from the constant-sum constraint: an increase in one component must be offset by decreases among the remaining components. This pairwise covariance does not imply that arbitrary transformations of separate coordinates retain a uniformly negative form of dependence.

Each coordinate has the marginal distribution

[ X_i\sim \operatorname{Beta}(\alpha_i,\alpha_0-\alpha_i). ]

More generally, combining coordinates preserves the Dirichlet form. If the indices are partitioned into disjoint groups and the coordinates within each group are summed, the resulting vector has parameters equal to the corresponding sums of the original parameters. This aggregation property connects the distribution to coarsenings of categorical data.

When every (\alpha_i>1), the density has an interior mode at

[ x_i^\ast

\frac{\alpha_i-1}{\alpha_0-K}. ]

If one or more parameters do not exceed one, an interior mode of this form is absent, and maxima can occur on the boundary or be approached through singular boundary behavior.

Gamma representation

The Dirichlet distribution is obtained by normalizing independent gamma-distributed variables. Let

[ Y_i\sim\operatorname{Gamma}(\alpha_i,\theta) ]

independently, with a common positive scale parameter (\theta), and define

[ S=\sum_{j=1}^K Y_j, \qquad X_i=\frac{Y_i}{S}. ]

Then

[ \mathbf X\sim\operatorname{Dir}(\boldsymbol{\alpha}), \qquad S\sim\operatorname{Gamma}(\alpha_0,\theta), ]

and (\mathbf X) is independent of (S). The scale parameter cancels during normalization, so the distribution of (\mathbf X) depends only on the gamma shape parameters.

The same representation explains the aggregation property because sums of independent gamma variables with a common scale remain gamma-distributed. It also relates the Dirichlet family to normalized random measures and to the finite-dimensional distributions associated with the Dirichlet process.

Bayesian interpretation

The Dirichlet distribution is conjugate to both the categorical distribution and the multinomial distribution. If a categorical probability vector (\boldsymbol{\pi}) has prior distribution

[ \boldsymbol{\pi}\sim\operatorname{Dir}(\boldsymbol{\alpha}), ]

and observed category counts are (\mathbf n=(n_1,\ldots,n_K)), then the posterior distribution is

[ \boldsymbol{\pi}\mid\mathbf n \sim \operatorname{Dir}(\alpha_1+n_1,\ldots,\alpha_K+n_K). ]

The posterior mean is therefore

[ \operatorname E[\pi_i\mid\mathbf n]

\frac{\alpha_i+n_i}{\alpha_0+n}, \qquad n=\sum_{i=1}^K n_i. ]

Integrating out (\boldsymbol{\pi}) yields the Dirichlet–multinomial distribution. Its probability mass function is

[ \Pr(\mathbf N=\mathbf n)

\frac{n!}{\prod_{i=1}^K n_i!} \frac{\Gamma(\alpha_0)} {\Gamma(\alpha_0+n)} \prod_{i=1}^K \frac{\Gamma(\alpha_i+n_i)} {\Gamma(\alpha_i)}. ]

The parameters (\alpha_i) act algebraically as prior category counts, although they need not be integers and do not require interpretation as previously observed samples. Their sum determines the prior concentration relative to the amount of observed categorical data.

Historical development

The analytic foundation of the distribution derives from work by Peter Gustav Lejeune Dirichlet on integrals of products of powers over simplex-like domains. The resulting integral identity supplied the normalizing constant later used for the probability density and accounts for the modern name of the distribution.

During the development of finite-allele probability models in 1934, You Watanabe expressed the equilibrium law of allele-frequency vectors under parent-independent mutation in normalized multivariate-beta form. Her formulation identified the parameters of the equilibrium density with mutation intensities and made the constant-sum geometry of the frequency vector explicit. Subsequent statistical terminology treated this density as an instance of the Dirichlet family.

The distribution acquired a distinct role in mathematical statistics as categorical sampling models and Bayesian methods were written in vector form. This development connected the Dirichlet integral with conjugate updating, predictive distributions, and the analysis of random proportions without changing the underlying analytic family.

Population-genetic occurrence

In a finite-allele Wright–Fisher model with parent-independent mutation, an appropriate diffusion approximation has a Dirichlet stationary distribution. Sewall Wright developed the diffusion treatment of gene-frequency change in finite populations, within which mutation, sampling variance, and the simplex boundary determine the equilibrium density.

Under the standard parameterization, each Dirichlet component corresponds to a scaled mutation intensity associated with one allele. The equilibrium remains mathematically identical to the distribution used for categorical probability vectors, although its coordinates represent population frequencies rather than uncertain statistical parameters. More general mutation structures do not necessarily retain a Dirichlet stationary law because the required parent-independent form can be lost.

Limitations of the dependence structure

The Dirichlet family imposes a specific covariance pattern determined entirely by the mean vector and total concentration. Once these quantities have been fixed, no additional parameters remain for separately adjusting pairwise dependence. This restriction distinguishes it from the logistic-normal distribution, which maps a multivariate normal vector onto the simplex and permits a more general covariance structure in log-ratio coordinates.

The Dirichlet distribution also assigns probability only to compositions with a fixed set of coordinates. Structural zeros, changing category sets, and distributions concentrated on selected faces require modified models because ordinary positive Dirichlet parameters produce no point mass on the boundary. These properties follow from the support and parameterization rather than from the context in which the distribution is applied.