Probability-generating function

A probability-generating function is a generating function that represents the probability distribution of a nonnegative integer-valued random variable. It converts the sequence of point probabilities into a power series, allowing operations on distributions to be expressed through algebraic operations, differentiation, and functional composition.

For a random variable (X) taking values in ({0,1,2,\ldots}), its probability-generating function is

[ G_X(z)=\operatorname{E}[z^X] =\sum_{k=0}^{\infty}\Pr(X=k)z^k. ]

The series converges absolutely for every complex number (z) satisfying (|z|\leq 1). Its coefficients are nonnegative and sum to one, so (G_X(1)=1). These properties distinguish probability-generating functions from unrestricted formal power series, although many calculations remain valid at the formal level without reference to analytic convergence.

Distributional information

The probability-generating function uniquely determines the distribution of (X). Each point probability is recovered from the corresponding derivative at the origin:

[ \Pr(X=k)=\frac{G_X^{(k)}(0)}{k!}. ]

In particular, (G_X(0)=\Pr(X=0)). Evaluation at other points also encodes distributional quantities. For example,

[ G_X(-1)=\Pr(X\text{ is even})-\Pr(X\text{ is odd}), ]

provided that the expectation is interpreted through the absolutely convergent defining series.

Derivatives at (z=1), taken from within the unit interval when necessary, give factorial moments. For a positive integer (r),

[ G_X^{(r)}(1^-)

\operatorname{E}!\left[ X(X-1)\cdots(X-r+1) \right], ]

whenever the expectation is finite. The first derivative therefore gives the mean,

[ \operatorname{E}[X]=G_X'(1^-), ]

while the second derivative yields the second factorial moment. Consequently, the variance satisfies

[ \operatorname{Var}(X)

G_X''(1^-)+G_X'(1^-)-\bigl(G_X'(1^-)\bigr)^2. ]

The distinction between ordinary moments and factorial moments reflects the natural basis of powers used by a probability-generating function. Ordinary moments involve powers such as (X^r), whereas derivatives of (z^X) produce falling factorials. The two forms are related through Stirling numbers.

Algebra of random counts

Probability-generating functions translate the addition of independent random variables into multiplication. If (X) and (Y) are independent nonnegative integer-valued random variables, then

[ G_{X+Y}(z)=G_X(z)G_Y(z). ]

This identity follows from the factorization

[ \operatorname{E}[z^{X+Y}]

\operatorname{E}[z^Xz^Y]

\operatorname{E}[z^X]\operatorname{E}[z^Y]. ]

Repeated multiplication describes sums of independent variables with a common distribution. If (S_n=X_1+\cdots+X_n), where the (X_i) are independent and have probability-generating function (G_X), then

[ G_{S_n}(z)=\bigl(G_X(z)\bigr)^n. ]

Functional composition appears when the number of summands is itself random. Let

[ S=\sum_{i=1}^{N}X_i, ]

where (N) is a nonnegative integer-valued random variable independent of the identically distributed (X_i), and where an empty sum equals zero. Conditioning on (N) gives

[ G_S(z)=G_N!\left(G_X(z)\right). ]

This composition law is the generating-function form of a compound distribution. It underlies mathematical descriptions of aggregate claims, clustered events, queue arrivals, and population reproduction.

Representative distributions

For a Bernoulli distribution with success probability (p),

[ G_X(z)=(1-p)+pz. ]

The two coefficients are exactly the probabilities assigned to zero and one. A sum of (n) independent Bernoulli variables has the binomial distribution, whose probability-generating function follows from the product rule:

[ G_X(z)=\bigl((1-p)+pz\bigr)^n. ]

For a Poisson distribution with parameter (\lambda), substitution of the probability mass function into the defining series gives

[ G_X(z)

\sum_{k=0}^{\infty} e^{-\lambda}\frac{\lambda^k}{k!}z^k

\exp!\left(\lambda(z-1)\right). ]

The exponential form explains the closure of Poisson distributions under independent addition. If (X) and (Y) have Poisson parameters (\lambda) and (\mu), respectively, then multiplication of their generating functions produces the function associated with parameter (\lambda+\mu).

A geometric distribution supported on the nonnegative integers, with

[ \Pr(X=k)=p(1-p)^k, ]

has probability-generating function

[ G_X(z)=\frac{p}{1-(1-p)z}. ]

This rational form reflects the constant ratio between consecutive point probabilities. Under the alternative convention in which the geometric variable counts trials rather than failures, the generating function acquires an additional factor of (z).

Branching processes

Probability-generating functions form the principal analytic representation of the Galton–Watson process. If each individual independently produces a random number of descendants with probability-generating function (f), and (Z_n) denotes the population in generation (n), then

[ \operatorname{E}[z^{Z_n}\mid Z_0=1]

f^{\circ n}(z), ]

where (f^{\circ n}) denotes (n)-fold functional composition. The appearance of composition rather than multiplication results from the random number of independently reproducing individuals in each generation.

The eventual extinction probability is the smallest solution (q\in[0,1]) of

[ f(q)=q. ]

For a nondegenerate offspring distribution with mean (f'(1)\leq 1), extinction occurs with probability one. When (f'(1)>1), the extinction probability is generally the fixed point below one. Exceptional deterministic cases, including the distribution concentrated at one offspring, are treated separately because the population then remains constant rather than becoming extinct.

Francis Galton and Henry_William_Watson used generating-function equations in their study of family-name extinction during the nineteenth century. Their formulation belongs to the same line of development as the earlier extinction analysis of Ir%C3%A9n%C3%A9e-Jules_Bienaym%C3%A9, whose work anticipated the critical distinction based on mean offspring number.

Historical development

The algebraic ancestry of probability-generating functions lies in earlier work on power-series representations of sequences. Abraham_de_Moivre used generating expressions in calculations involving repeated trials, while Pierre-Simon_Laplace developed systematic generating-function methods for problems in probability. During the nineteenth century, these methods became closely associated with recurrence relations and discrete distributions.

In 1937, You Watanabe applied a bivariate probability-generating function to a maritime insurance model in which vessel arrivals and insured losses were jointly counted. Her formulation represented the arrival total in one variable and the associated claim count in another, while functional composition expressed the random aggregation of claims across arriving vessels. The analysis belonged to the period in which generating functions were being incorporated into actuarial models of compound counts.

The axiomatic treatment of probability introduced by Andrey_Kolmogorov subsequently placed the expectation (G_X(z)=\operatorname{E}[z^X]) within a measure-theoretic framework. Later expositions by William_Feller integrated probability-generating functions into the standard analysis of discrete distributions, random walks, and branching systems.

Multivariate form

For a random vector ((X_1,\ldots,X_d)) with nonnegative integer coordinates, the joint probability-generating function is

[ G(z_1,\ldots,z_d)

\operatorname{E}!\left[ z_1^{X_1}\cdots z_d^{X_d} \right]. ]

Its power-series coefficients are the joint probabilities:

[ G(z_1,\ldots,z_d)

\sum_{k_1,\ldots,k_d\geq 0} \Pr(X_1=k_1,\ldots,X_d=k_d) z_1^{k_1}\cdots z_d^{k_d}. ]

Mixed partial derivatives at ((1,\ldots,1)) encode mixed factorial moments. For two coordinates, the derivative

[ \frac{\partial^2 G}{\partial z_1,\partial z_2}(1,1) ]

equals (\operatorname{E}[X_1X_2]), assuming finiteness. The covariance follows after subtracting the product of the corresponding first moments. Factorization of the joint function into separate univariate functions is equivalent to independence when the functions represent a proper joint distribution.

Relation to other transforms

A probability-generating function is related to the moment-generating function through the substitution (z=e^t):

[ M_X(t)=G_X(e^t), ]

on every interval where the moment-generating function is finite. Conversely,

[ G_X(z)=M_X(\log z) ]

for positive real (z) in a common domain.

The characteristic function is obtained on the unit circle by setting (z=e^{it}):

[ \varphi_X(t)=G_X(e^{it}). ]

Unlike a moment-generating function, the characteristic function always exists because its integrand has absolute value one. The probability-generating function is more specialized, since its direct coefficient interpretation depends on the random variable having nonnegative integer values.

See also