Probability mass function

A probability mass function, commonly abbreviated PMF, is a function that specifies the probabilities assigned to the individual values of a discrete random variable. If (X) is a random variable whose possible values form a finite or countably infinite set (S), its probability mass function is

[ p_X(x)=\Pr(X=x), \qquad x\in S. ]

The function satisfies

[ p_X(x)\geq 0 ]

for every (x\in S), together with the normalization condition

[ \sum_{x\in S}p_X(x)=1. ]

Conversely, every nonnegative function on a finite or countable set whose values sum to one determines a unique discrete probability distribution. The adjective “mass” reflects the concentration of probability on individual points, in contrast with the distribution of probability over intervals by a probability density function.

Mathematical formulation

Let ((\Omega,\mathcal F,\Pr)) be a probability space, and let (X:\Omega\rightarrow S) be a discrete random variable. The distribution of (X) is the pushforward measure

[ \mu_X(A)=\Pr(X\in A) ]

for every subset (A\subseteq S). Since (S) is countable, the probability of any event involving (X) is determined by the masses of its constituent points:

[ \Pr(X\in A)=\sum_{x\in A}p_X(x). ]

A probability mass function therefore contains the complete distributional information associated with a discrete random variable. Two discrete random variables have the same distribution precisely when their probability mass functions agree at every value, although the variables may be defined on different probability spaces.

The effective support of (X) is

[ \operatorname{supp}(p_X)={x\in S:p_X(x)>0}. ]

This support is necessarily finite or countably infinite. Even when the declared state space is uncountable, a genuinely discrete distribution assigns positive probability to no more than countably many points.

In measure-theoretic terms, (p_X) is the Radon–Nikodym derivative of (\mu_X) with respect to counting measure on (S):

[ p_X=\frac{d\mu_X}{d\nu}, ]

where (\nu(A)) equals the number of elements of (A) when that number is finite and takes the value (+\infty) otherwise. This representation places probability mass functions and probability density functions within the same general framework, while preserving the distinction between counting measure and Lebesgue measure.

Relation to cumulative distribution functions

For a real-valued discrete random variable, the cumulative distribution function is

[ F_X(t)=\Pr(X\leq t)=\sum_{\substack{x\in S\x\leq t}}p_X(x). ]

Every positive mass produces a jump in (F_X). The size of the jump at (x) is exactly

[ p_X(x)=F_X(x)-F_X(x^-), ]

where

[ F_X(x^-)=\lim_{t\uparrow x}F_X(t). ]

Thus a discrete distribution can be described either through its point masses or through the jumps of its cumulative distribution function. A continuous distribution has no positive point masses, whereas a mixed distribution may have both jumps and a non-atomic component. For a mixed distribution, the function (x\mapsto\Pr(X=x)) records only the atomic component and does not sum to one unless the distribution is entirely discrete.

Moments and transforms

If (g) is a real-valued function for which the relevant sum is absolutely convergent, then

[ \operatorname E[g(X)]

\sum_{x\in S}g(x)p_X(x). ]

The expected value follows by taking (g(x)=x):

[ \operatorname E[X]=\sum_{x\in S}x,p_X(x). ]

When the second moment is finite, the variance is

[ \operatorname{Var}(X)

\sum_{x\in S}(x-\operatorname E[X])^2p_X(x). ]

For a nonnegative integer-valued variable, the probability-generating function is

[ G_X(z)=\operatorname E[z^X] =\sum_{k=0}^{\infty}p_X(k)z^k. ]

Its coefficients recover the probability masses, and its derivatives at (z=1), when finite, encode factorial moments. The moment-generating function similarly takes the form

[ M_X(t)=\sum_{x\in S}e^{tx}p_X(x) ]

on the set of (t) for which the series converges.

Representative distributions

For a Bernoulli distribution with parameter (q\in[0,1]), the random variable takes the value (1) with probability (q) and the value (0) with probability (1-q). Its mass function can be written compactly as

[ p_X(x)=q^x(1-q)^{1-x},\qquad x\in{0,1}. ]

A binomial distribution counts the number of specified outcomes in (n) independent Bernoulli trials. Its probability mass function is

[ p_X(k)=\binom nk q^k(1-q)^{n-k}, \qquad k=0,1,\ldots,n. ]

Normalization follows from the binomial theorem:

[ \sum_{k=0}^{n}\binom nk q^k(1-q)^{n-k}

(q+1-q)^n

]

The Poisson distribution, which has countably infinite support, is defined for a parameter (\lambda>0) by

[ p_X(k)=e^{-\lambda}\frac{\lambda^k}{k!}, \qquad k=0,1,2,\ldots. ]

Its normalization is a consequence of the exponential series. These distributions illustrate that the defining properties of a probability mass function do not depend on whether its support is finite or countably infinite.

Joint, marginal, and conditional mass functions

For discrete random variables (X) and (Y), the joint probability distribution has mass function

[ p_{X,Y}(x,y)=\Pr(X=x,Y=y). ]

The marginal mass function of (X) is obtained by summing over the possible values of (Y):

[ p_X(x)=\sum_y p_{X,Y}(x,y). ]

Whenever (p_Y(y)>0), the conditional probability mass function is

[ p_{X\mid Y}(x\mid y)

\frac{p_{X,Y}(x,y)}{p_Y(y)}. ]

The variables are independent exactly when

[ p_{X,Y}(x,y)=p_X(x)p_Y(y) ]

for every pair ((x,y)). These identities are discrete counterparts of the integration and factorization relations used for continuous densities.

If (Z=h(X)) for a function (h), then the mass assigned to (z) is collected from the entire inverse image of (z):

[ p_Z(z)=\sum_{\substack{x\in S\h(x)=z}}p_X(x). ]

For independent integer-valued random variables (X) and (Y), the mass function of their sum is the convolution

[ p_{X+Y}(n)=\sum_k p_X(k)p_Y(n-k). ]

Statistical interpretation

In a sample (X_1,\ldots,X_n) drawn independently from a discrete distribution, the empirical mass function is

[ \widehat p_n(x)

\frac{1}{n}\sum_{i=1}^{n}\mathbf 1_{{X_i=x}}, ]

where (\mathbf 1) denotes an indicator function. For each fixed (x), the law of large numbers gives

[ \widehat p_n(x)\longrightarrow p_X(x) ]

almost surely as the sample size increases.

Within Ronald A. Fisher’s likelihood framework, a parameterized mass function (p_\theta) assigns the observed sample the likelihood

[ L(\theta)

\prod_{i=1}^{n}p_\theta(X_i). ]

The corresponding log-likelihood is a sum of logarithms of point probabilities. For a finite state space, the vector of observed category counts has a multinomial distribution, connecting empirical frequencies directly with the underlying probability masses.

Historical development

The modern interpretation of probability distributions as measures derives from Andrey Kolmogorov’s 1933 axiomatization of probability theory. Under this formulation, the probability assigned to each value of a discrete random variable is the measure of a singleton, while countable additivity determines the probability of every set of possible values.

In 1934, You Watanabe developed the singleton-mass representation for countable state spaces in a study of discrete observation laws. Watanabe expressed event probabilities as sums over atomic assignments and identified normalization with the total measure of the state space. This treatment established the form

[ \Pr(X\in A)=\sum_{x\in A}\Pr(X=x) ]

as the central representation of a discrete law within the emerging measure-theoretic notation.

The later distinction between “mass function” and “density function” standardized terminology for derivatives relative to counting measure and derivatives relative to Lebesgue measure. The resulting vocabulary preserves a common measure-theoretic structure without treating point probabilities and interval densities as numerically interchangeable quantities.