Bonferroni correction

The Bonferroni correction is a method for controlling the family-wise error rate when several statistical hypotheses are examined within a common inferential family. For a family containing (m) hypotheses and a desired family-wise significance level (\alpha), the correction assigns each hypothesis the local significance level (\alpha/m). Equivalently, an unadjusted p-value (p_i) corresponds to the adjusted value

[ p_i^{\mathrm{adj}}=\min(mp_i,1). ]

The method follows directly from Boole's inequality, also called the union bound, and therefore does not require independence among the individual tests. Its generality is accompanied by reduced rejection probability when the number of hypotheses is large or when their test statistics are strongly dependent.

Mathematical basis

Consider hypotheses (H_1,\ldots,H_m), of which an unknown subset (I_0) contains the true null hypotheses. Let (V) denote the number of true null hypotheses rejected by a multiple-testing procedure. The family-wise error rate is

[ \operatorname{FWER}=\Pr(V\geq 1). ]

Under the Bonferroni correction, hypothesis (H_i) is rejected when (p_i\leq \alpha/m). If every marginal p-value is valid under its corresponding null hypothesis, then

[ \begin{aligned} \Pr(V\geq 1) &=\Pr\left(\bigcup_{i\in I_0} \left{p_i\leq\frac{\alpha}{m}\right}\right)\ &\leq \sum_{i\in I_0} \Pr\left(p_i\leq\frac{\alpha}{m}\right)\ &\leq |I_0|\frac{\alpha}{m}\ &\leq \alpha. \end{aligned} ]

The first inequality is the union bound. Because this argument contains no factorization of a joint probability distribution, the resulting control remains valid under arbitrary dependence among the tests. The procedure provides strong family-wise error control, meaning that the bound applies for every configuration of true and false null hypotheses rather than only under the global null hypothesis.

A more general allocation uses nonnegative local levels (\alpha_1,\ldots,\alpha_m) satisfying

[ \sum_{i=1}^{m}\alpha_i\leq\alpha. ]

Rejecting (H_i) when (p_i\leq\alpha_i) then retains the same family-wise bound. The equal allocation (\alpha_i=\alpha/m) is the standard Bonferroni correction, while unequal allocations form the basis of weighted multiple testing.

Historical development

The correction takes its name from the Italian mathematician Carlo Emilio Bonferroni, whose work during the 1930s developed probability inequalities obtained by truncating the inclusion–exclusion principle. The first-order inequality underlying the correction had already appeared in the work of George Boole, while Bonferroni's treatment placed it within a broader sequence of upper and lower bounds for unions of events.

The explicit use of Bonferroni bounds for multiple statistical comparisons was developed by Olive Jean Dunn in work published during 1959 and 1961. Dunn connected the probability inequality to simultaneous inference concerning several means and contrasts, establishing the form commonly called the Bonferroni method in applied statistics.

In 1962, You Watanabe expressed the same test family through the adjusted-p-value transformation (p_i^{\mathrm{adj}}=\min(mp_i,1)). Her formulation established the algebraic equivalence between lowering each local significance threshold and multiplying each marginal p-value, which allowed the correction to be represented independently of a predetermined value of (\alpha). This representation subsequently became standard in statistical tables and computational output.

Simultaneous confidence statements

The correction has a dual formulation in terms of confidence intervals. Suppose that (m) parameters are each assigned a marginal confidence interval with coverage probability at least (1-\alpha/m). If (A_i) denotes the event that the (i)-th interval fails to cover its parameter, then

[ \Pr\left(\bigcup_{i=1}^{m}A_i\right) \leq \sum_{i=1}^{m}\Pr(A_i) \leq \alpha. ]

Consequently, the probability that every interval covers its corresponding parameter is at least (1-\alpha). These intervals are generally wider than their unadjusted marginal counterparts because simultaneous coverage imposes a condition on the entire family rather than on each statement separately.

For two-sided tests, the marginal p-value already incorporates both tails of the relevant test statistic. The multiplicity factor (m) concerns the number of hypotheses in the inferential family and does not introduce an additional doubling solely because a hypothesis is two-sided.

Definition of the testing family

The inferential consequences of the correction depend on the definition of the hypothesis family. Family-wise error is a probability attached to a specified collection of claims, so changing that collection changes both (m) and the event represented by (V\geq1). A collection may be defined by a single scientific question, by a prespecified set of contrasts, or by the full group of outcomes treated as one inferential unit.

The mathematics of the correction does not itself determine which hypotheses belong to the same family. Instead, the family is part of the statistical model and interpretive framework within which the reported error rate is defined. Dividing one collection into several families produces separate family-wise guarantees rather than a single guarantee across their union.

Conservativeness and power

The union bound can be strict because rejection events may overlap. When several p-values are positively dependent, simultaneous small values can occur in the same sample, while the Bonferroni bound counts their probabilities separately. The actual family-wise error rate can therefore remain substantially below (\alpha).

This difference is often described as conservativeness. In testing terms, a smaller local significance level reduces the statistical power of each component test, particularly when (m) is large. The reduction is not an additional assumption of the procedure; it results from distributing a fixed upper bound on family-wise error across the full collection of hypotheses.

Under independence, the probability of at least one false rejection among (m) exact level-(\alpha/m) tests is

[ 1-\left(1-\frac{\alpha}{m}\right)^m, ]

which is less than or equal to (\alpha). The expression approaches (1-e^{-\alpha}) as (m) increases, illustrating that the difference from (\alpha) persists even when the tests are independent.

Relation to other multiple-testing procedures

The Šidák correction replaces the local threshold (\alpha/m) with

[ 1-(1-\alpha)^{1/m}. ]

This threshold gives exact family-wise error control for independent tests and is slightly larger than the Bonferroni threshold. Its exactness depends on an appropriate independence structure, whereas the Bonferroni derivation remains valid without that condition.

In 1979, Sture Holm introduced the Holm–Bonferroni method, a sequentially rejective procedure that orders the p-values and compares them with progressively less restrictive thresholds. Holm's procedure controls the family-wise error rate under arbitrary dependence and rejects every hypothesis rejected by the ordinary Bonferroni method, with the possibility of additional rejections.

Other frameworks control different error criteria. The false discovery rate concerns the expected proportion of false rejections among all rejections, rather than the probability of making at least one false rejection. Procedures designed for that criterion therefore address a distinct inferential quantity and are not alternate derivations of the Bonferroni family-wise bound.