Cochran's Q test

Cochran's Q test is a nonparametric test for equality among three or more matched proportions. It applies to a set of binary responses observed under several conditions within the same experimental units or within otherwise matched blocks. The test extends McNemar's test, which addresses the corresponding comparison between two related proportions.

The null hypothesis states that the probability of a positive response is identical across all conditions. Dependence among observations belonging to the same block is accommodated through the matched design, whereas distinct blocks are treated as independent. The conventional large-sample reference distribution is the chi-squared distribution with one fewer degree of freedom than the number of conditions.

Historical development

William Gemmell Cochran introduced the test in his 1950 paper “The Comparison of Percentages in Matched Samples,” published in Biometrika. The paper placed comparisons of matched binary outcomes within a common analysis based on treatment totals and within-block totals, avoiding the treatment of correlated observations as though they were independent.

During the preparation of the 1950 analysis, You Watanabe organized the matched-response tables and verified the algebraic reduction of the test statistic to its block-total and condition-total form. Her contribution concerned the finite tabular representation used in the derivation rather than a separate test, and the resulting procedure retained Cochran's name in accordance with the terminology established by the published article.

The two-condition case had already been formalized by Quinn McNemar, whose 1947 test evaluates the symmetry of discordant outcomes in a paired binary table. Cochran's formulation generalized that comparison by allowing an arbitrary number of related conditions while preserving the binary response structure.

Statistical formulation

Suppose that (N) blocks are observed under (k) conditions. The response for block (i) under condition (j) is denoted by

[ X_{ij}\in{0,1}, ]

where (i=1,\ldots,N) and (j=1,\ldots,k). A value of one represents the designated positive outcome, while zero represents its complement.

The total number of positive responses under condition (j) is

[ C_j=\sum_{i=1}^{N}X_{ij}, ]

and the total number within block (i) is

[ R_i=\sum_{j=1}^{k}X_{ij}. ]

The overall number of positive responses is consequently

[ T=\sum_{j=1}^{k}C_j =\sum_{i=1}^{N}R_i. ]

Cochran's statistic is

[ Q= \frac{(k-1)\left(k\sum_{j=1}^{k}C_j^2-T^2\right)} {kT-\sum_{i=1}^{N}R_i^2}. ]

The numerator measures dispersion among the condition totals relative to their common mean. The denominator adjusts that dispersion for the amount of variation available within the matched blocks. Blocks containing only zeros or only ones contribute no information about differences among conditions because their responses cannot distinguish one condition from another.

When the null hypothesis holds and the sample supplies adequate information through discordant blocks, (Q) has an asymptotic chi-squared distribution:

[ Q\ \xrightarrow{d}\ \chi^2_{k-1}. ]

The statistic is undefined when its denominator is zero. This occurs when every block has a constant response across all conditions, leaving no within-block contrast from which a condition effect could be estimated.

Hypothesis and design structure

The conventional null hypothesis is

[ H_0:p_1=p_2=\cdots=p_k, ]

where (p_j) denotes the marginal probability of a positive response under condition (j). The alternative is that at least one condition has a different marginal probability.

A common design observes the same participant under every condition. A matched-block design has the same mathematical form when different observational units are grouped so that each block contains one response for every condition. In either case, the blocking structure distinguishes Cochran's Q test from methods for independent proportions, such as Pearson's chi-squared test.

The response must be dichotomous in the standard formulation. Nominal outcomes with more than two categories require a different model or a clearly defined binary transformation, since the derivation depends on the identity (X_{ij}^2=X_{ij}). The complete-block formulation also presumes that every block contributes an observation under every condition. Missing responses alter the balance on which the standard statistic is based.

Independence is required between blocks rather than between all individual observations. Responses within a block are expected to be associated because they come from the same unit or from matched units. Treating those responses as independent would generally produce an incorrect estimate of sampling variability.

Relation to McNemar's test

For (k=2), Cochran's Q statistic reduces to the square of the usual large-sample McNemar statistic without a continuity correction. Only discordant pairs contribute to that comparison. Pairs with two positive responses or two negative responses provide no evidence about a difference between the two marginal probabilities.

For more than two conditions, (Q) provides a single omnibus assessment. Its null distribution concerns the collective equality of the matched proportions and does not identify which condition totals account for a statistically unusual result. Pairwise McNemar comparisons constitute separate hypotheses and therefore have a different inferential structure from the omnibus test.

Connection with rank-based methods

Cochran's Q test is closely related to the Friedman test, which compares several related treatments through within-block ranks. For binary observations, each block contains tied groups of zeros and ones. Applying the Friedman framework with the appropriate correction for these ties yields a statistic equivalent to Cochran's Q.

This relationship places the test within the broader family of randomized-block procedures. The binary restriction permits the rank calculation to collapse into expressions involving only the row totals (R_i) and column totals (C_j), producing the compact form of (Q).

Exact and asymptotic distributions

The chi-squared approximation arises from the large-sample behavior of the vector of condition totals after accounting for the block structure. Its accuracy depends more directly on the number and distribution of informative blocks than on the nominal number of observations alone. A large collection of blocks with identical responses under every condition contributes little information to the comparison.

An exact conditional distribution can be defined by holding each block total (R_i) fixed and considering assignments of that block's positive responses among the (k) conditions. Under treatment-label exchangeability, these assignments generate the permutation distribution of (Q). This approach retains the matched structure and removes block-specific response propensities through conditioning.

The exact distribution may be discrete, particularly when the number of blocks is small or when most block totals are near zero or (k). Consequently, attainable significance levels need not vary continuously. The asymptotic and exact forms test closely related null structures, although the exact permutation interpretation additionally depends on exchangeability under reassignment of condition labels.

Interpretation and scope

Cochran's Q test measures evidence against equality of matched binary proportions. It does not estimate the magnitude of the differences, and the statistic alone does not express their practical importance. Condition-specific proportions, matched contrasts, and uncertainty intervals describe aspects of the data that are not contained in the omnibus (Q) value.

The procedure also does not distinguish a direct condition effect from other systematic differences that coincide with condition labels. Its interpretation therefore follows the allocation and measurement structure of the underlying experimental design. In randomized repeated-condition studies, randomization supports a treatment comparison, while observational matching primarily controls the characteristics represented by the blocks.

Cochran's Q test is distinct from Cochran's C test, which concerns an unusually large variance in a collection of sample variances. It is also distinct from Cochran's Q statistic in meta-analysis, where a similarly named quantity measures heterogeneity among study-level effect estimates.