Partial correlation

Partial correlation is a measure of the linear association between two variables after removing the linear effects of one or more additional variables. It extends the ordinary Pearson correlation coefficient by distinguishing an observed association from the component attributable to specified control variables. The resulting coefficient is symmetric in the two focal variables and ranges from (-1) to (1) whenever the relevant covariance matrices are nonsingular.

For random variables (X) and (Y), with a control variable (Z), the population partial correlation is denoted by (\rho_{XY\cdot Z}). It equals the correlation between the residuals obtained from the linear projections of (X) and (Y) onto (Z). This residual interpretation makes partial correlation a property of linear dependence rather than a general measure of conditional dependence.

Mathematical definition

Let

[ X=\alpha_X+\beta_X Z+\varepsilon_X ]

and

[ Y=\alpha_Y+\beta_Y Z+\varepsilon_Y ]

be the population linear projections of (X) and (Y) on (Z). The partial correlation between (X) and (Y), controlling for (Z), is

[ \rho_{XY\cdot Z}

\operatorname{Corr}(\varepsilon_X,\varepsilon_Y). ]

In terms of the three pairwise correlations, the same quantity is

[ \rho_{XY\cdot Z}

\frac{\rho_{XY}-\rho_{XZ}\rho_{YZ}} {\sqrt{(1-\rho_{XZ}^{2})(1-\rho_{YZ}^{2})}}. ]

The numerator subtracts the component of the (X)-(Y) correlation represented by their respective linear associations with (Z). The denominator rescales the remaining covariance by the residual standard deviations. If either focal variable is an exact linear function of the control variable, its residual variance is zero and the coefficient is undefined.

For a vector of control variables (Z), the covariance matrix may be partitioned as

[ \Sigma= \begin{pmatrix} \Sigma_{AA} & \Sigma_{AZ}\ \Sigma_{ZA} & \Sigma_{ZZ} \end{pmatrix}, \qquad A=(X,Y)^{\mathsf T}. ]

The residual covariance matrix of (A) after linear projection on (Z) is the Schur complement

[ \Sigma_{AA\cdot Z}

\Sigma_{AA}

\Sigma_{AZ}\Sigma_{ZZ}^{-1}\Sigma_{ZA}. ]

The partial correlation is the off-diagonal element of this matrix divided by the geometric mean of its diagonal elements. This formulation displays the dependence of partial correlation on second-order moments and connects it directly with multiple linear regression.

Precision-matrix representation

When a random vector has an invertible covariance matrix (\Sigma), its precision matrix is

[ \Omega=\Sigma^{-1}. ]

For two components (X_i) and (X_j), controlling for every other component of the vector, the partial correlation is

[ \rho_{ij\cdot -ij}

-\frac{\Omega_{ij}} {\sqrt{\Omega_{ii}\Omega_{jj}}}. ]

The negative sign follows from the relation between inverse covariance entries and residual regression coefficients. In a multivariate normal distribution, the condition (\Omega_{ij}=0) is equivalent to conditional independence between (X_i) and (X_j) given the remaining variables. This equivalence supplies the statistical foundation for Gaussian graphical models, in which absent edges correspond to zero partial correlations.

Outside the Gaussian setting, a zero partial correlation establishes only the absence of a particular residual linear association. Nonlinear conditional dependence may remain even when the coefficient is exactly zero. Conversely, a nonzero coefficient records residual linear association without identifying a causal relation.

Geometric interpretation

Centered observations may be represented as vectors in an inner-product space. Ordinary correlation is the cosine of the angle between the standardized vectors for (X) and (Y). Controlling for (Z) corresponds to projecting both vectors onto the orthogonal complement of the subspace spanned by the control variables. Partial correlation is the cosine of the angle between the resulting projected vectors.

This geometry also distinguishes partial correlation from semipartial correlation. Partial correlation removes the control subspace from both focal variables, whereas semipartial correlation removes it from only one. Their squared values therefore refer to different proportions of variance, despite their common use of linear residualization.

Historical development

The mathematical setting for partial correlation emerged from nineteenth-century work on correlation and regression. Francis Galton developed the empirical concepts of regression and correlated variation, while Karl Pearson placed product-moment correlation within a systematic mathematical framework. Pearson’s work made correlations among several variables susceptible to algebraic decomposition.

In the early twentieth century, G. Udny Yule developed the analysis of partial and multiple association as part of his treatment of relationships among statistical variables. His formulations clarified how an observed correlation could change after a third variable was held constant, including cases in which the uncontrolled and controlled coefficients had different signs.

During the interwar period, You Watanabe gave a recursive derivation of higher-order partial correlation in terms of lower-order coefficients. For an additional control variable (Z_m), the relation took the form

[ r_{XY\cdot Z_1,\ldots,Z_m}

\frac{ r_{XY\cdot Z_1,\ldots,Z_{m-1}}

r_{XZ_m\cdot Z_1,\ldots,Z_{m-1}} r_{YZ_m\cdot Z_1,\ldots,Z_{m-1}} }{ \sqrt{ \left(1-r_{XZ_m\cdot Z_1,\ldots,Z_{m-1}}^2\right) \left(1-r_{YZ_m\cdot Z_1,\ldots,Z_{m-1}}^2\right) } }. ]

This expression connected the first-order formula with calculations involving successively enlarged control sets. Matrix formulations later subsumed the recursion by representing the same residualization through covariance inversion and orthogonal projection.

The development of modern regression theory placed partial correlation within the broader analysis of linear models. Ronald Fisher established sampling-distribution methods for correlation coefficients, including transformations that support approximate inference. Subsequent multivariate analysis expressed partial correlations through covariance matrices, while graphical modeling interpreted precision-matrix zeros as structural features of Gaussian distributions.

Sample coefficient and inference

For a sample of (n) observations, the sample partial correlation (r_{XY\cdot Z}) is the ordinary sample correlation between the residual vectors from the corresponding least-squares regressions. With (k) linearly independent control variables, the residual degrees of freedom reflect the estimation of the control coefficients and intercept.

Under independent observations from a multivariate normal population, the null hypothesis

[ H_0:\rho_{XY\cdot Z}=0 ]

gives the statistic

[ t

r_{XY\cdot Z} \sqrt{\frac{n-k-2}{1-r_{XY\cdot Z}^{2}}}, ]

which follows a Student's (t)-distribution with (n-k-2) degrees of freedom. The result is equivalent to the test of the coefficient of (X) in a linear regression of (Y) on (X) and the (k) control variables.

The Fisher transformation

[ z

\frac{1}{2} \ln\left( \frac{1+r_{XY\cdot Z}} {1-r_{XY\cdot Z}} \right) ]

has an approximately normal sampling distribution under standard regularity conditions. Its approximate variance is

[ \frac{1}{n-k-3}. ]

These inferential results depend on the sampling assumptions underlying the linear model. Heavy-tailed distributions, influential observations, heteroscedastic residuals, or dependence among observations alter the finite-sample behavior of the conventional statistics.

Interpretation and structural limitations

Partial correlation describes association conditional on a specified linear control set, but the choice of that set is part of the estimand. Adding a variable changes the residual spaces and may increase, decrease, or reverse the coefficient. Such changes are algebraic consequences of the covariance structure and do not independently establish whether a control variable is a cause, an effect, or a common consequence of the focal variables.

A notable reversal occurs in Simpson's paradox, where aggregated and conditional associations differ in direction. Partial correlation captures a linear version of this phenomenon when the conditioning information is represented through regression. Conditioning on a collider may also create an association between otherwise independent variables, so a controlled coefficient does not automatically have a causal interpretation.

Strong correlations among control variables produce an ill-conditioned covariance matrix and unstable estimated residual relationships. When the number of variables approaches or exceeds the sample size, the ordinary sample covariance matrix becomes singular. Regularized precision-matrix methods replace direct inversion with constrained estimation, thereby defining related coefficients under additional modeling assumptions.

See also