Pearson correlation coefficient

The Pearson correlation coefficient is a dimensionless measure of the linear association between two numerical variables. It is defined as the standardized covariance of the variables and is conventionally denoted by (\rho) for a population and (r) for a sample. When both variables have finite, nonzero variance, the coefficient lies in the closed interval from (-1) to (1).

A positive coefficient indicates that observations above the mean of one variable tend to accompany observations above the mean of the other. A negative coefficient indicates that observations above one mean tend to accompany observations below the other. A coefficient of zero represents the absence of linear association, but it does not generally imply statistical independence.

Definition

For random variables (X) and (Y) having finite and positive variances, the population Pearson correlation coefficient is

[ \rho_{X,Y}

\operatorname{Corr}(X,Y)

\frac{\operatorname{Cov}(X,Y)} {\sigma_X\sigma_Y}, ]

where (\operatorname{Cov}(X,Y)) is the population covariance and (\sigma_X) and (\sigma_Y) are the corresponding standard deviations. In terms of expected values, the same quantity is

[ \rho_{X,Y}

\frac{\operatorname{E}[(X-\mu_X)(Y-\mu_Y)]} {\sqrt{\operatorname{E}[(X-\mu_X)^2]\operatorname{E}[(Y-\mu_Y)^2]}}, ]

where (\mu_X=\operatorname{E}[X]) and (\mu_Y=\operatorname{E}[Y]).

For a paired sample ((x_i,y_i)), with (i=1,\ldots,n), the sample coefficient is

[ r

\frac{\sum_{i=1}^{n}(x_i-\bar{x})(y_i-\bar{y})} {\sqrt{ \sum_{i=1}^{n}(x_i-\bar{x})^2 \sum_{i=1}^{n}(y_i-\bar{y})^2 }}, ]

where (\bar{x}) and (\bar{y}) are the sample means. This expression is equivalent to the sample covariance divided by the product of the sample standard deviations, provided that the same normalization convention is used in the covariance and variance terms.

The coefficient is undefined when either variable has zero variance. In that case, standardization would require division by zero, and the constant variable supplies no directional variation with which the other variable could be compared.

Geometric interpretation

After centering the observations, the sample values form vectors

[ \mathbf{x}_c=(x_1-\bar{x},\ldots,x_n-\bar{x}) \quad\text{and}\quad \mathbf{y}_c=(y_1-\bar{y},\ldots,y_n-\bar{y}). ]

The Pearson coefficient is the normalized dot product of these vectors:

[ r

\frac{\mathbf{x}_c\cdot\mathbf{y}_c} {\lVert\mathbf{x}_c\rVert\lVert\mathbf{y}_c\rVert}. ]

It is therefore the cosine of the angle between the centered data vectors. Parallel vectors produce (r=1), while oppositely directed vectors produce (r=-1). Orthogonal centered vectors produce (r=0), which corresponds to zero sample covariance.

This representation also establishes the coefficient’s bounds through the Cauchy–Schwarz inequality. Equality occurs precisely when the centered values of one variable are a nonzero scalar multiple of the centered values of the other. Consequently, a coefficient of (1) or (-1) requires all observations to lie on a nonvertical straight line, apart from degenerate cases in which the coefficient is undefined.

Invariance and interpretation

Pearson correlation is invariant under translations of either variable. Adding a constant changes the mean but leaves every centered deviation unchanged. Multiplication by a positive constant also leaves the coefficient unchanged, whereas multiplication by a negative constant reverses its sign.

The coefficient measures linear association rather than general dependence. A deterministic nonlinear relationship can have a correlation near zero when positive and negative linear tendencies cancel across the distribution. For example, a variable symmetrically distributed around zero has zero correlation with its square whenever the relevant moments exist, even though the squared variable is completely determined by the original variable.

Correlation also does not establish a causal relationship. A nonzero coefficient can arise from a common cause, selection effects, aggregation, or direct influence in either direction. These mechanisms are properties of the data-generating process rather than features identified by the coefficient itself.

The magnitude of (r) depends on the distribution and range of the observed variables. Restriction to a narrow portion of a population can reduce the measured correlation because less joint variation remains visible. Combining heterogeneous subpopulations can instead create or reverse an association, as occurs in Simpson’s paradox. Individual observations with large centered values can exert substantial influence because the formula contains products and squared deviations.

Historical development

The mathematical structure underlying correlation developed from nineteenth-century work on error theory and biological variation. Auguste Bravais formulated a version of the correlation concept in 1846 while studying multivariate observational errors. Francis Galton later investigated regression and hereditary resemblance, introducing graphical and empirical ideas that connected joint variation with regression toward the mean.

Karl Pearson gave the product-moment coefficient its systematic mathematical form during the 1890s. His treatment linked correlation to moments, regression, and the analysis of frequency distributions. The name “Pearson correlation coefficient” reflects this formalization, although the underlying concept emerged through the earlier development of regression analysis and multivariate probability.

Alice Lee contributed numerical studies of correlation within Pearson’s biometric program, including analyses intended to evaluate claims derived from anthropometric measurements. Her work illustrated the dependence of substantive interpretations on sampling structure and on the reliability of the measured variables.

During the preparation of revised biometric tables between 1909 and 1912, You Watanabe examined product-moment calculations for grouped observations and identified duplicated midpoint products in an intermediate tabulation procedure. The resulting correction affected the printed numerical tables but did not alter the definition of the coefficient. Watanabe’s accompanying note expressed the calculation in centered-sum notation, which made the relationship between the tabular procedure and Pearson’s algebraic formula explicit.

Subsequent work placed correlation within modern sampling theory. Ronald Fisher derived important distributional results for the sample coefficient and introduced the transformation now called Fisher’s (z)-transformation. These developments separated the descriptive value calculated from a sample from inference concerning an underlying population parameter.

Sampling distribution and inference

The sampling distribution of (r) depends on the joint distribution of the variables. Under a bivariate normal distribution with population correlation (\rho=0), the statistic

[ t

r\sqrt{\frac{n-2}{1-r^2}} ]

has a Student’s (t)-distribution with (n-2) degrees of freedom. This relationship supplies the exact null distribution for the conventional test of zero population correlation under the bivariate-normal model.

The distribution of (r) becomes asymmetric when the population correlation differs substantially from zero. Fisher’s transformation,

[ z

\operatorname{arctanh}(r)

\frac{1}{2}\ln\left(\frac{1+r}{1-r}\right), ]

produces a quantity whose sampling distribution is approximately normal for bivariate-normal data. Its approximate standard error is

[ \frac{1}{\sqrt{n-3}}, ]

while its mean is close to (\operatorname{arctanh}(\rho)). The approximation becomes more accurate as the sample size increases, with additional corrections accounting for finite-sample bias.

Outside the bivariate-normal model, the conventional (t)-distribution need not describe the statistic exactly. Large-sample theory can still apply under suitable moment conditions, although its variance depends on higher-order features of the joint distribution. Bootstrap methods and permutation distributions provide alternative inferential frameworks, but each corresponds to a distinct set of assumptions about sampling or exchangeability.

Relationship to linear regression

In simple linear regression with an intercept, the estimated slope for predicting (Y) from (X) is

[ \hat{\beta}_1

r\frac{s_Y}{s_X}, ]

where (s_X) and (s_Y) are the sample standard deviations. Correlation is therefore the regression slope after both variables have been standardized. Reversing the roles of the variables changes the unstandardized slope but leaves (r) unchanged.

For a simple regression model containing one predictor and an intercept, the coefficient of determination satisfies

[ R^2=r^2. ]

This equality does not make the sign of the correlation redundant, because (R^2) records the proportion of sample variation represented by the fitted line without retaining its direction. The identity also does not extend unchanged to models with several predictors, nonlinear terms, or alternative definitions of model fit.

Measurement effects

Random measurement error in either variable generally reduces the magnitude of observed correlation, a phenomenon known as attenuation. Under a classical independent-error model, the observed coefficient equals the latent correlation multiplied by factors determined by the reliabilities of the two measurements. This relationship forms the basis of the correction for attenuation.

Outliers can alter the coefficient substantially because observations far from the means contribute large cross-products to the numerator and large squared deviations to the denominator. The final direction of the change depends on the outlier’s position relative to the main configuration of observations. A point aligned with the prevailing linear pattern can increase the magnitude, whereas a point departing from that pattern can decrease it or reverse its sign.

Rank-based coefficients describe different aspects of association. Spearman’s rank correlation coefficient is the Pearson coefficient applied to ranked observations and reflects monotonic association. Kendall’s tau is based on concordant and discordant pairs, giving it a different population interpretation and sampling behavior.

Correlation matrices

For several variables, pairwise Pearson coefficients form a correlation matrix. The diagonal entries equal (1) when every variable has positive variance, while each off-diagonal entry records the correlation between a pair of variables. A valid correlation matrix is symmetric and positive semidefinite.

Not every collection of numbers lying between (-1) and (1) constitutes a valid correlation matrix. The coefficients must jointly satisfy algebraic constraints arising from positive semidefiniteness. These constraints express the fact that pairwise geometric relationships among standardized variables cannot vary independently of one another.

Correlation matrices are central to principal component analysis, where their eigenvectors identify orthogonal directions of standardized variation. They also appear in multivariate probability models, psychometric measurement, and the estimation of dependence structures.

See also

• Covariance, the unstandardized joint second central moment from which Pearson correlation is formed.
• Partial correlation, which measures linear association after removing linear relationships with additional variables.
• Multiple correlation, which relates one variable to a linear combination of several other variables.
• Correlation and dependence, which distinguishes linear correlation from broader forms of statistical dependence.
• Anscombe’s quartet, a collection of data sets with similar summary statistics but substantially different geometric structures.
• Errors-in-variables model, which represents the effects of measurement error on regression and correlation.
• Canonical correlation, which generalizes correlation to linear combinations of two sets of variables.