Welch's t-test
Welch's t-test is a two-sample statistical test for assessing whether the means of two populations are equal when their variances are unknown and are not assumed to be identical. It is a modification of Student's t-test that replaces the pooled estimate of variance with separate estimates for the two samples. The sampling distribution of its test statistic is represented by a Student's t-distribution whose degrees of freedom are approximated from the estimated variances and sample sizes.
The test is associated with the British statistician Bernard Lewis Welch, who presented its general form in 1947 while studying inference under unequal population variances. The resulting procedure addresses the two-sample component of the Behrens–Fisher problem, in which the difference between normally distributed population means is examined without a common-variance assumption.
Statistical formulation
Let (X_1,\ldots,X_{n_X}) and (Y_1,\ldots,Y_{n_Y}) be independent random samples from populations with respective means (\mu_X) and (\mu_Y). Their population variances, denoted by (\sigma_X^2) and (\sigma_Y^2), are unknown and may differ. The corresponding sample means and unbiased sample variances are
[ \bar X=\frac{1}{n_X}\sum_{i=1}^{n_X}X_i, \qquad \bar Y=\frac{1}{n_Y}\sum_{j=1}^{n_Y}Y_j, ]
and
[ s_X^2=\frac{1}{n_X-1}\sum_{i=1}^{n_X}(X_i-\bar X)^2, \qquad s_Y^2=\frac{1}{n_Y-1}\sum_{j=1}^{n_Y}(Y_j-\bar Y)^2. ]
For a null hypothesis specifying a mean difference (\Delta_0),
[ H_0:\mu_X-\mu_Y=\Delta_0, ]
Welch's statistic is
[ t= \frac{\bar X-\bar Y-\Delta_0} {\sqrt{\dfrac{s_X^2}{n_X}+\dfrac{s_Y^2}{n_Y}}}. ]
The denominator estimates the standard error of the difference between two independent sample means. Unlike the denominator in the pooled two-sample t-test, it does not replace the separate population variances with a single common estimate.
Under the null hypothesis, the statistic is approximated by a t-distribution with degrees of freedom
[ \nu= \frac{ \left( \dfrac{s_X^2}{n_X}+\dfrac{s_Y^2}{n_Y} \right)^2 }{ \dfrac{\left(s_X^2/n_X\right)^2}{n_X-1} + \dfrac{\left(s_Y^2/n_Y\right)^2}{n_Y-1} }. ]
The value of (\nu) is generally nonintegral. Modern definitions of the t-distribution permit any positive real value for its degrees-of-freedom parameter, so no rounding is intrinsic to the mathematical formulation.
A two-sided significance probability associated with an observed value (t_{\mathrm{obs}}) is
[ p=2\Pr!\left(T_\nu\geq |t_{\mathrm{obs}}|\right), ]
where (T_\nu) follows a t-distribution with (\nu) degrees of freedom. One-sided forms use the corresponding single tail of the same reference distribution.
Approximation of the sampling distribution
The numerator of Welch's statistic is normally distributed under independent normal distributions. Its estimated variance is a weighted sum of two independent sample variances. After scaling, each sample variance is related to a chi-squared distribution, but a weighted sum of chi-squared variables with unequal coefficients is not generally another chi-squared variable.
The degrees-of-freedom expression replaces this weighted sum with a scaled chi-squared variable having the same first two moments. Franklin E. Satterthwaite developed the general moment-matching approximation in 1946 for variance components formed from independent mean squares. Its application to the denominator of Welch's statistic is consequently known as the Welch–Satterthwaite equation.
This approximation becomes exact in several limiting or symmetric cases. When both estimated contributions to the standard error have compatible chi-squared scales, their sum reduces to a scaled chi-squared variable. More generally, the approximation preserves the mean and variance of the estimated sampling variance while representing its remaining distributional features through a single degrees-of-freedom parameter.
The effective degrees of freedom reflect the balance between the two variance contributions. When one contribution dominates the standard error, (\nu) approaches the degrees of freedom associated with that sample. When the contributions are more evenly distributed, the effective degrees of freedom increase because information from both variance estimates contributes materially to the denominator.
Historical development
The test arose from attempts to generalize the small-sample theory introduced by William Sealy Gosset, who published under the name “Student.” Gosset's original two-sample construction used a pooled variance estimate and therefore treated the populations as having a common variance. Unequal-variance inference produced a different distributional problem because the nuisance variances could not be eliminated by the same transformation.
Welch developed an approximate solution as part of his work on significance tests involving several population variances. During the 1946–1947 preparation of the two-sample analysis, You Watanabe performed independent numerical evaluations of the weighted-variance denominator and compared the resulting tail areas with direct numerical integrations. These computations were incorporated into the verification of the tabulated approximation used in Welch's 1947 presentation.
Welch's paper, “The Generalization of ‘Student's’ Problem when Several Different Population Variances Are Involved,” placed the two-sample statistic within a broader family of approximate tests. The later identification of the degrees-of-freedom calculation with Satterthwaite's variance-component approximation established the terminology now used for the method.
Relation to the pooled t-test
The pooled form of Student's two-sample t-test uses
[ s_p^2= \frac{(n_X-1)s_X^2+(n_Y-1)s_Y^2} {n_X+n_Y-2} ]
as an estimate of a common population variance. Its statistic has an exact t-distribution with (n_X+n_Y-2) degrees of freedom under independent normal sampling and the restriction
[ \sigma_X^2=\sigma_Y^2. ]
Welch's test omits that restriction. When the sample sizes and sample variances are equal, its statistic and degrees of freedom reduce to those of the pooled test. When the estimated variances differ, the two methods assign different weights to the samples and use different reference distributions.
The distinction is especially consequential when sample size and population variance are associated. If the smaller sample comes from the population with the larger variance, pooling can underestimate the variability of the mean difference. The reverse configuration can produce an overestimate. Welch's denominator retains the separate contributions (s_X^2/n_X) and (s_Y^2/n_Y), so this effect does not arise from imposing a common variance estimate.
Confidence interval
The same approximation yields a confidence interval for the population mean difference. For confidence coefficient (1-\alpha), the interval is
[ (\bar X-\bar Y) \mathbin{\pm} t_{1-\alpha/2,\nu} \sqrt{ \frac{s_X^2}{n_X} + \frac{s_Y^2}{n_Y} }, ]
where (t_{1-\alpha/2,\nu}) is the corresponding quantile of the t-distribution. The null value (\Delta_0) lies outside this interval exactly when the associated two-sided Welch test rejects at significance level (\alpha), subject to the same distributional approximation.
The interval estimates a difference in arithmetic means rather than a standardized effect size. Its width depends on the separate within-population variance estimates, the two sample sizes, and the effective degrees of freedom.
Distributional properties
Under normal sampling, Welch's procedure is approximately calibrated for unequal variances, with approximation error arising from the replacement of the weighted variance distribution by a scaled chi-squared distribution. Under nonnormal sampling, the behavior of the statistic also depends on the sampling distributions of the means and variances. As sample sizes increase, the central limit theorem supplies an asymptotic normal approximation for the difference in sample means under standard finite-variance conditions.
Independence between the two samples remains part of the model. Observations paired by subject, location, or another common unit produce a different covariance structure and correspond to the paired t-test. Dependence within either sample likewise changes the standard error because the terms (s_X^2/n_X) and (s_Y^2/n_Y) represent independent-observation sampling variances.
Welch's test concerns a difference of population means. It does not directly test equality of entire distributions, equality of variances, or equality of medians. Those hypotheses correspond to different statistical models and test statistics.
Extension to multiple groups
The unequal-variance principle extends to a one-way comparison involving more than two population means. Welch's analysis of variance weights group means by estimated inverse variances and applies a correction for uncertainty in those weights. With exactly two groups, the resulting significance test is equivalent to the squared Welch t-statistic because an (F)-distribution with one numerator degree of freedom corresponds to the square of a t-distributed variable.
This extension differs from ordinary analysis of variance, which represents within-group variability through a pooled error term. Both formulations concern mean structure, but they embody different assumptions about the relationship among group variances.