Choose Between Welch and Pooled T Procedures

Why Two Two-Sample t Procedures?

When comparing the means of two independent groups, the estimated difference in sample means is \(\bar{x}_1-\bar{x}_2\). The groups may, however, have different amounts of variability. The choice between a pooled procedure and Welch’s procedure determines how that variability is used to estimate the standard error of the difference.

The pooled procedure combines the two sample variances into one estimate of a shared population variance. Welch’s procedure keeps the variance estimates separate. The distinction matters most when the population variances differ and the sample sizes are unequal: in that setting, pooling can give a misleading standard error, while Welch’s method allows each group’s variance to contribute according to its own sample size.

Models and Formulas

Let group 1 and group 2 be independent samples from two populations. Their sample sizes are \(n_1\) and \(n_2\), their sample means are \(\bar{x}_1\) and \(\bar{x}_2\), and their sample standard deviations are \(s_1\) and \(s_2\). The parameter of interest is \(\mu_1-\mu_2\). The observations should be independent within and between groups, and the quantitative outcome should support a two-sample t analysis—for example, through reasonably well-behaved group distributions or sufficiently large samples. The study design and distributional conditions must be considered before either procedure is used.

The pooled method requires the additional model assumption that the two population variances are equal: \(\sigma_1^2=\sigma_2^2=\sigma^2\). Under that model, the pooled variance estimate is a degrees-of-freedom-weighted average of the two sample variances.

Here \((\mu_1-\mu_2)_0\) is the difference specified by a null hypothesis; for a test of equal means it is zero. For an interval estimate, the same standard error and degrees of freedom are used with the appropriate t critical value.

Welch’s method does not impose equal population variances. It estimates the variance of the difference by adding the estimated variance contributions from the two independent groups.

Because the standard error is built from two estimated variances, Welch’s degrees of freedom are generally not an integer and are approximated using the Welch–Satterthwaite formula:

How Unequal Sample Sizes Affect Pooling

The key issue is not simply that the sample standard deviations differ. It is how the two variance estimates enter the standard error. Welch’s estimated variance for the difference is \(s_1^2/n_1+s_2^2/n_2\). By contrast, the pooled method first averages the sample variances and then multiplies that common estimate by \(1/n_1+1/n_2\).

If the groups have equal sample sizes, these two estimated variances are equal, even if the observed sample variances differ. To see why, set \(n_1=n_2=m\). The pooled variance is \((s_1^2+s_2^2)/2\), so the pooled estimated variance of the difference is \(((s_1^2+s_2^2)/2)(2/m)=(s_1^2+s_2^2)/m\), which is exactly the Welch estimated variance. The degrees of freedom can still differ, but the standard errors match.

When the sample sizes are unequal, the standard errors need not match. The pooled estimate gives each sample variance a weight based on its degrees of freedom, then treats the result as if it applied to both groups. If the smaller group has the larger variance, that common estimate can understate the uncertainty contributed by the small, variable group. If the larger group has the larger variance, pooling can instead overstate the uncertainty. Welch’s method avoids requiring one variance to stand in for both.

This variance identity follows from independence: the variance of a difference of independent random variables is the sum of their variances, and the variance of a sample mean from group \(i\) is \(\sigma_i^2/n_i\). Under the equal-variance model, substituting the common \(\sigma^2\) gives \(\sigma^2(1/n_1+1/n_2)\), the structure used by pooling. If the variances are not equal, that common-variance expression is not the general variance of the difference.

Worked Examples

A Practical Choice

Welch’s procedure is generally a sound default when comparing two independent means because it does not require equal population variances. Choose the pooled procedure when a common variance is well justified by the scientific setting and the analysis model, not merely because two sample standard deviations look similar. Similar sample standard deviations do not prove equal population variances, especially with small samples.

A preliminary test of whether the variances differ is not a dependable procedure-selection rule. Such a test can have little ability to detect a meaningful variance difference in small samples, and choosing the mean procedure based on that preliminary result can make the overall analysis behave unpredictably. Consider the design, subject-matter knowledge, sample sizes, and observed variability instead.

Common Mistake

A common mistake is to pool whenever the sample standard deviations are close, or to assume the two procedures are interchangeable because they both use a t statistic. The correction is to state the model assumption explicitly: pooling treats the two population variances as equal. If that assumption is not defensible, especially with unequal sample sizes, use Welch’s method. Also avoid deciding solely from whether a variance test is statistically significant; failure to detect a difference is not evidence that the variances are equal.