Tutorials › Introduction to Statistics › Unequal Sample Sizes Under Unequal Variances

Home  /  Tutorials  /  Two-Sample t Inference

Two-Sample t Inference · Tutorial 738 of 996

Unequal Sample Sizes Under Unequal Variances

Learn why the pairing of group sizes and spreads matters, and compare pooled and Welch inference in concrete two-group examples.

Intermediate 12 min read

What You'll Learn

  • Explain why unequal group sizes make the pairing of each group’s sample size and spread important.
  • Calculate pooled and Welch standard errors from two groups’ summary statistics.
  • Describe how a pooled variance weights the group variances.
  • Identify when a pooled standard error may be too small or too large.
  • Calculate a Welch degrees of freedom value and interpret a two-sided test in context.
  • State the assumptions behind pooled and Welch two-sample t procedures.

Why the Pairing of Group Sizes and Spreads Matters

When two groups have different sample sizes and different spreads, it is not enough to say that the variances are unequal. It also matters which group has the larger spread. A group with a large spread contributes more uncertainty to its sample mean, but the size of that contribution also depends on how many observations it contains. Pooling the two spreads into one common variance can therefore understate or overstate the uncertainty in the difference of means.

Welch inference addresses this by estimating the uncertainty from each group separately. This tutorial focuses on how that distinction plays out when sample sizes are unequal. The groups are labelled 1 and 2, and the parameter of interest is the difference in population means, \(\mu_1-\mu_2\). A positive difference means population 1 has the greater mean.

Definition: For independent samples with sizes \(n_1\) and \(n_2\), sample means \(\bar{x}_1\) and \(\bar{x}_2\), and sample standard deviations \(s_1\) and \(s_2\), the Welch standard error is \(SE_W=\sqrt{s_1^2/n_1+s_2^2/n_2}\). The pooled procedure instead combines the sample variances into \(s_p^2=[(n_1-1)s_1^2+(n_2-1)s_2^2]/(n_1+n_2-2)\), then uses \(SE_P=s_p\sqrt{1/n_1+1/n_2}\). The pooled procedure assumes the two populations have equal variances; Welch’s procedure does not.

What Each Group Contributes

The two terms in \(SE_W^2\), \(s_1^2/n_1\) and \(s_2^2/n_2\), estimate the separate contributions of the groups to the variance of the difference between independent sample means. A larger variance increases a group’s contribution; a larger sample size decreases it. For example, dividing a variance by 10 rather than 40 makes its contribution four times as large.

The pooled variance is a weighted average of the two sample variances, with weights \(n_1-1\) and \(n_2-1\). Thus, when sample sizes are unequal, the larger sample generally has more influence on the pooled estimate. But the pooled standard error then applies that common estimate to both groups. If the smaller group has the larger population variance, this can assign too little uncertainty to the mean difference. If the larger group has the larger variance, pooling can assign too much.

Worked Example: The Smaller Group Has the Larger Spread

Suppose two independent hypothetical groups have sample summaries in millimetres: group 1 has \(n_1=10\), \(\bar{x}_1=68\), and \(s_1=12\); group 2 has \(n_2=40\), \(\bar{x}_2=62\), and \(s_2=5\). The question is whether the population means differ, so \(H_0:\mu_1-\mu_2=0\) and \(H_a:\mu_1-\mu_2\ne0\). Assume the samples were independently selected, observations within each sample are independent, and the distributions are reasonably well behaved with no severe outliers. In particular, the group of 10 should not have strong skewness or extreme outliers for t inference to be dependable.

The observed difference is \(68-62=6\) mm. Welch’s standard error keeps the groups separate:

$$ SE_W=\sqrt{\frac{12^2}{10}+\frac{5^2}{40}} =\sqrt{14.4+0.625} =\sqrt{15.025} \approx 3.876\text{ mm}. $$

The Welch statistic is \(t_W=6/3.876\approx1.548\). Its degrees of freedom are approximately

$$ df_W= \frac{(14.4+0.625)^2} {\frac{14.4^2}{10-1}+\frac{0.625^2}{40-1}} = \frac{15.025^2}{23.04+0.0100} \approx 9.80. $$

Using a t distribution with about 9.80 degrees of freedom gives a two-sided p-value of about 0.153. At significance level \(\alpha=0.05\), this is not sufficient evidence that the population mean measurements differ.

Now calculate the pooled standard error:

$$ s_p^2=\frac{9(144)+39(25)}{48} =\frac{2271}{48} =47.3125, \qquad SE_P=\sqrt{47.3125}\sqrt{\frac{1}{10}+\frac{1}{40}} \approx 2.432\text{ mm}. $$

The pooled statistic is \(t_P=6/2.432\approx2.467\), with \(df=10+40-2=48\). Its two-sided p-value is about 0.017. This differs sharply from the Welch result: pooling makes the evidence look stronger because the smaller group, which has the larger spread, receives too little influence in the common variance estimate relative to its contribution to the mean difference’s uncertainty. These calculations illustrate why the equal-variance assumption matters; the pooled result is not justified merely because it can be computed.

A Short Justification for the Direction of the Effect

The population variance of the difference between independent sample means is \(\sigma_1^2/n_1+\sigma_2^2/n_2\). Under the pooled model, the common population variance is estimated with \(s_p^2\). If the population variances actually differ, the expected pooled variance is

$$ E(s_p^2)= \frac{(n_1-1)\sigma_1^2+(n_2-1)\sigma_2^2}{n_1+n_2-2}. $$

This follows because each sample variance estimates its own population variance, and the pooled estimate is their weighted average. To see the approximate effect when sample sizes are reasonably large, replace the weights \(n_1-1\) and \(n_2-1\) by \(n_1\) and \(n_2\). The expected squared pooled standard error is then approximately

$$ \frac{n_1\sigma_1^2+n_2\sigma_2^2}{n_1+n_2} \left(\frac{1}{n_1}+\frac{1}{n_2}\right) = \frac{\sigma_1^2}{n_2}+\frac{\sigma_2^2}{n_1}. $$

Compare this with the actual variance of the difference, \(\sigma_1^2/n_1+\sigma_2^2/n_2\). If \(n_1<n_2\) and \(\sigma_1^2>\sigma_2^2\), the approximate pooled variance is smaller; if \(n_1<n_2\) and \(\sigma_1^2<\sigma_2^2\), it is larger. This explains the direction seen in the examples. It is a comparison of expected squared standard errors, not a guarantee about every pair of observed samples.

Worked Example: The Larger Group Has the Larger Spread

Consider a second hypothetical comparison, again in millimetres. Group 1 has \(n_1=40\), \(\bar{x}_1=68\), and \(s_1=12\); group 2 has \(n_2=10\), \(\bar{x}_2=62\), and \(s_2=5\). The samples are independent, and suppose the observations show no severe outliers or strong skewness, particularly in the group of 10. We test \(H_0:\mu_1-\mu_2=0\) against \(H_a:\mu_1-\mu_2\ne0\).

The Welch standard error is

$$ SE_W=\sqrt{\frac{12^2}{40}+\frac{5^2}{10}} =\sqrt{3.6+2.5} =\sqrt{6.1} \approx2.470\text{ mm}. $$

The observed difference is again 6 mm, so \(t_W=6/2.470\approx2.429\). The Welch degrees of freedom are

$$ df_W= \frac{6.1^2}{\frac{3.6^2}{39}+\frac{2.5^2}{9}} = \frac{37.21}{0.3323+0.6944} \approx36.24. $$

The two-sided p-value is approximately 0.020. At \(\alpha=0.05\), this is evidence that the population mean measurements differ.

For comparison, the pooled variance and standard error are

$$ s_p^2=\frac{39(144)+9(25)}{48} =\frac{5841}{48} =121.6875, \qquad SE_P=\sqrt{121.6875}\sqrt{\frac{1}{40}+\frac{1}{10}} \approx3.900\text{ mm}. $$

This gives \(t_P=6/3.900\approx1.538\), with 48 degrees of freedom and a two-sided p-value of about 0.130. Here the pooled standard error is much larger than Welch’s: the larger group’s high spread has substantial weight in the pooled variance, and that common variance is then used as if it also described the smaller, lower-spread group. The pooled calculation does not detect a difference at the 0.05 level, while Welch’s does. Neither this example nor the first shows that one method must always produce a smaller p-value; the mismatch can operate in either direction.

When Sample Sizes Are Equal

Unequal variances can matter even when the two sample sizes match, but the standard errors have a useful algebraic relationship. If \(n_1=n_2=n\), then

$$ SE_W^2=\frac{s_1^2+s_2^2}{n}, \qquad SE_P^2= \frac{(n-1)s_1^2+(n-1)s_2^2}{2n-2}\left(\frac{2}{n}\right) =\frac{s_1^2+s_2^2}{n}. $$

The two estimated standard errors are exactly equal for equal sample sizes. However, the pooled procedure still relies on equal population variances, and it uses \(n_1+n_2-2\) degrees of freedom rather than Welch’s Satterthwaite degrees of freedom. Equal standard errors alone do not make the pooled model assumption true.

Worked Example: Equal Sizes, Unequal Spreads

Suppose independent hypothetical samples have \(n_1=n_2=20\), means \(\bar{x}_1=54\) and \(\bar{x}_2=48\) hours, and standard deviations \(s_1=10\) and \(s_2=4\) hours. Assume observations are independent within and between groups, and the sample distributions show no serious outliers or extreme skewness. We test \(H_0:\mu_1-\mu_2=0\) against a two-sided alternative.

Welch’s standard error is

$$ SE_W=\sqrt{\frac{10^2}{20}+\frac{4^2}{20}} =\sqrt{5+0.8} =\sqrt{5.8} \approx2.408\text{ hours}. $$

The pooled variance is \(s_p^2=[19(100)+19(16)]/38=58\), so \(SE_P=\sqrt{58}\sqrt{1/20+1/20}=\sqrt{5.8}\approx2.408\) hours as well. Both statistics are \(t=6/2.408\approx2.491\). The pooled test uses 38 degrees of freedom. Welch’s degrees of freedom are

$$ df_W= \frac{(5+0.8)^2}{\frac{5^2}{19}+\frac{0.8^2}{19}} = \frac{33.64}{1.3495} \approx24.93. $$

The two-sided p-values are approximately 0.017 for the pooled test and 0.020 for Welch’s test. Both are below 0.05, but the reference distributions are not the same. Equal sample sizes make the standard-error estimates coincide; they do not erase the distinction between a common-variance model and a procedure that allows different variances.

Choosing and Interpreting the Comparison

For independent-group comparisons, Welch’s procedure is often a sensible default when there is no strong reason to assume equal population variances. It uses \(s_1^2/n_1\) and \(s_2^2/n_2\) directly and adjusts the degrees of freedom to reflect the uncertainty in both variance estimates. This adjustment can be especially important when sample sizes and spreads differ substantially.

A smaller standard error does not by itself prove that a procedure is better, and a larger p-value does not mean the null hypothesis is true. A p-value describes how unusual a test statistic at least as extreme as the observed one would be if the null hypothesis and the procedure’s assumptions held. Conclusions should also report the direction and units of the observed difference, and should recognize that sampling design and the assumptions of the method matter.

Key takeaway: With unequal sample sizes, check which group is larger and which group is more variable. When the smaller group has the larger population spread, pooling can underestimate uncertainty; when the larger group has the larger spread, pooling can overestimate it. Welch inference keeps the estimated contributions of the two groups separate.

Common Mistake

A common mistake is to compare the two sample standard deviations and decide that the pooled method is harmless because one spread is only “a little” larger—or to assume that pooling always gives a conservative result. Neither argument accounts for sample size. The correction is to examine both variance contributions, \(s_1^2/n_1\) and \(s_2^2/n_2\), and to use a procedure whose variance assumptions are appropriate. The effect of pooling can reverse when the group sizes are paired with the spreads in the opposite way.

Check Your Understanding

Answer each question using the definitions and examples above.

  1. Group 1 has the smaller sample size and the larger population variance. In which direction does the pooled squared standard error tend to differ from the true variance of the difference, and why?
  2. Why does a larger sample generally have greater weight in the pooled variance estimate?
  3. If both groups have the same sample size but different sample standard deviations, what is the relationship between the pooled and Welch standard errors? Does this establish equal population variances?
  4. For independent samples with \(n_1=12\), \(s_1=9\), \(n_2=36\), and \(s_2=4\), write the two terms that make up the squared Welch standard error. Which group contributes more to it?
  5. In a two-sided Welch test, what does a p-value of 0.03 mean in context, and what does it not mean?

Previous

Next

Back to Tutorials