Welch t Tests for Unequal Variances

Why Use a Welch t Test?

When comparing two population means from independent samples, the population variances are often unknown. If the variances are assumed equal, a pooled two-sample t procedure can combine the sample variances. That assumption is not always appropriate: the populations may have different variability, and their sample sizes may also differ. The Welch t test compares the means without assuming equal population variances.

The procedure uses a standard error built from each sample’s own variance and sample size. It then approximates the test statistic’s reference distribution with a t distribution whose degrees of freedom are estimated from both samples. Because those degrees of freedom need not be an integer, statistical software can use the computed value directly; a table-based calculation may use a suitable conservative rounding rule.

Definition: Let two independent random samples have sizes $n_1,n_2$, sample means $\bar{x}_1,\bar{x}_2$, and sample variances $s_1^2,s_2^2$. The Welch standard error for estimating $\mu_1-\mu_2$ is $$ SE_W=\sqrt{\frac{s_1^2}{n_1}+\frac{s_2^2}{n_2}}. $$ The Welch test statistic for $H_0:\mu_1-\mu_2=\Delta_0$ is $$ t_W=\frac{(\bar{x}_1-\bar{x}_2)-\Delta_0}{SE_W}, $$ with approximate degrees of freedom $$ \nu_W= \frac{\left(\frac{s_1^2}{n_1}+\frac{s_2^2}{n_2}\right)^2} {\frac{(s_1^2/n_1)^2}{n_1-1}+\frac{(s_2^2/n_2)^2}{n_2-1}}. $$

Assumptions and Test Procedure

The samples must be independent of one another, and observations within each sample must be obtained in a way that supports the intended population inference. The measurements should be quantitative. For small samples, each population should be approximately normal, without severe skewness or influential outliers. With sufficiently large samples, the sampling distributions of the sample means may be approximately normal under less restrictive population shapes, provided the observations are independent and the distributions have finite variances. Unequal population variances are allowed.

The test addresses a specified difference $\Delta_0$, usually zero. For a two-sided alternative, $H_A:\mu_1-\mu_2\ne\Delta_0$, compare $|t_W|$ with the two-sided critical value $t_{1-\alpha/2,\nu_W}$, or use the corresponding two-sided p-value. For the alternative $H_A:\mu_1-\mu_2>\Delta_0$, compare $t_W$ with $t_{1-\alpha,\nu_W}$. For a less-than alternative, use the lower-tail critical value. Reject the null hypothesis when the statistic falls in the relevant rejection region, or equivalently when the p-value is at most $\alpha$.

Write the hypotheses.
Specify the difference being tested and the direction of the alternative before looking at the sample results.

Calculate the standard error.
Use $s_1^2/n_1+s_2^2/n_2$ under the square root. Do not replace the two sample variances by a pooled variance.

Calculate the statistic and degrees of freedom.
Subtract the null difference from the observed difference, divide by $SE_W$, and apply the Welch degrees-of-freedom formula.

Make and explain the decision.
Use the appropriate tail and significance level. State the conclusion in the context of the two population means.

Worked Tests

Worked Example: Testing a Difference With Unequal Sample Variability

A hypothetical laboratory compares the time, in minutes, required by two independent procedures to complete a calibration. For procedure 1, $n_1=16$, $\bar{x}_1=52$, and $s_1=8$. For procedure 2, $n_2=9$, $\bar{x}_2=45$, and $s_2=3$. Assume independent random samples and approximately normal completion times in both populations. At the 5% level, test $H_0:\mu_1-\mu_2=0$ against $H_A:\mu_1-\mu_2\ne0$.

Method. Use a two-sided Welch t test. The observed difference is $\bar{x}_1-\bar{x}_2=52-45=7$ minutes.

The variance contributions to the squared standard error are $$ \frac{s_1^2}{n_1}=\frac{8^2}{16}=\frac{64}{16}=4,\qquad \frac{s_2^2}{n_2}=\frac{3^2}{9}=\frac{9}{9}=1. $$ Thus $SE_W=\sqrt{4+1}=\sqrt{5}\approx2.2361$, and $$ t_W=\frac{7-0}{\sqrt{5}}\approx3.1305. $$

For the degrees of freedom, the numerator is $(4+1)^2=25$, and the denominator is $$ \frac{4^2}{16-1}+\frac{1^2}{9-1} =\frac{16}{15}+\frac{1}{8} \approx1.1917. $$ Therefore $\nu_W=25/1.1917\approx20.98$. The two-sided 5% critical value is approximately $t_{0.975,20.98}=2.08$. Since $|3.1305|>2.08$, reject $H_0$. The sample provides evidence of a difference between the population mean calibration times.

Check. The statistic is positive because the first sample mean exceeds the second, and its magnitude exceeds the two-sided critical value. A common mistake is to use a one-sided critical value merely because the observed difference is positive; the stated alternative is two-sided, so the 2.5% tail critical value is required.

Welch Confidence Intervals

The same standard error and approximate degrees of freedom give a confidence interval for the difference in population means. Under the Welch approximation, a two-sided confidence interval with confidence level $1-\alpha$ is

(\bar{x}_1-\bar{x}_2)\ \pm\ t_{1-\alpha/2,\nu_W}SE_W.

This interval estimates $\mu_1-\mu_2$. Its endpoints are in the original measurement units. If the interval contains zero, the corresponding two-sided test of equal means does not reject at the matching significance level, apart from small differences caused by rounding.

Worked Example: Building a Welch Confidence Interval

Two independent samples measure the monthly number of service calls handled by staff in two hypothetical offices. Office 1 has $n_1=10$, $\bar{x}_1=15.6$, and $s_1=2$. Office 2 has $n_2=15$, $\bar{x}_2=14.1$, and $s_2=4$. Assume independent random samples and approximately normal populations. Construct an approximate 95% confidence interval for $\mu_1-\mu_2$.

Method. Construct a two-sided Welch interval; keep the sample variances separate. The point estimate is $15.6-14.1=1.5$ calls.

The variance contributions are $$ \frac{s_1^2}{n_1}=\frac{4}{10}=0.4,\qquad \frac{s_2^2}{n_2}=\frac{16}{15}\approx1.0667. $$ Hence $$ SE_W=\sqrt{0.4+1.0667}=\sqrt{1.4667}\approx1.2111. $$ The degrees-of-freedom numerator is $1.4667^2\approx2.1511$. Its denominator is $$ \frac{0.4^2}{10-1}+\frac{(1.0667)^2}{15-1} \approx\frac{0.16}{9}+\frac{1.1378}{14} \approx0.0178+0.0813=0.0990. $$ Thus $\nu_W\approx2.1511/0.0990\approx21.72$. The 97.5th percentile of the t distribution with about 21.72 degrees of freedom is approximately 2.076. The margin of error is $$ 2.076(1.2111)\approx2.514. $$ Therefore the interval is $$ 1.5\pm2.514=(-1.014,\ 4.014), $$ or approximately $(-1.01,\ 4.01)$ calls when each endpoint is rounded to two decimal places.

Interpretation and check. The interval is compatible with a range of differences from about 1.01 fewer calls to 4.01 more calls for Office 1. It includes zero, so equality of the means is not rejected by the corresponding two-sided 5% test. A common mistake is to round the endpoints prematurely or miscalculate the degrees of freedom; using the unrounded standard error and degrees of freedom gives endpoints $-1.01$ and $4.01$, not $-1.02$ and $4.02$.

Why the Degrees of Freedom Are Approximate

For independent samples, the estimated difference $\bar{X}_1-\bar{X}_2$ has variance $\sigma_1^2/n_1+\sigma_2^2/n_2$. Replacing the unknown population variances with their sample estimates gives the Welch estimated variance $$ V=\frac{s_1^2}{n_1}+\frac{s_2^2}{n_2}. $$ Each sample variance is itself variable. The Welch degrees-of-freedom formula approximates the variability of this sum by matching it to a scaled chi-square distribution, which leads to a t reference distribution after standardization.

To see the matching step, suppose the population samples are normal and independent. Then $(n_j-1)s_j^2/\sigma_j^2$ has a chi-square distribution with $n_j-1$ degrees of freedom for $j=1,2$. The two sample variances are independent. A scaled chi-square variable with $\nu$ degrees of freedom has variance equal to twice the square of its mean divided by $\nu$. Matching the mean and variance of the sum of the two estimated variance contributions yields

\nu_W= \frac{V^2} {\dfrac{(s_1^2/n_1)^2}{n_1-1}+ \dfrac{(s_2^2/n_2)^2}{n_2-1}}.

This is the Satterthwaite approximation. It is not generally an exact degrees-of-freedom result: the sum of two independently scaled chi-square variables with different scales need not itself be a scaled chi-square variable. Thus the Welch t distribution is an approximation, even when the populations are normal. The formula adapts the reference distribution to the observed contributions to the standard error rather than imposing a common variance.

Worked Example: A One-Sided Welch Test

A hypothetical analyst compares processing times for two independent types of data batch. Type 1 has $n_1=12$, $\bar{x}_1=103$, and $s_1=6$ seconds. Type 2 has $n_2=20$, $\bar{x}_2=98$, and $s_2=10$ seconds. Assume independent random samples and approximately normal population times. At the 5% level, test whether type 1 has a greater population mean processing time: $H_0:\mu_1-\mu_2=0$ against $H_A:\mu_1-\mu_2>0$.

Method. Use a right-tailed Welch test. The observed difference is $103-98=5$ seconds. The variance contributions are $$ \frac{6^2}{12}=\frac{36}{12}=3,\qquad \frac{10^2}{20}=\frac{100}{20}=5. $$ Therefore $SE_W=\sqrt{3+5}=\sqrt{8}\approx2.8284$, and $t_W=5/\sqrt{8}\approx1.7678$.

The degrees-of-freedom numerator is $8^2=64$. The denominator is $$ \frac{3^2}{12-1}+\frac{5^2}{20-1} =\frac{9}{11}+\frac{25}{19} \approx0.8182+1.3158=2.1340. $$ Thus $\nu_W\approx64/2.1340\approx30.00$. The upper 5% critical value is approximately $t_{0.95,30}=1.697$. Since $1.7678>1.697$, reject $H_0$ at the 5% level. There is evidence that type 1 has a greater population mean processing time.

Check. The alternative is directional, so this is a right-tail comparison, not a two-sided comparison. The statistic is positive and exceeds the right-tail critical value. Reversing the group order would reverse the sign of both the estimated difference and the statistic; the hypotheses and direction must be kept consistent.

What the Procedure Does and Does Not Say

Welch’s method is useful when equal population variances cannot be justified, particularly when the sample sizes are unequal. Its estimated standard error reflects both the size of each sample and its observed variability. A group with greater variability can make a substantial contribution even if it has a larger sample size.

A statistically significant result is evidence against the specified null difference under the model assumptions; it does not establish that the difference is practically important or identify its cause. Conversely, failure to reject does not establish that the means are equal. Consider the confidence interval as well as the test decision to assess the range of differences consistent with the data.

Do not pool by habit. Welch’s standard error uses $s_1^2/n_1+s_2^2/n_2$; a pooled variance belongs to a procedure that assumes equal population variances.
Do not confuse variance and standard deviation. The formula requires $s_1^2$ and $s_2^2$; square the reported standard deviations before dividing by sample sizes.
Do not use the wrong tail. The alternative hypothesis determines whether the critical region is one-sided or two-sided.
Do not interpret a confidence interval as a probability statement about a fixed parameter. The interval is produced by a method with a stated long-run coverage rate under its assumptions.

Key takeaway: For two independent means with unknown, potentially unequal population variances, calculate each variance contribution separately, use the Welch standard error and Satterthwaite degrees of freedom, and match the critical value or p-value to the stated alternative.

Check Your Understanding

For each problem, use a Welch procedure and retain sufficient precision until the final answer.

Two independent samples have $n_1=25,\bar{x}_1=31,s_1=5$ and $n_2=16,\bar{x}_2=28,s_2=4$. Test $H_0:\mu_1-\mu_2=0$ against a two-sided alternative at $\alpha=0.05$. Give the standard error, approximate degrees of freedom, and decision. (Answer: $SE_W=\sqrt{2}\approx1.4142$, $\nu_W\approx36.92$, $t_W\approx2.1213$; reject $H_0$ at 5%.)
Independent samples have $n_1=9,\bar{x}_1=12.4,s_1=3$ and $n_2=25,\bar{x}_2=10.0,s_2=5$. Find an approximate 95% Welch confidence interval for $\mu_1-\mu_2$, rounding endpoints to two decimals. (Answer: $SE_W=\sqrt{2}\approx1.4142$, $\nu_W=24$, and the interval is approximately $(-0.52,5.32)$.)
For two independent populations, sample 1 has $n_1=10,\bar{x}_1=7.8,s_1=2$, and sample 2 has $n_2=12,\bar{x}_2=6.2,s_2=3$. At $\alpha=0.05$, test $H_A:\mu_1-\mu_2>0$ against a zero-difference null. (Answer: $SE_W=\sqrt{1.15}\approx1.0724$, $\nu_W\approx19.19$, $t_W\approx1.4920$; do not reject $H_0$, since the upper 5% critical value is about 1.729.)
A two-sided Welch test has observed difference $\bar{x}_1-\bar{x}_2=-4$, $SE_W=1.5$, and $\nu_W=18$. Test a zero difference at $\alpha=0.05$, and state whether the matching 95% confidence interval contains zero. (Answer: $t_W=-4/1.5\approx-2.6667$; since $|t_W|>t_{0.975,18}\approx2.101$, reject $H_0$, and the matching interval does not contain zero.)

What You'll Learn