Welch Confidence Intervals for a Mean Difference

Why Use a Welch Confidence Interval?

When two groups are measured on the same quantitative outcome, a natural question is how far apart their population means are. The answer depends on the order of subtraction: the difference $\mu_1-\mu_2$ is positive when population 1 has the larger mean and negative when population 2 does. A confidence interval gives a range of plausible values for that difference, while accounting for the variability and sizes of both samples.

The Welch confidence interval is designed for two independent samples when the population variances are not assumed equal. It combines the observed difference in sample means with a standard error that reflects both samples. The critical value comes from a $t$ distribution using Satterthwaite degrees of freedom.

Model, Assumptions, and Notation

Let $X_1$ and $X_2$ represent the quantitative outcomes in populations 1 and 2. Their population means are $\mu_1$ and $\mu_2$; the target parameter is $\mu_1-\mu_2$. From independent samples, record the sample sizes $n_1,n_2$, sample means $\bar{x}_1,\bar{x}_2$, and sample standard deviations $s_1,s_2$.

This interval is appropriate when each sample is obtained through a design that supports inference to its target population, and the two samples are independent of one another. Observations within each sample should also be independent. For small samples, the outcome distributions within each population should be reasonably close to normal; with larger samples, moderate departures from normality are generally less consequential. Strong skewness or extreme outliers can make a $t$-based interval unreliable, particularly with small samples. The procedure does not require equal population variances.

Definition: The Welch confidence interval for $\mu_1-\mu_2$ is $$ (\bar{x}_1-\bar{x}_2)\ \pm\ t^* SE, $$ where $$ SE=\sqrt{\frac{s_1^2}{n_1}+\frac{s_2^2}{n_2}} $$ is the Welch standard error, and $t^*$ is the $1-\alpha/2$ quantile of a $t$ distribution with Satterthwaite degrees of freedom $$ df=\frac{\left(\frac{s_1^2}{n_1}+\frac{s_2^2}{n_2}\right)^2} {\frac{(s_1^2/n_1)^2}{n_1-1}+\frac{(s_2^2/n_2)^2}{n_2-1}}. $$ For a confidence level of $100(1-\alpha)\%$, the interval endpoints are $(\bar{x}_1-\bar{x}_2)-t^*SE$ and $(\bar{x}_1-\bar{x}_2)+t^*SE$.

The degrees of freedom may be fractional. Use that value, or an appropriate software calculation, to obtain $t^*$; rounding it down to an integer is not part of the formula. The same construction works for different confidence levels by changing $\alpha$. For example, a 95% interval has $\alpha=0.05$ and uses the 0.975 quantile.

Why This Formula Has This Form

The sample mean difference estimates the population mean difference. For independent samples, the variance of that difference is the sum of the variances of the two sample means. Replacing unknown population variances with sample variances gives the squared Welch standard error:

\widehat{\operatorname{Var}}(\bar{x}_1-\bar{x}_2) =\frac{s_1^2}{n_1}+\frac{s_2^2}{n_2}=SE^2.

The standardized difference is therefore $(\bar{x}_1-\bar{x}_2-(\mu_1-\mu_2))/SE$. Because the variances are estimated and may differ, this statistic is treated as approximately following a $t$ distribution. The Satterthwaite formula supplies the degrees of freedom for that approximation. For a central $100(1-\alpha)\%$ interval, the approximate probability that the standardized statistic lies between the two critical values is $1-\alpha$:

P\left(-t^* \leq \frac{(\bar{x}_1-\bar{x}_2)-(\mu_1-\mu_2)}{SE} \leq t^*\right)\approx 1-\alpha.

Rearranging the left inequality gives $(\bar{x}_1-\bar{x}_2)-t^*SE \leq \mu_1-\mu_2$. Rearranging the right gives $\mu_1-\mu_2 \leq (\bar{x}_1-\bar{x}_2)+t^*SE$. Together, these yield the stated interval. The approximation is why the distributional and sampling assumptions matter.

Constructing the Interval Step by Step

Fix the order of groups.
Write the target as $\mu_1-\mu_2$, and use $\bar{x}_1-\bar{x}_2$ for the estimate. Keep the same order throughout.

Calculate the standard error.
Compute $\sqrt{s_1^2/n_1+s_2^2/n_2}$. Each sample contributes its estimated variance divided by its own size.

Find the degrees of freedom and critical value.
Use the Satterthwaite formula, then find $t^*$ for the selected confidence level and those degrees of freedom.

Calculate and report the endpoints.
Multiply $t^*$ by $SE$, then subtract and add that margin of error to the sample mean difference. Keep the outcome units.

Worked Example: A 95% Interval for Two Independent Groups

Suppose a hypothetical exercise compares weekly activity time for two independent groups. In group 1, $n_1=16$, $\bar{x}_1=12.4$ hours, and $s_1=3.2$ hours. In group 2, $n_2=12$, $\bar{x}_2=10.1$ hours, and $s_2=2.4$ hours. Assume the sampling and distributional conditions for Welch inference are reasonable. Construct a 95% interval for $\mu_1-\mu_2$.

The sample mean difference, in the stated order, is $12.4-10.1=2.3$ hours. The standard error is

SE=\sqrt{\frac{3.2^2}{16}+\frac{2.4^2}{12}} =\sqrt{\frac{10.24}{16}+\frac{5.76}{12}} =\sqrt{0.64+0.48} =\sqrt{1.12} \approx 1.058\text{ hours}.

The degrees of freedom are

df=\frac{(0.64+0.48)^2}{0.64^2/15+0.48^2/11} =\frac{1.2544}{0.02731+0.02095} \approx 26.00.

For a 95% interval with about 26 degrees of freedom, $t^*\approx2.056$. The margin of error is $2.056(1.058)\approx2.176$ hours. Thus,

2.3\pm2.176 \quad\Longrightarrow\quad (0.124,\ 4.476)\text{ hours}.

The resulting interval estimates the difference in mean weekly activity time, group 1 minus group 2. Its endpoints are in hours, the same units as the measured outcome.

Worked Example: A Difference That Is Not Precise

Imagine a hypothetical comparison of two independent groups’ scores, measured in points. Group 1 has $n_1=20$, $\bar{x}_1=84$, and $s_1=10$; group 2 has $n_2=25$, $\bar{x}_2=78$, and $s_2=15$. Assuming the conditions for the procedure are reasonable, find a 95% Welch interval for $\mu_1-\mu_2$.

The observed difference is $84-78=6$ points. The standard error is

SE=\sqrt{\frac{10^2}{20}+\frac{15^2}{25}} =\sqrt{5+9} =\sqrt{14} \approx3.742\text{ points}.

Using the Satterthwaite formula,

df=\frac{(5+9)^2}{5^2/19+9^2/24} =\frac{196}{25/19+81/24} \approx41.78.

The 95% critical value is approximately $2.018$. The margin of error is $2.018(3.742)\approx7.551$ points, so

6\pm7.551 \quad\Longrightarrow\quad (-1.551,\ 13.551)\text{ points}.

The estimate is positive, but the interval also includes negative differences. The calculation itself does not change: a margin of error larger than the observed difference simply places the lower endpoint below zero.

Worked Example: Changing the Confidence Level

Suppose a hypothetical clinical measurement, in milligrams per litre, is taken from two independent groups. Group 1 has $n_1=9$, $\bar{x}_1=5.8$, and $s_1=1.5$; group 2 has $n_2=11$, $\bar{x}_2=4.9$, and $s_2=1.2$. Construct a 90% Welch interval for $\mu_1-\mu_2$, assuming the conditions are reasonable.

The estimated difference is $5.8-4.9=0.9$ milligrams per litre, and

SE=\sqrt{\frac{1.5^2}{9}+\frac{1.2^2}{11}} =\sqrt{0.25+0.1309} \approx0.6172\text{ milligrams per litre}.

The degrees of freedom are

df=\frac{(0.25+0.1309)^2}{0.25^2/8+0.1309^2/10} \approx15.23.

For a 90% interval, $\alpha=0.10$, so the critical value is the 0.95 quantile, approximately $t^*=1.752$. The margin of error is $1.752(0.6172)\approx1.081$ milligrams per litre. The interval is

0.9\pm1.081 \quad\Longrightarrow\quad (-0.181,\ 1.981)\text{ milligrams per litre}.

This example uses a different confidence level but the same standard error and degrees-of-freedom calculations. Only the critical value, and therefore the margin of error, changes when the confidence level changes.

Common Mistakes and Practical Checks

Reversing the groups midway. If the target is $\mu_1-\mu_2$, the point estimate must be $\bar{x}_1-\bar{x}_2$. Reversing the order changes the sign of both endpoints.
Using a pooled standard error. Welch’s procedure uses $s_1^2/n_1+s_2^2/n_2$. Do not combine the sample variances under an equal-variance assumption for this interval.
Using the wrong critical value. A two-sided $100(1-\alpha)\%$ interval uses the $1-\alpha/2$ quantile, not the $1-\alpha$ quantile.
Treating $df$ as the number of observations. Satterthwaite degrees of freedom are calculated from both sample sizes and variances; they are not $n_1+n_2-2$ in general.
Dropping the units. A difference in means has the same units as the original outcome. Report those units with the interval.
Applying the method to paired observations. This formula is for independent samples. When observations are paired, the variability of within-pair differences is relevant instead.

A useful final check is to confirm that the interval is centered on $\bar{x}_1-\bar{x}_2$, has a nonnegative margin of error, and uses the same measurement units throughout. These checks often reveal arithmetic or sign errors before the result is reported.

Check Your Understanding

Use the definitions and procedure above to answer the following questions.

For a 90% two-sided Welch interval, what quantile of the $t$ distribution is used to find $t^*$?
Why does the Welch standard error contain one variance term for each sample?
If the target is $\mu_1-\mu_2$, what must be the order of subtraction in the sample estimate?
What information, in addition to the sample means, is needed to calculate the Satterthwaite degrees of freedom?
Which assumption is violated if the two samples consist of paired measurements on the same individuals?