Why Use a Welch Confidence Interval?
When two groups have different sample sizes and different spreads, a confidence interval for the difference in their population means should account for both differences. The Welch confidence interval does this without assuming that the population variances are equal. It combines the two sample variances in a standard error and uses an approximate degrees of freedom to select a t critical value.
The target is the difference between the two population means, written in a stated order. For example, if group 1 is a treatment group and group 2 is a comparison group, the parameter is \(\mu_1-\mu_2\). Reversing the order reverses the signs of the estimate and interval, so the group order must be clear.
Conditions and the Reason for the Formula
Welch inference concerns two independent groups, not paired measurements. The observations within each group should be independent, and the two groups should be independent of each other. Random sampling supports generalizing to the populations sampled; random assignment in an experiment supports a causal comparison for the units represented by that experiment. Neither design feature can be inferred from the arithmetic alone.
The response must be quantitative. For small samples, each group’s population distribution should be approximately normal, with no severe skewness or influential outliers. With larger samples, the sampling distributions of the means are generally more robust to moderate departures from normality, but extreme skewness and outliers still warrant attention. The two population variances do not need to be equal. This is the defining advantage of Welch’s method over a pooled analysis.
The standard error adds the estimated variance contributions from the two sample means. A sample mean has estimated variance \(s_i^2/n_i\), so the estimated variance of their difference, for independent groups, is the sum of those two quantities. The degrees of freedom adjust for the uncertainty in estimating each variance separately.
Worked Welch Confidence Interval
Worked Example: Difference in Mean Concentration
In a hypothetical study, independent samples from two groups are measured for a concentration in milligrams. Group 1 has \(n_1=20\), \(\bar{x}_1=12.4\) mg, and \(s_1=6.2698\) mg. Group 2 has \(n_2=18\), \(\bar{x}_2=9.1\) mg, and \(s_2=4.9480\) mg. The observations are described as roughly symmetric with no extreme outliers, and the samples are independent. Construct a 95% Welch interval for \(\mu_1-\mu_2\).
Check the conditions. Concentration is quantitative. The two samples are independent, and the stated distributional description supports t inference for these sample sizes. The sample standard deviations and sizes differ, so using Welch’s separate variance contributions is appropriate; equality of population variances is not required.
Estimate the difference. Keeping group 1 minus group 2 throughout, \[ \bar{x}_1-\bar{x}_2=12.4-9.1=3.3\text{ mg}. \]
Calculate the standard error. Substitute each group’s sample standard deviation and sample size: \[ SE=\sqrt{\frac{6.2698^2}{20}+\frac{4.9480^2}{18}} =\sqrt{\frac{39.31039204}{20}+\frac{24.482704}{18}} =\sqrt{1.965519602+1.360150222} =\sqrt{3.325669824} \approx 1.823642\text{ mg}. \] The standard error is in milligrams, like the estimated difference.
Calculate the approximate degrees of freedom. Let \(a=1.965519602\) and \(b=1.360150222\), the two variance contributions just calculated. Then \[ df=\frac{(a+b)^2}{a^2/19+b^2/17} =\frac{(3.325669824)^2}{(1.965519602)^2/19+(1.360150222)^2/17} \approx 35.43. \]
For a 95% two-sided interval, \(\alpha=0.05\), so the critical value is the 0.975 quantile of the t distribution with about 35.43 degrees of freedom. From a t table, \(t^*\approx2.03\). The margin of error is \[ t^*SE=2.03(1.823642)\approx3.701993\text{ mg}. \] Thus the interval is \[ 3.3\pm3.701993=(-0.401993,\ 7.001993)\text{ mg}, \] or, to three decimal places, approximately \((-0.402,\ 7.002)\) mg.
Interpretation. The data are consistent with a 95% confidence interval from about 0.402 mg lower to 7.002 mg higher for the population mean concentration in group 1 compared with group 2. Because zero lies in the interval, the results do not rule out equal population means at the corresponding two-sided 5% level. This does not establish that the means are equal.
More Examples of the Calculation
Worked Example: A 95% Interval for Two Independent Mean Times
Suppose hypothetical independent samples have time measurements in minutes. Group 1 has \(n_1=16\), \(\bar{x}_1=52\) minutes, and \(s_1=8\) minutes. Group 2 has \(n_2=25\), \(\bar{x}_2=47\) minutes, and \(s_2=5\) minutes. Each sample is described as roughly symmetric without influential outliers. Find a 95% Welch interval for \(\mu_1-\mu_2\).
The response is quantitative, the groups are independent, and the distribution descriptions support the procedure. The standard error is \[ SE=\sqrt{\frac{8^2}{16}+\frac{5^2}{25}} =\sqrt{4+1} =\sqrt{5} \approx2.2361\text{ minutes}. \] The degrees of freedom are \[ df=\frac{(4+1)^2}{4^2/15+1^2/24} =\frac{25}{16/15+1/24} =\frac{25}{133/120} =\frac{3000}{133} \approx22.56. \] For 95% confidence, \(t^*\approx2.07\). The estimated difference is \(52-47=5\) minutes, and the margin of error is \(2.07(2.2361)\approx4.6287\) minutes. Therefore, \[ 5\pm4.6287=(0.3713,\ 9.6287)\text{ minutes}, \] or approximately \((0.37,\ 9.63)\) minutes. The interval suggests that the population mean time in group 1 is higher, with plausible differences of about 0.37 to 9.63 minutes under this model.
Worked Example: A 90% Interval With Unequal Sample Sizes
In a hypothetical independent comparison of two groups, a quantitative outcome is measured in points. Group 1 has \(n_1=12\), \(\bar{x}_1=108\) points, and \(s_1=12\) points. Group 2 has \(n_2=30\), \(\bar{x}_2=102\) points, and \(s_2=6\) points. Assume the measurements in each group are roughly symmetric and have no extreme outliers. Construct a 90% Welch interval for \(\mu_1-\mu_2\).
The outcome is quantitative, and the independent samples and stated distribution shape support Welch inference. For a 90% interval, \(\alpha=0.10\), and the critical value is the 0.95 t quantile. First, \[ SE=\sqrt{\frac{12^2}{12}+\frac{6^2}{30}} =\sqrt{12+1.2} =\sqrt{13.2} \approx3.6332\text{ points}. \] The degrees of freedom are \[ df=\frac{(12+1.2)^2}{12^2/11+1.2^2/29} =\frac{174.24}{144/11+1.44/29} \approx13.26. \] For about 13.26 degrees of freedom, \(t^*\approx1.77\). The estimated difference is \(108-102=6\) points, and the margin of error is \(1.77(3.6332)\approx6.4308\) points. The interval is \[ 6\pm6.4308=(-0.4308,\ 12.4308)\text{ points}, \] or approximately \((-0.43,\ 12.43)\) points. The interval includes zero, so these data are compatible with no population mean difference as well as with differences in either direction. A 90% interval is narrower than a 95% interval calculated from the same data because its critical value is smaller.
Reading and Reporting the Interval
The confidence level describes the long-run performance of the procedure: if the same sampling process were repeated many times and a Welch interval were constructed each time, about 95% of the resulting intervals would contain the fixed difference \(\mu_1-\mu_2\), when the model assumptions hold. It is not correct to say that there is a 95% probability that the fixed parameter lies in this particular calculated interval.
The interval gives a range of parameter values reasonably compatible with the data and the model. Its units are the response’s units, not the units of the standard deviation squared. If zero is outside a two-sided 95% interval, the interval indicates a nonzero difference at the corresponding 5% level; if zero is inside, the interval does not provide that evidence. Inclusion of zero is not proof that the means are equal.
A clear report names the outcome, the group order, the confidence level, the estimated difference, and the interval with units. For example: “The estimated group 1 minus group 2 mean concentration difference was 3.3 mg; the 95% Welch confidence interval was \((-0.402,\ 7.002)\) mg.” A reader can then see both the estimate and the uncertainty around it.
Common Mistake
A common mistake is to combine the two sample variances into one pooled variance simply because both groups measure the same outcome, or to divide each variance by the total sample size. The Welch standard error instead uses the separate contributions \(s_1^2/n_1\) and \(s_2^2/n_2\); its degrees of freedom also use each group’s own \(n_i-1\). Pooling imposes a different variance assumption and is not the Welch calculation.
Another mistake is to interpret an interval containing zero as proof of no difference. The interval reports uncertainty: values inside it, including zero when applicable, remain compatible with the data at the stated confidence level. The right conclusion is limited to what the interval supports.
Check Your Understanding
Use the definitions and examples above to answer these questions.
- In a Welch interval for \(\mu_1-\mu_2\), what does the sign of the estimated difference \(\bar{x}_1-\bar{x}_2\) mean?
- Why does the Welch standard error use \(s_1^2/n_1+s_2^2/n_2\) rather than a single pooled variance?
- What role do the Welch-Satterthwaite degrees of freedom play in choosing the critical value?
- A 95% Welch interval for a difference in mean scores is \((-2,\ 5)\) points. What does its inclusion of zero indicate, and what does it not prove?
- State the long-run meaning of a 95% confidence level without assigning a probability to the fixed parameter after the interval has been calculated.