Deriving The Standard Error For Independent Mean Differences

Why the Standard Error for a Difference Needs Deriving

When comparing two population means, the observed difference between sample means varies from sample to sample. Its uncertainty depends on the variability in both samples. For independent samples, the two sources of variance add. Deriving that result makes clear why the standard error has its familiar form—and why combining the two sample sizes as though they were one sample is not correct.

This tutorial treats the population variances as known. The earlier material on normal-theory Z intervals gives the general confidence-interval structure; here the focus is how to obtain and use the standard error for a difference of means. The formula requires independent samples. If observations are matched or otherwise dependent across samples, the covariance term cannot simply be discarded.

Definition: Let $X_1,\ldots,X_{n_1}$ be an independent, identically distributed random sample from a population with mean $\mu_1$ and variance $\sigma_1^2$. Let $Y_1,\ldots,Y_{n_2}$ be an independent, identically distributed random sample from a second population with mean $\mu_2$ and variance $\sigma_2^2$. Assume the two samples are independent of one another. Their sample means and their difference are $$ \bar{X}=\frac{1}{n_1}\sum_{i=1}^{n_1}X_i,\qquad \bar{Y}=\frac{1}{n_2}\sum_{j=1}^{n_2}Y_j,\qquad D=\bar{X}-\bar{Y}. $$ The parameter of interest is $\Delta=\mu_1-\mu_2$.

Deriving the Variance and Standard Error

First, a sample mean has the same expected value as its population mean and, for independent observations with common variance, has variance equal to the population variance divided by the sample size. The difference $D$ is a linear combination of the two sample means. The variance rule for a difference is $\operatorname{Var}(\bar{X}-\bar{Y})=\operatorname{Var}(\bar{X})+\operatorname{Var}(\bar{Y})-2\operatorname{Cov}(\bar{X},\bar{Y})$. Independence makes the covariance zero.

Find each sample-mean variance.
Independence within each sample gives $\operatorname{Var}(\bar{X})=\sigma_1^2/n_1$ and $\operatorname{Var}(\bar{Y})=\sigma_2^2/n_2$.

Apply the variance rule.
For any two random variables, subtracting them gives a covariance term: $\operatorname{Var}(D)=\operatorname{Var}(\bar{X})+\operatorname{Var}(\bar{Y})-2\operatorname{Cov}(\bar{X},\bar{Y})$.

Use independence between samples.
Because the two samples are independent, their sample means are independent and $\operatorname{Cov}(\bar{X},\bar{Y})=0$. The variances therefore add.

E[D]=E[\bar{X}]-E[\bar{Y}]=\mu_1-\mu_2=\Delta

\begin{aligned} \operatorname{Var}(D) &=\operatorname{Var}(\bar{X})+\operatorname{Var}(\bar{Y})-2\operatorname{Cov}(\bar{X},\bar{Y})\\ &=\frac{\sigma_1^2}{n_1}+\frac{\sigma_2^2}{n_2}. \end{aligned}

Thus $D$ is an unbiased estimator of the population mean difference, and its standard deviation—the standard error when the population variances are known—is the square root of this variance.

Theorem: Under the independent-sample assumptions above, $$ \operatorname{SE}(D)=\sqrt{\frac{\sigma_1^2}{n_1}+\frac{\sigma_2^2}{n_2}}. $$ If both populations are normal, $D$ is normally distributed exactly, with mean $\Delta$ and the variance shown above. If the populations are not normal, a normal approximation may be appropriate for sufficiently large samples under the conditions of the central limit theorem; it is then an approximation, not an exact distributional statement.

The formula is a sum of variances of sample means, not a pooled variance divided by a combined sample size. Each sample mean has its own variance, determined by that population’s variance and sample size. The square root is taken after adding those two variance contributions.

Using the Standard Error in a Confidence Interval

For independent normal populations with known variances, the standardized difference $(D-\Delta)/\operatorname{SE}(D)$ has a standard normal distribution. Let $z_{1-\alpha/2}$ be the standard normal critical value with probability $1-\alpha/2$ to its left. A two-sided confidence interval with confidence level $1-\alpha$ is

D\ \pm\ z_{1-\alpha/2} \sqrt{\frac{\sigma_1^2}{n_1}+\frac{\sigma_2^2}{n_2}}.

The same interval is exact if both populations are normal. With nonnormal populations, it is generally used as an approximate interval when the samples are large enough for the difference of sample means to be approximately normal. If the sampling assumptions or the normal approximation are not defensible, this interval may not have its stated coverage.

Worked Example: A 90% Interval With Unequal Sample Sizes

Two independent normal populations have known standard deviations of 6 and 8. Independent samples of sizes 36 and 64 give sample means 14 and 10, respectively. Find a 90% confidence interval for $\mu_1-\mu_2$, using $z_{0.95}=1.645$. Round endpoints to two decimal places.

Method and assumptions. The populations are normal, the samples are independent, and both population standard deviations are known. Therefore the known-variance Z interval applies. The sample difference estimates $\mu_1-\mu_2$, in the order stated.

Step 1: Calculate the observed difference. $$ D=\bar{x}-\bar{y}=14-10=4. $$

Step 2: Calculate the standard error. The first sample contributes $6^2/36=36/36=1$. The second contributes $8^2/64=64/64=1$. Hence $$ \operatorname{SE}(D)=\sqrt{1+1}=\sqrt{2}\approx1.4142. $$

Step 3: Calculate the margin of error. $$ 1.645\sqrt{2}\approx1.645(1.4142)\approx2.3264. $$

Step 4: Form and round the endpoints. $$ 4-2.3264=1.6736,\qquad 4+2.3264=6.3264. $$ The 90% confidence interval is $(1.67,\ 6.33)$.

Check. The interval is centered at 4, since $(1.67+6.33)/2=4.00$, and its half-width is approximately 2.33, matching the calculated margin of error. The common mistake this example avoids is treating the observations as one combined sample and using the total sample size in a single variance term.

Worked Example: A 95% Interval With Different Population Variances

A hypothetical laboratory compares the mean output, in milliliters, of two independent production methods. The populations are normal, with known standard deviations 3 milliliters and 5 milliliters. Samples of sizes 25 and 100 have means 52 and 49 milliliters. Construct a 95% confidence interval for the first method’s population mean minus the second method’s mean. Use $z_{0.975}=1.96$, and round endpoints to two decimal places.

Method and assumptions. Since both populations are normal, the samples are independent, and the variances are known, use the exact known-variance Z interval.

Step 1: Find the sample difference. $$ D=52-49=3\text{ milliliters}. $$

Step 2: Find each contribution to the variance. $$ \frac{\sigma_1^2}{n_1}=\frac{3^2}{25}=\frac{9}{25}=0.36,\qquad \frac{\sigma_2^2}{n_2}=\frac{5^2}{100}=\frac{25}{100}=0.25. $$ Therefore, $$ \operatorname{SE}(D)=\sqrt{0.36+0.25}=\sqrt{0.61}\approx0.7810\text{ milliliters}. $$

Step 3: Find the margin of error. $$ 1.96(0.7810)\approx1.5308\text{ milliliters}. $$

Step 4: Calculate the interval. $$ 3-1.5308=1.4692,\qquad 3+1.5308=4.5308. $$ The 95% confidence interval is $(1.47,\ 4.53)$ milliliters.

Check. The midpoint is 3.00 milliliters, the observed difference, and the endpoints are about 1.53 milliliters from that midpoint. The two variance contributions are not equal: the second population is more variable, but its larger sample partly offsets that variability.

How Sample Size Affects the Standard Error

For fixed population variances, increasing either sample size reduces its contribution $\sigma_i^2/n_i$ to the variance of the difference. It does not remove the other sample’s contribution. In particular, making one sample extremely large cannot drive the overall standard error below the contribution from the other sample.

Worked Example: Comparing Two Sampling Plans

Two independent normal populations have known standard deviations 4 and 6 units. In the initial plan, sample sizes are 16 and 36, and the sample means are 31 and 28. Find a 90% confidence interval for $\mu_1-\mu_2$, using $z_{0.95}=1.645$. Then calculate the standard error and interval margin if the first sample size is doubled to 32 while all other quantities, including the observed difference, are held fixed. Round the intervals to two decimal places.

Method and assumptions. Use the known-variance Z interval for the initial plan. For the comparison, change only $n_1$ in the standard-error formula; the second sample’s variance contribution remains unchanged.

Initial plan, step 1: Find the observed difference and standard error. $$ D=31-28=3. $$ The two variance contributions are $4^2/16=16/16=1$ and $6^2/36=36/36=1$, so $$ \operatorname{SE}(D)=\sqrt{1+1}=\sqrt{2}\approx1.4142. $$

Initial plan, step 2: Find the margin and interval. $$ 1.645\sqrt{2}\approx2.3264, \qquad (3-2.3264,\ 3+2.3264)=(0.6736,\ 5.3264). $$ The initial 90% interval is $(0.67,\ 5.33)$.

Revised plan: Recalculate the standard error. Doubling $n_1$ from 16 to 32 changes the first contribution to $$ \frac{4^2}{32}=\frac{16}{32}=0.5. $$ The second contribution remains $36/36=1$. Thus $$ \operatorname{SE}_{\mathrm{new}}(D)=\sqrt{0.5+1}=\sqrt{1.5}\approx1.2247. $$ The revised margin is $1.645(1.2247)\approx2.0147$, giving the interval $$ (3-2.0147,\ 3+2.0147)=(0.9853,\ 5.0147), $$ or $(0.99,\ 5.01)$ to two decimal places.

Check. The revised standard error is smaller than the initial standard error, and the revised interval is narrower. Its width decreases from approximately $2(2.3264)=4.6528$ to $2(2.0147)=4.0294$. It does not decrease by half, because only one of the two variance contributions was reduced.

Common Mistakes and Conditions to Check

The formula is simple, but applying it correctly requires checking the sampling structure and keeping the two sample contributions distinct.

Using one combined sample size. Do not add the sample sizes and divide a combined variance by that total. The formula has one term for each sample: $\sigma_1^2/n_1+\sigma_2^2/n_2$.
Adding standard errors instead of variances. The correct calculation is the square root of the sum of the two variances of the sample means. It is not $\sigma_1/\sqrt{n_1}+\sigma_2/\sqrt{n_2}$.
Forgetting the covariance condition. The variance formula for a difference includes $-2\operatorname{Cov}(\bar{X},\bar{Y})$. Independence makes this term zero. Without independence, the stated standard-error formula does not generally apply.
Mixing up the direction of the difference. An interval for $\mu_1-\mu_2$ must use $\bar{x}-\bar{y}$. Reversing the order changes the sign of the center and the meaning of the interval.
Calling an approximation exact. The interval is exact under independent normal sampling with known variances. With nonnormal populations, the normal-based interval may be approximate when sample sizes justify a central limit theorem approximation.

Key takeaway: For independent samples, the variance of the difference of sample means is the sum of the two sample-mean variances. Take the square root for the standard error, then multiply by the appropriate normal critical value for a known-variance confidence interval.

Check Your Understanding

For each problem, keep the order of the population means as stated and use the given normal critical value where needed.

Independent normal populations have known standard deviations 2 and 3. Samples of sizes 16 and 36 have means 20 and 18. Find a 90% confidence interval for $\mu_1-\mu_2$, using $z_{0.95}=1.645$. Round endpoints to two decimals. (Answer: $(0.84,\ 3.16)$)
Independent normal populations have known standard deviations 10 and 6. Sample sizes are 25 and 36, and the sample means are 75 and 70. Find a 95% confidence interval for $\mu_1-\mu_2$, using $z_{0.975}=1.96$. Round endpoints to two decimals. (Answer: $(0.62,\ 9.38)$)
Two independent normal populations have known standard deviations 8 and 12. Sample sizes are 64 and 36, and sample means are 42 and 45. Find a 99% confidence interval for $\mu_1-\mu_2$, using $z_{0.995}=2.576$. Round endpoints to two decimals. (Answer: $(-8.76,\ 2.76)$)
Two independent normal populations have known standard deviations 5 and 10. The first sample size is fixed at 25. What is the smallest second sample size that makes the margin of error for a 95% confidence interval no greater than 3? Use $z_{0.975}=1.96$. (Answer: $n_2=75$)

What You'll Learn