Two-Sample Z Tests for Independent Means

Why Compare Two Independent Means?

A comparison of two means asks whether the observed difference between two samples is persuasive evidence of a difference between their population means. For example, a hypothetical actuary might compare average processing times under two independent procedures. Even if the sample means differ, sampling variation can produce a difference when the population means are equal. A two-sample Z test measures the observed difference relative to its expected sampling variability.

This procedure is specifically for independent samples with known population variances. It is aligned with the SOA VEE Mathematical Statistics syllabus. If the population variances are unknown, this is not the appropriate procedure; a t procedure is needed instead.

Definition: Let \(X_1,\ldots,X_{n_1}\) be a random sample from a population with mean \(\mu_1\) and known variance \(\sigma_1^2\), and let \(Y_1,\ldots,Y_{n_2}\) be a random sample from a population with mean \(\mu_2\) and known variance \(\sigma_2^2\). The two samples must be independent of one another, and observations within each sample must be independent. A two-sample Z test tests a hypothesis about \(\mu_1-\mu_2\), using the known variances to standardize the difference in sample means.

Model, Assumptions, and Test Statistic

The observations in each sample must have the stated population mean and variance. Exact Z-test calculations apply when both populations are normally distributed. If the populations are not normal, the test can be used as an approximation when both sample sizes are sufficiently large for the sample means to be approximately normal. A small sample from a substantially nonnormal population does not automatically justify this test.

The samples must also be independent. For instance, measuring the same individuals under two conditions does not produce independent samples: the two measurements from each individual are related. That design requires a paired analysis, not the independent-means procedure described here.

For a hypothesized difference \(\Delta_0\), the usual hypotheses are \(H_0:\mu_1-\mu_2=\Delta_0\) against one of three alternatives: \(H_1:\mu_1-\mu_2\ne\Delta_0\), \(H_1:\mu_1-\mu_2>\Delta_0\), or \(H_1:\mu_1-\mu_2<\Delta_0\). The null value is often zero, but it need not be. The standard error uses the population variances, not sample estimates.

\operatorname{SE}(\bar X-\bar Y) = \sqrt{\frac{\sigma_1^2}{n_1}+\frac{\sigma_2^2}{n_2}}, \qquad Z= \frac{(\bar X-\bar Y)-\Delta_0} {\sqrt{\sigma_1^2/n_1+\sigma_2^2/n_2}}.

Under the null hypothesis and the model assumptions, \(Z\) has a standard normal distribution. For two-sided alternatives, extreme values in either direction count against the null; for a greater-than alternative, evidence lies in the upper tail; for a less-than alternative, it lies in the lower tail. The p-value is the probability, calculated under the null, of a test statistic at least as unfavorable to the null as the observed statistic.

Why the Standard Error Has This Form

The standard error follows from the variance of a difference of independent sample means. Each sample mean averages independent observations, so its variance is its population variance divided by its sample size. Independence between the two samples means the covariance of their sample means is zero. Thus the variances add when one sample mean is subtracted from the other.

E[\bar X-\bar Y]=\mu_1-\mu_2, \qquad \operatorname{Var}(\bar X-\bar Y) = \operatorname{Var}(\bar X)+\operatorname{Var}(\bar Y) = \frac{\sigma_1^2}{n_1}+\frac{\sigma_2^2}{n_2}.

Under \(H_0\), the mean of \(\bar X-\bar Y\) is \(\Delta_0\). If both populations are normal, the difference of the independent sample means is normal. Subtracting its null mean and dividing by its standard deviation therefore gives a standard normal variable. With nonnormal populations and sufficiently large samples, the central limit theorem gives the corresponding approximate justification.

Theorem: If the two samples are independent and the sample means are normal (exactly, under normal populations, or approximately under suitable large-sample conditions), then under \(H_0:\mu_1-\mu_2=\Delta_0\), the statistic \(Z\) above has a standard normal distribution. Its standard error contains \(\sigma_1^2/n_1\) and \(\sigma_2^2/n_2\); the variances are not pooled.

Carrying Out the Test

A reliable solution keeps the direction of the difference fixed throughout: sample 1 minus sample 2 corresponds to population 1 minus population 2. Then compare the resulting statistic with the appropriate tail, or calculate the p-value and compare it with the chosen significance level \(\alpha\).

Define the difference and hypotheses.
Write the population quantity as \(\mu_1-\mu_2\), state the null difference \(\Delta_0\), and select the alternative that matches the question.

Check the model.
Confirm independent samples, known population variances, and normal populations or an adequate large-sample approximation.

Calculate the standard error and statistic.
Use the two separate variance contributions, then standardize the observed difference relative to \(\Delta_0\).

Find the p-value or critical value.
Use one tail for a directional alternative and both tails for a two-sided alternative.

Conclude in context.
Reject \(H_0\) when the p-value is at most \(\alpha\); otherwise, do not reject it. A non-rejection is not proof that the means are equal.

Worked Example: Two-Sided Test of Equal Means

Method: Use the independent two-sample Z test with \(\Delta_0=0\). In a hypothetical study of processing times, sample 1 has \(n_1=36\), \(\bar x=52\) minutes, and known population standard deviation \(\sigma_1=12\) minutes. An independent sample 2 has \(n_2=49\), \(\bar y=47\) minutes, and known population standard deviation \(\sigma_2=14\) minutes. Assume both populations are normal. Test \(H_0:\mu_1-\mu_2=0\) against \(H_1:\mu_1-\mu_2\ne0\) at \(\alpha=0.05\).

The observed difference is \(\bar x-\bar y=52-47=5\) minutes. The first variance contribution to the standard error is \(12^2/36=144/36=4\), and the second is \(14^2/49=196/49=4\). Therefore,

\operatorname{SE}=\sqrt{4+4}=\sqrt{8}=2.8284, \qquad Z=\frac{5-0}{2.8284}=1.7678.

For a two-sided test, the p-value is \(2P(Z_{\text{standard}}\ge 1.7678)\), approximately \(0.0772\). Since \(0.0772>0.05\), do not reject \(H_0\). The data do not provide sufficient evidence at the 5% level that the population mean processing times differ. This conclusion does not establish that the means are equal.

Check: The 5% two-sided critical values are approximately \(-1.96\) and \(1.96\). The observed statistic, \(1.7678\), lies between them, agreeing with the p-value decision. Common mistake avoided: treating a positive observed difference as automatically one-sided. The stated alternative is two-sided, so both tails must be counted.

Worked Example: A Directional Test

Method: Use an upper-tail Z test because the alternative predicts that population 1 has the larger mean. In a hypothetical comparison of two independent calibration methods, the first sample has \(n_1=25\), \(\bar x=108\) units, and known \(\sigma_1=10\) units. The second has \(n_2=16\), \(\bar y=100\) units, and known \(\sigma_2=8\) units. Assume normal populations. Test \(H_0:\mu_1-\mu_2=0\) against \(H_1:\mu_1-\mu_2>0\) at \(\alpha=0.01\).

The observed difference is \(108-100=8\) units. The variance contributions are \(10^2/25=100/25=4\) and \(8^2/16=64/16=4\), so the standard error is \(\sqrt{4+4}=\sqrt{8}=2.8284\). The test statistic is

Z=\frac{8-0}{2.8284}=2.8284.

The upper-tail p-value is \(P(Z_{\text{standard}}\ge2.8284)\), approximately \(0.0023\). Since \(0.0023<0.01\), reject \(H_0\). The data provide evidence that the first population mean exceeds the second. Check: The 1% upper-tail critical value is approximately \(2.326\), and \(2.8284>2.326\), confirming the decision.

Common mistake avoided: using a two-sided p-value just because a difference was calculated. The direction was specified before evaluating the data, so the relevant p-value is the upper-tail probability only.

Worked Example: Testing a Nonzero Difference

Method: Center the observed difference at the hypothesized difference, not at zero. In a hypothetical comparison of two independent service populations, sample 1 has \(n_1=64\), \(\bar x=31.5\) units, and known \(\sigma_1=8\) units. Sample 2 has \(n_2=25\), \(\bar y=28\) units, and known \(\sigma_2=5\) units. Assume normal populations. Test \(H_0:\mu_1-\mu_2=1\) against \(H_1:\mu_1-\mu_2\ne1\) at \(\alpha=0.05\).

The observed difference is \(31.5-28=3.5\). The variance contributions are \(8^2/64=64/64=1\) and \(5^2/25=25/25=1\), giving \(\operatorname{SE}=\sqrt{1+1}=\sqrt{2}=1.4142\). Under the null, the expected difference is 1, so

Z=\frac{3.5-1}{1.4142}=1.7678.

The two-sided p-value is approximately \(0.0772\), so do not reject \(H_0\) at the 5% level. As a check, a 95% confidence interval for \(\mu_1-\mu_2\) is

3.5\mathbin{\pm}1.96(1.4142) = 3.5\mathbin{\pm}2.7718 = [0.7282,\ 6.2718].

The hypothesized difference of 1 lies inside this interval, consistent with the test decision. Common mistake avoided: using \(3.5/1.4142\), which tests a zero difference. The correct numerator subtracts the null value: \(3.5-1\).

Common Mistakes and Interpretation

The known-variance assumption is substantive. In this procedure, \(\sigma_1^2\) and \(\sigma_2^2\) are specified population variances. Replacing them with sample variances changes the procedure and its reference distribution. Likewise, independence is not a cosmetic condition: if sample means are dependent, the variance of their difference includes a covariance term, so the displayed standard error may be wrong.

A frequent error is to pool the two known variances or average them before calculating the standard error. The samples may have different sizes and different population variances, and each contributes its own variance of a sample mean. For the first worked example, the incorrect shortcut \(\sqrt{(12^2+14^2)/(36+49)}=\sqrt{340/85}=2\) would give \(Z=5/2=2.5\), rather than the correct standard error \(\sqrt{12^2/36+14^2/49}=\sqrt{8}=2.8284\). That incorrect statistic would also lead to a different p-value and an erroneous rejection at the 5% level.

Other checks help prevent avoidable errors:

Keep the order consistent. If the statistic uses \(\bar x-\bar y\), define the parameter as \(\mu_1-\mu_2\) and write the hypotheses in the same order.
Match the tail to the alternative. A two-sided alternative uses both tails; a directional alternative uses the tail in its stated direction.
Keep units consistent. Both means and both standard deviations must refer to the same measurement scale.
Separate statistical evidence from practical importance. A small p-value does not by itself say whether a difference matters in the application.

A confidence interval can provide a useful companion to a two-sided test: at the corresponding confidence level, it contains the null difference exactly when the two-sided test does not reject that difference, subject to the same model and standard error. Neither a p-value nor a confidence interval repairs a violation of the independence or known-variance assumptions.

Key takeaway: For independent sample means with known population variances, standardize \((\bar x-\bar y)-\Delta_0\) by \(\sqrt{\sigma_1^2/n_1+\sigma_2^2/n_2}\). The null difference, alternative direction, assumptions, and tail calculation all matter to the conclusion.

Check Your Understanding

For each problem, identify the appropriate tail and calculate the requested result using the stated known variances. Round probabilities to four decimals.

A hypothetical two-sided comparison has independent normal samples with \(n_1=25\), \(\bar x=18\), \(\sigma_1=5\), \(n_2=36\), \(\bar y=16\), and \(\sigma_2=6\). Test equal means at \(\alpha=0.05\): find the standard error, Z statistic, p-value, and decision. (Answer: \(\operatorname{SE}=\sqrt{2}=1.4142\), \(Z=1.4142\), two-sided \(p=0.1573\); do not reject.)
For independent normal samples, \(n_1=49\), \(\bar x=72\), \(\sigma_1=7\), \(n_2=25\), \(\bar y=75\), and \(\sigma_2=10\). Test \(H_0:\mu_1-\mu_2=0\) against \(H_1:\mu_1-\mu_2<0\) at \(\alpha=0.05\). (Answer: \(\operatorname{SE}=\sqrt{5}=2.2361\), \(Z=-1.3416\), lower-tail \(p=0.0898\); do not reject.)
Independent normal samples give \(n_1=100\), \(\bar x=42\), \(\sigma_1=10\), \(n_2=64\), \(\bar y=39\), and \(\sigma_2=8\). Test \(H_0:\mu_1-\mu_2=2\) against a two-sided alternative. Find the statistic and p-value. (Answer: \(\operatorname{SE}=\sqrt{2}=1.4142\), \(Z=(3-2)/1.4142=0.7071\), two-sided \(p=0.4795\); do not reject at \(\alpha=0.05\).)
Independent normal samples have \(n_1=16\), \(\bar x=11.2\), \(\sigma_1=2\), \(n_2=25\), \(\bar y=9.8\), and \(\sigma_2=3\). Construct a 95% confidence interval for \(\mu_1-\mu_2\) and determine whether it includes zero. (Answer: \(\operatorname{SE}=\sqrt{0.61}=0.7810\); interval \(1.4\mathbin{\pm}1.96(0.7810)=[-0.1308,\ 2.9308]\), which includes zero.)
A hypothetical one-sided comparison has independent normal samples with \(n_1=40\), \(\bar x=103\), \(\sigma_1=12\), \(n_2=36\), \(\bar y=99\), and \(\sigma_2=9\). Test \(H_0:\mu_1-\mu_2=0\) against \(H_1:\mu_1-\mu_2>0\) at \(\alpha=0.05\). (Answer: \(\operatorname{SE}=\sqrt{5.85}=2.4187\), \(Z=4/2.4187=1.6538\), upper-tail \(p=0.0491\); reject, narrowly.)

What You'll Learn