Unequal Sample Sizes In Difference Intervals

Why Unequal Group Sizes Matter

When two independent groups are used to compare proportions, their sample sizes do not have to match. For example, a hypothetical benefits administrator might compare the proportions of two independently sampled member groups who completed an optional wellness screening. If one group has many more observations than the other, the larger group will generally estimate its proportion more precisely. The uncertainty in the comparison depends on both groups, but not necessarily in equal amounts.

This tutorial develops the Wald confidence interval for the difference between two independent population proportions with unequal sample sizes. The central calculation is the standard error: it combines a separate estimated variance from each group. Knowing the two contributions helps identify which sample is limiting the precision of the comparison.

Definition: Let \(p_1\) and \(p_2\) be the success probabilities in two populations. From independent random samples of sizes \(n_1\) and \(n_2\), let \(X_1\) and \(X_2\) be the observed numbers of successes, and define \(\hat{p}_1=X_1/n_1\) and \(\hat{p}_2=X_2/n_2\). The parameter of interest is \(p_1-p_2\), estimated by \(\hat{p}_1-\hat{p}_2\).

A success must be defined in the same way for both populations. The samples must be independent of one another, and each sample must be suitable for estimating its population proportion. For the Wald interval, the normal approximation must also be adequate in each group; a common working check is that each group's expected success and failure counts, \(n_j\hat{p}_j\) and \(n_j(1-\hat{p}_j)\), are not small. This interval can perform poorly when these counts are small or a sample proportion is near zero or one.

The Unequal-Size Wald Interval

For independent samples, estimate the variance of each sample proportion separately. The estimated variance for group \(j\) is \(\hat{p}_j(1-\hat{p}_j)/n_j\). Independence means that the variance of the difference is the sum of the two variances, so the estimated standard error is

\widehat{\operatorname{SE}}(\hat{p}_1-\hat{p}_2) = \sqrt{ \frac{\hat{p}_1(1-\hat{p}_1)}{n_1} + \frac{\hat{p}_2(1-\hat{p}_2)}{n_2} }.

For a confidence level of \(100(1-\alpha)\%\), the Wald interval is the estimated difference plus and minus the standard normal critical value \(z_{1-\alpha/2}\) times this standard error:

(\hat{p}_1-\hat{p}_2) \ \pm\ z_{1-\alpha/2} \widehat{\operatorname{SE}}(\hat{p}_1-\hat{p}_2).

For a 95% interval, \(z_{0.975}\) is approximately 1.96. The order of subtraction matters: the interval estimates \(p_1-p_2\), so switching the group labels reverses the sign of both the estimate and the interval's endpoints.

Key result: The estimated variance contribution from group \(j\) is \(\hat{p}_j(1-\hat{p}_j)/n_j\). Its share of the total estimated variance is that contribution divided by the sum of both contributions. The larger share identifies the group contributing more to the interval's uncertainty.

Why the Variances Add

The standard error formula follows from the variance of a difference of independent random variables. If the population indicators in group \(j\) are independent Bernoulli observations with success probability \(p_j\), then the sample proportion is their average. Its variance is \(p_j(1-p_j)/n_j\). The two sample proportions are independent because they come from independent samples.

\begin{aligned} \operatorname{Var}(\hat{p}_1-\hat{p}_2) &=\operatorname{Var}(\hat{p}_1)+\operatorname{Var}(\hat{p}_2) -2\operatorname{Cov}(\hat{p}_1,\hat{p}_2)\\ &=\frac{p_1(1-p_1)}{n_1}+\frac{p_2(1-p_2)}{n_2}, \end{aligned}

The covariance term is zero by independence. Replacing each unknown population proportion \(p_j\) with its sample estimate \(\hat{p}_j\) gives the estimated variance used in the Wald standard error. Taking its square root gives the estimated standard deviation of the difference. The interval then uses the normal approximation to place a margin of error around the observed difference.

Worked Calculations

Worked Example: A Larger Second Sample

In a hypothetical study, 54 of 120 members in group 1 and 70 of 200 members in group 2 complete a voluntary screening. Construct a 95% Wald interval for \(p_1-p_2\), and calculate each group's share of the estimated variance. Treat the two samples as independent and assume the normal approximation is adequate.

Method. Calculate each sample proportion, form the unpooled standard error using the two separate terms, and apply the 95% critical value. This avoids treating the unequal groups as if they had the same sample size.

The sample proportions and estimated difference are

\hat{p}_1=\frac{54}{120}=0.45,\qquad \hat{p}_2=\frac{70}{200}=0.35,\qquad \hat{p}_1-\hat{p}_2=0.10.

The group 1 variance contribution is \(0.45(0.55)/120=0.2475/120=0.0020625\). The group 2 contribution is \(0.35(0.65)/200=0.2275/200=0.0011375\). Therefore,

\widehat{\operatorname{SE}} =\sqrt{0.0020625+0.0011375} =\sqrt{0.0032} =0.05657.

The 95% margin of error is \(1.96(0.05657)=0.1109\). Thus the interval is \(0.10\pm0.1109\), or \((-0.0109,\ 0.2109)\), rounded to four decimals.

The total estimated variance is 0.0032. Group 1 contributes \(0.0020625/0.0032=0.6445\), or 64.45%; group 2 contributes \(0.0011375/0.0032=0.3555\), or 35.55%. The smaller first sample contributes more uncertainty here. Check: the variance shares sum to 100%, and the interval midpoint is 0.10, the estimated difference.

Worked Example: A Small Group Dominates the Uncertainty

A hypothetical audit compares two independent groups. In group 1, 48 of 80 sampled files meet a specified documentation standard. In group 2, 80 of 200 sampled files meet it. Find a 95% Wald interval for \(p_1-p_2\) and assess the variance contributions. Assume the normal approximation is adequate.

Method. Use the unpooled standard error because each group's proportion is estimated from its own sample. The common mistake to avoid is dividing both variance terms by one sample size or averaging the sample sizes.

The proportions are \(48/80=0.60\) and \(80/200=0.40\); the estimated difference is \(0.60-0.40=0.20\). The group 1 contribution is \(0.60(0.40)/80=0.24/80=0.0030\). The group 2 contribution is \(0.40(0.60)/200=0.24/200=0.0012\). Hence,

\widehat{\operatorname{SE}} =\sqrt{0.0030+0.0012} =\sqrt{0.0042} =0.06481.

The 95% margin is \(1.96(0.06481)=0.1270\), so the interval is \(0.20\pm0.1270=(0.0730,\ 0.3270)\), to four decimals. The group 1 share is \(0.0030/0.0042=0.7143\), or 71.43%; group 2 contributes 28.57%.

Both estimated proportions are equally far from 0.5, so their \(\hat{p}(1-\hat{p})\) factors are equal. The smaller sample's variance contribution is larger because its denominator is smaller. Check: the standard error exceeds either contribution's square root alone, as required when independent variance contributions are added.

Worked Example: The Smaller Group Has the Larger Proportion

In a hypothetical independent survey, 45 of 150 participants in group 1 and 12 of 60 participants in group 2 report using a particular service. Construct a 90% Wald interval for \(p_1-p_2\), and determine which group contributes more to estimated variance. Assume the normal approximation is adequate.

Method. Calculate the two proportions and their separate contributions. This example shows that the smaller sample does not automatically dominate: its estimated proportion also affects the variance factor.

The estimates are \(\hat{p}_1=45/150=0.30\) and \(\hat{p}_2=12/60=0.20\), so their difference is 0.10. The group 1 term is \(0.30(0.70)/150=0.21/150=0.0014\). The group 2 term is \(0.20(0.80)/60=0.16/60=0.0026667\). Thus,

\widehat{\operatorname{SE}} =\sqrt{0.0014+0.0026667} =\sqrt{0.0040667} =0.06377.

For a 90% interval, \(z_{0.95}\) is approximately 1.645. The margin is \(1.645(0.06377)=0.1049\), giving \(0.10\pm0.1049=(-0.0049,\ 0.2049)\), to four decimals.

Group 2's share is \(0.0026667/0.0040667=0.6557\), or 65.57%; group 1 contributes 34.43%. Group 2 contributes more because its sample is smaller and its estimated \(\hat{p}(1-\hat{p})\) factor is also larger: 0.16 rather than 0.21? In this case 0.16 is smaller; the smaller denominator more than offsets that. Check: the two variance shares sum to 100%, and the 90% margin is less than the 95% margin would be for the same data.

Reading the Contributions Correctly

For each group, uncertainty is governed jointly by sample size and the estimated variability of a Bernoulli observation, \(\hat{p}_j(1-\hat{p}_j)\). This factor is largest at a proportion of 0.5 and becomes smaller as the proportion approaches 0 or 1. A larger group usually has a smaller contribution if its estimated proportion is comparable to that of the smaller group, but sample size alone does not determine the ranking.

The contribution shares describe the estimated variance, not the standard error itself. If a contribution is 70% of total variance, it does not mean that group is responsible for 70% of the standard error. The standard error is the square root of total variance. For comparing groups' influence on the variance, divide each variance term by the sum of both terms.

Quantity	Calculation	Meaning
Group 1 estimated variance	\(\hat{p}_1(1-\hat{p}_1)/n_1\)	Group 1's contribution to uncertainty in the difference
Group 2 estimated variance	\(\hat{p}_2(1-\hat{p}_2)/n_2\)	Group 2's contribution to uncertainty in the difference
Total estimated variance	Sum of the two contributions	Square of the unpooled standard error
Contribution share	Group term divided by total	Proportion of total estimated variance attributable to that group

Common Exam Trap

A frequent error is to use the pooled proportion in the confidence-interval standard error. For an interval estimating \(p_1-p_2\), the unpooled formula uses \(\hat{p}_1\) in the first variance term and \(\hat{p}_2\) in the second. A pooled estimate is associated with a different calculation and is not substituted into this Wald interval. Another error is to assume equal group sizes merely because the samples are being compared.

For example, in the first worked example, using a pooled proportion of \(124/320=0.3875\) and then dividing by a common size would discard the actual group-specific sample sizes and sample proportions. The correct variance is \(0.45(0.55)/120+0.35(0.65)/200=0.0032\). Always retain \(n_1\) with group 1's term and \(n_2\) with group 2's term.

A Wald interval can extend beyond the possible parameter range \([-1,1]\), and its approximation may be unreliable with sparse counts. These limitations do not change the arithmetic, but they are reasons to check the method's conditions rather than treating every computed interval as equally dependable.

Key takeaway: For independent proportions with unequal sample sizes, estimate \(p_1-p_2\) by \(\hat{p}_1-\hat{p}_2\), add the two separate estimated variance contributions, and multiply the square root of their sum by the appropriate normal critical value. Compare the variance terms to see which group contributes more uncertainty.

Check Your Understanding

For each problem, use the unpooled Wald interval and round interval endpoints and probabilities to four decimals.

A hypothetical study observes 40 successes among 100 sampled units in group 1 and 90 among 150 in group 2. Find the 95% interval for \(p_1-p_2\). (Answer: \((-0.3240,\ -0.0760)\))
In two independent samples, group 1 has 35 successes among 50 observations and group 2 has 50 among 100. Find a 90% interval for \(p_1-p_2\). (Answer: \((0.0654,\ 0.3346)\))
Group 1 has 90 successes among 300 observations; group 2 has 30 among 75. Calculate each group's percentage contribution to the estimated variance. (Answer: group 1, 17.95%; group 2, 82.05%)
A hypothetical comparison records 60 successes among 120 in group 1 and 24 among 80 in group 2. Find a 95% interval for \(p_1-p_2\), and identify the larger variance contribution. (Answer: \((0.0655,\ 0.3345)\); group 2 contributes more.)
Group 1 has 27 successes among 90 observations, while group 2 has 90 among 180. For \(p_1-p_2\), find the 95% interval and each group's approximate variance share. (Answer: \((-0.3196,\ -0.0804)\); group 1, 62.7%, group 2, 37.3%)

What You'll Learn