Common Errors With Pooled and Unpooled Standard Errors

Why These Errors Matter

Two-proportion inference compares the proportions of individuals with an outcome in two groups. The arithmetic can look similar for a confidence interval and a hypothesis test, but their standard errors answer different questions. Using a pooled standard error in a standard confidence interval, or an unpooled one in a test of equal proportions, changes the calculation and can change the conclusion.

Other mistakes start before the standard error is calculated. The difference depends on which group is subtracted from which, and counts cannot be compared as if they were proportions when the group sizes differ. A reliable calculation therefore begins by naming the groups, writing their sample proportions, and stating the inferential task.

The Setup and the Two Standard Errors

Let group 1 have $x_1$ individuals with the outcome among $n_1$ observations, and let group 2 have $x_2$ among $n_2$. Their sample proportions are $\hat{p}_1=x_1/n_1$ and $\hat{p}_2=x_2/n_2$. Throughout, define the estimated difference in the order group 1 minus group 2:

\hat{p}_1-\hat{p}_2

The target parameter is $p_1-p_2$, where $p_1$ and $p_2$ are the corresponding population proportions. The standard errors below use the normal approximation for two independent groups. This approximation is appropriate when the observations are independent within and between groups and the relevant numbers of outcomes and non-outcomes are sufficiently large. In practice, the sampling or assignment design must also support the intended population or treatment interpretation.

Definition: For the usual Wald confidence interval for $p_1-p_2$, the unpooled standard error is $$ SE_{\text{unpooled}}= \sqrt{\frac{\hat{p}_1(1-\hat{p}_1)}{n_1} +\frac{\hat{p}_2(1-\hat{p}_2)}{n_2}}. $$ For a test of $H_0:p_1=p_2$, the pooled standard error is $$ SE_{\text{pooled}}= \sqrt{\hat{p}_{\text{pool}}(1-\hat{p}_{\text{pool}}) \left(\frac{1}{n_1}+\frac{1}{n_2}\right)}, \qquad \hat{p}_{\text{pool}}=\frac{x_1+x_2}{n_1+n_2}. $$

The usual two-sided $100(1-\alpha)\%$ confidence interval is $(\hat{p}_1-\hat{p}_2)\pm z^*SE_{\text{unpooled}}$, where $z^*$ is the standard normal critical value for the chosen confidence level. For example, $z^*\approx1.96$ for a 95% interval. For a test of $H_0:p_1=p_2$, the test statistic is $z=(\hat{p}_1-\hat{p}_2)/SE_{\text{pooled}}$. A one-sided alternative uses the appropriate tail of the standard normal distribution; a two-sided alternative uses both tails.

Why the Choice Changes

The difference between the formulas follows from the question each procedure asks. For independent Bernoulli observations within a group, a sample proportion has variance $p(1-p)/n$. Independence between groups means the variance of the difference is the sum of the two group variances:

\operatorname{Var}(\hat{p}_1-\hat{p}_2) =\operatorname{Var}(\hat{p}_1)+\operatorname{Var}(\hat{p}_2) =\frac{p_1(1-p_1)}{n_1}+\frac{p_2(1-p_2)}{n_2}.

For a confidence interval, the difference $p_1-p_2$ is not assumed to be zero. The two population proportions are estimated separately by $\hat{p}_1$ and $\hat{p}_2$, giving the unpooled standard error.

For a test of $H_0:p_1=p_2$, the null hypothesis says both groups share a common proportion, call it $p$. Under that restriction, the variance of the difference becomes $p(1-p)(1/n_1+1/n_2)$. Since $p$ is unknown, the null model estimates it using all observed outcomes: $\hat{p}_{\text{pool}}=(x_1+x_2)/(n_1+n_2)$. Substituting this estimate gives the pooled standard error. Pooling is therefore justified by the equality imposed in the null hypothesis; it is not a general rule for any two-group calculation.

Worked Example: Use Different Standard Errors for an Interval and a Test

Suppose a hypothetical comparison records the outcome for 30 of 200 individuals in group 1 and 12 of 150 in group 2. Assume the groups are independent and the observations within each group are independent. Each group has at least 10 outcomes and 10 non-outcomes, so the usual large-count check is met.

The sample proportions and estimated difference are $$ \hat{p}_1=\frac{30}{200}=0.15,\qquad \hat{p}_2=\frac{12}{150}=0.08,\qquad \hat{p}_1-\hat{p}_2=0.07. $$ The estimated outcome proportion is 7 percentage points higher in group 1.

Confidence interval. Use the separate sample proportions in the unpooled standard error: $$ SE_{\text{unpooled}} =\sqrt{\frac{0.15(0.85)}{200}+\frac{0.08(0.92)}{150}} =\sqrt{0.0006375+0.000490667} \approx0.03359. $$ A 95% interval is $$ 0.07\pm1.96(0.03359) =0.07\pm0.06583 \approx(0.00417,\ 0.1358). $$ On the difference scale, this is approximately 0.417 to 13.58 percentage points. The interval estimates the population difference with group 1 first.

Test of equal proportions. Under $H_0:p_1=p_2$, pool the outcomes: $$ \hat{p}_{\text{pool}}=\frac{30+12}{200+150}=\frac{42}{350}=0.12. $$ Thus $$ SE_{\text{pooled}} =\sqrt{0.12(0.88)\left(\frac{1}{200}+\frac{1}{150}\right)} =\sqrt{0.001232}=0.03510, \qquad z=\frac{0.07}{0.03510}\approx1.994. $$ For a two-sided test, the standard normal p-value is approximately $0.0461$. At significance level $\alpha=0.05$, this is evidence against equal population proportions, under the stated model and conditions. The test uses the pooled standard error because it evaluates the difference under the equality null; the interval uses the unpooled standard error because it estimates a difference that need not be zero.

Common Errors and How to Correct Them

A useful check is to ask what assumption is being used at the point where the standard error is calculated. If the calculation is for an interval estimating an unrestricted difference, use the two separate sample proportions. If it is for a test under $H_0:p_1=p_2$, calculate the pooled proportion first.

Pooling for the standard confidence interval. The mistaken reasoning is, “There are two groups, so I should combine their outcomes before estimating variability.” That imposes a common proportion even though the interval does not assume equality. For the usual Wald interval, calculate each group’s variance contribution using its own $\hat{p}$.
Using separate estimates for a test of equality. The mistaken reasoning is, “Each group has its own sample proportion, so those must go into the test standard error.” Under the null of equal population proportions, the test estimates one common proportion from the combined data. Use the pooled standard error for this null-based test.
Reversing the group order midway. The mistaken reasoning is to calculate $\hat{p}_1-\hat{p}_2$, but describe the result as group 2 minus group 1. Write the group order explicitly and keep it fixed. Reversing the order negates the estimated difference and swaps and negates the interval endpoints; it does not change a two-sided test’s p-value.
Subtracting counts instead of proportions. A difference such as $x_1-x_2$ is a difference in numbers of observed outcomes, not a difference in population proportions. When sample sizes differ, counts especially can give a misleading comparison. Calculate $x_1/n_1$ and $x_2/n_2$ first.
Pooling because the sample proportions look close. Similar observed proportions do not by themselves justify pooling for an interval. For the standard test, pooling follows from the specified equality null, not from a preliminary visual judgment about the data.

Worked Example: Do Not Compare Counts When Group Sizes Differ

In a hypothetical comparison, 40 of 200 individuals in group 1 and 30 of 100 in group 2 have an outcome. The raw counts differ by 10, but the group sizes differ too. Both groups have at least 10 outcomes and 10 non-outcomes, and assume independent observations and groups.

The proportions, in the order group 1 minus group 2, are $$ \hat{p}_1=\frac{40}{200}=0.20,\qquad \hat{p}_2=\frac{30}{100}=0.30,\qquad \hat{p}_1-\hat{p}_2=-0.10. $$ The estimated proportion is 10 percentage points lower in group 1. The count difference $40-30=10$ does not express that comparison: group 1 has twice the sample size.

For a 95% confidence interval, use the unpooled standard error: $$ SE_{\text{unpooled}} =\sqrt{\frac{0.20(0.80)}{200}+\frac{0.30(0.70)}{100}} =\sqrt{0.0008+0.0021} \approx0.05385. $$ Therefore, $$ -0.10\pm1.96(0.05385) =-0.10\pm0.1055 \approx(-0.2055,\ 0.00555). $$ The interval is approximately -20.55 to 0.555 percentage points for group 1 minus group 2. It includes zero.

For a test of equal proportions, pool the outcomes instead: $$ \hat{p}_{\text{pool}}=\frac{40+30}{200+100}=\frac{70}{300}\approx0.2333, $$ $$ SE_{\text{pooled}} =\sqrt{0.2333(0.7667)\left(\frac{1}{200}+\frac{1}{100}\right)} \approx0.05180, \qquad z=\frac{-0.10}{0.05180}\approx-1.931. $$ The two-sided p-value is approximately $0.0535$. At $\alpha=0.05$, this test does not provide sufficient evidence against equal proportions. The conclusion is not that the proportions are proven equal; the data do not cross the chosen significance threshold.

Worked Example: Reverse the Group Order Consistently

Return to the first comparison: group 1 had 30 outcomes among 200, and group 2 had 12 among 150. If the research question instead defines the difference as group 2 minus group 1, the estimate is $$ \hat{p}_2-\hat{p}_1=0.08-0.15=-0.07. $$ The unpooled standard error is unchanged because its two variance terms are added. Thus the 95% interval in the reversed order is $$ -0.07\pm1.96(0.03359)\approx(-0.1358,\ -0.00417). $$ This is exactly the sign-reversed version of the first interval: its endpoints are reversed and negated. The estimated outcome proportion is 7 percentage points lower in group 2, rather than 7 percentage points higher in group 1. For a two-sided equality test, reversing the order changes $z$ from about $1.994$ to $-1.994$, but leaves the p-value unchanged because the two-sided p-value depends on the magnitude of $z$.

A Practical Check Before Reporting

Before interpreting a result, verify that the numerical procedure matches the question. In particular, do not describe a difference without saying which group comes first, and include the proportion scale and outcome in context. A difference of $0.07$ means 7 percentage points, not a 7% relative increase. Ratio measures such as relative risk answer a different question and are not interchangeable with a difference in proportions.

Name the groups and order.
Write “group 1 minus group 2” (or the reverse) before calculating.

Convert counts to proportions.
Calculate $x_1/n_1$ and $x_2/n_2$, then subtract in the stated order.

Match the standard error to the task.
Use unpooled estimates for the usual difference confidence interval; use the pooled estimate for a test of equal proportions.

Check conditions and interpretation.
Confirm independent observations and adequate counts for the normal approximation, then report the result in context and with units.

Common pitfall: “I pooled for both calculations because the groups are being compared.” The correction is that the procedure depends on the inferential claim. A confidence interval estimates a potentially nonzero difference and uses separate estimates for its standard error. A test of equality evaluates data under a null model that gives the groups a common proportion, so its standard error uses a pooled estimate. In either case, counts must first be converted to proportions, and the group order must be maintained through the calculation and its interpretation.

Check Your Understanding

For each question, state the reasoning as well as the calculation choice.

For the usual Wald interval estimating $p_1-p_2$, should the standard error use separate sample proportions or a pooled proportion? Why?
A test has $H_0:p_1=p_2$. Write the pooled estimate in terms of $x_1,x_2,n_1,n_2$.
Two groups have outcome counts 18 and 12, but their sample sizes are 60 and 30. Why is $18-12$ not the estimated difference in proportions?
If an interval for $p_1-p_2$ is $(0.02,0.10)$, what is the interval for $p_2-p_1$?
For a two-sided test, what happens to the test statistic and p-value if the group order is reversed?