Two-Proportion Z Tests With a Pooled Null Estimate

Why Pool the Estimate Under the Null?

A comparison of two proportions asks whether the success probabilities in two populations differ. Examples include comparing the proportions of applicants who complete a hypothetical training program, or the proportions of policyholders who select one of two hypothetical coverage options. When the null hypothesis says the population proportions are equal, the data should be analyzed under that shared-proportion assumption. That is why the standard error for the test uses a pooled estimate, rather than estimating a separate proportion for each group.

The method is designed for two independent groups with a binary outcome. It is a large-sample test: before interpreting its normal-approximation p-value, check that the pooled expected success and failure counts are sufficiently large in each group.

Definition: Let \(n_1\) and \(n_2\) be the sample sizes from two independent populations, and let \(x_1\) and \(x_2\) be the numbers of successes. The sample proportions are \(\hat p_1=x_1/n_1\) and \(\hat p_2=x_2/n_2\). The two-proportion pooled z test assesses a hypothesis about \(p_1-p_2\), where \(p_1\) and \(p_2\) are the population success proportions. Under the null hypothesis \(H_0:p_1=p_2\), its pooled estimate is \(\hat p=(x_1+x_2)/(n_1+n_2)\).

Model, Assumptions, and Test Statistic

Model each observation as a success or failure. Within group \(i\), the \(n_i\) observations must be independent and have the same success probability \(p_i\), so the success count has a binomial distribution. The two groups must also be independent of one another. In particular, this test is not for matched pairs, repeated measurements on the same individuals, or clustered observations treated as if they were independent.

For a two-sided test, the hypotheses are \(H_0:p_1=p_2\) against \(H_A:p_1\ne p_2\). For a directional question, use \(H_A:p_1>p_2\) or \(H_A:p_1<p_2\). Choose the direction based on the question before examining the results. The null hypothesis specifies a common proportion \(p\), which is estimated by combining the successes and sample sizes from both groups.

Theorem: Under \(H_0:p_1=p_2=p\), estimate the common proportion by \(\hat p=(x_1+x_2)/(n_1+n_2)\). The pooled standard error and test statistic are

SE_0=\sqrt{\hat p(1-\hat p)\left(\frac{1}{n_1}+\frac{1}{n_2}\right)}, \qquad z=\frac{\hat p_1-\hat p_2}{SE_0}.

When the null model and large-sample conditions are appropriate, \(z\) is compared with the standard normal distribution. For a two-sided alternative, the p-value is \(2P(Z\ge |z|)\), where \(Z\sim N(0,1)\). For \(H_A:p_1>p_2\), it is \(P(Z\ge z)\); for \(H_A:p_1<p_2\), it is \(P(Z\le z)\).

The pooled expected counts under the null are \(n_i\hat p\) successes and \(n_i(1-\hat p)\) failures in group \(i\). A commonly used large-sample rule is that all four expected counts are at least 5; some settings use the stricter cutoff of 10. Follow the stated convention when one is specified. If the counts are too small, do not rely on the normal-approximation p-value from this procedure.

Why the Pooled Standard Error Is Used

Under the null hypothesis, a Bernoulli observation with success probability \(p\) has variance \(p(1-p)\). A sample proportion from \(n_i\) independent observations therefore has variance \(p(1-p)/n_i\). Because the groups are independent, the variance of the difference of the two sample proportions is the sum of those variances.

\operatorname{Var}(\hat p_1-\hat p_2\mid H_0) =\frac{p(1-p)}{n_1}+\frac{p(1-p)}{n_2} =p(1-p)\left(\frac{1}{n_1}+\frac{1}{n_2}\right).

The common \(p\) is unknown, so the test estimates it using the combined success count divided by the combined sample size. Substituting \(\hat p\) for \(p\) gives \(SE_0\). The normal approximation to the standardized difference then gives the z test. Pooling is justified here by the null restriction \(p_1=p_2\); it is not a general instruction to combine proportions when the null does not impose equality.

The pooled estimate is also the maximum likelihood estimate of the common probability under \(H_0\). To see this, write \(N=n_1+n_2\) and \(x=x_1+x_2\). Ignoring factors that do not depend on \(p\), the likelihood under the null is \(L(p)=p^x(1-p)^{N-x}\). If \(0<x<N\), its log-likelihood has derivative \(x/p-(N-x)/(1-p)\). Setting this derivative equal to zero gives \(x(1-p)=p(N-x)\), and hence \(p=x/N=\hat p\). If \(x=0\), then \(L(p)=(1-p)^N\), which is maximized at the boundary \(p=0\). If \(x=N\), then \(L(p)=p^N\), which is maximized at \(p=1\). Thus the pooled estimate also applies in these boundary cases, but it is not obtained by solving the interior derivative equation there.

Applying the Test

Worked Example: A Two-Sided Comparison of Completion Rates

Two independent groups take part in a hypothetical online certification program. In group 1, 72 of 120 participants complete the program; in group 2, 48 of 120 complete it. Test \(H_0:p_1=p_2\) against \(H_A:p_1\ne p_2\) at significance level \(\alpha=0.05\). Use the pooled expected-count rule of at least 5.

Method and conditions. This is a two-sided pooled two-proportion z test. The groups are stated to be independent, and completion is a binary outcome. The observed proportions are

\hat p_1=\frac{72}{120}=0.6000,\qquad \hat p_2=\frac{48}{120}=0.4000.

Pool under the null. There are \(72+48=120\) successes among \(120+120=240\) participants, so

\hat p=\frac{120}{240}=0.5000.

Check counts and calculate. Each group has \(120(0.5)=60\) pooled expected successes and \(120(1-0.5)=60\) pooled expected failures, all at least 5. The null standard error is

SE_0=\sqrt{0.5(0.5)\left(\frac{1}{120}+\frac{1}{120}\right)} =\sqrt{0.25\left(\frac{2}{120}\right)} =\sqrt{0.0041667} \approx 0.06455.

Test and conclude. The observed difference is \(0.6000-0.4000=0.2000\), so \(z=0.2000/0.06455\approx3.10\). The two-sided p-value is \(2P(Z\ge3.10)\approx0.0019\), to four decimal places. Since \(0.0019<0.05\), reject \(H_0\); the data provide evidence that the population completion proportions differ. The method avoids using separate sample proportions in the standard error even though the null asserts one common proportion.

Worked Example: A Directional Test With Unequal Sample Sizes

In a hypothetical pilot study, 84 of 200 users assigned to a redesigned reminder system complete a scheduled task, while 60 of 180 users assigned to a standard reminder complete it. Test whether the redesigned system has a higher completion proportion, using \(\alpha=0.05\) and the expected-count cutoff of 5.

Method and hypotheses. Use a one-sided pooled z test with \(H_0:p_1=p_2\) and \(H_A:p_1>p_2\). The observed proportions are

\hat p_1=\frac{84}{200}=0.4200,\qquad \hat p_2=\frac{60}{180}=0.3333.

Pool and check. The pooled estimate is \((84+60)/(200+180)=144/380\approx0.3789\). The pooled expected counts are approximately \(200(0.3789)=75.79\) successes and \(200(0.6211)=124.21\) failures in group 1; for group 2 they are \(180(0.3789)=68.21\) successes and \(180(0.6211)=111.79\) failures. All exceed 5.

Calculate the statistic.

SE_0=\sqrt{\frac{144}{380}\left(1-\frac{144}{380}\right) \left(\frac{1}{200}+\frac{1}{180}\right)} \approx\sqrt{0.378947(0.621053)(0.0105556)} \approx0.04983.

The difference is \(0.4200-0.3333=0.0867\), giving \(z\approx0.0867/0.04983=1.74\). For the upper-tailed alternative, the p-value is \(P(Z\ge1.74)\approx0.0410\), to four decimal places. Because \(0.0410<0.05\), reject \(H_0\) in favor of evidence that the redesigned system has a higher completion proportion. The method avoids doubling a one-sided p-value: the stated alternative is directional, so only the upper tail is used.

Worked Example: A Statistic Is Not Enough When Expected Counts Are Sparse

Two independent groups are each observed for 12 hypothetical trials. Group 1 has 1 success and group 2 has 7 successes. Consider a two-sided equality test and use the expected-count cutoff of 5. Determine whether the pooled z approximation is suitable.

Method. The sample proportions are \(\hat p_1=1/12\approx0.0833\) and \(\hat p_2=7/12\approx0.5833\). The pooled estimate is \((1+7)/(12+12)=8/24=1/3\). Under the null, each group has expected successes \(12(1/3)=4\) and expected failures \(12(2/3)=8\). The expected success counts are below 5, so the large-sample count condition fails.

For diagnosis, the formula would give

SE_0=\sqrt{\frac{1}{3}\frac{2}{3}\left(\frac{1}{12}+\frac{1}{12}\right)} =\sqrt{\frac{2}{27}}\approx0.2722,\qquad z=\frac{1/12-7/12}{0.2722}\approx-1.84.

Conclusion. Do not interpret the normal-approximation p-value as a reliable test result here: the pooled expected successes are only 4 in each group. The method avoids treating the act of calculating a z statistic as proof that its approximation is adequate. A procedure designed for small samples is needed instead.

Common Traps and Interpretation

Using the wrong standard error. For this test, the standard error is based on the pooled estimate under \(H_0\). Substituting \(\hat p_1\) and \(\hat p_2\) separately changes the procedure.
Pooling before stating the null. Pool because the null imposes \(p_1=p_2\), not merely because there are two samples.
Ignoring the direction of the alternative. A two-sided test uses both tails; a one-sided test uses only the tail specified in advance.
Checking only the total number of successes. Check pooled expected successes and failures separately in each group.
Using the method for dependent data. Matched outcomes and clustered observations do not satisfy the independent-group model used to derive this standard error.

A p-value measures how unusual a test statistic at least as extreme as the observed one would be if the null hypothesis and model assumptions held. It is not the probability that the null hypothesis is true, and a rejection does not by itself establish that the difference is practically important.

Key takeaway: State the hypotheses, verify the independent binary-sample model, pool the observed successes only for the null estimate, check all four pooled expected counts, and then calculate the z statistic and the p-value appropriate to the alternative.

Check Your Understanding

For each problem, identify the appropriate tail and check the pooled expected counts before interpreting a normal-approximation result. Answers are given at the end of each question.

Two independent samples have 56 successes out of 100 and 44 successes out of 100. For a two-sided equality test, calculate the pooled estimate and z statistic. (Answer: \(\hat p=0.5000\), \(z\approx1.697\).)
A hypothetical service receives 90 successful ratings among 150 clients in region 1 and 63 among 150 in region 2. Test \(p_1\ne p_2\) at \(\alpha=0.05\). (Answer: \(\hat p=0.5100\), \(SE_0\approx0.05772\), \(z\approx3.119\), two-sided p-value \(\approx0.0018\); reject \(H_0\).)
In independent samples of 20 each, there are 8 successes in group 1 and 2 in group 2. For \(H_A:p_1>p_2\), calculate the pooled z statistic and decide at \(\alpha=0.05\). (Answer: \(\hat p=0.2500\), \(SE_0\approx0.1369\), \(z\approx2.191\), upper-tail p-value \(\approx0.0142\); reject \(H_0\).)
Group 1 has 2 successes in 30 observations and group 2 has 10 in 30. Does the pooled expected-count rule of at least 5 pass? (Answer: Yes. The pooled estimate is \(12/60=0.2000\); each group has 6 expected successes and 24 expected failures.)
Two groups have 0 successes in 15 observations and 0 successes in 25 observations. What pooled estimate results, and can the usual pooled z statistic be calculated? (Answer: \(\hat p=0\); no, \(SE_0=0\), so the usual z statistic is undefined and the large-sample count condition fails.)

What You'll Learn