Tutorials › Introduction to Statistics › Worked Tests for Differences in Proportions

Home  /  Tutorials  /  Two-Proportion Inference

Two-Proportion Inference · Tutorial 709 of 996

Worked Tests for Differences in Proportions

Work through two-proportion z tests from start to finish, choosing the right hypotheses and pooled standard error and interpreting the result in context.

Intermediate 10 min read

What You'll Learn

  • State null and alternative hypotheses for a difference in two population proportions.
  • Calculate sample proportions and the pooled estimate under the null hypothesis.
  • Check the basic design and count conditions for a two-proportion z test.
  • Compute the pooled standard error and z test statistic from group counts.
  • Choose the correct p-value tail from the alternative hypothesis.
  • Report a test conclusion without treating a nonsignificant result as proof of equality.

From Group Counts to a Test

When a study records a binary outcome in each of two independent groups, the observed success proportions will usually differ somewhat, even if the population proportions are equal. A two-proportion z test asks whether the observed difference is sufficiently incompatible with equality to provide evidence of a population difference. A complete test connects the study question to hypotheses, checks the conditions for the approximation, calculates a test statistic and p-value, and ends with a conclusion in context.

Let \(p_1\) and \(p_2\) be the population proportions with the outcome of interest in groups 1 and 2. In the observed data, let \(x_1\) of \(n_1\) individuals in group 1 and \(x_2\) of \(n_2\) individuals in group 2 have the outcome. The sample proportions are \(\hat{p}_1=x_1/n_1\) and \(\hat{p}_2=x_2/n_2\). The quantity being tested is the population difference \(p_1-p_2\).

Definition: A two-proportion z test evaluates a null hypothesis that the two population proportions are equal, \(H_0:p_1-p_2=0\), against a specified alternative. Under this null hypothesis, the test estimates the common proportion by pooling the successes and sample sizes: \(\hat{p}=(x_1+x_2)/(n_1+n_2)\). The pooled standard error is \(SE=\sqrt{\hat{p}(1-\hat{p})(1/n_1+1/n_2)}\), and the test statistic is \(z=(\hat{p}_1-\hat{p}_2)/SE\).

The alternative hypothesis determines which departures from zero count as evidence. A two-sided test uses \(H_a:p_1-p_2\ne0\); a directional test might use \(H_a:p_1-p_2>0\) or \(H_a:p_1-p_2<0\). The direction must be chosen from the research question before examining the results. The p-value is calculated from the standard normal distribution and is the probability, assuming \(H_0\), of a test statistic at least as extreme in the direction specified by \(H_a\).

Model and Test Conditions

The test treats each outcome as a success or failure and relies on independent observations within each group and independence between groups. The groups should arise from an appropriate random sample or randomized design if the intended conclusion concerns population proportions or a treatment effect. The normal approximation also needs enough expected successes and failures. For the pooled test, assess these counts under the null using \(\hat{p}\): \(\hat{p}n_1,\hat{p}n_2,(1-\hat{p})n_1,\) and \((1-\hat{p})n_2\) should each be at least 10.

A randomized experiment can support a causal interpretation of a treatment difference, subject to the experiment's design and limitations. For observational or sampling studies, the test can assess evidence of a difference in proportions, but the result alone does not establish causation. These conditions and the study design determine what the final conclusion can responsibly say.

Why the Test Pools the Estimate

Under \(H_0\), the two groups share a common population success probability \(p\). The combined successes \(x_1+x_2\) among \(n_1+n_2\) observations therefore estimate that shared probability with \(\hat{p}\). For independent Bernoulli outcomes, the variance of a sample proportion from a group of size \(n\) is \(p(1-p)/n\). Independence makes the variance of a difference the sum of the variances:

$$ \operatorname{Var}(\hat{p}_1-\hat{p}_2) =\frac{p(1-p)}{n_1}+\frac{p(1-p)}{n_2} =p(1-p)\left(\frac{1}{n_1}+\frac{1}{n_2}\right). $$

Because the null hypothesis specifies a shared \(p\), substituting the pooled estimate \(\hat{p}\) for that unknown probability gives the pooled standard error. Under \(H_0\), the expected difference is zero, so dividing the observed difference by this standard error gives the z statistic. The standard normal approximation then supplies the p-value. The test uses the pooled estimate for its standard error because equality is assumed when calibrating the statistic; the observed difference in the numerator remains \(\hat{p}_1-\hat{p}_2\).

Worked Tests

Worked Example: A Two-Sided Test of Different Event Rates

Suppose a hypothetical survey independently samples 120 people in Group 1 and 100 people in Group 2. The binary outcome is reporting a particular experience: 42 people in Group 1 and 24 in Group 2 report it. Test at \(\alpha=0.05\) whether the population proportions differ.

1. Hypotheses. Let \(p_1\) and \(p_2\) be the population proportions reporting the experience. The question is whether they differ, so \(H_0:p_1-p_2=0\) and \(H_a:p_1-p_2\ne0\).

2. Conditions. The hypothetical groups are independently sampled, and each person's response is binary. Under the null, the pooled estimate is \(\hat{p}=(42+24)/(120+100)=66/220=0.3000\). The expected success counts are \(120(0.3000)=36.0\) and \(100(0.3000)=30.0\); expected failure counts are 84.0 and 70.0. All exceed 10, so the pooled normal approximation is reasonable under the stated sampling setup.

3. Sample proportions and test statistic. The observed proportions are \(\hat{p}_1=42/120=0.3500\) and \(\hat{p}_2=24/100=0.2400\), giving a difference of \(0.1100\). The pooled standard error is

$$ SE=\sqrt{0.3000(1-0.3000)\left(\frac{1}{120}+\frac{1}{100}\right)} =\sqrt{0.21(0.0183333)} =\sqrt{0.003850} \approx0.06205. $$

Thus \(z=(0.3500-0.2400)/0.06205\approx1.773\). For the two-sided alternative, the p-value is \(2P(Z\ge |1.773|)\approx0.0762\) (approximately 0.0763 using the unrounded statistic).

4. Conclusion. Since \(0.0762>0.05\), the result is not statistically significant at the 5% level. These data do not provide sufficient evidence that the two population proportions reporting the experience differ. This conclusion does not establish that the proportions are equal.

Worked Example: A One-Sided Test for a Lower Event Proportion

In a hypothetical randomized study, 18 of 150 participants assigned to Treatment have an event, compared with 31 of 140 assigned to Standard care. Test whether Treatment reduces the population event proportion, using \(\alpha=0.05\).

1. Hypotheses. Define \(p_1\) as the event proportion under Treatment and \(p_2\) as the event proportion under Standard care. A reduction under Treatment means \(p_1<p_2\), so \(H_0:p_1-p_2=0\) and \(H_a:p_1-p_2<0\).

2. Conditions. The hypothetical randomized assignment creates independent groups, and the event indicator is binary. The pooled estimate under the null is \(\hat{p}=(18+31)/(150+140)=49/290\approx0.1690\). Expected successes are \(150(49/290)\approx25.34\) and \(140(49/290)\approx23.66\); expected failures are about 124.66 and 116.34. Each is at least 10.

3. Test statistic and p-value. The sample proportions are \(\hat{p}_1=18/150=0.1200\) and \(\hat{p}_2=31/140\approx0.2214\); their difference is approximately \(-0.1014\). The pooled standard error is

$$ SE=\sqrt{\frac{49}{290}\left(1-\frac{49}{290}\right) \left(\frac{1}{150}+\frac{1}{140}\right)} \approx\sqrt{0.001939} \approx0.04403. $$

Therefore \(z=(-0.1014)/0.04403\approx-2.303\). Because the alternative is left-sided, the p-value is \(P(Z\le-2.303)\approx0.0106\); it is not doubled.

4. Conclusion. Since \(0.0106<0.05\), reject \(H_0\) at the 5% significance level. The hypothetical study provides evidence that the population event proportion is lower under Treatment than under Standard care. Random assignment supports interpreting this comparison as evidence about the treatment effect in the study context, though it does not by itself guarantee that the result generalizes to every population.

Worked Example: A Small Observed Difference

Suppose a hypothetical independent-group study observes an outcome in 28 of 100 people in Group 1 and 25 of 100 people in Group 2. Test \(H_0:p_1-p_2=0\) against \(H_a:p_1-p_2\ne0\) at \(\alpha=0.05\).

Conditions. Assume independent sampling for the two groups and a binary outcome. Pooling gives \(\hat{p}=(28+25)/(100+100)=53/200=0.2650\). Expected successes are 26.5 in each group and expected failures are 73.5 in each, so the normal approximation condition is met.

Calculation. The sample proportions are \(0.2800\) and \(0.2500\), with difference \(0.0300\). The pooled standard error and test statistic are

$$ SE=\sqrt{0.2650(0.7350)\left(\frac{1}{100}+\frac{1}{100}\right)} =\sqrt{0.0038955} \approx0.06241, \qquad z=\frac{0.0300}{0.06241}\approx0.481. $$

The two-sided p-value is \(2P(Z\ge0.481)\approx0.631\). Since this exceeds 0.05, the result is not statistically significant. The data do not provide sufficient evidence of a difference between the population proportions. The sample proportions are not identical, but the observed 3-percentage-point difference is small relative to its estimated sampling variability.

Common Mistakes and Reporting

A frequent error is to use an unpooled standard error in the test statistic because the two observed sample proportions differ. That reasoning ignores the null model: the test is calibrated assuming \(p_1=p_2\), so it estimates their common value with the pooled proportion. The test statistic uses the pooled standard error shown here. An unpooled standard error is used for a different purpose, such as estimating uncertainty for a confidence interval for \(p_1-p_2\).

Another mistake is to choose the p-value tail after seeing the sign of the result. The alternative must reflect the question specified before examining the data. For \(H_a:p_1-p_2\ne0\), include extreme results in either direction; for \(H_a:p_1-p_2<0\), use the lower tail; for \(H_a:p_1-p_2>0\), use the upper tail. A small p-value is evidence against the null under the model, not the probability that the null hypothesis is true.

Key takeaway: A complete two-proportion z test reports the groups and outcome, hypotheses, relevant design and count checks, pooled estimate, standard error, z statistic, correctly tailed p-value, and a conclusion in context. A nonsignificant result means the data did not provide sufficient evidence against equality; it does not prove that the population proportions are equal.

Check Your Understanding

For each question, identify what the test requires or what its result means.

  1. In a test of \(H_0:p_1-p_2=0\) against \(H_a:p_1-p_2>0\), the observed difference is negative. Which tail determines the p-value, and why?
  2. Two independent groups have 30 successes among 100 observations and 20 successes among 100 observations. Calculate the pooled estimate and the four expected success and failure counts under the null.
  3. Why does the pooled standard error use the combined success proportion when testing equality of two population proportions?
  4. A two-sided test gives \(p=0.08\) at \(\alpha=0.05\). State an appropriate conclusion and one conclusion that would overstate the evidence.
  5. Explain why a small p-value in a randomized experiment and a small p-value in an observational study may support different kinds of conclusions.

Previous

Next

Back to Tutorials