Choose a Two-Proportion Procedure

When Do Two-Proportion Procedures Apply?

Many questions compare how often an outcome occurs in two groups: What fraction of patients improve under each of two treatments? What proportion of products pass inspection at each of two facilities? These are questions about rates of a binary outcome. The relevant comparison is not a mean but a difference between two population proportions.

A two-proportion procedure is appropriate when there are two groups, the outcome for each observational unit is classified as a success or failure, and the data provide a count of successes and a total number of observations in each group. “Success” is a label for the outcome of interest; it need not mean a beneficial result. The group order must be specified because reversing it reverses the sign of the difference.

Definition: Let \(p_1\) be the population proportion with the outcome of interest in group 1, and let \(p_2\) be the corresponding population proportion in group 2. The target parameter for a two-proportion comparison is the difference \(p_1-p_2\). If \(x_1\) of \(n_1\) observations in group 1 and \(x_2\) of \(n_2\) observations in group 2 are successes, the sample proportions are \(\hat{p}_1=x_1/n_1\) and \(\hat{p}_2=x_2/n_2\). The estimate of the difference is \(\hat{p}_1-\hat{p}_2\).

The parameter describes a difference in the underlying populations or groups, while \(\hat{p}_1-\hat{p}_2\) describes the observed difference in the samples. For example, a difference of \(0.08\) means an estimated rate in group 1 that is 8 percentage points higher than the rate in group 2. It is a difference in percentage points, not an 8 percent relative increase.

Match the Question to the Procedure

A confidence interval for \(p_1-p_2\) estimates the size and direction of the difference. A significance test assesses a specified claim about that difference; the common null hypothesis of equal proportions is \(H_0:p_1-p_2=0\). Both are two-proportion procedures, but they answer different inferential questions. In either case, use the group order in the question consistently in the parameter, calculations, and interpretation.

For the standard large-sample methods, the data are modeled as two independent binomial counts:

X_1\sim \operatorname{Bin}(n_1,p_1), \qquad X_2\sim \operatorname{Bin}(n_2,p_2), \qquad X_1\ \text{and}\ X_2\ \text{independent}.

This model requires binary outcomes and independent observations within each group, as well as independence between the groups. A random sample from each population or random assignment to two groups can support the relevant inference, though those designs justify different kinds of conclusions. The normal-approximation methods also require enough successes and failures for the approximation to be reasonable. For an interval, assess the observed counts in each group; for a test of equal proportions, assess the expected counts under the null model. If the design is paired, observations are clustered, or counts are too sparse, the ordinary independent two-proportion method may not be appropriate.

The usual confidence interval uses the separate observed proportions to estimate uncertainty. A test of equal proportions instead pools the observed successes under the null hypothesis, because that null says both groups share one proportion. The formulas reflect that difference.

Procedure choice: For estimating \(p_1-p_2\), use a two-proportion \(z\) interval under the stated model and large-sample conditions. For testing \(H_0:p_1=p_2\), use a two-proportion \(z\) test, with the pooled estimate under the null. The estimated difference in either analysis is \(\hat{p}_1-\hat{p}_2\).

Worked Examples

Worked Example: Comparing Two Independent Treatment Groups

In a hypothetical randomized study, 42 of 100 participants assigned to treatment A and 30 of 100 assigned to treatment B meet a prespecified binary improvement criterion. The research question asks for an estimate of the difference in improvement rates, treatment A minus treatment B.

The outcome is binary, and the groups contain different participants. Random assignment supports comparison of the groups, and the counts are large enough for a normal-approximation interval: A has 42 successes and 58 failures, while B has 30 successes and 70 failures. Thus a two-proportion interval is suitable for estimating \(p_A-p_B\).

The observed proportions and their difference are

\hat{p}_A=\frac{42}{100}=0.42,\qquad \hat{p}_B=\frac{30}{100}=0.30,\qquad \hat{p}_A-\hat{p}_B=0.12.

The estimated standard error for the interval uses the two separate sample proportions:

SE=\sqrt{\frac{\hat{p}_A(1-\hat{p}_A)}{n_A} +\frac{\hat{p}_B(1-\hat{p}_B)}{n_B}} =\sqrt{\frac{0.42(0.58)}{100}+\frac{0.30(0.70)}{100}} =\sqrt{0.004536} \approx 0.06735.

A 95% interval is the estimate plus or minus \(1.96\) standard errors:

0.12\pm1.96(0.06735) =0.12\pm0.1320 =(-0.0120,\ 0.2520).

The interval estimates that the improvement rate under treatment A is between 1.20 percentage points lower and 25.2 percentage points higher than under treatment B. Because the interval includes zero, these data do not rule out equal population improvement rates at this confidence level. The interval alone does not show that the treatments are equivalent.

Worked Example: Testing Whether Two Success Rates Differ

In a hypothetical randomized evaluation, 72 of 120 participants in group 1 meet a binary response criterion, compared with 45 of 100 participants in group 2. The question is whether the response proportions differ. The individuals in the two groups are distinct, assignment is random, and each response is classified once as success or failure.

The sample proportions are \(72/120=0.600\) and \(45/100=0.450\), so the observed difference, group 1 minus group 2, is \(0.150\). For the test of equal proportions, the null hypothesis says the two groups share a common success probability. The pooled estimate is therefore

\hat{p}=\frac{x_1+x_2}{n_1+n_2} =\frac{72+45}{120+100} =\frac{117}{220} \approx0.5318.

The pooled standard error and test statistic are

SE=\sqrt{\hat{p}(1-\hat{p})\left(\frac{1}{n_1}+\frac{1}{n_2}\right)} =\sqrt{0.5318(0.4682)\left(\frac{1}{120}+\frac{1}{100}\right)} \approx0.06756, $$ $$ z=\frac{\hat{p}_1-\hat{p}_2}{SE} =\frac{0.600-0.450}{0.06756} \approx2.220.

Under the null hypothesis, the expected success and failure counts are approximately 63.82 and 56.18 in group 1, and 53.18 and 46.82 in group 2. These counts support the normal approximation. For a two-sided test, \(z=2.220\) gives \(P\)-value approximately \(0.0264\). At significance level \(\alpha=0.05\), there is evidence that the population response proportions differ. The observed rate is higher in group 1 by 15.0 percentage points, but the test does not establish that the population difference is exactly 15.0 percentage points.

Worked Example: Checking Whether a Two-Proportion Method Fits

A hypothetical survey independently samples 40 adults from region A and 90 adults from region B. In response to the same yes-or-no question, 18 people in region A and 33 in region B answer “yes.” The goal is to estimate the difference in “yes” rates, region A minus region B.

There are two groups and a binary outcome, and the samples are independent. Suppose each is a random sample from its region, with no repeated individuals. The success and failure counts are 18 and 22 in region A, and 33 and 57 in region B, so the observed counts are adequate for a standard large-sample interval.

The estimated difference is

\hat{p}_A-\hat{p}_B =\frac{18}{40}-\frac{33}{90} =0.4500-0.3667 =0.0833.

Its standard error is

SE=\sqrt{\frac{0.45(0.55)}{40} +\frac{0.3667(0.6333)}{90}} \approx\sqrt{0.008768} \approx0.09364.

A 95% interval is

0.0833\pm1.96(0.09364) =0.0833\pm0.1835 \approx(-0.1002,\ 0.2669).

The estimated “yes” rate in region A is 8.33 percentage points higher, but the interval is wide and includes zero. It is compatible with rates in region A up to about 10.0 percentage points lower or 26.7 percentage points higher than those in region B. Since this is an observational survey, the comparison describes a difference between populations; it does not show that living in one region causes a different response rate.

Why Independence and the Group Structure Matter

Two proportions are not automatically a two-proportion problem. The standard method treats the group counts as independent. If the same people are measured twice, or individuals are deliberately matched, the two observations for a person are related. For example, recording whether each patient improves before and after treatment produces paired binary data. Although one can calculate a proportion at each time point, treating those proportions as if they came from independent groups discards the pairing and can give the wrong standard error. Use a method for paired binary data instead.

The same caution applies when observations are collected in clusters, such as many patients within a small number of clinics. Outcomes from patients at the same clinic may be related, so the independent-observation model may fail. Procedure choice depends on how the data were collected, not just on whether two percentages appear in a table.

The subtraction order also carries meaning. If group 1 is treatment A and group 2 is treatment B, then \(p_1-p_2\) is the rate for A minus the rate for B. Reversing the labels changes the sign but not the underlying comparison. State the order explicitly so a positive estimate is not mistaken for a higher rate in the other group.

Common Mistakes and a Short Justification

A frequent error is to use the pooled standard error for a confidence interval simply because a pooled formula appears in the two-proportion test. Pooling is justified for the test of equal proportions because its null model imposes \(p_1=p_2\). For an interval, the difference is not assumed to be zero; the standard error estimates the variability of two separately observed proportions.

The distinction follows from the variance of independent sample proportions. Under the binomial model, the variance of a sample proportion is \(p(1-p)/n\). Independence makes the variance of a difference the sum of the two variances:

\operatorname{Var}(\hat{p}_1-\hat{p}_2) =\operatorname{Var}(\hat{p}_1)+\operatorname{Var}(\hat{p}_2) =\frac{p_1(1-p_1)}{n_1}+\frac{p_2(1-p_2)}{n_2}.

For an interval, each unknown population proportion is estimated by its own sample proportion, producing the separate-proportion standard error. Under \(H_0:p_1=p_2\), both proportions are represented by one common value, estimated by the pooled proportion \(\hat{p}\); substituting that value in the variance gives the pooled test standard error. Pooling is not appropriate when estimating uncertainty without that null restriction.

Wrong: “There are two percentages, so this must be a two-proportion procedure.” Correction: First verify that outcomes are binary and that the groups are independent; paired or clustered observations need different treatment.
Wrong: “The estimated difference is 0.08, so the first rate is 8% higher.” Correction: The difference is 8 percentage points; a relative percentage increase is a different quantity.
Wrong: “A confidence interval containing zero proves the rates are equal.” Correction: It means the interval includes zero as a plausible difference at that confidence level; it does not establish equality.
Wrong: “A significant result proves the group assignment caused the difference.” Correction: Causal interpretation depends on the design, especially whether assignment was randomized.

Key takeaway: Choose a two-proportion procedure when two independent groups have binary outcomes and the question concerns \(p_1-p_2\). Use an interval to estimate that difference or a test to assess a null claim about it, and keep the group order fixed throughout.

Check Your Understanding

For each situation, identify whether an ordinary two-proportion procedure fits and state the target parameter if it does.

A survey independently samples 250 residents of town A and 300 residents of town B and records whether each supports a proposal. What is the natural difference parameter?
A clinic records whether each of 80 patients has a symptom before and after treatment. Should these measurements be treated as two independent proportions? Explain.
In a two-proportion test of equal population rates, why is a pooled estimate used in the standard error?
A 95% interval for \(p_1-p_2\) is \((-0.04,0.13)\). What does the inclusion of zero indicate, and what does it not prove?
In a randomized study, group 1 has a success rate of 0.70 and group 2 has a success rate of 0.62. Express the estimated difference in percentage points when the parameter is defined as group 1 minus group 2.