Planning Sample Sizes for a Proportion Difference

Planning for Precision

When comparing two population proportions, it is useful to decide in advance how precisely their difference should be estimated. For example, a researcher might want a confidence interval for the difference in event rates to have a margin of error no larger than 5 percentage points. That target can be translated into a planned sample size for each group.

This calculation is a planning approximation. It uses anticipated proportions to estimate the variability of the difference in sample proportions, then finds the equal sample size that makes the confidence interval's margin of error meet the target. The result depends on the anticipated proportions, the desired confidence level, and the chosen margin of error.

Definition: The margin of error for a confidence interval is the amount added to and subtracted from its point estimate. For a confidence interval for \(p_1-p_2\), a target margin of error \(M\) is an intended maximum half-width, measured in proportion units. For instance, \(M=0.05\) means 5 percentage points.

Suppose the two groups are independent, have the same planned sample size \(n\), and each observation records whether an event occurred. The underlying event probabilities are anticipated to be \(p_1\) and \(p_2\). The planning values need not be equal. They represent informed estimates of the proportions expected in the two groups, not values guaranteed to appear in the eventual data.

The Planning Formula

For independent Bernoulli observations, the approximate standard error of the difference between the sample proportions is \(\sqrt{p_1(1-p_1)/n+p_2(1-p_2)/n}\), when the anticipated proportions are used to plan. A two-sided confidence interval with confidence level \(1-\alpha\) uses the standard normal critical value \(z_{1-\alpha/2}\). Thus, its planned margin of error is that critical value multiplied by the planned standard error.

Planning result: With equal-sized independent groups and anticipated proportions \(p_1\) and \(p_2\), the approximate sample size needed in each group for target margin of error \(M\) is

n=\frac{z_{1-\alpha/2}^{\,2}\left[p_1(1-p_1)+p_2(1-p_2)\right]}{M^2}.

Here \(1-\alpha\) is the confidence level, \(z_{1-\alpha/2}\) is the corresponding standard normal critical value, and \(M\) is expressed as a proportion rather than a percentage. Round the calculated \(n\) up to the next whole number.

The formula gives the size of each group, not the combined total. If the calculated requirement is \(n=150\) per group, the planned total is \(2n=300\). Rounding up matters because a fractional participant cannot be recruited and rounding down could leave the planned margin of error above the target.

Why the Formula Has This Form

For one independent Bernoulli observation with event probability \(p_i\), the variance is \(p_i(1-p_i)\). For a sample proportion based on \(n\) independent observations, the variance is \(p_i(1-p_i)/n\). Because the two groups are independent, the variance of the difference of their sample proportions is the sum of those two variances. Taking the square root gives the standard error used in the planning approximation.

The approximate confidence interval has the form “estimate plus or minus critical value times standard error.” For planning, substitute the anticipated proportions into that standard error and set the margin of error equal to the target \(M\). The equal group sizes allow \(1/n\) to be factored out. Squaring both sides and rearranging isolates \(n\):

\begin{aligned} M &=z_{1-\alpha/2} \sqrt{\frac{p_1(1-p_1)}{n}+\frac{p_2(1-p_2)}{n}}\\ &=z_{1-\alpha/2} \sqrt{\frac{p_1(1-p_1)+p_2(1-p_2)}{n}},\\[4pt] M^2 &=\frac{z_{1-\alpha/2}^{\,2}\left[p_1(1-p_1)+p_2(1-p_2)\right]}{n},\\[4pt] n &=\frac{z_{1-\alpha/2}^{\,2}\left[p_1(1-p_1)+p_2(1-p_2)\right]}{M^2}. \end{aligned}

This derivation also shows how the inputs affect the answer. A higher confidence level uses a larger critical value and therefore requires a larger sample. A smaller target margin of error requires a larger sample because \(M\) is squared in the denominator. The anticipated proportions determine the variance contribution from each group.

Worked Example: A 95% Interval With an 8-Percentage-Point Target

Suppose a hypothetical planning team expects event proportions of \(p_1=0.30\) and \(p_2=0.20\). It wants a two-sided 95% confidence interval for \(p_1-p_2\) with a margin of error no greater than 0.08, or 8 percentage points. Find the required equal sample size per group.

For a 95% confidence level, use \(z_{1-\alpha/2}=1.96\). The anticipated variance contributions are \(0.30(1-0.30)=0.21\) and \(0.20(1-0.20)=0.16\). Substitution gives:

n=\frac{1.96^2[0.21+0.16]}{0.08^2} =\frac{3.8416(0.37)}{0.0064} =\frac{1.421392}{0.0064} =222.09.

Round up to \(n=223\) observations in each group, for a total planned sample of 446. As a check, the planned standard error is approximately \(\sqrt{0.37/223}=0.04073\), giving a margin of error \(1.96(0.04073)\approx0.0798\), just below 0.08. The planning calculation therefore meets the stated target under its assumptions.

Worked Applications

The following examples use hypothetical planning values. In each case, the anticipated proportions describe assumptions for sizing the study; they are not observed results or promises about what a later sample will contain.

Worked Example: Changing the Confidence Level and Target

For a hypothetical comparison, planners anticipate proportions \(p_1=0.12\) and \(p_2=0.18\). They seek a 90% confidence interval with margin of error at most 0.05. Find the number required per group.

The critical value for a two-sided 90% confidence level is approximately 1.645. The variance contributions are \(0.12(0.88)=0.1056\) and \(0.18(0.82)=0.1476\), whose sum is 0.2532. Therefore:

n=\frac{1.645^2(0.2532)}{0.05^2} =\frac{2.706025(0.2532)}{0.0025} =\frac{0.6851655}{0.0025} =274.07.

Rounding up gives 275 participants per group, or 550 in total. At this size, the anticipated event counts are \(275(0.12)=33\) and \(275(0.18)=49.5\); the anticipated non-event counts are 242 and 225.5. These expected counts are comfortably above a common rough guideline of 10 in each outcome category, supporting the normal approximation for planning. Such a guideline is a check, not a guarantee that every study design has adequate inference conditions.

Worked Example: Planning Conservatively When Proportions Are Unknown

Suppose a hypothetical study has no useful prior information about either event proportion. The team wants a 99% confidence interval with margin of error at most 0.06 and plans equal-sized independent groups. As a conservative variance choice, use \(p_1=p_2=0.50\).

For a two-sided 99% confidence level, \(z_{1-\alpha/2}\approx2.576\). Each variance contribution is \(0.50(0.50)=0.25\), so their sum is 0.50. The calculation is:

n=\frac{2.576^2(0.50)}{0.06^2} =\frac{6.635776(0.50)}{0.0036} =\frac{3.317888}{0.0036} =921.64.

The study should plan for 922 observations per group, or 1,844 overall. At the planning values, each group has expected counts of 461 events and 461 non-events, so the normal approximation is not threatened by small expected counts. This conservative choice protects against underestimating the variance when the proportions are unknown: \(p(1-p)\) is largest at \(p=0.50\). It may result in a larger sample than a calculation based on more informative anticipated proportions.

Assumptions and Limits

The formula is appropriate for planning an approximate confidence interval for a difference in two proportions under a particular design. Its assumptions should be considered before treating the computed value as a useful target.

Independent groups: the observations in one group are independent of those in the other. The calculation does not describe paired observations or matched groups.
Independent observations within groups: each group is modeled as independent Bernoulli outcomes with a common event probability. Clustered observations can have more variability than this model allows.
Representative sampling or appropriate assignment: the planned observations must correspond to the population and comparison the study intends to address. Sample size alone cannot correct selection bias or a flawed design.
Reasonable normal approximation: the anticipated numbers of events and non-events, \(np_i\) and \(n(1-p_i)\), should be sufficiently large in both groups. A common rough check is that each is at least 10, though exact adequacy depends on the setting.
Planning proportions are informative: the formula's standard error uses the anticipated \(p_1\) and \(p_2\). If these differ substantially from the eventual proportions, the realized interval's width may differ from the target.

The result is an approximate sample size for precision. It does not establish that a study has a particular probability of detecting a specified difference. A power-based calculation instead requires a meaningful effect to detect and a power target; those are different planning questions. Likewise, the target margin of error is not the difference the study expects to observe. A target of 0.05 means a desired interval half-width of 5 percentage points, not that \(p_1-p_2\) is expected to equal 0.05.

Common Mistakes and Practical Checks

Using percentages directly in the formula. Substituting \(30\) for a 30% proportion or \(8\) for an 8-point margin produces an invalid result. Use \(0.30\) and \(M=0.08\).
Reporting the total as the per-group requirement. The formula returns \(n\) for each group. For equal group sizes, the overall total is \(2n\).
Rounding to the nearest integer or down. The sample size must be rounded up, since the calculated value is the minimum continuous solution meeting the approximate target.
Treating the target width as guaranteed. The achieved interval uses the actual sample proportions. The planned margin is based on anticipated proportions, so the realized width can be somewhat larger or smaller.
Confusing precision with power. A narrow-interval goal addresses estimation precision; it does not by itself promise detection of an effect or a particular probability of rejecting a null hypothesis.

Key takeaway: For equal-sized independent groups, calculate the anticipated sum \(p_1(1-p_1)+p_2(1-p_2)\), multiply by the squared confidence critical value, divide by the squared target margin of error, and round up. The answer is the required sample size in each group, subject to the model and approximation checks.

Check Your Understanding

Use the formula and the interpretation of its inputs to answer these questions.

What does a target margin of error of 0.04 mean when estimating a difference in proportions?
A calculation gives \(n=180.2\). How many observations should be planned for each group, and what is the total for two equal groups?
Why does reducing the target margin of error generally increase the required sample size?
For anticipated proportions of 0.40 and 0.25, write the sum of the two variance contributions used in the formula.
Does meeting a precision-based sample-size target guarantee a particular power to detect a difference? Explain briefly.