Sample-Size Calculations for Binary Outcomes

Why Binary Outcomes Need Their Own Sample-Size Calculation

For a binary outcome, each participant either experiences the event of interest or does not. Examples include response versus no response, or recurrence versus no recurrence by a specified time. Planning therefore begins with assumed response probabilities in each arm, not with a mean and standard deviation as for a continuous outcome. Those probabilities determine the variability of the treatment contrast as well as its expected size.

In a multi-arm multi-stage (MAMS) design, the basic calculation can be made for each treatment-control contrast using the planned sample sizes at a stage or by the end of the trial. The resulting numbers are planning approximations for that contrast. They do not, by themselves, establish the power or type I error of the complete multi-stage procedure: those also depend on its sequential decision rules and the dependence among comparisons that share a control.

Definition: Let arm \(0\) be the control and arm \(j\) a treatment arm. Write \(p_0\) and \(p_j\) for their assumed probabilities of the binary outcome, and let \(\delta=p_j-p_0\) be the treatment-control difference in response probabilities. If \(n_0\) and \(n_j\) participants are independently observed in the two arms, the estimated difference is \(\hat{p}_j-\hat{p}_0\). Under independent Bernoulli outcomes, its variance is \(p_j(1-p_j)/n_j+p_0(1-p_0)/n_0\).

Planning Formula and Its Assumptions

Suppose the planned allocation ratio for this comparison is \(r=n_j/n_0\): for each control participant, the design plans \(r\) treatment participants. The variance of the estimated difference can then be written as \(V/n_0\), where \(V\) is the variance coefficient for the chosen allocation ratio and assumed response probabilities.

\operatorname{Var}(\hat{p}_j-\hat{p}_0) = \frac{p_j(1-p_j)}{n_j}+\frac{p_0(1-p_0)}{n_0} = \frac{1}{n_0} \left\{ \frac{p_j(1-p_j)}{r}+p_0(1-p_0) \right\} = \frac{V}{n_0}.

For a one-sided superiority calculation, let \(c\) be the planned critical value on the standard-normal scale and let \(1-\beta\) be the target power under the assumed alternative. Here \(z_q\) denotes the \(q\) quantile of the standard normal distribution. Under a large-sample Wald approximation, the sample size follows from setting the approximate power at least \(1-\beta\). This calculation assumes independent participants, fixed response probabilities, the stated allocation ratio, and sufficiently large arm sizes for the normal approximation to be reasonable. It also assumes that the outcome is observed as defined for every participant included in the calculation.

n_0 \approx \frac{(c+z_{1-\beta})^2 V}{\delta^2}, \qquad n_j = r n_0.

This is an unpooled Wald planning approximation: the variance is evaluated at the assumed alternative probabilities. A particular testing procedure may instead use a different variance estimate or critical value. In that case, the calculation must match the planned test. For a two-sided test, the corresponding normal critical value is \(z_{1-\alpha/2}\) rather than a one-sided value; for a one-sided test with nominal level \(\alpha\), the usual marginal critical value is \(z_{1-\alpha}\). In a MAMS design, \(c\) must reflect the intended comparison and decision rule. A marginal critical value alone does not account for all sequential or multiple-comparison features.

Why the Formula Has This Form

Under the alternative, the estimated difference has approximate mean \(\delta\) and standard error \(\sqrt{V/n_0}\). If the test rejects for a sufficiently large standardized difference, its approximate probability of rejection is the probability that a standard-normal variable exceeds \(c-\delta/\sqrt{V/n_0}\). Requiring this probability to be at least \(1-\beta\) gives the following steps.

Express power under the assumed alternative.
With the large-sample approximation, power is approximately \(\Phi\!\left(\delta/\sqrt{V/n_0}-c\right)\), where \(\Phi\) is the standard-normal cumulative distribution function.

Impose the target power.
Requiring \(\Phi\!\left(\delta/\sqrt{V/n_0}-c\right)\geq 1-\beta\) is equivalent to \(\delta/\sqrt{V/n_0}-c\geq z_{1-\beta}\).

Solve for the control-arm size.
Rearranging gives \(\delta/\sqrt{V/n_0}\geq c+z_{1-\beta}\), and squaring both positive sides gives \(n_0\geq (c+z_{1-\beta})^2V/\delta^2\).

The result is an approximate requirement for the contrast under the assumed response probabilities. It is not a guarantee that the full MAMS design attains the target power: the design's stagewise boundaries, number of comparisons, and joint operating characteristics still matter.

Worked Example: Equal Allocation for a Response-Rate Difference

Suppose a hypothetical two-arm comparison plans equal allocation, assumes a 40% response rate under control and a 55% response rate under treatment, and seeks 90% power for a one-sided calculation with critical value \(c=1.960\). Find the approximate number required in each arm. The assumptions describe the planning scenario; they are not observed study results.

The target difference is \(\delta=0.55-0.40=0.15\), and \(r=1\). The variance coefficient is \(V=0.55(0.45)+0.40(0.60)=0.2475+0.2400=0.4875\). For 90% power, \(z_{1-\beta}=z_{0.90}\approx1.282\).

n_0 = \frac{(1.960+1.282)^2(0.4875)}{(0.15)^2} = \frac{(3.242)^2(0.4875)}{0.0225} \approx 227.7.

The calculation gives approximately 227.7 control participants, so round up to 228. With equal allocation, plan for 228 treatment participants as well, giving a maximum of 456 participants for this two-arm comparison if everyone completes the planned follow-up and no other enrollment adjustments are applied.

Translating a Total into MAMS Stage Sizes

Once a total requirement has been calculated for a contrast, it must be expressed in the design's stage increments. If the same arms remain active throughout and the per-stage increments are equal, divide the planned total by the number of stages, then round and check that the cumulative sample size at the final stage meets the requirement. If stages have unequal increments, specify each increment directly and sum them to obtain the cumulative size. For several treatment-control comparisons, each can have a different variance coefficient and requirement; a common planned size must be adequate for the comparisons it is intended to support.

The following example uses equal increments to show how the calculation becomes a maximum-enrollment plan. It does not imply that every arm necessarily continues to the final stage. If an arm stops early, actual enrollment can be lower than maximum enrollment; that distinction is separate from the fixed maximum calculated here.

Worked Example: Unequal Allocation Across Four Stages

Consider a hypothetical comparison with assumed response probabilities \(p_0=0.20\) and \(p_j=0.35\), and a treatment-to-control allocation ratio \(r=2\). Use a one-sided critical value \(c=1.960\) and target power of 90%. Calculate a stage plan with four equal increments.

Here \(\delta=0.35-0.20=0.15\), and \(V=0.20(0.80)+0.35(0.65)/2=0.1600+0.11375=0.27375\). Thus:

n_0 = \frac{(1.960+1.282)^2(0.27375)}{(0.15)^2} = \frac{(3.242)^2(0.27375)}{0.0225} \approx 127.9.

Round the control requirement up to 128. The allocation ratio calls for 256 treatment participants. For four equal increments, assign 32 control and 64 treatment participants per stage. The cumulative planned sizes are 32 and 64 after stage 1, 64 and 128 after stage 2, 96 and 192 after stage 3, and 128 and 256 after stage 4, listed as control and treatment respectively. The maximum enrollment for this comparison is \(128+256=384\) participants.

This arithmetic treats the allocation ratio as applying to the planned sizes for this contrast. A full multi-arm design may need to account for shared-control enrollment and the way different arms enter or leave stages; the arm-by-stage plan must make those design rules explicit.

Allocation Ratios and a Second Calculation

Changing the allocation ratio changes the variance coefficient, so a sample size calculated for equal allocation cannot simply be redistributed while assuming its precision is unchanged. For a fixed total, putting more participants in treatment can reduce the treatment component of variance, but the control component then reflects fewer control participants. The best ratio depends on the response probabilities, the number of comparisons, and the design objective; the formula quantifies the variance for a specified ratio rather than selecting an optimal one.

Worked Example: A Non-Equal Allocation Ratio

Suppose a hypothetical comparison assumes \(p_0=0.25\) and \(p_j=0.40\), with the treatment-to-control ratio \(r=64/39\), approximately 1.641. Use a critical value \(c=2.20\) and 80% target power. This larger critical value could represent a more stringent planned marginal comparison. Calculate the approximate sample sizes.

The target difference is \(\delta=0.15\). The variance coefficient is:

V = 0.25(0.75)+\frac{0.40(0.60)}{64/39} = 0.1875+0.2400\left(\frac{39}{64}\right) = 0.33375.

For 80% power, \(z_{0.80}\approx0.842\). Therefore:

n_0 = \frac{(2.20+0.842)^2(0.33375)}{(0.15)^2} = \frac{(3.042)^2(0.33375)}{0.0225} \approx 137.3.

Round the control requirement up to 138 participants. The nominal treatment requirement is \(138(64/39)\approx226.5\), so at least 227 treatment participants are needed to meet that ratio-based target after integer rounding. The resulting total is 365 participants. The realized integer ratio is slightly different from the nominal ratio, so a final design calculation should use the actual integer arm sizes and its planned test.

Diagnostics, Alternatives, and Limitations

Before relying on the result, check that the inputs describe the actual planning contrast and the outcome definition. In particular, confirm which response is counted as an event, whether the contrast is treatment minus control, and whether the target difference is absolute rather than relative. Check the allocation ratio using the same convention as the formula: here \(r=n_j/n_0\), not \(n_0/n_j\). Finally, verify the variance coefficient term by term and recompute the final rounded arm sizes against the ratio.

The normal approximation can be poor when an arm is small or an assumed response probability is close to zero or one. A useful warning sign is a small expected number of responses or nonresponses in an arm, since the distribution of a sample proportion may then be notably discrete or skewed. A Wald approximation is also sensitive to its variance assumptions. If these conditions are concerning, use a method whose test and variance assumptions better match the design, and verify the planned operating characteristics rather than relying on this approximation alone.

The calculation targets a difference in response proportions. A risk ratio or odds ratio is a different estimand and requires a corresponding variance calculation. A score-based or exact procedure may also produce a different sample-size requirement, particularly in smaller samples. These approaches are alternatives, not interchangeable labels for the same calculation: state the estimand and planned test, then use a calculation aligned with both.

Planning approach	What it targets	Main caution
Wald approximation for a difference	An absolute treatment-control difference in response probabilities	Relies on large-sample approximation and alternative-based variance
Score-based planning	A binary-outcome contrast using a score-test variance structure	Must match the test and null-variance specification used in the design
Exact or other small-sample planning	A specified binary-outcome test with finite-sample behavior represented more directly	Can yield a different requirement; exactness for one comparison does not by itself address the full MAMS procedure

For multiple treatment arms sharing a control, each marginal variance uses the control sample size allocated to that comparison. However, the contrast estimates are correlated because they contain the same control estimate. Consequently, applying a separate single-comparison formula to every treatment does not establish the joint power or error rate of the multi-arm design. Likewise, a stage-size calculation is not an expected-enrollment calculation: maximum enrollment sums the planned increments across all arms and stages, whereas early stopping can reduce the realized total.

Common Mistake

Common pitfall: A tempting but incorrect argument is, “The treatment arm is larger, so its variance is smaller; therefore I can use the equal-allocation variance and simply add treatment participants.” The variance of a difference contains both arms: \(p_j(1-p_j)/n_j+p_0(1-p_0)/n_0\). With unequal allocation, calculate both terms using their own arm sizes, or use \(V=p_j(1-p_j)/r+p_0(1-p_0)\). Then round the arm sizes and check the actual allocation and total.

Check Your Understanding

Answer each question using the stated assumptions. Complete solutions follow the questions.

For \(p_0=0.30\), \(p_j=0.50\), and equal allocation, calculate \(\delta\) and \(V\).
If a comparison requires 150 control participants and the planned treatment-to-control ratio is \(3:2\), how many treatment participants are required before any rounding issue?
Why does a separate sample-size calculation for each treatment-control contrast not establish the joint power of a MAMS design with a shared control?
Name two conditions that can make the normal approximation for binary-outcome planning unreliable.

Solutions. (1) The difference is \(\delta=0.50-0.30=0.20\). Under equal allocation, \(V=0.50(0.50)+0.30(0.70)=0.25+0.21=0.46\). (2) The ratio is \(r=3/2\), so \(n_j=150(3/2)=225\) treatment participants. (3) The treatment-control contrast estimates share the control estimate, so they are correlated; marginal requirements do not determine joint operating characteristics. (4) Examples include small arm sizes, or assumed response probabilities near zero or one that produce small expected counts of responses or nonresponses.