Fixed-Design Benchmarks For Adaptive Sample Sizes

Why Use a Fixed-Design Benchmark?

A sample size re-estimation rule is judged against a reference: what sample size and power would a trial need if its design were fixed in advance under the same target effect, error rate, outcome assumptions, and allocation plan? That reference helps separate the cost of the adaptation rule from the amount of information the scientific question requires. It is a benchmark, not a prediction of the adaptive trial’s realized sample size or a guarantee of its power.

The calculations below focus first on a two-arm parallel trial with a continuous outcome, then show how allocation and outcome type change the calculation. All numerical studies are hypothetical. A benchmark is meaningful only when its assumptions and the adaptive design’s assumptions are comparable.

Continuous Outcomes: Model and Power

Suppose participants are independently assigned to treatment and control, and the outcome is measured once per participant. Assume the outcome is normally distributed in each arm, with a common known standard deviation \(\sigma\). Let \(\delta=\mu_T-\mu_C\) be the treatment-minus-control mean difference. For sample sizes \(n_T\) and \(n_C\), the observed difference \(D=\bar{x}_T-\bar{x}_C\) has distribution

For a two-sided test of \(H_0:\delta=0\) against \(H_a:\delta\ne0\), the known-variance test rejects when the absolute standardized difference exceeds \(z_{1-\alpha/2}\). Here \(z_q\) is the \(q\)-quantile of the standard normal distribution, and \(\Phi\) is its cumulative distribution function. Under a particular nonzero \(\delta\), the standardized statistic has mean \(\lambda=\delta/SE\) and variance 1, where \(SE=\sigma\sqrt{1/n_T+1/n_C}\). Its power is therefore

The two terms are the probabilities of crossing the lower and upper rejection boundaries, respectively. For a positive effect, the lower-tail probability is often very small, but it belongs in the exact normal-theory expression.

Deriving the Equal-Allocation Sample Size

For equal allocation, set \(n_T=n_C=n\). Then \(SE=\sigma\sqrt{2/n}\), so the noncentrality parameter for an effect of magnitude \(|\delta|\) is \(|\delta|\sqrt{n}/(\sigma\sqrt{2})\). A standard planning approximation chooses \(n\) so that this parameter is at least \(z_{1-\alpha/2}+z_{1-\beta}\). This places the alternative mean far enough beyond the upper rejection boundary to achieve the target upper-tail probability; the other tail is negligible for the usual positive effect and conventional power targets. Solving for \(n\) gives:

This is a planning approximation, not an instruction to round to the nearest integer. Round up to a whole participant per arm, then, when useful, recalculate power using the rounded sample size and the two-tail formula. The calculation assumes the specified effect and standard deviation are appropriate, observations are independent, the variance is common across arms, and the test’s normal reference distribution is justified.

Calculating Power for a Specified Fixed Design

Sometimes the planned sample size is already set and the benchmark question is how much power it provides for a chosen effect. Use the same model and rejection rule, but calculate \(SE\), then \(\lambda\), and then both tail probabilities. This is preferable to comparing sample sizes alone when rounding, caps, or different allocation ratios affect the design.

Unequal Allocation and Binary Outcomes

Unequal allocation changes the standard error and therefore the benchmark. For a known common standard deviation, suppose the control-to-treatment allocation ratio is \(k\), so \(n_C=k n_T\). Then \(SE=\sigma\sqrt{(1+1/k)/n_T}\), and the approximate treatment-arm sample size is

The formula’s \(n_T\) is specifically the treatment-arm count because \(n_C=k n_T\). Changing which arm is in the numerator of the allocation ratio changes the formula’s interpretation. Always state the ratio in words as well as symbols.

For a binary outcome, a common planning approximation compares two independent proportions. Let \(p_T\) and \(p_C\) be the anticipated event probabilities, \(\Delta=p_T-p_C\), and \(\bar{p}=(p_T+p_C)/2\). With equal allocation, a normal-approximation formula for the number per arm is

This approximation uses the pooled proportion under the null in the critical-value component and the two anticipated arm-specific variances under the alternative. It relies on a large-sample normal approximation; sparse expected event or non-event counts can make it unreliable. The design’s actual test and planned analysis should be consistent with the formula used for its benchmark.

Using the Benchmark to Evaluate Re-Estimation

A fixed-design calculation supplies a reference point for the adaptation’s performance. Compare designs that target the same effect, use comparable type I error control, and make the same assumptions about the outcome and primary test. Depending on the re-estimation rule, useful comparisons include the fixed total sample size versus the adaptive design’s expected total, the distribution of the adaptive final sample size, and the adaptive design’s power under the same effect assumptions. If the adaptive rule has a maximum sample size, report how often that limit is reached as well as the average; an average alone can conceal a substantial chance of a much larger trial.

Do not infer the adaptive design’s power simply from its average sample size. Its final sample size can depend on interim data, and that dependence may affect the distribution of the final test statistic. The fixed-design power formula describes a fixed sample size under its stated model; it does not automatically describe an adaptive procedure. A fair evaluation calculates or simulates the adaptive rule’s operating characteristics separately, while retaining the fixed design as a transparent reference.

The normal formulas also have limits. In practice, a continuous-outcome analysis commonly estimates the variance rather than knowing \(\sigma\); a \(t\)-based calculation may then be more appropriate, especially for small samples. Unequal variances, clustering, repeated measurements, noncompliance, or a different primary estimand can also make these simple formulas an unsuitable benchmark unless the calculation is modified to reflect the design. For complex analyses, a justified numerical or simulation-based calculation may be preferable to a closed-form approximation.

Diagnostics and Common Pitfalls

Before treating a number as a benchmark, check the assumptions and the arithmetic. Confirm that the effect and standard deviation use the same outcome units, that the target is stated as a two-sided or one-sided test, and that the allocation ratio matches the sample-size formula. For binary outcomes, check that the assumed event probabilities make the normal approximation credible. After rounding up, recalculate power when the target is important or the planned sample size is close to the requirement.

• Wrong: “The formula returned 63, so the trial needs 63 participants.” Correction: In the equal-allocation continuous formula, \(n\) is the number per arm; report both arm counts and total.
• Wrong: Round a calculated sample size to the nearest integer. Correction: Round up so the planned sample does not fall below the calculated requirement, then check power at that rounded value.
• Wrong: Treat the fixed-design power as the adaptive design’s power because their average sample sizes are similar. Correction: Adaptation can link sample size to interim data; assess the adaptive rule’s operating characteristics under its own procedure.
• Wrong: Compare an adaptive design with a fixed design using different effect assumptions or error rates. Correction: Hold the scientific target and inferential standards constant so the comparison is interpretable.