Efficiency Requires More Than One Operating Characteristic
A re-estimation rule can reduce the average number of participants enrolled relative to a fixed design, but that does not by itself show that the adaptive design is more efficient. The relevant question is what the design achieves for that enrollment: in particular, its power under specified treatment-effect scenarios, its type I error control, and its maximum sample size. Because an unblinded rule responds to interim results, its expected sample size can change substantially across scenarios.
This tutorial uses a two-arm normal model to make those comparisons precise. The calculations focus on expected enrollment under a prespecified sample-size rule. Power values, when used for comparison, must come from evaluating the complete adaptive design and its prespecified final analysis—not from assuming that a random final sample size can be inserted into an ordinary fixed-sample test.
For a clear comparison, report expected enrollment together with power for each treatment-effect scenario, and also report the design’s maximum enrollment. If two designs have different power, a smaller expected sample size is a trade-off, not automatically an efficiency improvement. If their power is comparable at the effects that matter, expected enrollment can help show which design reaches that operating point with fewer participants on average.
A Two-Stage Model and an Expansion Rule
Suppose the outcome is continuous, the treatment and control groups have equal allocation, and individual outcomes are independent and normally distributed with a common known standard deviation \(\sigma\). Let \(\Delta\) denote the true difference in population means, measured in outcome units. At the interim analysis there are \(n_1\) participants per arm. The interim standardized statistic is
Under these assumptions, \(Z_1\sim N(\Delta\sqrt{n_1/2}/\sigma,1)\). Consider a prespecified rule that completes the trial with \(n_L\) participants per arm unless \(Z_1\) lies in a promising interval \((c_L,c_U)\); if it does, the trial instead completes with \(n_H\) per arm, where \(n_1\leq n_L<n_H\). Write \(q(\Delta)=P_{\Delta}(c_L<Z_1<c_U)\) for the probability of expansion in the scenario with effect \(\Delta\). Then expected total enrollment is
The factor of two converts per-arm enrollment to total enrollment. This expression assumes that everyone enrolled at the interim continues and that there are no other stopping rules or losses to follow-up. If a design can stop early or has other sample-size outcomes, its expectation must include those outcomes too.
Why Expected Enrollment Changes With the Effect
The mean of \(Z_1\) moves with the true effect, while its standard deviation remains one under the stated model. Therefore the expansion probability is
where \(\Phi\) is the standard normal cumulative distribution function. This is the probability that the interim statistic falls in the specified interval. As the treatment effect changes, that probability can increase or decrease: it is not generally true that stronger effects always lead to more expansions. In particular, if a promising interval is bounded above, a large positive effect can move the interim statistic beyond it.
The expected-enrollment formula follows directly by writing the final per-arm sample size as \(n_L+(n_H-n_L)I\), where \(I=1\) when the rule expands and \(I=0\) otherwise. Taking expectations gives \(n_L+(n_H-n_L)P(I=1)\). Multiplying by two gives total enrollment. This derivation also shows why the rule and the scenario-specific probability of expansion must be specified before interpreting the expectation.
Worked Example: Expected Enrollment at a Moderate Effect
Consider a hypothetical trial with \(\sigma=10\) outcome units, \(n_1=100\) participants per arm at interim, \(n_L=200\), and \(n_H=300\) participants per arm at completion. Expand when \(0.5<Z_1<1.5\). Calculate expected total enrollment if the true treatment effect is \(\Delta=2\) outcome units.
The interim statistic has mean
Thus the expansion probability is
The expected final sample size per arm is \(200+(300-200)(0.3540)=235.4\). Total expected enrollment is therefore
A fixed design enrolling 300 per arm would enroll 600 participants. Relative to that maximum, the adaptive design’s expected enrollment is \(1-470.8/600=0.2153\), or about 21.5% lower in this treatment-effect scenario. This is an enrollment comparison; it does not establish that the adaptive design has the same power as the fixed design.
Compare Expected Enrollment and Power Together
Power is the probability that the prespecified analysis rejects \(H_0\) under a specified alternative. For an adaptive design, that probability must account for the interim data, the resulting sample size, and the final decision rule. The ordinary fixed-sample power formula generally does not give the adaptive design’s power, because the final sample size depends on interim results. In particular, choosing a sample size from unblinded interim results and then applying an unadjusted fixed-sample test can affect type I error.
A valid comparison therefore assumes that the adaptive design has a prespecified final analysis that controls the type I error at the chosen significance level, and that power has been calculated for that complete design. The calculation may use a suitable analytic evaluation or simulation. Expected enrollment and power answer different questions: the first concerns average resource use; the second concerns the chance of a statistically significant result at a specified effect.
Worked Example: The Same Rule in Two Effect Scenarios
Use the rule and assumptions from the previous example. Compare expected enrollment under a null effect, \(\Delta=0\), and a larger effect, \(\Delta=4\) outcome units. These are hypothetical scenarios; the calculation concerns enrollment, not power.
Under the null, \(Z_1\sim N(0,1)\), so
Expected total enrollment is \(2\{200+100(0.2417)\}=448.3\) participants.
For \(\Delta=4\), the interim mean is \(4\sqrt{100/2}/10=2.828\). Hence
Expected total enrollment is \(2\{200+100(0.0821)\}=416.4\) participants. The stronger effect produces fewer expansions here because the interim statistic is often above the promising interval. The corresponding expected enrollments are about 448 participants under the null and 416 under the larger effect, despite the same prespecified rule.
These figures do not tell us which scenario has greater power, nor whether the adaptive rule is preferable. For that, power must be evaluated under each scenario using the design’s actual final analysis. A rule that rarely expands can still have inadequate power at an effect of interest.
Reading Efficiency Comparisons Carefully
Relative expected-enrollment reduction is a useful descriptive quantity when a fixed comparator is specified:
It must be labelled with the scenario \(\Delta\), the comparator, and the design assumptions. When powers differ, report the difference in power as well. A reduction in expected enrollment accompanied by a substantial loss of power may not represent a desirable trade-off. Conversely, similar power with lower expected enrollment can be practically valuable, provided type I error is controlled and the maximum sample size is acceptable.
Worked Example: Separating Enrollment Savings From Power
Suppose a hypothetical, fully prespecified adaptive design has \(n_L=200\) and \(n_H=300\) per arm, and an operating-characteristic evaluation gives \(q(\Delta)=0.600\) at the effect of interest. The evaluation of its valid final analysis gives power \(0.688\) at that effect. A fixed design enrolls 300 per arm and has power \(0.688\) at the same effect. Compare enrollment and interpret the result.
For the adaptive design,
The fixed design enrolls \(2(300)=600\) participants. The adaptive design’s expected enrollment is \(1-520/600=0.1333\), or 13.3% lower, while the stated powers are equal at this scenario. Under these hypothetical operating characteristics, the adaptive design uses fewer participants on average to achieve the same power at the specified effect. Its maximum enrollment remains 600, however, and the comparison does not imply equal power at other effects or establish lower type I error; those characteristics must also be checked.
The power values in this example are assumed outputs from a valid design evaluation, not values derived from the expected-enrollment calculation. This distinction is essential: knowing \(q(\Delta)\) determines expected enrollment under this rule, but does not determine power.
Diagnostics, Alternatives, and Limitations
Before using expected sample size as an efficiency measure, verify that its inputs match the implemented design. The interim timing, expansion interval, sample-size floor and cap, allocation, and any rounding rules all affect the distribution of enrollment. For an unblinded rule, the scenario-specific treatment effect and variance assumptions also affect the expansion probability. If the assumptions used to calculate \(q(\Delta)\) are implausible, the expected enrollment may be misleading.
Useful diagnostics include checking the expansion probability under the null and across a range of clinically relevant effects; confirming that the expected enrollment lies between the minimum and maximum possible enrollment; and comparing analytic calculations with a simulation of the prespecified rule. A simulation should apply the same interim decision and final analysis as the planned trial. If the simulated type I error or power differs materially from the intended operating characteristics, do not interpret the expected-enrollment comparison as evidence of efficiency until the design evaluation is resolved.
The principal alternative is a fixed design, whose enrollment is nonrandom once planned and whose power is evaluated at prespecified sample sizes. A fixed design makes the resource commitment more predictable. An adaptive design can vary enrollment in response to interim evidence, potentially reducing average enrollment in some scenarios while retaining the option to expand in others. That flexibility comes with scenario-dependent enrollment and requires careful evaluation of the final analysis. Maximum sample size is therefore a necessary companion to the expectation: it describes the upper enrollment commitment that an average alone conceals.
Expected sample size is also not the same as expected cost or trial duration. If costs per participant vary by arm, site, or follow-up, expected enrollment alone will not capture total cost. If calendar time depends on recruitment rates or the timing of interim decisions, it will not capture duration either. Those objectives require their own explicitly defined measures.
Common Mistake
- Wrong reasoning: “The adaptive design has lower expected enrollment than the fixed design, so it is more efficient.”
- Correction: State the treatment-effect scenario and compare power, type I error, expected enrollment, and maximum enrollment. Lower expected enrollment is an advantage only in relation to the design’s operating characteristics and the study’s goals.
- Wrong reasoning: “A large effect must cause more sample-size increases.”
- Correction: The expansion probability is the chance that the interim statistic lies in a particular region. If that region is bounded, a large effect may move the statistic beyond it and reduce the chance of expansion.
Check Your Understanding
Answer each question, then check the short solutions below.
- A design has \(n_L=150\) and \(n_H=250\) participants per arm, and its expansion probability at a specified effect is 0.30. What is its expected total enrollment?
- Why can a bounded promising interval have a lower expansion probability under a very large effect than under a moderate effect?
- An adaptive design has lower expected enrollment but lower power than a fixed comparator at the effect of interest. Is it correct to call the adaptive design more efficient without qualification?
- Name two diagnostics that help check an expected-enrollment comparison.
Solutions. (1) The expected per-arm sample size is \(150+(250-150)(0.30)=180\), so expected total enrollment is \(2(180)=360\) participants. (2) A large effect can shift the interim statistic above the interval, making it less likely to fall inside the region that triggers expansion. (3) No. The enrollment reduction must be reported with the power difference and other relevant operating characteristics; whether the trade-off is acceptable depends on the study goal. (4) Examples include checking expansion probabilities under the null and relevant alternatives, confirming the expectation is within the design’s enrollment limits, and comparing analytic calculations with simulation of the prespecified rule and final analysis.