Why Compare Designs Across Scenarios?
A promising-zone design is not described by a single power figure or a single sample-size figure. Its operating characteristics depend on the true treatment effect: the interim statistic falls in the promising zone with different probabilities, and the effect also determines how informative the additional participants will be. Comparing designs across several effects makes that trade-off visible. In particular, it distinguishes performance under the null, at the effect used for planning, and at a different effect that may occur in practice.
The comparison below uses a deliberately simple two-stage normal model. It is hypothetical, and the calculations illustrate how to make an internally valid comparison rather than provide a recommendation for a particular trial. The central principle is to evaluate the whole prespecified rule—including its final test—under each scenario.
Model, Designs, and Test
Suppose a randomized trial compares two groups with a continuous outcome. Assume independent observations, equal allocation, normally distributed outcomes in both groups, and a common known standard deviation \(\sigma=10\) outcome units. Let \(\delta\) denote the treatment-minus-control difference in population means. At a stage with \(n\) total participants, there are \(n/2\) participants per group, and the standard error of the difference in sample means is \(2\sigma/\sqrt{n}\). These assumptions make the stage-specific standardized treatment-effect statistic normal. For this illustration, take the stages to use disjoint participants, so their statistics are independent before adaptation.
The first stage enrolls \(n_1=100\) participants. The second stage enrolls at least 100 more. The fixed baseline design always enrolls 100 in stage 2, for a total of 200. A fixed maximum-size reference design always enrolls 200 in stage 2, for a total of 300. The adaptive design begins with the baseline plan and increases stage 2 from 100 to 200 only if the first-stage statistic \(Z_1\) lies between 0.5 and 1.5. This interval is the promising zone in this example; outside it, stage 2 remains at 100.
To account for the fact that the stage-2 size depends on \(Z_1\), use a fixed-weight combination test. Let \(Z_2\) be the stage-2 treatment-effect statistic, standardized using the stage-2 sample size actually selected. Reject \(H_0\) when \(C=(Z_1+Z_2)/\sqrt{2}>1.96\), a one-sided test with nominal \(\alpha=0.025\). Under \(H_0\), \(Z_1\) and \(Z_2\) are independent standard normal variables, even though the stage-2 sample size is selected using \(Z_1\): conditional on \(Z_1\), the stage-2 statistic remains standard normal under the null. The fixed combination weights therefore make \(C\) standard normal under \(H_0\), preserving the stated type I error rate.
Power and Expected Sample Size Formulas
Under a true difference \(\delta\), a stage-specific statistic based on \(n\) participants has mean \(\lambda(n)=\delta\sqrt{n}/(2\sigma)\) and variance 1. In particular, the first-stage mean is \(\lambda_1=\delta/2\). The stage-2 mean is \(\lambda_0=\delta/2\) when stage 2 enrolls 100, and \(\lambda_+=\delta\sqrt{200}/20\) when it enrolls 200.
Write \(\phi(z;\lambda,1)\) for the normal density at \(z\) with mean \(\lambda\) and variance 1, and \(\Phi\) for the standard normal cumulative distribution function. Conditional on \(Z_1=z\), rejection occurs when \(Z_2>1.96\sqrt{2}-z\). Thus the adaptive power is the integral of the conditional rejection probability over the distribution of \(Z_1\), using the appropriate stage-2 mean in each region. If \(\mathcal{P}(z)\) denotes the stage-2 mean selected by the rule, then
The formula for expected sample size is simpler. The trial always enrolls 200 participants, and it enrolls 100 additional participants exactly when \(Z_1\) is in the promising zone. Since \(Z_1\sim N(\delta/2,1)\),
Worked Example: The Null Scenario
Under \(H_0\), \(\delta=0\), so \(Z_1\sim N(0,1)\). The chance of expansion is
The adaptive design's expected total enrollment is therefore \(200+100(0.2417)=224.2\) participants. The fixed baseline design enrolls 200, and the fixed maximum-size design enrolls 300. Under the null, both fixed designs have power equal to their type I error rate, 0.025. So does the adaptive design: under the null, \(C\) is standard normal by the fixed-weight combination argument, and \(P(C>1.96)=0.025\). Expansion under the null increases expected enrollment, but it does not increase the false-positive probability for this specified test.
Comparing the Planned and Larger Effects
For the fixed designs, the stage-specific statistics have equal variances and are independent. With 100 participants in stage 1 and \(n_2\) in stage 2, the combination statistic has mean \((\lambda_1+\lambda(n_2))/\sqrt{2}\) and variance 1. Its power is the standard normal probability above 1.96 with that mean. The adaptive power, by contrast, uses the conditional integral above: the stage-2 mean changes in the promising zone. These calculations use the same one-sided combination test and critical value for all three designs.
Worked Example: The Planned Effect
Take \(\delta=3\) outcome units as the effect used for planning. Then \(\lambda_1=\lambda_0=3/2=1.5\). The fixed baseline design has combination-statistic mean \((1.5+1.5)/\sqrt{2}=2.121\), giving power \(\Phi(2.121-1.96)=\Phi(0.161)\approx0.564\).
For the adaptive design, stage-2 mean in the promising zone is \(3\sqrt{200}/20=2.121\), rather than 1.5. Its expansion probability is \(\Phi(0)-\Phi(-1)=0.5000-0.1587=0.3413\), so expected enrollment is \(200+100(0.3413)=234.1\) participants.
Evaluating the adaptive power integral gives approximately 0.643. One way to see the increment over the fixed baseline power is to integrate, over \(0.5\le z<1.5\), the interim density times \(\Phi(z+2.121-1.96\sqrt{2})-\Phi(z+1.5-1.96\sqrt{2})\). Numerical integration gives an increment of about 0.079, so the adaptive power is about \(0.564+0.079=0.643\). The fixed maximum-size design has combination mean \((1.5+2.121)/\sqrt{2}=2.561\), and power \(\Phi(2.561-1.96)\approx0.726\).
Worked Example: A Larger True Effect
Now take \(\delta=4\) outcome units. The fixed baseline design has combination mean \((2+2)/\sqrt{2}=2.828\), so its power is \(\Phi(2.828-1.96)\approx0.807\). The fixed maximum-size design has stage-2 mean \(4\sqrt{200}/20=2.828\), combination mean \((2+2.828)/\sqrt{2}=3.414\), and power \(\Phi(3.414-1.96)\approx0.927\).
The expansion probability is \(\Phi(-0.5)-\Phi(-1.5)\approx0.3085-0.0668=0.2417\), giving expected adaptive enrollment of 224.2 participants. Numerical evaluation of the adaptive integral gives power about 0.866. Equivalently, the increase over fixed-baseline power is the integral over the promising zone of the interim density times \(\Phi(z+2.828-1.96\sqrt{2})-\Phi(z+2-1.96\sqrt{2})\); the increment is about 0.059. The adaptive design therefore has more power than the fixed baseline design in this scenario, but less power than the fixed maximum-size design.
The approximate results are collected below. Expected sample size is in participants; power is a probability of rejection. Small differences from the displayed values can arise from rounding.
| True effect \(\delta\) (outcome units) | Design | Expected sample size | Power |
|---|---|---|---|
| 0 (null) | Fixed baseline | 200 | 0.025 |
| 0 (null) | Promising-zone adaptive | 224.2 | 0.025 |
| 0 (null) | Fixed maximum size | 300 | 0.025 |
| 3 (planned) | Fixed baseline | 200 | 0.564 |
| 3 (planned) | Promising-zone adaptive | 234.1 | 0.643 |
| 3 (planned) | Fixed maximum size | 300 | 0.726 |
| 4 (larger) | Fixed baseline | 200 | 0.807 |
| 4 (larger) | Promising-zone adaptive | 224.2 | 0.866 |
| 4 (larger) | Fixed maximum size | 300 | 0.927 |
How to Read the Comparison
The adaptive design lies between the fixed baseline and fixed maximum-size designs in both expected enrollment and power in these two non-null scenarios. It spends additional participants only in the interim region selected for expansion, gaining power relative to the baseline design while using fewer expected participants than the maximum-size design. These are scenario-specific results, not a guarantee that the adaptive design is uniformly more efficient. The probability of expansion—and therefore expected enrollment—depends on the true effect. Power also depends on how much information the added stage contributes and on where the interim statistic falls.
Evaluate the null, the effect used to plan, and substantively plausible departures from that effect.
Use the distribution of the interim statistic under each effect to find the chance of entering the promising zone.
Expected enrollment follows from the expansion probability; power requires the conditional stage-2 rejection probability as well.
Use the same valid testing procedure in each comparison, and verify its type I error under the sample-size adaptation.
A key limitation of this illustration is its known common variance and normally distributed endpoint. Real planning may involve estimated variance, binary outcomes, unequal allocation, discrete sample-size choices, or binding maximum sizes. The numerical values should not be carried over to those settings without adapting the model and calculations. The fixed equal combination weights also deliberately do not change with the chosen second-stage size; that makes the null calibration transparent, but may not be the most efficient valid procedure for every design.
Common Pitfall: Comparing Power Without Checking Type I Error
A tempting shortcut is to select the final sample size using \(Z_1\), calculate a pooled final \(Z\)-statistic, and compare it with 1.96 as if the sample size had been fixed in advance. That reasoning treats the final statistic as having its fixed-design null distribution despite the data-dependent choice of sample size. Selection can change that distribution, so the nominal 0.025 cutoff need not control the type I error rate. A power comparison based on that shortcut may credit the adaptive design with an apparent gain that partly comes from an inflated false-positive rate.
The correction is to specify a valid final test as part of the adaptive design and evaluate operating characteristics for that complete procedure. Here, the fixed-weight combination statistic has a standard normal null distribution because the two stage-specific statistics are independent standard normals under \(H_0\), and the weights do not depend on \(Z_1\). This justification is specific to the stated model and test; a different adaptation or final test requires its own calibration argument.
Check Your Understanding
Use the model and rule in this tutorial. Answers are included so that each calculation can be checked.
- At \(\delta=2\), what is the probability of entering the promising zone, and what is expected enrollment? Solution: Here \(Z_1\sim N(1,1)\), so the probability is \(\Phi(0.5)-\Phi(-0.5)\approx0.6915-0.3085=0.3830\). Expected enrollment is \(200+100(0.3830)=238.3\) participants.
- Why does the adaptive design have power 0.025 under the null in this example? Solution: Under \(H_0\), the stage-specific statistics are independent standard normals. Thus \(C=(Z_1+Z_2)/\sqrt{2}\sim N(0,1)\), and \(P(C>1.96)=0.025\).
- For \(\delta=3\), which design has the greatest power, and which has the smallest expected sample size? Solution: The fixed maximum-size design has the greatest power, about 0.726. The fixed baseline has the smallest expected sample size, 200 participants; the adaptive design averages about 234.1 and has power about 0.643.
- Why is expected enrollment under the adaptive design larger under \(\delta=3\) than under \(\delta=0\)? Solution: The interim mean shifts from 0 to 1.5, moving more probability into the interval \([0.5,1.5)\). The expansion probability rises from about 0.2417 to 0.3413, increasing expected enrollment.
- What must be checked before interpreting an apparent adaptive power gain? Solution: The final test must control the type I error rate under the sample-size selection rule. Otherwise, a higher rejection probability may reflect an inflated false-positive rate rather than a valid power gain.