Choosing Cutoffs Is a Design Decision
A promising-zone rule uses interim information to decide whether to increase the planned sample size. Its cutoffs determine which interim results trigger an increase. A narrow zone may conserve participants but miss opportunities to improve power; a wider zone may raise power in some scenarios while increasing expected enrollment and the frequency of increases. There is rarely one cutoff pair that is best for every possible treatment effect.
The choice should therefore be tied to design goals stated before comparing candidates. Typical goals include a minimum power at a target effect, an upper limit on expected sample size, and a maximum acceptable proportion of trials that increase sample size. Prespecifying a set of plausible treatment-effect scenarios makes those trade-offs visible instead of letting one favorable scenario determine the choice.
The calculations below assume that the interim score, the candidate cutoffs, the base and maximum sample sizes, and the final testing procedure are fully specified for each candidate design. Operating characteristics can be estimated by simulation under each treatment-effect scenario. The final procedure must be appropriate for the adaptive design being evaluated; the calculations here do not establish type I error control.
The score \(R\) could, for example, be a prespecified conditional-power measure. Its exact definition and the way it is computed must be held fixed while comparing cutoffs. This tutorial concerns choosing the cutoffs from design goals, not deriving a new interim score or a complete operational re-estimation algorithm.
Operating Characteristics and a Selection Principle
For each candidate pair \((c_L,c_U)\) and each scenario \(\theta\), evaluate at least three quantities: power, expected sample size, and the probability of a sample-size increase. If the design has only two possible total sample sizes, the expected sample size follows directly from the increase probability.
This identity follows by conditioning on the two possible sample sizes: with probability \(1-q(\theta)\), the total is \(n_0\); with probability \(q(\theta)\), it is \(n_{\max}\). It also makes a practical point clear: with \(n_0\) and \(n_{\max}\) fixed, the increase rate determines expected sample size. The two measures still answer different planning questions. Expected sample size describes average burden, while the increase rate describes how often the larger enrollment will be needed.
With several possible treatment effects, a design goal can be written as a set of constraints. For example, require power to be at least \(p_{\min}\) at a target effect, expected sample size not to exceed \(B\), and the increase rate not to exceed \(q_{\max}\) in specified scenarios. Among candidates satisfying all constraints, one might minimize the largest expected sample size across those scenarios. The precise objective should reflect the study’s scientific and operational priorities.
Worked Example: Selecting a Feasible Pair
Worked Example: Selecting a Feasible Pair
Hypothetical design. Suppose the base total sample size is \(n_0=400\) participants and the increased total is \(n_{\max}=520\). An evaluation simulates 10,000 trials for each treatment-effect scenario using the same prespecified final procedure. The planning goals are power of at least 0.85 at \(\theta=0.30\), power of at least 0.90 at \(\theta=0.40\), and an increase rate no greater than 0.30 at either scenario. Here \(\theta\) is measured in the study’s prespecified effect units.
Three candidate cutoff pairs produce the following hypothetical simulation results. “Power” is the number of simulated rejections divided by 10,000; “increase rate” is the number of trials increased divided by 10,000.
| Candidate cutoffs | Power at \(\theta=0.30\) | Increase rate at \(\theta=0.30\) | Power at \(\theta=0.40\) | Increase rate at \(\theta=0.40\) |
|---|---|---|---|---|
| \([0.20,0.80)\) | 8,420/10,000 = 0.842 | 3,600/10,000 = 0.360 | 9,140/10,000 = 0.914 | 3,000/10,000 = 0.300 |
| \([0.30,0.70)\) | 8,610/10,000 = 0.861 | 2,800/10,000 = 0.280 | 9,200/10,000 = 0.920 | 2,100/10,000 = 0.210 |
| \([0.15,0.65)\) | 8,750/10,000 = 0.875 | 4,500/10,000 = 0.450 | 9,350/10,000 = 0.935 | 3,500/10,000 = 0.350 |
The first candidate misses the power goal at \(\theta=0.30\), since \(0.842<0.85\), and its increase rate there also exceeds 0.30. The third meets both power goals but exceeds the increase-rate limit at both scenarios. The middle candidate meets all four requirements.
Its expected sample size at \(\theta=0.30\) is
At \(\theta=0.40\), the calculation is \(400+0.210(120)=425.2\) participants. These are scenario-specific averages, not guaranteed sample sizes. Under these stated point-estimate constraints, the middle pair is the feasible choice among the three candidates.
Worked Example: A Different Constraint Changes the Choice
Worked Example: A Different Constraint Changes the Choice
Continue with the same hypothetical sample sizes, but suppose an operational limit requires the increase rate at \(\theta=0.30\) to be no greater than 0.20. Compare the middle candidate above with a fourth candidate, \([0.35,0.65)\), whose simulated results at \(\theta=0.30\) are 8,250 rejections and 1,700 increases in 10,000 trials. At \(\theta=0.40\), suppose its simulated power is 0.889 and its increase rate is 0.120.
For the fourth candidate, the power estimate at \(\theta=0.30\) is \(8{,}250/10{,}000=0.825\), its increase rate is \(1{,}700/10{,}000=0.170\), and its expected sample size there is
If the power floor is 0.82 and the increase-rate ceiling is 0.20 at \(\theta=0.30\), this candidate meets both limits. It also has power 0.889 and increase rate 0.120 at \(\theta=0.40\). By contrast, the middle candidate’s increase rate of 0.280 at \(\theta=0.30\) violates the new operational limit, even though it has higher power there.
If the required power at \(\theta=0.30\) remains 0.85, however, neither candidate meets both that power requirement and the 0.20 increase-rate ceiling: the fourth candidate has power 0.825, and the middle candidate’s increase rate is too high. The correct conclusion is that these candidate designs do not satisfy the joint goals. A lower expected sample size alone does not make a candidate acceptable.
Worked Example: Check Whether a Small Difference Is Meaningful
Worked Example: Check Whether a Small Difference Is Meaningful
Suppose two candidate pairs are evaluated in independent sets of 10,000 hypothetical trials at one treatment-effect scenario. Candidate A increases sample size in 2,800 trials; candidate B does so in 2,860 trials. The estimated increase rates are 0.280 and 0.286, a difference of 0.006.
For an estimated probability \(\hat q\) from \(M\) independent simulated trials, its Monte Carlo standard error is approximately \(\sqrt{\hat q(1-\hat q)/M}\). Thus the two standard errors are
Because the simulations are independent, the standard error of the difference is approximately
The observed difference, 0.006, is smaller than this standard error. These simulations do not provide a precise basis for claiming that A has a meaningfully lower increase rate than B. If that distinction affects the decision, run more simulations or report that the estimated rates are too close to distinguish reliably at the current simulation size. A similar check applies to estimated power.
Diagnostics, Alternatives, and Limitations
Cutoffs should be compared over the full prespecified scenario set, not just at one target effect. The probability of entering the promising zone depends on the distribution of the interim score, which changes with the true effect and can also depend on variance assumptions, interim timing, and other design features. Earlier material on sensitivity to interim timing and instability from noisy interim effects is relevant when defining scenarios and interpreting an apparently favorable cutoff pair.
A useful diagnostic is to inspect a grid of candidate cutoff pairs and the resulting operating characteristics. If small changes in a cutoff produce large changes in power or increase rate, the selection may be fragile. Check whether the grid is sufficiently fine near the feasible region, whether results are stable under plausible scenario changes, and whether the reported Monte Carlo uncertainty could change which candidates appear feasible. A single rounded estimate just above a power threshold should not be treated as decisive evidence of adequate power.
There are several reasonable comparison strategies. A fixed-sample design avoids sample-size increases but may have lower power under some effects or require a larger sample from the outset. A cutoff pair chosen to maximize power can spend more participants than an operationally acceptable design. Conversely, choosing the pair with the smallest expected sample can sacrifice power. A weighted average across treatment-effect scenarios is another possible objective, but its answer depends on the weights; it can conceal poor performance in a scenario that matters clinically. Constraint-based selection makes such trade-offs explicit, while a Pareto comparison can show candidates for which no other candidate is at least as good on all chosen measures and strictly better on one.
A cutoff pair is not universally optimal: it is optimal, if at all, relative to the model assumptions, scenario set, constraints, and objective used to select it. The simulation must reflect the intended enrollment and analysis procedures. If those assumptions are implausible, more simulation precision will not repair the design evaluation. Further, power, expected sample size, and increase frequency do not by themselves establish valid error control or address every operational issue; those require their own evaluation.
Putting the Selection in Order
Choose treatment-effect scenarios and define power, expected-sample-size, and increase-rate requirements before comparing candidates.
For each cutoff pair and scenario, estimate power and the probability of an increase using the same prespecified design and final procedure.
Use the increase probability to calculate expected sample size when there are two possible sample sizes, and assess whether Monte Carlo uncertainty could alter the comparison.
Identify feasible candidates, compare them using the stated objective, and document any scenario sensitivity or unmet constraint.
Check Your Understanding
Answer each question using the definitions and selection principles above. Solutions are included below the questions.
- A design has \(n_0=300\), \(n_{\max}=420\), and estimated increase probability 0.25 at a specified treatment effect. What is its expected sample size in that scenario?
- Candidate A has power 0.88 and an increase rate of 0.35; candidate B has power 0.84 and an increase rate of 0.20. If the prespecified minimum power is 0.85 and maximum increase rate is 0.25, which candidate is feasible?
- Why can a design with the smallest average sample size across scenarios still be unacceptable?
- Two estimated increase rates are 0.31 and 0.32 from 2,000 simulations each. Should the difference alone be treated as decisive? What should be checked?
Solutions. (1) \(300+0.25(420-300)=330\) participants. (2) Neither: A exceeds the increase-rate limit, while B falls below the power requirement. (3) An average can hide failure to meet a power or increase-rate requirement in an important scenario; feasibility must be checked scenario by scenario. (4) No. The difference is small relative to the likely Monte Carlo variability, so calculate or estimate the uncertainty and use more simulations if the distinction would affect selection.