Planned Stopping and What Actually Happens
A monitoring plan can include a lower futility boundary: if the interim statistic falls below it, the plan recommends stopping because the treatment appears unlikely to provide sufficient benefit. When the rule is nonbinding, however, crossing that boundary does not have to end the trial. A team may continue monitoring, for example, because the result is uncertain or because stopping would have other costs.
The key distinction is between the stopping behavior specified in advance and the stopping behavior actually followed. If a nonbinding futility rule is ignored, the realized design has fewer futility stops than planned. Its conclusions about efficacy still depend on the efficacy boundaries and on how those boundaries were calibrated. “Nonbinding” is not a general guarantee that any change in monitoring leaves every operating characteristic unchanged.
For the usual one-sided efficacy design, \(H_0\) is the null hypothesis of no treatment benefit and \(H_a\) is the alternative of benefit in the direction of larger \(Z_i\). The type I error rate is the probability of rejecting \(H_0\) when \(H_0\) is true. In a group sequential design, that probability concerns the complete rule across all looks, not the error probability at any one look considered in isolation.
A common model for deriving the joint distribution of the statistics is the canonical group sequential model. Under \(H_0\), the statistics have standard normal marginal distributions and their dependence is determined by the information fractions \(t_i\). In particular, for looks \(i<j\), the canonical model gives \(\operatorname{Corr}(Z_i,Z_j)=\sqrt{t_i/t_j}\). The probability arguments below require that this joint null model, the look schedule, and the efficacy boundaries are the ones used for calibration. The logic about stopping events does not require that specific normal model, but numerical operating characteristics do.
Why an Efficacy Guarantee Can Survive Ignored Futility
Suppose the upper efficacy boundaries were chosen to control the one-sided type I error at level \(\alpha\) without relying on futility stops. Under that condition, continuing after a futility crossing does not invalidate the efficacy-based guarantee: the realized rule is precisely the efficacy-only rule for which the boundaries were calibrated. If futility is enforced, it can only remove some paths that might otherwise reach an efficacy boundary, so the probability of rejection cannot increase, provided the efficacy boundaries and analysis remain unchanged.
Here \(R_{\text{enforced}}\) is the event of rejecting \(H_0\) when the futility rule is followed, and \(R_{\text{efficacy-only}}\) is the event of rejecting \(H_0\) using the upper boundaries without stopping for futility. The inclusion holds because enforcing futility can prevent a later rejection, but cannot create a rejection on a path that would not cross an efficacy boundary. Ignoring futility restores the efficacy-only rejection event, so its type I error is controlled at the level achieved by that efficacy-only design.
This argument is about the type I error guarantee, not about identical trial behavior. Enforcing futility can reduce the chance of a later efficacy rejection and can shorten the expected trial duration. Ignoring it changes those operating characteristics: more trials continue, and some that would have stopped may eventually reject \(H_0\). The guarantee says that the probability of such a rejection under \(H_0\) remains controlled under the stated calibration; it does not say the two stopping rules produce the same decisions.
Worked Example: A Futility Crossing Is Ignored
Consider a hypothetical two-look trial with an interim futility boundary \(f_1=-0.40\), interim efficacy boundary \(u_1=2.80\), and final efficacy boundary \(u_2=2.00\). At the interim analysis, suppose \(Z_1=-0.62\). The planned rule says to stop for futility because \(-0.62\leq-0.40\). The team instead continues, and the final statistic is \(Z_2=2.12\).
Under the planned rule: the trial stops at the interim look and does not reject \(H_0\). No final efficacy decision is reached.
Under the realized rule: the futility crossing is not enforced. At the final look, \(2.12\geq2.00\), so the efficacy boundary is crossed and the trial rejects \(H_0\).
The two rules give different realized decisions for this hypothetical path. That difference is not, by itself, evidence that type I error control failed. To assess that guarantee, one must ask whether the upper boundaries were calibrated for the efficacy-only rule and whether the trial otherwise followed the assumptions used in that calibration.
What the Guarantee Does—and Does Not—Cover
The qualification about calibration matters. A design may instead choose its efficacy boundaries while treating futility as binding. In that case, the calculated type I error can depend on stopping trials at the lower boundaries. If a team later ignores futility, additional paths can continue and cross an upper boundary. The set-inclusion argument still shows that enforcing futility cannot increase rejection relative to the same efficacy-only rule, but it does not establish that the efficacy-only rule itself has type I error at most \(\alpha\). That must be checked under the actual stopping rule.
Thus, the label “nonbinding” must correspond to the design’s calibration, not merely to a later decision to disregard a boundary. The crucial question is whether the efficacy boundaries were calibrated to control type I error even when no futility stops occur. If they were, the efficacy guarantee remains applicable when futility is ignored, as long as the rest of the design is unchanged. If they were not, recompute the operating characteristic for the realized rule rather than assuming the original guarantee carries over.
Worked Example: Checking the Type I Error Claim
Suppose a hypothetical one-sided design has nominal \(\alpha=0.025\). Its upper efficacy boundaries were calibrated under the null distribution for the full efficacy-only monitoring rule, and that calculation gives \(P_{H_0}(R_{\text{efficacy-only}})=0.025\). A lower futility rule is also listed, but the calibration did not use it to remove paths.
If the team ignores futility and follows the upper boundaries at every scheduled look, then the realized rejection event is \(R_{\text{efficacy-only}}\). Therefore,
If instead the team enforces the futility rule while leaving the upper boundaries unchanged, its rejection event is a subset of the efficacy-only event. Hence its type I error is at most \(0.025\), though it need not equal \(0.025\).
Now consider a different hypothetical design in which the upper boundaries were calibrated only for a rule that stops at futility. The calculation \(P_{H_0}(R_{\text{enforced}})\leq0.025\) does not imply \(P_{H_0}(R_{\text{efficacy-only}})\leq0.025\). The latter probability requires a separate calculation under the actual joint null distribution and the actual efficacy-only rule. The nominal value alone cannot supply the missing guarantee.
Compare the Planned and Realized Designs
| Feature | Futility enforced | Futility ignored |
|---|---|---|
| Interim \(Z_i\leq f_i\) | Stop for futility | Continue unless an efficacy rule independently requires stopping |
| Efficacy boundaries | Use the prespecified \(u_i\) | Use the same \(u_i\), if the plan is followed |
| Type I error implication | Cannot exceed the efficacy-only error when stopping only removes paths | Has the efficacy-only error; it is controlled at \(\alpha\) only if calibrated accordingly |
| Stopping and power | May stop earlier and may lose opportunities for later rejection | More paths continue; later decisions may differ |
The main alternative to ignoring a nonbinding boundary is to follow it as a stopping recommendation, while retaining the same efficacy boundaries. This may reduce expected sample size, but it can also reduce power because some trials that would have crossed a later efficacy boundary are stopped early. The earlier material on power loss from enforcing a nonbinding rule treats that trade-off in more detail. Neither choice changes the meaning of the efficacy boundary; it changes which paths reach later looks.
Diagnostics and Practical Limits
Before relying on the original efficacy guarantee, verify the design as implemented, not only as described in an early draft. Confirm that the upper boundaries, analysis times, test statistic, and multiplicity procedure match the calibration. Check that continuing after a futility crossing does not trigger an unplanned sample-size change, a different endpoint, an altered analysis population, or a revised efficacy boundary. Such changes define a different monitoring rule and may require new operating-characteristic calculations.
A useful diagnostic is to compute or simulate the null rejection probability under the realized rule. In a group sequential design, simulation should preserve the joint dependence across looks; independently simulating each \(Z_i\) would generally give the wrong crossing probabilities. Compare the estimated rejection probability with the target and report Monte Carlo uncertainty if simulation is used. Simulation can check a calculation, but it cannot repair a mismatch between the simulated rule and the rule actually followed.
The argument also has limits. It addresses efficacy-based type I error for a fixed set of efficacy boundaries. It does not establish that ignoring futility preserves power, expected sample size, confidence-interval properties, or any planned probability of stopping early. Nor does it cover a rule that changes the efficacy test after a futility crossing. Those features need to be evaluated under their own specified procedures.
Common Mistake
- Wrong reasoning: “The futility boundary is called nonbinding, so ignoring it cannot affect any error rate or operating characteristic.”
- Correction: Ignoring it changes which trials continue and can change decisions, power, and duration. The type I error guarantee remains available only when the efficacy boundaries control that error under the efficacy-only rule and the realized analysis otherwise matches the calibration.
- Wrong reasoning: “Because enforcing futility can only remove rejection paths, the efficacy boundaries must still control type I error if futility is ignored.”
- Correction: The set-inclusion result bounds the enforced rule by the efficacy-only rule. It does not show that the efficacy-only rule is controlled unless its error probability was calibrated or verified to be at most \(\alpha\).
Check Your Understanding
Answer each question, then check the solutions below.
- If \(Z_i\leq f_i\) and the futility rule is ignored, what does that observation alone require the team to do?
- Why can enforcing futility not increase the rejection probability when the efficacy boundaries are fixed?
- Under what calibration condition does ignoring futility retain the efficacy-based type I error guarantee?
- A design’s upper boundaries were calibrated only while assuming futility stops would be enforced. Can the same type I error guarantee automatically be claimed after futility is ignored? Explain.
Solutions. (1) It does not require stopping; the team continues unless another prespecified rule, such as an efficacy boundary, calls for stopping. (2) Enforcing futility removes paths that might otherwise reach a later efficacy boundary, so the enforced rejection event is a subset of the efficacy-only rejection event. (3) The upper boundaries must control type I error at the target level under the efficacy-only monitoring rule, and the realized procedure must otherwise match the calibrated design. (4) No. The enforced-rule calculation does not establish the error rate for the efficacy-only rule; that probability must be recalculated or otherwise verified under the actual procedure.