Two Different Reasons to Stop
At an interim analysis, a trial may stop because the accumulating evidence is sufficiently strong to support efficacy, or because continuing is judged not worthwhile. These are different decisions. An efficacy boundary defines evidence strong enough to stop early in favor of efficacy. A futility rule defines evidence or a projection of future evidence weak enough to recommend stopping without a claim of efficacy. A design with an efficacy boundary does not automatically include a futility rule.
This distinction matters when interpreting an efficacy-only Pocock, O’Brien-Fleming, or Wang-Tsiatis design. Such a design specifies when the trial may stop early for efficacy, given the planned analyses and the chosen boundary family. It does not, by itself, specify when to stop for lack of promise. Nor does failing to cross an efficacy boundary establish futility: it means only that the efficacy stopping criterion was not met at that analysis.
Model and Decision Language
Consider a one-sided group sequential test in which positive values of the interim statistic favor the treatment. Let \(K\) be the planned number of analyses, \(Z_k\) the standardized test statistic at look \(k\), and \(I_k\) the information available at that look. Assume the information increases across looks and that the analysis schedule and the efficacy boundary are specified in advance. In the usual canonical model under \(H_0\), the statistics are jointly normal with mean zero, unit variances, and
This model describes the dependence caused by reusing accumulating data across analyses. The boundary values must be calibrated for the planned look schedule and the chosen error-rate target; they are not ordinary independent-test cutoffs. A design also depends on the direction of the test, the definition of information, and the assumption that the interim statistics and analysis timing correspond to the plan. Changes to these features can make the planned boundary sequence inappropriate.
For an efficacy-only design, the decision rule is straightforward: stop and declare efficacy at the first look \(k\) for which \(Z_k \ge u_k\). If no look crosses its upper boundary, the trial does not stop early for efficacy under this rule. Depending on the protocol, it continues to the next planned analysis or completes at the final look. The rule says nothing about stopping for futility.
What the Classical Boundary Families Specify
Pocock, O’Brien-Fleming, and Wang-Tsiatis are ways to specify the shape of an upper efficacy boundary across looks. Once a family and a look schedule are selected, the boundary must be calibrated to the intended error-rate criterion. The precise finite-look cutoffs depend on that calibration and schedule.
- Pocock: uses approximately constant upper cutoffs on the standardized \(Z\)-scale across the analyses. The early cutoff is therefore not as demanding as in an O’Brien-Fleming design, while the final cutoff is typically more demanding than a single final analysis would be.
- O’Brien-Fleming: uses a boundary that is very demanding at early looks and becomes less demanding as information accumulates. Its characteristic shape is approximately proportional to \(1/\sqrt{t_k}\), where \(t_k=I_k/I_K\) is the information fraction, with calibrated constants and finite-look adjustments.
- Wang-Tsiatis: provides a family of shapes between these patterns. A common shape representation is proportional to \(1/t_k^{1/2-\Delta}\), where \(\Delta\) controls the shape; \(\Delta=0\) gives an O’Brien-Fleming-like shape, while \(\Delta=0.5\) gives a constant, Pocock-like shape. The calibrated cutoffs, rather than this proportionality alone, determine the actual stopping rule.
These descriptions concern upper efficacy stopping. They do not specify lower futility boundaries, conditional-power thresholds, or a recommendation to stop because the treatment effect appears too small. A complete protocol may add such a rule, but it is an additional design choice.
Worked Example: Applying an Efficacy-Only Boundary
Suppose a hypothetical trial has three planned looks. Its calibrated upper efficacy cutoffs are \(u_1=3.20\), \(u_2=2.55\), and \(u_3=2.05\). The observed statistics are \(Z_1=2.70\), \(Z_2=2.60\), and \(Z_3\) is not observed if the trial stops early. Apply the efficacy-only rule.
At look 1, \(Z_1=2.70\), and \(2.70<3.20\). There is no efficacy crossing, so this rule does not stop the trial at look 1.
At look 2, \(Z_2=2.60\), and \(2.60\ge2.55\). This is the first crossing of an upper efficacy boundary. The trial stops for efficacy at look 2 under the stated rule. The third analysis is not reached.
The conclusion follows from the first-crossing rule, not from treating the second analysis as an independent test. The example assumes the displayed cutoffs are the calibrated values for this hypothetical schedule and that the trial follows the planned analyses. It contains no futility rule; none is needed to make this efficacy decision.
How a Futility Rule Changes the Decision Structure
A futility rule adds another possible stopping decision. One simple form uses lower boundaries \(l_k\), with \(l_k<u_k\). At look \(k\), the trial stops for futility if \(Z_k\le l_k\), stops for efficacy if \(Z_k\ge u_k\), and otherwise continues if another analysis is planned. The region between the boundaries is a continuation region. A futility rule may instead be based on conditional power—the probability of meeting a specified future efficacy criterion conditional on the data so far—or on another pre-specified measure. The rule and its threshold must be stated; “stop if results look poor” is not an operational decision rule.
Futility boundaries are often chosen to limit the chance of spending resources on a trial unlikely to succeed. Their choice involves tradeoffs: a more aggressive futility rule may reduce expected sample size but can also stop a trial that might have crossed an efficacy boundary later. A futility stop is therefore a decision under uncertainty, not proof that the treatment has no effect.
Worked Example: Adding a Futility Boundary
Use the hypothetical three-look efficacy cutoffs \(3.20\), \(2.55\), and \(2.05\) from the preceding example. Now suppose the protocol also specifies lower futility cutoffs \(l_1=-1.50\) and \(l_2=-0.50\). The observed values are \(Z_1=-1.70\) and \(Z_2\) would have been \(0.30\) if measured.
At look 1, \(Z_1=-1.70\le -1.50\), so the stated futility rule recommends stopping. Also, \(Z_1=-1.70<3.20\), so there is no efficacy crossing. If the protocol’s futility rule is followed, look 2 does not occur; the hypothetical \(Z_2=0.30\) is not available to the decision-maker.
If this same data path were considered under the efficacy-only design, the first look would not trigger efficacy because \(-1.70<3.20\). That design alone would provide no basis to stop for futility, so the trial would proceed according to its planned schedule. The difference is the added lower rule, not a change in what an upper efficacy crossing means.
The assumptions here are that the upper and lower cutoffs are pre-specified, \(l_1<u_1\), and the statistic is computed in the planned direction. The lower cutoff is an operational stopping criterion; it does not convert the observed value into evidence that the null hypothesis is true.
Justifying the First-Crossing Rule
The efficacy stopping time can be written as the first look at which the statistic reaches or exceeds its upper cutoff. If the set is empty, there is no early efficacy stop. This representation makes clear why a design is a sequence of decisions rather than a collection of unrelated tests.
The justification is direct. At each planned look, compare the observed statistic with that look’s cutoff. If it is below the cutoff, the efficacy condition has not been met at that look. If it is at or above the cutoff, the condition is met, and the first such look defines the efficacy stopping time. With a futility rule, the same comparison is made against the lower cutoff as well; because \(l_k<u_k\), a statistic cannot satisfy both stopping criteria at the same look. It may satisfy neither, in which case the planned continuation decision applies.
This rule does not by itself show that the overall type I error has its intended value. That depends on the joint distribution of the interim statistics and the calibrated boundary sequence. Nor does the first-crossing notation tell us whether an efficacy boundary is well chosen for the desired power or expected sample size; those are design properties that require their own evaluation.
Worked Example: Interpreting Boundary Shapes
A hypothetical design has information fractions \(t_1=0.25\), \(t_2=0.60\), and \(t_3=1.00\). For illustration, consider the uncalibrated shape terms \(1\), \(1/\sqrt{t_k}\), and \(1/t_k^{1/4}\). These are shape comparisons only, not usable critical values.
For a constant, Pocock-like shape, the three terms are \(1.00, 1.00, 1.00\). For an O’Brien-Fleming-like shape, they are \(1/\sqrt{0.25}=2.00\), \(1/\sqrt{0.60}\approx1.291\), and \(1/\sqrt{1.00}=1.00\). For a Wang-Tsiatis shape with \(\Delta=0.25\), the exponent is \(1/2-\Delta=0.25\), so the terms are \(1/(0.25^{0.25})\approx1.414\), \(1/(0.60^{0.25})\approx1.136\), and \(1.00\).
The arithmetic shows the relative shapes: the O’Brien-Fleming-like term is steepest early, the constant shape is flat, and the selected Wang-Tsiatis shape lies between them. These terms must not be read as the actual \(Z\)-cutoffs. A calibrated design needs constants or adjusted cutoffs appropriate to the planned schedule and error-rate target. None of the three shapes supplies a futility cutoff.
Assumptions, Diagnostics, and Limitations
Before applying a boundary rule, verify that the interim analysis matches the planned design. In particular, check that the statistic has the intended direction and scale, that the information at each look is as defined in the design, and that the look schedule and analysis population are consistent with the calibration. The canonical joint-normal model is an approximation in many applications; substantial departures from its assumptions can undermine operating-characteristic calculations.
Warning signs include an unplanned analysis, a changed endpoint or test direction, a statistic computed with a different variance method, or information fractions that do not correspond to the planned schedule. These issues do not automatically determine the correct replacement cutoff. They call for review of the design and analysis plan rather than informal adjustment of a boundary after seeing results.
Compared with an efficacy-only design, adding futility can reduce the chance of continuing a low-promise trial, but it introduces another rule whose consequences need assessment. A binding futility rule is part of the formal stopping design and must be followed as specified. A rule designated nonbinding may be ignored without invalidating the efficacy boundary’s formal calibration under its stated design assumptions; if it is followed, it can change the trial’s actual stopping behavior. The designation and its operational meaning should be explicit.
Finally, no boundary family is universally best. Pocock-like, O’Brien-Fleming-like, and intermediate Wang-Tsiatis shapes distribute the strictness of the efficacy criterion differently across looks. Their comparison concerns the timing and shape of efficacy evidence thresholds, not a choice between efficacy and futility. The appropriate design depends on the priorities and assumptions of the trial, and the actual calibrated values must be consulted for decisions.
- Mistake: “The statistic did not cross the upper boundary, so the trial should stop for futility.” Correction: Non-crossing means only that the efficacy condition was not met at that look. A futility decision requires a separate rule.
- Mistake: “An O’Brien-Fleming design specifies both an early efficacy cutoff and an early futility cutoff.” Correction: The named boundary family describes the efficacy boundary shape unless a futility rule is separately specified.
- Mistake: “A futility stop proves there is no treatment effect.” Correction: It is a stopping decision based on limited information or projected promise, not proof of a null effect.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- A trial statistic is below its upper efficacy cutoff at the first look. Does that fact alone trigger futility?
- At a look, \(Z_k=2.40\), the upper cutoff is \(2.30\), and the futility cutoff is \(-0.60\). What is the decision under the two-boundary rule?
- Which boundary shape is approximately constant in standardized \(Z\)-units: Pocock, O’Brien-Fleming, or Wang-Tsiatis with \(\Delta=0\)?
- Why should the proportionality terms for a Wang-Tsiatis shape not be used directly as critical values?
Solutions. (1) No. Failure to cross the efficacy boundary is not itself a futility rule; the trial continues unless another specified rule calls for stopping. (2) Since \(2.40\ge2.30\), stop for efficacy. The statistic is also above \(-0.60\), so it does not meet the futility criterion. (3) Pocock is approximately constant; Wang-Tsiatis with \(\Delta=0\) has an O’Brien-Fleming-like shape. (4) The proportionality terms describe relative shape only. Actual cutoffs require calibration for the planned looks and error-rate criterion.