Why the Decision to Stop Is Not a Rejection
In a multi-endpoint group sequential trial, an interim decision can stop testing an endpoint because further evidence of benefit appears unlikely. That may be a sensible operational choice: it can avoid collecting data that are unlikely to change the endpoint’s conclusion. But a futility decision does not establish that the endpoint’s null hypothesis is false. Nor does it establish that the Type I error risk assigned to that endpoint has vanished.
This distinction matters when considering another endpoint that has not crossed its efficacy boundary. Passing alpha from one endpoint to another can make a valid design more powerful, but an unplanned transfer can also make the probability of at least one false rejection exceed the intended level. The key question is not simply whether an endpoint has been dropped. It is whether the complete, prespecified testing rule—including its branches and interim looks—controls the error rate.
For a family of endpoint hypotheses, the family-wise error rate (FWER) is the probability of rejecting at least one true null hypothesis. Strong FWER control at level \(\alpha\) means this probability is at most \(\alpha\) for every configuration of true and false null hypotheses. In a sequential design, this definition includes the possibility of rejection at any monitored look. The endpoint-specific local levels and boundaries must therefore be understood as parts of a jointly calibrated procedure, not as isolated numbers that can be reassigned whenever a test stops.
What Alpha Recycling Presumes
In a multiplicity graph, alpha can be reassigned according to a specified rule after an endpoint’s null hypothesis is rejected. Rejection is a formal inferential event: the endpoint has crossed its efficacy boundary under the planned testing procedure. A futility stop has a different meaning. It says to stop pursuing that endpoint under the chosen design rule, not that the endpoint has been shown to have no effect or that its testing risk has been removed.
Some designs specify binding futility rules: following the rule is part of the testing procedure. Other designs use nonbinding futility guidance, which permits the trial to continue despite a futility signal. Neither label, on its own, authorizes transfer of alpha. The relevant issue is the error control of the full procedure that will actually be used. If futility is ignored in calibration but followed in practice, or if it is used to trigger a new testing level that was not calibrated, the procedure may no longer match the error-control argument.
A p-value threshold is also not a free-standing amount of error that becomes available when a test stops. In a group sequential design, error is allocated across looks through the planned boundaries. An endpoint that has not rejected at an interim look has still contributed to the sequential decision process; its remaining opportunities and their error spending cannot simply be treated as an unspent pot. In addition, a futility rule may depend on observed data. The event that triggers the transfer can therefore be associated with the test statistics for other endpoints, changing the probability of a false rejection after transfer.
A Simple Justification for the Caution
The following argument illustrates why the branch of a rule matters. Suppose two null hypotheses, \(H_A\) and \(H_B\), are both true. Let \(p_A\) and \(p_B\) be independent p-values that are each uniformly distributed on \([0,1]\) under their null hypotheses. Consider a one-look rule that calls endpoint A futile when \(p_A>0.50\). On that branch, it tests B at level \(0.05\); otherwise it tests B at level \(0.025\). A is rejected at level \(0.025\) if the futility branch is not taken. This rule is deliberately simple; it isolates the risk of changing B’s level based on A’s result.
The FWER is the probability of an A rejection or a B rejection when A has not already been rejected. These disjoint events give:
The independent uniform p-values justify multiplying the branch probabilities. The resulting FWER, about \(0.0619\), exceeds \(0.05\). The fact that the transfer occurs only when A is futile does not prevent inflation. The branch-specific testing levels combine with the opportunities to reject A in a way that yields more than the intended error probability. Dependence between endpoints, additional looks, or a different futility rule can change the calculation; that is precisely why the full rule needs its own justification.
Worked Examples
Worked Example: A Futility Stop Does Not Change B’s Boundary
Hypothetical design. Two one-sided endpoint hypotheses, \(H_A\) and \(H_B\), are tested at an interim look. The prespecified multiplicity procedure assigns each endpoint a local level of \(0.025\), and the design’s interim efficacy boundaries are calibrated to those allocations. Endpoint A also has a futility rule: stop A if its conditional power under the design alternative is below \(10\%\). At the interim, that conditional power is \(7\%\). Endpoint B has an interim \(z\)-statistic of \(1.80\), with a one-sided p-value of approximately \(0.0359\).
Because \(7\%<10\%\), the rule calls for stopping A for futility. This is not an efficacy rejection of \(H_A\). The assigned threshold for B remains \(0.025\), unless a different, valid reallocation was specified in advance. B’s p-value does not cross that threshold:
Therefore B is not rejected at this look under the stated procedure. It would be incorrect to replace \(0.025\) by \(0.05\) merely because A stopped. A one-sided \(z\)-test boundary at level \(0.05\) is about \(1.645\), so \(1.80\) would exceed that boundary—but that calculation does not authorize use of the \(0.05\) level. The design assumptions used here are that the specified interim test and conditional-power rule are applied as planned and that the \(0.025\) boundary is valid for this look within the sequential procedure.
Worked Example: Checking an Unplanned Transfer Under the Global Null
Hypothetical one-look calculation. Suppose both endpoint null hypotheses are true, and their p-values \(p_A\) and \(p_B\) are independent uniform random variables. Use the unplanned rule described above: if \(p_A>0.50\), stop A for futility and test B at \(0.05\); otherwise test B at \(0.025\). Test A at \(0.025\) whenever \(p_A\le 0.025\).
The possible ways to make at least one false rejection are:
- A is rejected, when \(p_A\le 0.025\), with probability \(0.025\).
- A is not rejected and is called futile, while B is rejected: \(p_A>0.50\) and \(p_B\le 0.05\), with probability \((0.50)(0.05)=0.025\).
- A is not rejected and is not called futile, while B is rejected: \(0.025<p_A\le 0.50\) and \(p_B\le 0.025\), with probability \((0.475)(0.025)=0.011875\).
These events are disjoint, so adding their probabilities gives:
Thus the rule exceeds a \(0.05\) family-wise error target under the global null, despite the fact that each p-value is valid and the endpoints are independent. The problem is the uncalibrated change in B’s threshold, not a defect in either individual p-value. This calculation checks only the stated one-look global-null scenario; it is not a substitute for evaluating strong error control across all configurations of true null hypotheses in an actual sequential design.
Worked Example: A Prespecified Conditional Allocation
Hypothetical one-look illustration. Suppose A is tested at level \(0.01\). Before the trial, the design specifies that if A meets its futility rule, B will be tested at level \(0.04\); otherwise B will be tested at level \(0.02\). Assume that under each true endpoint null, its p-value is uniform, and that B’s p-value is independent of A’s interim data. For illustration, suppose the futility branch has probability \(0.60\) under the global null.
Under the global null, the probability of rejecting B is:
The probability of rejecting A is \(0.01\). By the union bound, the probability of at least one rejection is no more than:
This is a valid illustration under the stated one-look assumptions, not a general recipe for sequential designs. In particular, independence makes B’s null p-value valid conditional on the branch selected using A’s data. The rule also has to be checked when only one of the two null hypotheses is true. The example shows the distinction: a transfer-like change in B’s level can be part of a defensible procedure when it is specified in advance and its error control is justified. The futility outcome alone does not supply that justification.
How to Assess a Proposed Transfer
When a design proposes to increase one endpoint’s alpha after another endpoint stops for futility, assess the entire decision rule rather than only the proposed new threshold. A useful review asks what data determine the futility branch, whether the endpoint that stops has already had efficacy-testing opportunities, what boundaries apply to every endpoint at every look, and how the FWER is controlled in each configuration of true null hypotheses.
Separate efficacy rejection from futility stopping. Record which null hypotheses can still be rejected at each look and which hypotheses are simply no longer pursued.
State the futility rule, any changed local levels, and the efficacy boundaries in the design. Do not infer a new threshold from the fact that an endpoint has stopped.
Account for interim looks, branch-dependent thresholds, dependence among endpoints, and every relevant configuration of true null hypotheses. Validity of each test considered separately is not enough.
In practice, a warning sign is that the argument for a new level is only “the other endpoint is no longer being tested.” Another is that a nominal threshold is increased without recalibrating the sequential boundaries or checking its conditional behavior on the branch that triggers it. A documented, prespecified allocation with a valid error-control argument is different from an informal post-interim decision to spend whatever appears unused.
Common Mistake, Alternatives, and Limits
Common mistake: “Endpoint A stopped for futility and will never be rejected, so its alpha can be added to endpoint B’s alpha.” The correction is that stopping A is not an efficacy rejection and does not automatically remove the error already associated with its testing history. The transfer changes B’s rejection rule, potentially on a data-selected branch, and must be part of a procedure whose family-wise error is controlled.
The main alternative is to retain the prespecified allocation after futility: stop collecting or testing A as planned, but leave B’s boundaries unchanged. This avoids an uncalibrated transfer. Another option is to design a branch-dependent allocation in advance and evaluate it as one joint procedure. Such a design may use a multiplicity graph or another calibrated sequential rule, but it must account for the endpoint statistics’ dependence and the timing of the looks. A rule that is defensible under independence may not remain valid under a different dependence structure.
Futility decisions also have practical limits. They depend on the futility criterion and assumptions used to assess the chance of a meaningful result. A futility stop is not proof of no clinically important effect, and it does not settle the scientific question about the endpoint. Its operational value can be real even when it yields no transferable alpha. Conversely, a planned reallocation can be statistically valid but still require careful documentation so that investigators do not confuse a futility stop with evidence of efficacy or with a formal rejection.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- In one sentence, distinguish a futility stop from an efficacy rejection.
- Two independent true-null p-values are tested at \(0.025\) each. Explain why changing B’s level to \(0.05\) only when A meets a data-dependent futility rule still requires an error-control calculation.
- In the prespecified illustration above, calculate the probability of rejecting B when the futility branch has probability \(0.60\), with B tested at \(0.04\) on that branch and \(0.02\) otherwise.
- Does a binding futility rule automatically permit alpha transfer? Briefly explain.
Solutions. (1) A futility stop discontinues further pursuit under a rule; an efficacy rejection crosses a planned efficacy boundary and rejects the null hypothesis. (2) The branch changes B’s threshold according to observed data, so the joint probability of rejecting a true hypothesis may exceed the target; valid individual p-values do not establish validity of this combined rule. (3) The probability is \((0.60)(0.04)+(0.40)(0.02)=0.032\), or \(3.2\%\), under the stated conditional-validity assumptions. (4) No. Binding status describes whether the futility rule is followed; transfer still requires a prespecified procedure with an error-control justification.