Efficacy And Futility Boundaries For Arm Selection

Why Use Two Boundaries?

At an interim analysis, an arm may look sufficiently promising to retain, clearly unpromising, or uncertain. A single retention threshold can blur these cases: it may treat an arm that barely misses the threshold like one with little prospect of success. Separate efficacy and futility criteria make the intended decisions explicit. The upper criterion identifies arms with strong interim evidence; the lower criterion identifies arms with weak evidence or low projected chance of eventual success. Between them, an arm can continue without being labelled either a success or a failure.

These are rules for selecting arms to continue or test, not automatically formal conclusions about treatment effects. In particular, an arm that crosses an interim efficacy boundary has not necessarily met the design’s requirements for an early confirmatory claim. If the design permits such a claim, the interim testing rule must be calibrated for repeated looks and the intended error rate. Here the focus is how the two criteria classify arms and change the set of hypotheses carried forward.

The precise boundaries depend on the design. A rule may use a statistic such as \(Z_{k,I}\), or define futility through conditional power, the probability of eventual statistical success given the interim data and a stated assumption about the future effect. The two criteria need not be numerically symmetric: the cost of carrying an implausible arm and the cost of dropping an arm that could ultimately succeed are different design concerns.

Assumptions and Decision Logic

A \(Z\)-based rule is easiest to interpret when the interim estimator is approximately normal and its standard error is appropriate for the interim data. Under a null effect of zero, this gives \(Z_{k,I}\) an approximately standard normal distribution. The direction of benefit must be set consistently: if lower outcome values are beneficial, define the contrast so that positive \(\theta_k\) still represents benefit, or reverse the inequalities accordingly. The rule also requires a prespecified analysis time, boundary values, and treatment of missing or unavailable interim estimates.

For several candidate arms, apply the rule to each arm’s statistic and define the retained set before looking at results. If an arm meets the futility criterion, its hypothesis is removed from the set eligible for the later confirmatory testing stage under this selection plan. An efficacy-qualified arm remains eligible and may receive priority under a prespecified selection strategy. An indeterminate arm also remains eligible if the plan allows it to continue. Selection does not itself specify how the remaining hypotheses will be tested or how multiplicity will be controlled; those are separate design decisions.

A useful distinction is between selection status and hypothesis-testing status. A boundary crossing can determine which arms proceed without rejecting a null hypothesis. If the protocol instead treats an efficacy crossing as a formal early rejection, that conclusion needs an appropriate sequential testing procedure. Similarly, dropping for futility is generally a decision to stop gathering data because success is judged unlikely or not worth pursuing; it is not proof that the treatment has no effect.

Futility Based on Conditional Power

A futility rule need not be a fixed lower \(Z\) boundary. It can instead drop an arm when its conditional power is below a prespecified cutoff. Conditional power is the probability, conditional on the interim data and a specified model for future data, that the final test will cross its success threshold. Since this probability depends on the assumed future effect, the assumption must be stated; it is not a probability that the treatment is truly effective.

For a simple canonical normal model, suppose the interim information fraction is \(t\), where \(0<t<1\), and the final success rule is \(Z_F>c\). Let \(\delta\) be the assumed final standardized effect. Under the model of independent information increments, the final statistic conditional on the interim statistic \(Z_I=z\) has distribution

The variance \(1-t\) is the variance of the unobserved increment, and its conditional mean contribution is \(\delta(1-t)\) under the assumed effect. Standardizing the remaining distance to \(c\) therefore gives

where \(\Phi\) is the standard normal cumulative distribution function. A design might drop an arm if this quantity is at most a chosen cutoff \(\gamma\). This rule is only as relevant as its assumptions: the information fraction, final threshold, and assumed effect must match the intended design and analysis.

How the Boundaries Change the Testing Set

The simplest way to track the consequences is to map every interim classification to a status and then form the retained set. Let \(\mathcal{K}\) be the set of candidate arms and let \(\mathcal{R}\) be the subset retained after applying the prespecified rule. Hypotheses for arms outside \(\mathcal{R}\) are not part of the planned later testing family; hypotheses for arms inside it remain eligible, subject to the later testing plan. The efficacy boundary can distinguish high-priority arms from merely continuing arms, while the futility boundary determines which arms are removed.

Why the Two-Boundary Design Matters

Using separate boundaries makes the decision policy more transparent than a single threshold. A futility boundary can protect resources by stopping arms with little projected chance of success, while an efficacy boundary can identify arms that are especially promising for continuation or priority. The interval between the boundaries avoids forcing every non-futile arm to be described as promising, or every arm that misses an efficacy criterion to be dropped.

Boundaries should be evaluated as parts of the whole design. A lower boundary that is too aggressive can remove an arm whose interim estimate is low because of random variation. A futility cutoff based on conditional power can also be misleading if the assumed effect is unrealistic or if the information fraction is miscalculated. An efficacy boundary that is too permissive may give an unstable ranking or encourage an unjustified claim of success. Operating characteristics should therefore be assessed under plausible treatment effects, variability, accrual patterns, and missing-data conditions, with the actual decision algorithm implemented as prespecified.

When comparing approaches, a fixed lower \(Z\) boundary is simple and depends mainly on the observed interim statistic. Conditional-power futility uses the expected remaining information and an explicit effect assumption, so its decision has a more direct connection to projected final success but is more assumption-sensitive. A one-threshold retention rule is simpler still, but does not separate strong interim evidence from mere non-futility. None of these choices by itself resolves multiplicity or establishes valid final inference after selection.

Diagnostics and Common Pitfalls

• Mixing scales. A \(Z\)-boundary and a conditional-power cutoff are not interchangeable numbers. State which quantity is compared with each boundary and apply the criterion on that scale.
• Reversing benefit direction. If beneficial effects correspond to negative contrasts but the rule treats positive \(Z\) as beneficial, the arms can be classified backwards. Verify the contrast definition before applying boundaries.
• Calling futility proof of no effect. A futility decision means the arm is dropped under a prespecified continuation rule. It does not establish \(H_0\) as true.
• Calling qualification a formal efficacy finding. Meeting a selection boundary alone does not establish a confirmatory result. A formal interim claim needs an analysis calibrated for the design’s repeated testing.
• Ignoring boundary sensitivity. Recalculate decisions under plausible changes in the assumed effect, information fraction, or standard error. If a small change reverses a decision, report that sensitivity and ensure the rule was fixed in advance.
• Forgetting the hypothesis set. After dropping an arm, explicitly record which hypotheses remain eligible. Do not silently include dropped-arm results in the later testing family or treat retained arms as automatically significant.