Posterior Futility Rules for Arm Removal

Why Futility Is About the Target, Not Just Superiority

At an interim analysis, an experimental arm may look unlikely to be better than control. But “better” can mean any positive difference, including one too small to matter for the trial’s prespecified objective. A futility rule instead asks whether the arm still has a sufficiently plausible chance of reaching a specified target effect. If that posterior probability is very small, the protocol may stop assigning new participants to the arm.

This tutorial uses a binary endpoint, such as response by a defined assessment time. Let \(p_E\) denote the response probability in an experimental arm and \(p_C\) the response probability in the control arm. Define the risk difference as \(p_E-p_C\), measured in probability units. A positive target \(\delta\) represents the effect level the rule asks the arm to achieve. The futility threshold \(\gamma\) is a small probability chosen before applying the rule.

The target \(\delta\) and threshold \(\gamma\) answer different questions. The target says what effect counts as reaching the goal; the threshold says how little posterior support for that goal is enough to trigger the rule. Neither should be selected after looking at the interim result. A value of \(\gamma=0.10\), for example, means the decision criterion is a posterior probability below 10%; it does not assert a 10% frequentist type I error rate.

Binary-Outcome Model and Calculation

Suppose an interim analysis contains \(x_E\) responses among \(n_E\) participants assigned to the experimental arm and \(x_C\) responses among \(n_C\) participants assigned to control. Conditional on the response probabilities, model the counts as \(X_E\sim\mathrm{Binomial}(n_E,p_E)\) and \(X_C\sim\mathrm{Binomial}(n_C,p_C)\). A convenient specification assigns independent beta priors, \(p_E\sim\mathrm{Beta}(a_E,b_E)\) and \(p_C\sim\mathrm{Beta}(a_C,b_C)\). With these assumptions, the interim posteriors are:

The beta prior parameters are positive. They describe prior distributions on probabilities, not extra observed participants unless a separate interpretation has been explicitly justified. The binomial likelihood assumes a common response probability within each arm for the endpoint and assessment definition being analyzed. The model also treats participants’ outcomes as conditionally independent given their arm-specific probabilities. The independent priors encode prior independence of the two arm probabilities; they are not an automatic requirement of all Bayesian models.

Write \(f_E(u)\) for the experimental-arm posterior density and \(F_C(v)\) for the control-arm posterior cumulative distribution function. Conditional posterior independence lets us integrate over the control probability for each possible experimental probability:

This expression follows because, for a fixed \(p_E=u\), the event \(p_E-p_C\geq\delta\) is equivalent to \(p_C\leq u-\delta\). If \(u-\delta\leq 0\), that conditional probability is zero; if \(u-\delta\geq 1\), it is one. Numerical integration or posterior simulation can evaluate \(q\). The resulting posterior probability is compared with \(\gamma\); it is not itself a test statistic with a frequentist reference distribution.

Worked Interim Decisions

Derivation, Application, and Diagnostics

The posterior update and the futility decision have two distinct steps. First, the observed counts update the distribution of each response probability. Second, the joint posterior distribution is used to calculate the probability of the target event. Keeping these steps separate helps prevent a common error: comparing posterior means alone with the target.

The calculation has a useful basic check. If \(\delta\) increases while the posterior distributions stay fixed, the event \(p_E-p_C\geq\delta\) becomes narrower, so its posterior probability cannot increase. Similarly, if \(\gamma\) increases, more arms can satisfy \(q<\gamma\). A software result that violates either property may indicate a coding, sign, or threshold error.

Other diagnostics focus on whether the model answers the intended question. Check arm-specific denominators, response counts, outcome timing, and any missing or pending outcomes included in the interim data. Review whether the beta priors are defensible for the setting and whether a common response probability per arm is plausible. If outcomes vary substantially across sites or patient groups, or if dependence is important, an unadjusted binomial model may misrepresent uncertainty. A sensitivity analysis using prespecified alternative priors or a more suitable model can reveal whether the decision is fragile.

At multiple interim looks, repeatedly applying a threshold does not make the decision automatically calibrated to a desired frequentist operating characteristic. The protocol should specify when the rule is evaluated and the decision threshold at each permitted look. This tutorial’s posterior calculation uses observed interim data to describe uncertainty about the arm probabilities; it does not calculate the chance that future enrollment will eventually cross a stopping boundary.

How Futility Differs From Other Arm-Dropping Rules

A posterior superiority rule, as in the earlier material on Posterior Superiority Rules for Dropping Arms, evaluates posterior support for a specified ordering such as \(p_E>p_C\). The futility rule here evaluates support for a target-sized difference, \(p_E-p_C\geq\delta\). These events are not generally the same: evidence for any positive difference need not imply much evidence for reaching a larger target. Conversely, a target can be defined as zero, in which case the probability event reduces to superiority, though the decision threshold still determines the action.

The posterior probability in this rule is also distinct from a probability about future trial outcomes. It conditions on the current data and concerns the unknown arm probabilities. A calculation about future enrollment would require modeling future outcomes and the planned continuation of the trial, which is a different decision quantity.

The rule’s limitations follow from its inputs. A posterior probability depends on the likelihood, prior, target, and interim data; changing any of them can change the decision. A small \(q\) is not proof of no treatment effect, and failing to meet the futility criterion is not proof of success. The rule also does not, by itself, determine whether an arm is removed permanently, how participants already assigned to it are followed, or how randomization changes afterward. Those operational details belong in the trial protocol.

Common Mistake