Predictive Probability of Futility and Early Stopping

Why Predictive Futility Is a Different Interim Question

An interim analysis can ask whether an arm currently looks promising. A different question is whether the arm is likely to meet its final success criterion after the remaining data are collected. Predictive probability of futility answers the second question: it estimates, conditional on the data observed so far, the chance that the final decision rule will not declare the arm successful.

This distinction matters because the interim posterior describes uncertainty about the treatment effect now, whereas predictive futility averages over both uncertainty about the effect and the outcomes that may still be observed. An arm can have a posterior probability of benefit that is neither very high nor very low, yet have little chance of reaching a demanding final criterion. The introductory material on predictive probability of final trial success establishes the corresponding success calculation; here the focus is its complement and how to use it for stopping.

Definition: Let $D_I$ denote the data observed at an interim analysis, and let $Y_F$ denote the remaining data that could be collected by the planned final analysis. Let $S(Y_F,D_I)$ be the event that the prespecified final success criterion is met after combining interim and future data. The predictive probability of futility is $$ q_F=P\{S(Y_F,D_I)^c\mid D_I\}. $$ If success and failure are exhaustive and mutually exclusive, $q_F=1-P\{S(Y_F,D_I)\mid D_I\}$. The predictive probability is conditional on the interim data and on the assumed model for future data.

For arm $j$, write the corresponding quantity as $q_{F,j}$. An early stopping rule may specify that arm $j$ stops for futility when $q_{F,j}\geq c_F$, where $c_F$ is a threshold fixed in the design. Equality is included here by convention; a protocol may choose another explicit boundary convention. The threshold is a decision parameter, not a probability that the treatment is ineffective.

Calculating the Predictive Probability

The calculation has two parts. First, specify the final success event exactly, including its posterior probability threshold or other decision criterion. Then average over all possible future data, using their predictive distribution given $D_I$. For discrete future outcomes $y$, this gives

P\{S(Y_F,D_I)\mid D_I\} =\sum_y I\{S(y,D_I)\}\,P(Y_F=y\mid D_I), \qquad q_F=\sum_y I\{S(y,D_I)^c\}\,P(Y_F=y\mid D_I).

Here $I\{\cdot\}$ is 1 when the condition inside braces is true and 0 otherwise. The sum works because the possible future outcomes form a partition: each outcome either leads to final success or does not. For continuous future data, replace the sum by an integral over the region that does not satisfy the final criterion. In either case, the predictive distribution must account for uncertainty in the treatment parameter as well as sampling variation in future observations.

The final success event might, for example, be that the final posterior probability of benefit exceeds a prespecified value. It is not enough to check whether the interim posterior probability already exceeds that value: the future data can change the final posterior, and the predictive calculation integrates over that possibility.

Model requirements: The likelihood, prior, and planned future sample size or accrual process must be specified well enough to determine $P(Y_F\mid D_I)$. The interim data must be incorporated once, not counted again as future data. If future outcomes depend on shared controls or other arms, their dependence must be represented when it affects the arm’s predictive distribution. Missingness, nonadherence, or changes in follow-up must be handled consistently with the model and trial plan.

Worked Examples With Discrete Outcomes

Worked Example: Beta-Binomial Predictive Futility

Hypothetical data and assumptions. Suppose a binary response probability $p$ has prior distribution $p\sim\operatorname{Beta}(1,1)$. At interim, 3 of 4 participants in an arm have responded. The planned final analysis will include 2 more participants. Assume their responses are conditionally independent Bernoulli observations with probability $p$, and the beta-binomial model is appropriate. Define final success as $P(p>0.5\mid\text{all data})\geq0.80$.

The interim posterior is $\operatorname{Beta}(4,2)$. Let $K$ be the number of responses among the 2 remaining participants. Integrating the binomial likelihood over this interim posterior gives a beta-binomial predictive distribution:

P(K=k\mid D_I)=\binom{2}{k} \frac{B(4+k,\,2+2-k)}{B(4,2)},\qquad k=0,1,2.

Substitution gives $P(K=0\mid D_I)=1/7=0.1429$, $P(K=1\mid D_I)=8/21=0.3810$, and $P(K=2\mid D_I)=10/21=0.4762$. These probabilities sum to 1.

If $K=0$, the final posterior is $\operatorname{Beta}(4,3)$, for which $P(p>0.5\mid D_I,K=0)=42/64=0.6563$; this does not meet 0.80. If $K=1$, the posterior is $\operatorname{Beta}(5,2)$, and the probability is $57/64=0.8906$. If $K=2$, it is $\operatorname{Beta}(6,1)$, and the probability is $63/64=0.9844$. Thus final success occurs exactly when $K\geq1$.

The predictive probability of final success is $8/21+10/21=18/21=0.8571$. Therefore,

q_F=P(K=0\mid D_I)=\frac{1}{7}=0.1429.

Under a prespecified stopping rule $q_F\geq0.20$, this arm would continue: its predictive futility is below the 0.20 threshold. This conclusion concerns the decision rule, not certainty that the arm will succeed.

Worked Example: Normal Posterior Predictive Futility

Hypothetical data and assumptions. Suppose the interim posterior for a treatment benefit $\theta$, measured in outcome units, is $N(0.10,0.04)$. A future estimate $Z$, based on the remaining participants, satisfies $Z\mid\theta\sim N(\theta,0.01)$. Assume this normal likelihood and posterior are appropriate. After observing $Z$, define final success as $P(\theta>0\mid D_I,Z)\geq0.90$.

Normal conjugacy gives the final posterior mean $M=0.2(0.10)+0.8Z$ and variance $V=(0.04)(0.01)/(0.04+0.01)=0.008$. The final posterior probability of benefit reaches 0.90 when $M/\sqrt{V}\geq1.2816$. Hence the criterion is met when

0.02+0.8Z\geq1.2816\sqrt{0.008}=0.11463\ldots, \qquad Z\geq\frac{0.11463\ldots-0.02}{0.8}=0.11828\ldots.

Before observing the future data, integrating over the interim posterior gives $Z\mid D_I\sim N(0.10,0.04+0.01)$. Thus the predictive probability of final success is

P(Z\geq0.11828\ldots\mid D_I) =1-\Phi\!\left(\frac{0.11828\ldots-0.10}{\sqrt{0.05}}\right) \approx 1-\Phi(0.0818) \approx 0.4674.

Consequently $q_F\approx1-0.4674=0.5326$. With a futility threshold of 0.55, the arm would continue because $0.5326<0.55$. This is a close decision, so the cutoff and calculations should be implemented at sufficient precision rather than rounded before comparing with the threshold.

From Predictive Probability to an Early Stopping Rule

Worked Example: Applying a Prespecified Stopping Threshold

Hypothetical data and assumptions. Return to a binary response outcome with $p\sim\operatorname{Beta}(1,1)$, 2 responses among 4 participants at interim, and 2 participants remaining. Assume the same conditionally independent beta-binomial model. Now define final success as $P(p>0.5\mid\text{all data})\geq0.70$, and set the futility threshold at $c_F=0.70$.

The interim posterior is $\operatorname{Beta}(3,3)$. For $K=0,1,2$ future responses, the final posteriors are respectively $\operatorname{Beta}(3,5)$, $\operatorname{Beta}(4,4)$, and $\operatorname{Beta}(5,3)$. Their posterior probabilities that $p>0.5$ are $29/128=0.2266$, $0.5000$, and $99/128=0.7734$. Only $K=2$ meets the 0.70 final success criterion.

Under the interim posterior, the predictive probability of two future responses is

P(K=2\mid D_I)=\frac{3\cdot4}{6\cdot7}=\frac{2}{7}=0.2857.

Therefore $q_F=P(K\leq1\mid D_I)=1-2/7=5/7=0.7143$. Since $0.7143\geq c_F=0.70$, the arm stops for futility under the stated rule. The interim posterior probability of benefit is $P(p>0.5\mid D_I)=0.5000$; that number is not the predictive futility probability and does not by itself determine this stopping decision.

The threshold should be chosen before applying the rule and considered in light of the consequences of stopping too soon versus continuing an arm with little prospect of success. A lower $c_F$ makes stopping less frequent; a higher $c_F$ makes it easier to stop. These are design choices, not universal constants. A predictive futility rule is Bayesian and conditional on the model and the interim data. It does not, by itself, guarantee a particular frequentist type I error rate, power, or expected sample size.

Justification, Diagnostics, and Limitations

The complement calculation follows directly from conditioning on the possible future data. For discrete $Y_F$, the probability of final success is the sum of the predictive probabilities of outcomes that satisfy the final rule. The remaining outcomes are disjoint from that success set and together make up its complement. Because the predictive probabilities sum to one,

\sum_{y\notin S}P(Y_F=y\mid D_I) =1-\sum_{y\in S}P(Y_F=y\mid D_I).

This identity explains why predictive futility is the complement of predictive success only when those two outcomes exhaust the final decision possibilities. If the final rule includes a separate indeterminate category, report its predictive probability explicitly rather than silently classifying it as success or failure.

Useful checks include verifying that the probabilities over future outcomes sum to one, recomputing the final criterion for each possible outcome, and confirming that success and futility are complements under the stated rule. In a continuous model, check that the cutoff follows from the final posterior criterion and that the predictive distribution uses the correct variance. When $q_F$ is close to $c_F$, numerical precision can change the action; retain unrounded values for the comparison and report rounded values afterward.

The calculation can be misleading if the prior or likelihood poorly represents the arm, if the remaining sample size or follow-up differs from the plan, or if dependence between arms or shared controls is ignored. Predictive futility also depends on the selected final success criterion: a different posterior threshold can change which future outcomes count as success and therefore change $q_F$. It is not a model-free forecast, nor does a high value prove that the treatment has no effect.

The main alternatives answer different questions. An interim posterior futility rule uses the current posterior probability of benefit or another current-data summary. Predictive futility instead asks about the chance of missing the final criterion after future data arrive. A fixed information-based stopping boundary may be calibrated through frequentist operating characteristics, while a predictive rule is explicitly conditional on a Bayesian model and chosen threshold. These approaches can be compared during design assessment, but their numerical thresholds are not interchangeable.

Common Pitfalls

Using current posterior futility as predictive futility. A small interim $P(\theta>0\mid D_I)$ is not generally the probability of failing the final rule; future data may change the posterior.
Stopping when the number is below the threshold. Under the convention here, stop when $q_F\geq c_F$. If $q_F<c_F$, continue under this rule.
Using an unspecified final criterion. The predictive probability has no unique value until the final success event is defined.
Treating a predictive probability as certainty. A value such as $q_F=0.70$ means a 0.70 model-based predictive chance of missing the criterion, conditional on interim data; it does not mean the arm is certainly futile.

Key takeaway: To calculate predictive futility, define the final success criterion, derive the predictive distribution of the remaining data given the interim data, and sum or integrate the predictive probability over outcomes that fail that criterion. Apply a stopping threshold only after this calculation.

Check Your Understanding

Answer each question, then compare your reasoning with the brief solutions.

If the predictive probability of final success is 0.68 and success and failure are exhaustive, what is predictive futility?
A rule stops when predictive futility is at least 0.80. If the calculated value is 0.76, should the arm stop?
Why must the final success criterion be specified before predictive futility can be calculated?
In the normal example, why is the predictive distribution of the future estimate wider than the sampling distribution conditional on a fixed $\theta$?

Solutions. (1) $q_F=1-0.68=0.32$. (2) No. Since $0.76<0.80$, the stated stopping boundary is not reached. (3) The criterion determines which possible future outcomes count as success; without it, the success set and its complement are undefined. (4) Conditional on fixed $\theta$, $Z$ has variance 0.01. Given only the interim data, uncertainty about $\theta$ also contributes variance 0.04, so the predictive variance is $0.01+0.04=0.05$.