Predictive Probability Rules for Futility

Why Use Predictive Probability for Futility?

An interim result can look unpromising without making it clear whether the trial should stop. A futility rule makes the decision forward-looking: given the responses observed so far, how likely is the trial to meet its prespecified terminal success criterion if enrollment continues as planned? If that predictive probability is below a chosen threshold, the design recommends stopping for futility.

The rule combines the observed data, a probability model for future responses, and a decision threshold. It does not say that the treatment has no effect, nor does it establish that terminal success is impossible. It says that, under the model and the plan to continue, the chance of meeting the defined terminal criterion is too small to justify the remaining enrollment according to the design’s prespecified threshold.

Model and Futility Rule

For a single-arm binary-response design, suppose each patient’s response is represented by a Bernoulli outcome with common response probability \(p\). Conditional on \(p\), patient outcomes are assumed independent. Use a proper beta prior, \(p\sim\operatorname{Beta}(a,b)\), with \(a>0\) and \(b>0\). After observing \(x\) responses among \(n\) patients, the posterior distribution is \(\operatorname{Beta}(a+x,b+n-x)\).

Let \(Y\) be the number of responses among the \(N-n\) patients still planned for enrollment. The rule predicts the final total as \(x+Y\). Thus, terminal success occurs if \(Y\ge r^*-x\). If \(r^*-x\le 0\), the criterion has already been met; if \(r^*-x>N-n\), it cannot be met with the remaining patients. Otherwise, calculate the probability by summing the posterior predictive probabilities for all qualifying values of \(Y\).

Here \(B(\cdot,\cdot)\) is the beta function. The predictive probability of terminal success is

The design then applies the prespecified rule \(\operatorname{PP}_{\mathrm{succ}}(x,n)<c_F\). Because the rule says “below,” a value exactly equal to \(c_F\) does not trigger stopping. The look, terminal criterion, prior, and threshold should be defined in advance. The calculation also assumes that the outcomes available at the look are the outcomes used to update the model; pending or otherwise unavailable outcomes need an explicit treatment in the design.

Why the Beta-Binomial Calculation Applies

Conditional on \(p\), the future response count has a binomial distribution. The predictive distribution averages that binomial probability over uncertainty about \(p\) in the interim posterior. For a particular future count \(y\), this gives

Collecting powers of \(p\) and \(1-p\) inside the integral produces a beta-function integral. Dividing by the posterior beta normalizing constant gives the beta-binomial formula above. The averaging is essential: using only a single estimated value of \(p\) would omit posterior uncertainty about the response rate.

Worked Futility Calculations

Consequences, Comparisons, and Diagnostics

The threshold \(c_F\) expresses how much chance of eventual terminal success the design requires before it continues. A larger threshold generally makes stopping more likely and reduces expected enrollment, but it also increases the chance of stopping in a state from which terminal success could still have occurred. A smaller threshold tends to preserve more opportunities to meet the terminal criterion, at the cost of enrolling more patients in states with low predictive probability. These trade-offs should be examined for the planned design rather than inferred from a single interim calculation.

Predictive probability is different from two commonly used quantities. The posterior probability \(\Pr(p>p_0\mid x,n)\) concerns the unknown response rate at the interim; it does not directly ask whether the final trial will meet its success criterion. Conditional power instead evaluates future success under a specified value or estimate of the response rate. Predictive probability averages over uncertainty about \(p\) using the interim posterior. Each quantity answers a different question, so they should not be substituted for one another without changing the interpretation of the rule.

Before applying the rule, check the inputs and assumptions. Confirm that \(N\), \(r^*\), the prior, the interim timing, and \(c_F\) are the intended prespecified values. Check that \(0\le x\le n\le N\), that the remaining number is \(N-n\), and that the success condition has been translated correctly into \(Y\ge r^*-x\). A probability outside \([0,1]\), or a predictive distribution whose probabilities do not sum to one, signals a computational or indexing error. Also assess whether a common response probability and conditionally independent outcomes are plausible for the patient population and conduct of the trial.

A predictive futility rule is model-dependent. If the prior or the assumed response model poorly represents plausible response rates, the stopping decisions and expected enrollment can be misleading. The rule also depends on the chosen terminal criterion: changing \(r^*\) changes the predicted event even if the interim data are unchanged. Planned operating characteristics should therefore be assessed across plausible response rates and prior choices before the trial begins. A single interim predictive probability describes the current state under the model; it does not by itself characterize how often the design will stop or succeed over repeated trials.

Common Mistake

• Wrong reasoning: “The predictive probability is 8%, so there is an 8% probability that the treatment works.” This confuses the chance of meeting a specified terminal criterion, conditional on interim data and continued enrollment, with a posterior probability about \(p\).
• Correction: State the event and conditioning explicitly: “Given the interim outcomes and the beta-binomial model, the probability of reaching at least \(r^*\) total responses by \(N\), if enrollment continues, is 8%.” Whether that warrants stopping depends on the prespecified futility threshold.