Predictive Probability of Futility Before Trial Completion

What Predictive Futility Measures

Before a trial is complete, the observed data do not determine whether the final success criterion will be met. Future participants may produce outcomes that strengthen or weaken the current evidence. The predictive probability of futility quantifies the chance, given the data observed so far and the model for future outcomes, that the trial's final data will fail its prespecified success criterion.

This quantity is the complement of predictive success when the final criterion has exactly two outcomes: met or not met. It is not the same as the current posterior probability that a treatment effect is absent or harmful. It concerns a future trial result, and therefore includes uncertainty about both unknown parameters and future observations. A high predictive probability of failure is evidence that eventual success is unlikely under the specified model; by itself, it does not establish that stopping is the right action.

Definition: Let \(D_t\) denote the data available at an interim time, \(Y_{\mathrm{future}}\) the data yet to be observed, and \(D_{\mathrm{final}}=(D_t,Y_{\mathrm{future}})\) the completed data. Let \(S(D_{\mathrm{final}})=1\) if the prespecified final success criterion is met and \(S(D_{\mathrm{final}})=0\) otherwise. The predictive probability of futility is \(P\{S(D_{\mathrm{final}})=0\mid D_t\}\), calculated under the posterior predictive distribution for future data.

The success criterion must be specified in advance and applied to the completed data in a definite way. It might require a minimum number of responders, a final estimate beyond a target, or another prespecified endpoint rule. This tutorial focuses on predicting whether a given criterion will fail, not on choosing among possible criteria.

Posterior Predictive Calculation

Write \(\theta\) for the unknown trial parameters. The posterior distribution \(p(\theta\mid D_t)\) describes uncertainty about those parameters after the interim data. Conditional on \(\theta\) and the observed data, the trial model gives a distribution for future outcomes. The probability of failure is found by averaging the conditional probability of failure over the posterior distribution of \(\theta\).

P\{S(D_{\mathrm{final}})=0\mid D_t\} = \int_{\Theta} \left[ \sum_{y_{\mathrm{future}}} \mathbf{1}\{S(D_t,y_{\mathrm{future}})=0\} p(y_{\mathrm{future}}\mid\theta,D_t) \right] p(\theta\mid D_t)\,d\theta

This expression applies when the future data take discrete values: for each possible future outcome, its conditional predictive probability contributes if that outcome leads to failure. For continuous future data, replace the sum with an integral over the future outcomes:

P\{S(D_{\mathrm{final}})=0\mid D_t\} = \int_{\Theta}\int_{\mathcal{Y}} \mathbf{1}\{S(D_t,y_{\mathrm{future}})=0\} p(y_{\mathrm{future}}\mid\theta,D_t)\,dy_{\mathrm{future}}\, p(\theta\mid D_t)\,d\theta

In both cases, the calculation has two sources of uncertainty. Future outcomes vary even if \(\theta\) were known, and \(\theta\) itself remains uncertain after observing \(D_t\). Conditioning only on a point estimate of \(\theta\) omits the second source and generally does not give the posterior predictive probability.

Justification: The posterior predictive distribution is obtained by averaging the future-data distribution over the posterior distribution of \(\theta\). For a failure event \(A\), the law of total probability gives \(P(A\mid D_t)=\int P(A\mid\theta,D_t)p(\theta\mid D_t)d\theta\). For discrete outcomes, \(P(A\mid\theta,D_t)\) is the sum of the probabilities of the failure-producing outcomes; for continuous outcomes, it is the corresponding integral.

Worked Examples

Worked Example: Binary Endpoint With a Beta-Binomial Model

Suppose, hypothetically, that a single-arm trial records a binary response. The final criterion is at least 8 responses among 12 participants. At an interim look, 5 of the first 8 participants have responded, leaving 4 outcomes unknown. Assume independent responses with response probability \(p\), and use the prior \(p\sim\mathrm{Beta}(1,1)\). These assumptions make the posterior and predictive calculation explicit; they do not guarantee that the model suits every clinical endpoint.

After observing 5 responses and 3 nonresponses, the posterior is \(p\mid D_t\sim\mathrm{Beta}(6,4)\). The number \(Y\) of responses among the 4 remaining participants has a beta-binomial posterior predictive distribution. A final total below 8 occurs when \(5+Y<8\), or \(Y\leq 2\). Thus:

P(\text{futility}\mid D_t) = \sum_{y=0}^{2}{4\choose y} \frac{B(6+y,\,4+4-y)}{B(6,4)}

For clarity, the three predictive probabilities can be written using rising factorials, where \((a)_k=a(a+1)\cdots(a+k-1)\). The common denominator is \((10)_4=10\cdot11\cdot12\cdot13=17{,}160\):

P(Y=0\mid D_t)=\frac{4\cdot5\cdot6\cdot7}{17{,}160} =\frac{840}{17{,}160}=0.0490

P(Y=1\mid D_t)=\frac{4(6)(4\cdot5\cdot6)}{17{,}160} =\frac{2{,}880}{17{,}160}=0.1678

P(Y=2\mid D_t)=\frac{6(6\cdot7)(4\cdot5)}{17{,}160} =\frac{5{,}040}{17{,}160}=0.2937

Adding these probabilities gives \(P(\text{futility}\mid D_t)=0.5105\). Under this model and this final criterion, the chance of ending with fewer than 8 responses is about 51.05%. The complementary predictive probability of meeting the criterion is about 48.95%. Neither number is the posterior probability that the response rate is below a particular value; each concerns the result of the still-incomplete trial.

Worked Example: Continuous Endpoint With a Normal Model

Consider a hypothetical trial in which a larger continuous outcome is better. The final criterion is that the mean for all 10 participants exceed 5 units. After 6 participants, the observed mean is 4.2 units. Assume observations are conditionally independent with \(X_i\mid\mu\sim N(\mu,3^2)\), and use the prior \(\mu\sim N(3,2^2)\). The variance \(3^2=9\) is assumed known. The normal likelihood and prior are conjugate, giving posterior variance \(V\) and mean \(\mu_t\):

V=\left(\frac{1}{4}+\frac{6}{9}\right)^{-1}=1.0909, \qquad \mu_t=V\left(\frac{3}{4}+\frac{6(4.2)}{9}\right)=3.8727

There are 4 participants left. Conditional on \(\mu\), their sample mean has variance \(9/4\). Averaging over the posterior for \(\mu\), the predictive variance of their mean is \(V+9/4=3.3409\). The final mean is the weighted average of the interim and future means. Therefore its posterior predictive mean is \((6(4.2)+4(3.8727))/10=4.0691\), and its variance is \((4/10)^2(3.3409)=0.5345\). The final mean consequently has a normal posterior predictive distribution with standard deviation \(\sqrt{0.5345}=0.7311\) units.

P(\text{futility}\mid D_t) = P(\bar X_{\mathrm{final}}\leq 5\mid D_t) = \Phi\left(\frac{5-4.0691}{0.7311}\right) = \Phi(1.273)\approx 0.8985

The model assigns about an 89.85% probability to a final mean at or below 5 units, so the final success criterion is unlikely to be met under these assumptions. This probability accounts for uncertainty about \(\mu\) as well as variation in the four future measurements. It is not a claim that the true mean has an 89.85% probability of being below 5.

Worked Example: Interpreting a Prespecified Futility Trigger

Suppose the team in the binary example has a protocol rule to review the trial for possible early stopping if predictive futility is at least 0.80. The calculated probability is 0.5105. Comparing the two numbers, \(0.5105<0.80\), so the stated trigger is not met at this interim look.

That comparison answers only whether this particular trigger has been reached. It does not prove that continuing is optimal, nor does a probability above 0.80 automatically make stopping appropriate in every trial. A stopping decision also depends on the protocol, clinical consequences, resource use, safety information, and the costs of stopping when success was still possible. The predictive probability supplies evidence for that decision rather than replacing it.

Assumptions, Diagnostics, and Failure Modes

A predictive futility calculation is meaningful only relative to its model and design assumptions. Before interpreting it, check that the following are reasonably represented:

The final criterion is fixed and unambiguous. The same rule used in the calculation must be applied to the final data. Changing the endpoint threshold, analysis population, or missing-data rule changes the predicted failure event.
The future-data model is credible. For example, the beta-binomial illustration assumes a common response probability and conditionally independent binary outcomes. Treatment heterogeneity or changes in clinical practice may violate that assumption.
The posterior reflects relevant information. Prior choices and interim data jointly determine parameter uncertainty. A prior that puts little mass near plausible response rates can affect the prediction, especially early in a trial.
The planned future data are represented correctly. The number of participants remaining, allocation, follow-up, and event ascertainment must match the design. If recruitment may stop early or follow-up varies, those features need to be included in the predictive model.
Interim access and adaptation are handled consistently. If future sample size or other design features will change in response to interim information, the prediction must model that rule rather than assume a fixed plan.

Useful diagnostics include checking whether the interim observations are plausible under the assumed model, verifying that the predicted future sample size and endpoint distribution match the protocol, and examining whether the answer changes substantially under reasonable alternative assumptions. A probability near 0 or 1 deserves particular scrutiny: it can reflect genuinely strong information, but it can also result from a coding error, an incorrectly applied success threshold, or an overly restrictive model.

Common mistakes include treating predictive futility as the probability that the null hypothesis is true, or treating a probability such as 0.80 as a universal stopping boundary. Both interpretations are incorrect. Predictive futility is a probability about a future trial outcome conditional on current data and a specified model. A threshold is a separate policy choice that requires justification. Also avoid calculating future outcomes only at the posterior mean of a parameter: this discards posterior uncertainty and can misstate the predictive tail probability.

How It Differs From Related Quantities

Quantity	Question answered	What it conditions on
Current posterior probability	How much posterior probability supports a statement about an unknown parameter now?	The interim data and parameter posterior
Predictive probability of futility	How likely is failure of the final criterion after future outcomes are observed?	The interim data, posterior uncertainty, and future-data model
Futility stopping threshold	At what predictive probability will a specified action be considered?	A decision policy, including consequences of actions

Predictive futility is related to predictive success, but expresses the complementary event when the final rule classifies outcomes as either success or failure. It also differs from a frequentist operating characteristic: the predictive probability is conditional on the current data and Bayesian model, whereas an operating characteristic describes performance across repeated trials under specified conditions. Neither quantity is a substitute for the other.

Key takeaway: Predictive futility is the posterior predictive probability that future trial data will fail a prespecified final success criterion. Its value depends on the model, prior, interim data, and remaining design, and it is evidence for decision-making—not a stopping decision by itself.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions below.

A binary trial has a final criterion of at least 7 responses in 9 participants. After 7 participants, 4 have responded. How many responses among the 2 remaining participants lead to final failure?
If predictive futility is 0.27 and a protocol trigger is 0.80, what follows from the comparison? What does not follow?
In a continuous-outcome model, why does the posterior predictive variance include both posterior parameter variance and future sampling variance?
Explain why predictive futility of 0.90 is not the same statement as a 90% posterior probability that the treatment is ineffective.

Solutions. (1) Failure means fewer than 7 total responses. Since 4 have already responded, failure occurs if the remaining count \(Y\) satisfies \(4+Y<7\), or \(Y\leq2\). With two participants remaining, that means either zero, one, or two responses all lead to failure. (2) Since \(0.27<0.80\), the specified trigger is not reached. This does not establish that continuing is optimal or that the treatment will succeed; it only answers the stated trigger comparison. (3) Future outcomes vary even for a fixed parameter, and the parameter is uncertain given interim data. Averaging over both sources produces predictive variance that includes both components. (4) The 0.90 probability concerns failure of the final data criterion under the predictive model. A posterior probability of ineffectiveness concerns an event about the unknown treatment effect. They are different events and need not have similar probabilities.