Predictive Probability of Final Success

From Intermediate Outcomes to a Probability of Final Success

An intermediate outcome can be informative about a final time-to-event endpoint without being the endpoint itself. At an interim look, the practical question may be: given the intermediate outcomes observed so far, how likely is this arm to satisfy its prespecified final success criterion? The answer depends both on what the intermediate data imply about the final treatment effect and on how much final-endpoint information remains to be collected.

Predictive probability makes both sources of uncertainty explicit. It averages over uncertainty about the effect, as updated by the intermediate outcomes, and over the distribution of future final-endpoint data. It is therefore not simply the probability of success calculated at one chosen effect value. The result is conditional on the interim data and on the stated models and design assumptions; it is not a guarantee that the arm will succeed.

For illustration, write \(\theta\) for an arm’s effect on the final endpoint, with larger values defined to be more beneficial. For a time-to-event endpoint, one possible convention is \(\theta=-\log(HR)\), where \(HR\) is the hazard ratio comparing the arm with control. Under that convention, a hazard ratio below 1 corresponds to \(\theta>0\). The final success boundary may be a one-sided critical value for a test of no benefit, or a different prespecified boundary that accounts for the design’s testing procedure.

A Normal Posterior-Predictive Calculation

Consider a simple normal approximation. Suppose the interim intermediate data provide an estimate \(X\) that is linked to the final effect \(\theta\) by the model \(X\mid\theta\sim N(\theta,s_I^2)\). Suppose the analysis assigns \(\theta\) a normal prior distribution \(N(m_0,v_0)\). At the final analysis, let \(Y\) be an estimator of \(\theta\), with \(Y\mid\theta\sim N(\theta,V_F)\). Here \(V_F\) is the variance of the final estimator under the planned final information. Assume, for this calculation, that \(X\) and \(Y\) are conditionally independent given \(\theta\), and that the variances are known.

If final success requires \(Y/\sqrt{V_F}\ge c\), where \(c\) is the prespecified one-sided critical value, then success is equivalent to \(Y\ge c\sqrt{V_F}\). Normal updating gives a posterior distribution for \(\theta\mid X\); the predictive distribution for \(Y\mid X\) then averages the future sampling variation over that posterior uncertainty.

To see the predictive variance, write \(Y=\theta+\varepsilon_F\), where \(\varepsilon_F\sim N(0,V_F)\) is independent of \(\theta\mid X\). Thus \(E(Y\mid X)=m\), and \(\operatorname{Var}(Y\mid X)=\operatorname{Var}(\theta\mid X)+\operatorname{Var}(\varepsilon_F)=v+V_F\). Consequently, the predictive probability is

where \(\Phi\) is the standard normal cumulative distribution function. The denominator contains two distinct uncertainties: \(V_F\), the future sampling variance, and \(v\), the remaining uncertainty about the effect after seeing the intermediate data. This calculation relies on the assumed link between intermediate outcomes and the final effect, as well as on the normal, variance, and independence assumptions stated above.

Why Predictive Probability Differs From Conditional Power

Conditional power calculates the chance of final success under a specified value of the true effect, conditional on the interim information included in its calculation. Predictive probability instead averages over uncertainty about that effect. In the normal example, using the posterior mean \(m=0.132353\) as though it were the true effect gives a conditional power of

That is lower than the predictive probability of about 0.329. The predictive calculation includes the possibility that the true effect is above the posterior mean, as well as the possibility that it is below. Averaging success probabilities over this uncertainty need not equal the success probability at the mean because the probability function is nonlinear. The two quantities answer different questions; neither is universally preferable. Their usefulness depends on the decision being considered and on whether the assumptions about the effect are credible.

In multi-arm designs, this calculation is initially an arm-specific probability for a specified final boundary. A marginal probability for one arm does not by itself describe the probability that any arm succeeds, the probability of selecting the correct arm, or a familywise error rate. If final success uses an adjusted or otherwise design-specific boundary, use that boundary in the predictive calculation. Dependence between arms, for example from a shared control group, also matters when calculating joint probabilities.

Uncertain Future Information

The final information may itself be uncertain at the interim look. For a time-to-event endpoint, for example, the eventual number of observed events can vary with accrual, follow-up, and event occurrence. If future information \(I\) has a specified distribution given the interim data, predictive success should average over that distribution as well. Under the normal approximation \(Y\mid\theta,I\sim N(\theta,1/I)\), the final statistic is \(Z=Y\sqrt{I}\), and the success event is \(Z\ge c\). Conditional on \(I\) and the posterior \(\theta\mid D_I\sim N(m,v)\), \(Z\) has mean \(m\sqrt I\) and variance \(1+vI\). Thus the probability conditional on \(I\) is

Assumptions, Diagnostics, and Alternatives

The calculation is only as credible as its ingredients. Check that the final success criterion is stated on the correct scale and direction; that the intermediate-to-final relationship is supported by an appropriate model; and that the model reflects the trial’s accrual, follow-up, and endpoint maturity. For the normal calculations, also check the normal approximation, variance assumptions, and any claimed conditional independence. In survival settings, censoring patterns, competing events, or changing risk over follow-up can make a simple fixed-information approximation inadequate.

Useful diagnostics include comparing predicted and observed final outcomes in relevant historical or simulated settings, checking calibration across ranges of intermediate results, and examining sensitivity to plausible alternative effect distributions and future-information assumptions. A prediction that changes sharply under small reasonable changes to the intermediate-to-final link or prior distribution is fragile. Predictions made when few outcomes are mature may also be strongly driven by assumptions rather than by observed evidence.

The main alternatives answer related but distinct questions. Conditional power evaluates a fixed assumed effect; predictive probability averages over specified uncertainty about effects and, if needed, future information. A plug-in forecast based only on a point estimate is simpler but omits uncertainty in that estimate. Simulation can handle more complex time-to-event mechanisms, but it still requires a defensible model and must implement the actual final criterion. None of these methods removes the need to prespecify the final testing rule or to account for the design’s multiplicity.

Predictive probability is not automatically a frequentist operating characteristic. It depends on the distribution used for effect uncertainty and on the future-data model. It should not be interpreted as the probability that the null hypothesis is true, nor as a replacement for error-rate evaluation of the complete design. It is a conditional forecast useful for reasoning about an arm given interim information, subject to explicit assumptions.

Common Mistake

• Wrong reasoning: “The posterior mean effect is \(0.132\), so the predictive probability is the conditional power computed at \(0.132\).” This treats the uncertain effect as known and omits its posterior variance.
• Correction: Integrate the final success probability over the effect distribution. In the normal model, this adds \(v\) to \(V_F\) in the predictive variance; with mixtures or uncertain information, average over the relevant components or information values as well.
• Another warning: A high predictive probability is not proof that the arm is effective. It is conditional on the stated model and does not replace the prespecified final test.