Why Predict Success Before the Trial Is Complete?
In a confirmatory trial, an interim analysis can show encouraging evidence without establishing whether the trial is likely to meet its final success rule. A useful question is therefore not just “What is the current posterior probability of benefit?” but “If enrollment continues as planned, how likely is the final analysis to satisfy the prespecified criterion?” The latter probability incorporates both the evidence already observed and the uncertainty in the remaining data.
This is the predictive probability of success, abbreviated PPOS. It is conditional on the interim data, the model and prior, the planned remaining enrollment, and the final decision rule. It is not the posterior probability that the treatment works, nor is it a guarantee about the trial’s eventual result.
A Normal-Model Calculation
Consider a randomized two-arm trial with a continuous outcome. Let \(\delta\) be the treatment-minus-control effect, coded so that positive values favor treatment. Suppose the final rule declares success when the posterior probability that \(\delta\) exceeds a clinically relevant threshold \(\delta^*\) is at least \(\gamma\). The threshold could be zero when any positive effect is relevant, or a positive value when a minimum clinically important effect is required.
For a transparent calculation, assume a normal model with known sampling variances and independent information from the interim and future portions of the trial. After observing the interim data, suppose the posterior distribution is \( \delta\mid I\sim N(m_I,v_I) \), where \(I\) denotes the interim data. Let \(X_F\) be the estimated treatment effect from the future data, with \(X_F\mid\delta\sim N(\delta,V_F)\). The future estimate is independent of the interim data conditional on \(\delta\). The prior and interim likelihood are summarized by \(m_I\) and \(v_I\); thus, they are not being discarded when the final analysis is formed.
Normal conjugacy gives the final posterior variance \(v_N\) and posterior mean \(M_N\), conditional on the future estimate:
Given the final data, the posterior probability of exceeding \(\delta^*\) is \( \Phi((M_N-\delta^*)/\sqrt{v_N}) \), where \(\Phi\) is the standard normal cumulative distribution function. Therefore, the final success criterion is equivalent to a threshold on the final posterior mean:
where \(z_\gamma\) is the \(\gamma\)-quantile of the standard normal distribution. The PPOS is the probability, given the interim data, that the future data will make \(M_N\) reach or exceed \(c_N\).
Why the Predictive Distribution Has This Form
The future estimate is not predicted by treating the interim posterior mean as the known true treatment effect. Instead, uncertainty about \(\delta\) remains. Integrating over the interim posterior gives \(X_F\mid I\sim N(m_I,v_I+V_F)\): the variance includes both posterior uncertainty about the true effect and sampling variation in the future data. Since \(M_N\) is a linear function of \(X_F\), its interim predictive distribution is also normal.
For the variance identity, the coefficient of \(X_F\) in \(M_N\) is \(v_N/V_F\). Thus its predictive variance is \((v_N/V_F)^2(v_I+V_F)\). Substituting \(v_N=v_IV_F/(v_I+V_F)\) yields \(v_I^2/(v_I+V_F)\), which is also \(v_I-v_N\). Consequently, the calculation reduces to a normal tail probability:
The predictive variance \(v_I-v_N\) is the uncertainty in the final posterior mean induced by the future data, conditional on the interim evidence. It is not the final posterior variance \(v_N\). Confusing these two variances gives an incorrect predictive probability.
Worked Example: An 11.3% Predictive Probability of Success
Hypothetical trial. In a randomized confirmatory trial, positive \(\delta\) denotes improvement on a continuous endpoint, measured in points. The interim posterior is \(N(6.00,77.333)\) points squared. The planned future data have sampling variance \(V_F=102.08\) points squared. The final criterion is \(P(\delta>0\mid\text{all data})\geq0.975\). These posterior and variance summaries are assumed to arise from the stated normal model.
First combine interim and future information. The final posterior variance is \(v_N=(1/77.333+1/102.08)^{-1}\approx44.00\) points squared. The final posterior mean must be at least \(c_N=0+1.96\sqrt{44.00}\approx13.00\) points to meet the 0.975 criterion.
The predictive standard deviation of that final posterior mean is \(\sqrt{v_I-v_N}=\sqrt{77.333-44.00}=\sqrt{33.333}\approx5.774\) points. Therefore,
Conditional on the interim evidence and continued enrollment as planned, the probability of meeting the final posterior rule is about 11.3%. This is not an 11.3% posterior probability that the treatment is beneficial; the current posterior probability of benefit is a different quantity.
Further Worked Calculations
Worked Example: A Borderline Interim Result
Hypothetical trial. Treatment benefit is measured in mmHg, with positive values favoring treatment. The interim posterior is \(N(0.400,0.250)\) mmHg squared, future sampling variance is \(0.250\) mmHg squared, and final success requires \(P(\delta>0\mid\text{all data})\geq0.95\).
The final variance is \(v_N=(1/0.250+1/0.250)^{-1}=0.1250\) mmHg squared. The posterior-mean success threshold is \(c_N=1.645\sqrt{0.1250}\approx0.5813\) mmHg. The predictive standard deviation is \(\sqrt{0.250-0.1250}=0.3536\) mmHg. Hence \(z=(0.5813-0.400)/0.3536\approx0.513\), and the PPOS is \(1-\Phi(0.513)\approx0.304\).
The trial has about a 30.4% predictive probability of satisfying the final rule. The interim estimate is positive, but its direction alone does not make success likely; the final rule is demanding relative to the posterior mean and remaining information.
Worked Example: A Favorable Interim Result
Hypothetical trial. For another randomized trial, the interim posterior for a treatment benefit measured in points is \(N(1.200,0.360)\) points squared. Future data have sampling variance \(V_F=0.160\) points squared. Success requires posterior probability at least \(0.975\) that the benefit exceeds zero.
The final variance is \(v_N=(1/0.360+1/0.160)^{-1}\approx0.1108\) points squared. Therefore, the success threshold is \(c_N=1.96\sqrt{0.1108}\approx0.6524\) points. The predictive standard deviation is \(\sqrt{0.360-0.1108}\approx0.4992\) points. The standardized threshold is \((0.6524-1.200)/0.4992\approx-1.097\), so
The conditional predictive probability of meeting the final rule is about 86.4%. It is high, but it is not certainty: future observations can still shift the final posterior mean below the required threshold.
Interpretation, Diagnostics, and Alternatives
PPOS is most useful when reported with its conditions: the interim data, the prior and likelihood, the planned future information, and the final success criterion. It can support a practical discussion of whether continuing enrollment is likely to deliver a result meeting that criterion. The probability itself does not prescribe a decision; a decision also depends on the costs and consequences of continuing, stopping, or changing the trial.
Several checks help detect a calculation that is going wrong. Confirm that \(V_F\) represents the remaining information only, and that the final analysis includes the interim information once, not twice. Check that \(v_N<v_I\) when valid future information is added, and that the predictive variance \(v_I-v_N\) is positive. Confirm that the direction of the effect, threshold \(\delta^*\), and one-sided posterior criterion agree. Finally, compare the calculated PPOS with the locations of \(m_I\) and \(c_N\): if the interim posterior mean is far below the success threshold and future information is modest, an unexpectedly high PPOS should prompt a review of inputs and signs.
The normal calculation is exact under its assumptions, but clinical trial data may not satisfy them. Unknown outcome variance, skewed or bounded outcomes, missing measurements, nonadherence, or a changed analysis population can make the normal summary inadequate. If the final model is not normal-conjugate, the same definition still applies, but the predictive probability may need numerical integration or simulation from the specified posterior predictive distribution. The model and future-data assumptions then need careful checking rather than being hidden by a closed-form formula.
PPOS also differs from several related quantities. The current posterior probability \(P(\delta>\delta^*\mid I)\) assesses the effect using observed data only; it does not average over future trial outcomes. Frequentist conditional power instead calculates the chance of meeting a frequentist endpoint under specified assumptions about the true effect, rather than applying a posterior distribution and final posterior criterion. Prior predictive assurance averages success probability over prior uncertainty before observing interim trial data; PPOS conditions on the interim data and is therefore a different, later-stage quantity.
| Quantity | Conditions on interim data? | What is being evaluated? |
|---|---|---|
| Current posterior probability | Yes | Whether the current posterior supports effects beyond a threshold |
| Predictive probability of success | Yes | Whether future data are likely to make the final posterior rule succeed |
| Frequentist conditional power | Yes, with a specified effect assumption | Whether the planned trial is likely to meet its frequentist endpoint |
| Prior predictive assurance | No interim data are yet available | Success probability averaged over prior uncertainty |
A common mistake is to read a high current posterior probability of benefit as a high PPOS. The reasoning is wrong because current evidence does not determine the final result: the amount of future information and the final threshold also matter. Calculate the predictive distribution of the final decision statistic, then take the probability that it crosses the final success boundary.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- In the normal model, why is the predictive variance of the final posterior mean not the same as the final posterior variance?
- A trial has interim posterior \(N(0.500,0.160)\), future sampling variance \(0.160\), and success rule \(P(\delta>0\mid\text{final data})\geq0.95\). Calculate its PPOS.
- What does a PPOS of 0.20 mean, and what does it not mean?
- Name two assumptions or inputs that should be checked if a PPOS appears implausibly high.
Solutions. (1) The final posterior variance describes remaining uncertainty about \(\delta\) after all data are observed; the predictive variance describes how much the final posterior mean varies across possible future data sets, conditional on interim data. The former is \(v_N\), while the latter is \(v_I-v_N\). (2) The final variance is \(v_N=(1/0.160+1/0.160)^{-1}=0.0800\). The success threshold is \(1.645\sqrt{0.0800}\approx0.4651\), and the predictive standard deviation is \(\sqrt{0.160-0.0800}=0.2828\). Thus \(z=(0.4651-0.500)/0.2828\approx-0.123\), giving PPOS \(=1-\Phi(-0.123)=\Phi(0.123)\approx0.549\). (3) It means that, conditional on the interim data, model, planned future data, and final rule, the predictive probability of meeting that rule is 20%. It does not mean there is a 20% posterior probability that the treatment works. (4) Examples include checking that future variance refers only to remaining data, that interim information is incorporated exactly once, that the effect direction and threshold are correct, and that the normal model and planned future enrollment are appropriate.