Two Probabilities, Two Questions
At an interim analysis, a Bayesian trial team may ask whether the accumulating evidence favors treatment benefit and whether the trial is likely to finish with a result that meets its prespecified success rule. These questions are related, but they are not interchangeable. One concerns the unknown treatment effect given data observed so far; the other concerns future data and a decision rule applied after those data arrive.
The difference matters in practice. A high current probability of benefit does not guarantee that the final analysis will cross a stricter posterior threshold: future observations may be noisy or may weaken the interim signal. Conversely, predictive success is not a probability that treatment works. It is a probability about what a specified future analysis will conclude, under the model and assumptions used to predict it.
A probability of benefit is about the parameter under the current posterior distribution. Predictive success averages over uncertainty about that parameter and the possible future observations. The event being predicted must be stated explicitly—for example, that the final posterior probability of benefit will be at least 0.95. Changing that criterion changes the predictive probability.
A Normal Model and a Final Go Rule
The examples use a hypothetical continuous endpoint, such as symptom-score reduction, measured in points. Positive \(\delta\) means the treatment reduces symptoms more than control. Assume the current posterior distribution is normal, \(\delta\mid D\sim N(m,v)\), where \(m\) is its mean and \(v\) its variance. Assume the future estimate \(W\), based on the remaining participants, satisfies \(W\mid\delta\sim N(\delta,\tau^2)\). Here \(\tau\) is its known standard error.
For this calculation, assume the current and future data are conditionally independent given \(\delta\), the future standard error is known and fixed, the normal likelihood is appropriate, and the current posterior adequately represents uncertainty about the effect. These assumptions make the future-data calculation explicit; they may not hold exactly in a real trial.
Suppose the final go rule is \(P(\delta>0\mid D,W)\geq\gamma\), with \(\gamma=0.95\). More generally, the calculation below works for any stated \(\gamma\) between 0 and 1. Combining the current posterior with the future normal likelihood gives a final posterior with variance \(s_f^2\) and mean \(m_f\):
For a normal posterior, \(P(\delta>0\mid D,W)=\Phi(m_f/s_f)\), where \(\Phi\) is the standard normal cumulative distribution function. Thus the final go rule is equivalent to requiring \(m_f\geq c s_f\), where \(c=\Phi^{-1}(\gamma)\). For \(\gamma=0.95\), \(c\) is approximately 1.645.
This inequality can be rearranged to find the minimum future estimate \(W\) that would produce a go decision. Since \(s_f^2/\tau^2>0\), rearranging does not reverse the inequality:
Given the current data, \(W\) has a normal predictive distribution. Integrating over the current posterior for \(\delta\) gives \(W\mid D\sim N(m,v+\tau^2)\): its predictive mean is \(m\), and its variance is the sum of posterior variance \(v\) and future sampling variance \(\tau^2\). Therefore,
This is a predictive probability for the specified final decision rule. It is not obtained by simply re-labelling the current posterior probability of benefit.
Worked Comparisons
Worked Example: Strong Current Evidence, Lower Predictive Success
Hypothetical trial. The interim posterior for the symptom-score benefit is \(N(0.33,0.04)\) points, so \(m=0.33\) points and \(v=0.04\) points squared. The standard error for the future estimate is \(\tau=0.20\) points. The final go rule requires at least 0.95 posterior probability that benefit is positive.
Current probability of benefit. The current posterior standard deviation is \(\sqrt{0.04}=0.20\) points. Hence
Predictive probability of final success. The final posterior standard deviation is
The final posterior mean must be at least \(1.645(0.1414)\approx0.2326\) points. With equal current and future variances, \(m_f=(0.33+W)/2\), so the final rule requires \(W\geq 2(0.2326)-0.33=0.1352\) points. The predictive distribution is \(W\mid D\sim N(0.33,0.08)\), with standard deviation \(\sqrt{0.08}\approx0.2828\) points. Therefore,
The interim posterior probability of benefit is about 0.951, but the probability of satisfying the final go rule is about 0.755. The first number describes current evidence about benefit; the second reflects the possibility that future data will or will not leave enough posterior evidence to cross the final threshold.
Worked Example: Same Current Evidence, Different Remaining Information
Hypothetical trial. Keep the same current posterior, \(N(0.33,0.04)\), and the same 0.95 final posterior go rule. Suppose, however, that the future estimate has standard error \(\tau=0.05\) points rather than 0.20 points. The current posterior probability of benefit remains \(\Phi(1.65)\approx0.9505\); only the amount of remaining information has changed.
The final posterior variance and standard deviation are
The final posterior mean must be at least \(1.645(0.04851)\approx0.07980\) points. In this case \(m_f=(25(0.33)+400W)/425\). Solving \(m_f\geq0.07980\) gives \(W\geq0.06415\) points. The predictive distribution is \(W\mid D\sim N(0.33,0.0425)\), with standard deviation about \(0.2062\) points. Thus,
The predictive success probability is now about 0.901 rather than 0.755, even though the current posterior is unchanged. Under these assumptions, the more informative future estimate makes the eventual decision more closely reflect the current evidence. The example shows why the current posterior alone cannot determine predictive success: the amount of information remaining also matters.
Worked Example: Favorable Interim Evidence Need Not Mean High Predictive Success
Hypothetical trial. Suppose the interim posterior is \(N(0.1683,0.04)\) points, the future standard error is \(0.20\) points, and the final go rule still requires a 0.95 posterior probability of benefit. Current evidence is favorable because
The final posterior standard deviation is again \(0.1414\) points, so the final posterior mean must reach approximately \(0.2326\) points. Since \(m_f=(0.1683+W)/2\), the future estimate must be at least \(W_{\mathrm{crit}}=2(0.2326)-0.1683=0.2969\) points. The predictive distribution is \(W\mid D\sim N(0.1683,0.08)\). Therefore,
An 0.800 current probability of benefit may sound favorable, but it is not the probability of meeting a more demanding final go rule. With the stated amount of remaining information, the predictive probability of final success is about 0.325.
Interpretation, Checks, and Limitations
The distinction can be summarized by the event each probability refers to:
| Quantity | Event being evaluated | What it depends on |
|---|---|---|
| Current posterior probability of benefit | \(\delta>0\), given interim data \(D\) | The current posterior distribution and the definition of benefit |
| Predictive probability of final success | The final analysis satisfies a stated success rule | The current posterior, future-data model, remaining information, and final rule |
A useful diagnostic is to write down the event before interpreting the number. If the event is “the treatment effect is positive,” the probability is about benefit under the current posterior. If it is “the final posterior probability of benefit will be at least 0.95,” the probability is predictive success. The latter depends on both future sampling variability and parameter uncertainty; in the normal model, these appear together as \(v+\tau^2\).
The calculation also makes important failure modes visible. If future observations are not conditionally independent of the current data as assumed, or if the future standard error is not known and fixed, the predictive distribution used above may be wrong. Uncertain nuisance parameters, changes to the analysis plan, unmodeled missingness, or adaptation of the design can also alter the distribution of future data or the final rule. In those settings, the prediction must reflect those features rather than use the simple normal formula unchanged.
A common mistake is to report a current posterior probability of 0.95 as “a 95% probability that the final trial will succeed.” That conclusion does not follow: the current probability concerns \(\delta>0\), whereas final success is an event involving future observations and a specified decision criterion. Calculate or otherwise evaluate the predictive probability under the future-data model before making a claim about the chance of a final go decision.
Another pitfall is comparing predictive probabilities from trials without checking that their final criteria and prediction assumptions are the same. A value of 0.80 under a 0.90 posterior threshold is not directly comparable to 0.80 under a 0.99 threshold. Likewise, predictive success is conditional on the current data and the assumed model; it does not replace assessment of the trial procedure's performance across possible true effects.
The normal examples isolate the conceptual contrast, but their numerical results depend on the assumptions. Other outcome models require their own posterior and predictive distributions. The general distinction remains: current posterior evidence describes uncertainty about the treatment effect now, while predictive success concerns a future decision. The introductory material on predictive probability of success establishes the averaging-over-future-data idea; here the key interpretive step is to keep that future decision event separate from the current probability of benefit.
Check Your Understanding
Answer each question, then compare with the solutions below.
- In one sentence each, state the event described by the current posterior probability of benefit and by predictive probability of final success.
- A current posterior is \(N(0.20,0.25^2)\). What is the current posterior probability that \(\delta>0\)?
- If no future data will be collected and the final rule is the same as the current rule, what does predictive success become?
- Suppose \(m=0.10\) points, \(v=0.04\) points squared, \(\tau=0.20\) points, and final success requires a 0.95 posterior probability of benefit. Find the predictive probability of final success.
Solutions. (1) Current posterior probability of benefit is \(P(\delta>0\mid D)\). Predictive success is the probability, conditional on current data, that future data will make the final analysis meet its stated success rule. (2) The posterior standard deviation is \(0.25\), so \(P(\delta>0\mid D)=\Phi(0.20/0.25)=\Phi(0.80)\approx0.7881\). This is the current probability of benefit, not the chance of satisfying a final rule. (3) If the analysis and rule do not change and there are no future observations, the final decision is already determined by the current data. Predictive success is 1 if the current result meets that rule and 0 otherwise. (4) The final standard deviation is \(s_f=\sqrt{1/50}\approx0.1414\) points, and the required final posterior mean is \(1.645(0.1414)\approx0.2326\) points. Since \(m_f=(0.10+W)/2\), success requires \(W\geq2(0.2326)-0.10=0.3652\) points. Under the predictive distribution \(W\mid D\sim N(0.10,0.08)\), the standardized cutoff is \((0.3652-0.10)/0.2828\approx0.9379\). The predictive probability is \(1-\Phi(0.9379)\approx0.1743\).