From an Interim Posterior to a Predictive Decision
An interim result is incomplete evidence: the trial has observed some participants, but more data could change the final conclusion. A Bayesian predictive decision rule asks a practical question about that remaining uncertainty: given the data so far, how likely is the trial to meet its final success criterion if it continues? That probability can guide whether to continue enrollment, stop for futility, or—in a separately specified rule—stop for efficacy.
The probability used for this decision is not simply the current posterior probability that the treatment works. It averages over possible future data and asks whether those data would produce a specified final decision. Simulation then addresses a different question: across repeated trials under specified conditions, how often does the rule continue, stop, or ultimately declare success?
A Normal Model for an Interim Treatment Contrast
For a transparent derivation, suppose the endpoint is a continuous treatment contrast, such as a difference in mean change from baseline. Let \(\Delta\) be the treatment-minus-control effect, with positive values favoring treatment. Assume an interim estimate \(Y_I\) and a future estimate \(Y_F\) satisfy
and are conditionally independent given \(\Delta\). Use the prior \(\Delta\sim N(\mu_0,\tau^2)\), and treat the variances as known. This stylized model is suitable when the estimates are approximately normal and the planned future information is specified. It does not automatically account for unplanned changes in enrollment, missing data, non-normal endpoints, or changes to the analysis model; those features must be represented in the simulation if they matter to the design.
After observing \(Y_I=y_I\), normal-normal updating gives interim posterior variance \(v_I\) and mean \(m_I\). After future data arrive, the final posterior variance is \(v_N\), and the final posterior mean \(m_N\) depends on \(Y_F\).
Suppose final success is defined as a posterior probability of at least \(\gamma\) that \(\Delta>0\). Since the final posterior is normal, this is equivalent to \(m_N\ge z_\gamma\sqrt{v_N}\), where \(z_\gamma\) is the standard normal quantile with cumulative probability \(\gamma\). The interim posterior also gives a predictive distribution for \(Y_F\): its mean is \(m_I\) and its variance is \(v_I+s_F^2\). Substituting this distribution into the final success condition yields a probability that can be calculated before seeing the future data.
where \(\Phi\) is the standard normal cumulative distribution function. This expression follows because \(m_N\), conditional on the interim data, is normal with mean \(m_I\) and standard deviation \((v_N/s_F^2)\sqrt{v_I+s_F^2}\). The continuation threshold \(c\) is a design choice, not a mathematical constant. A rule might continue when the displayed probability is at least \(c\); its consequences must be evaluated over trial scenarios.
Worked Example: Calculating the Predictive Probability
Consider a hypothetical trial with interim and future contrast estimates each based on information corresponding to \(s_I^2=s_F^2=5\) squared score units. Let the prior be \(\Delta\sim N(0,100)\), the observed interim estimate be \(y_I=5\) score units, and final success require \(P(\Delta>0\mid\text{all data})\ge0.95\). Use a futility rule that continues if the predictive probability of final success is at least \(0.20\).
The interim posterior variance and mean are
At completion, \(v_N=(0.01+0.20+0.20)^{-1}=2.4390\), so the final posterior mean must be at least \(1.645\sqrt{2.4390}=2.569\) score units. The predictive standard deviation of that final posterior mean is
Therefore,
Because \(0.925\ge0.20\), the rule continues. This is a probability conditional on the interim data and model; it is not a guarantee that the final criterion will be met.
Why the Predictive Calculation Works
The key step is to propagate uncertainty in both the unknown effect and the future estimate. Conditional on interim data, \(\Delta\sim N(m_I,v_I)\). Since \(Y_F=\Delta+\epsilon_F\), with independent \(\epsilon_F\sim N(0,s_F^2)\), adding the two variances gives \(Y_F\mid y_I\sim N(m_I,v_I+s_F^2)\). The final posterior mean is a linear function of \(Y_F\), so it is also normally distributed.
For completeness, rewriting the final posterior mean using the interim posterior gives \(m_N=m_I+(v_N/s_F^2)(Y_F-m_I)\). Its predictive mean is therefore \(m_I\), and multiplying the standard deviation of \(Y_F\) by the absolute value of the coefficient gives \((v_N/s_F^2)\sqrt{v_I+s_F^2}\). Standardizing the final-success boundary \(m_N\ge z_\gamma\sqrt{v_N}\) gives the stated normal tail probability. The calculation assumes that the future information and analysis are as planned; if the future sample size is random, its uncertainty must also be included.
Worked Example: Prior-Data Conflict at Interim
Suppose a hypothetical program uses a prior \(\Delta\sim N(5,25)\) score units squared, expressing a prior center of 5 score units. The interim estimate is \(y_I=-8\), with \(s_I^2=5\); planned future information has \(s_F^2=5\). Use the same final success criterion, \(P(\Delta>0\mid\text{all data})\ge0.95\), and the same continuation threshold of 0.20.
The interim posterior is
The prior predictive distribution of the interim estimate is \(Y_I\sim N(5,25+5)\). Thus the observed estimate is \((-8-5)/\sqrt{30}=-2.373\) prior-predictive standard deviations below the prior mean. This is a diagnostic signal of tension between the prior and observed data, although a tail-area diagnostic alone does not determine whether the prior is appropriate.
For the final analysis, \(v_N=(1/25+1/5+1/5)^{-1}=2.2727\), and the success boundary is \(1.645\sqrt{2.2727}=2.480\) score units. The predictive standard deviation of the final posterior mean is
The predictive probability is \(\Phi((-5.833-2.480)/1.376)=\Phi(-6.04)\), approximately \(8\times10^{-10}\). The rule stops for futility because this is below 0.20. The result illustrates why a prior should be examined against plausible data before it is used in a decision rule: a prior centered on benefit does not ensure that conflicting interim evidence is ignored, but prior strength affects how much that evidence shifts the posterior.
Simulating Decision Rates and Calibration
A single interim calculation describes one observed trial. Design evaluation requires repeated simulated trials under specified data-generating conditions. For each replicate, generate interim data, calculate the predictive probability using the planned prior and analysis, apply the decision rule, and—if the trial continues—generate the future data and apply the final criterion. Repeat under a range of true treatment effects, including no effect and clinically relevant alternatives. Keep the simulated truth distinct from the prior: the truth generates data, while the prior is part of the analysis rule.
Set the true treatment effect, sample sizes or information levels, outcome variability, prior, final criterion, and continuation threshold. Include plausible prior-data conflict scenarios.
Generate each replicate under its chosen truth, then calculate the interim posterior and predictive probability using the prespecified analysis.
Record continuation or stopping. For continued trials, generate future observations and apply the final success criterion.
Estimate continuation, futility stopping, and final success rates, with Monte Carlo uncertainty. Check calibration separately by comparing predicted probabilities with observed outcomes.
Worked Example: Comparing Decision Rates Across Effects
As a hypothetical illustration, use the model and rule from the first worked example: \(s_I^2=s_F^2=5\), prior \(N(0,100)\), final posterior probability threshold 0.95, and continuation threshold 0.20. A simulation of 10,000 trials per scenario might produce the following illustrative counts. “Final successes” counts successes among all initiated trials; trials stopped at interim are not counted as successes.
| True effect (score units) | Continued trials | Continuation rate | Final successes | Success rate per initiated trial |
|---|---|---|---|---|
| -2 | 670 | 670/10,000 = 6.70% | 4 | 4/10,000 = 0.04% |
| 0 | 2,740 | 2,740/10,000 = 27.40% | 390 | 390/10,000 = 3.90% |
| 3 | 7,710 | 7,710/10,000 = 77.10% | 5,660 | 5,660/10,000 = 56.60% |
| 6 | 9,820 | 9,820/10,000 = 98.20% | 9,805 | 9,805/10,000 = 98.05% |
These are hypothetical simulation results, not empirical trial findings. They show how the same rule can have different continuation and success rates as the true effect changes. Under the zero-effect scenario, the 3.90% success rate estimates the probability of a false success under this particular model and design. For a rate near 0.50 from 10,000 independent replicates, the approximate Monte Carlo standard error is \(\sqrt{0.5(1-0.5)/10{,}000}=0.005\), or 0.50 percentage points; smaller rates have different Monte Carlo uncertainty. Reporting uncertainty helps distinguish a genuine design difference from simulation noise.
Calibration, Diagnostics, and Alternatives
Calibration concerns whether predicted probabilities correspond to observed frequencies under the assumptions used to generate and analyze data. One check is to group simulated trials by their interim predictive probability, then compare the average prediction in each group with the proportion that achieves final success. Under a correctly specified prior-predictive model, predictions should be calibrated on average over that model’s distribution of effects and data. Conditional on a fixed true effect, however, the prior-predictive probability need not equal the success frequency. Therefore, assess calibration under the prior-predictive model and assess operating characteristics separately at fixed effects, including alternatives and null scenarios.
Useful diagnostics include checking the distribution of simulated interim estimates against prior-predictive expectations, verifying that predictive probabilities lie between zero and one and respond sensibly to data, and comparing a small set of simulated cases with direct calculations. Examine results under weakened, shifted, or otherwise scientifically defensible priors when prior-data conflict is plausible. Also check that simulated sample sizes, variances, endpoint timing, and interim information match the intended trial. Numerical convergence and model misspecification can undermine results even when the decision rule is coded correctly.
A posterior probability rule based only on interim evidence is a simpler alternative, but it does not directly quantify the chance that the final criterion will be met after collecting more data. A frequentist futility boundary is another approach: it can be calibrated to specified operating characteristics without interpreting a prior probability, but its boundary is not itself a posterior predictive probability. Predictive rules make the future decision explicit, while their conclusions depend on the likelihood, prior, planned future information, and final success definition. No approach removes the need to simulate scenarios that matter for the development decision.
Check Your Understanding
Answer each question, then compare with the solutions below.
- What is the difference between an interim posterior probability that \(\Delta>0\) and a predictive probability of final success?
- If the future estimate has variance \(s_F^2\), what is the predictive variance of \(Y_F\) given interim data in the normal model?
- Why should a simulation assess both a zero-effect scenario and clinically relevant nonzero effects?
- What does a prior-predictive interim estimate several standard deviations from its prior-predictive mean suggest, and what does it not establish by itself?
Solutions. (1) The posterior probability concerns the effect given currently observed data. The predictive probability concerns future data and whether the completed trial will meet a prespecified success criterion. (2) It is \(v_I+s_F^2\), because uncertainty in the effect after interim data and future sampling variance add under conditional independence. (3) The zero-effect scenario evaluates false success under that scenario, while nonzero effects show continuation and success behavior when treatment benefit is present; together they characterize different operating properties. (4) It signals possible conflict between observed data and the prior predictive distribution. It does not alone prove that the prior or model is wrong; the scientific plausibility of the prior and the data-generating assumptions must also be assessed.