From Current Evidence to a Future Trial Outcome
A development decision often depends on a question that is different from whether the current evidence looks promising: if the next trial were run as planned, how likely is it to meet its prespecified success criterion? The answer must account both for uncertainty about the treatment effect given the evidence available now and for the sampling variability of the future trial. A favorable observed response alone does not provide that probability.
The success criterion must be specified before calculating the probability. The earlier material on defining success by estimand and endpoint provides the key principle: the target, analysis, and decision rule must describe the same clinical question. Here, treat those elements and the future trial design as fixed inputs. The calculation then concerns the probability that the future data will satisfy the stated rule, conditional on current evidence.
For example, suppose a future trial succeeds if its estimated favorable treatment effect \(\hat{\theta}_{\mathrm{new}}\) exceeds a cutoff \(c\). If, conditional on \(\theta\), the estimate has a normal sampling distribution with standard error \(\sigma_{\mathrm{new}}\), then \(P(S\mid\theta)=P(\hat{\theta}_{\mathrm{new}}>c\mid\theta)\). The predictive probability averages this quantity over the distribution of \(\theta\) given current evidence. This is a probability about a future trial result, not simply the probability that the true effect exceeds a clinical threshold.
Assumptions and a Basic Calculation
A useful first model makes the following assumptions: the success rule and trial design are fixed; \(\theta\) is the relevant effect on the prespecified estimand scale; the future estimate is unbiased for \(\theta\); and its sampling standard error is known or adequately approximated. For a normal approximation, assume \(\hat{\theta}_{\mathrm{new}}\mid\theta\sim N(\theta,\sigma_{\mathrm{new}}^2)\). Also specify a probability distribution for \(\theta\) given current evidence, \(p(\theta\mid D)\). That distribution may represent a posterior distribution from a Bayesian analysis or a discrete set of evidence-weighted scenarios. If these assumptions are not reasonable, the calculation needs a more suitable model.
For a success rule \(\hat{\theta}_{\mathrm{new}}>c\), the conditional success probability at a fixed \(\theta\) is \(1-\Phi((c-\theta)/\sigma_{\mathrm{new}})\), where \(\Phi\) is the standard normal cumulative distribution function. Therefore,
The integral expresses the central idea: first calculate the future trial’s success probability for a particular effect, then average across plausible effects. A plug-in calculation that substitutes a single estimated effect for \(\theta\) leaves out uncertainty in that effect.
Why the Normal Predictive Distribution Has Added Variance
A common special case has a normal distribution for the effect given current evidence: \(\theta\mid D\sim N(\mu_D,\tau_D^2)\), and the future estimate follows the normal sampling model above. The predictive distribution of the future estimate is then normal with mean \(\mu_D\) and variance \(\tau_D^2+\sigma_{\mathrm{new}}^2\). The variance includes both uncertainty about the effect and the future trial’s sampling variability.
To justify this result, write the future estimate as \(\hat{\theta}_{\mathrm{new}}=\theta+\varepsilon\), where \(\varepsilon\) is future sampling error, independent of \(\theta\) given the stated model, with \(\varepsilon\sim N(0,\sigma_{\mathrm{new}}^2)\). The sum of independent normal variables is normal. Its mean is \(E(\theta\mid D)+E(\varepsilon)=\mu_D\); its variance is \(\operatorname{Var}(\theta\mid D)+\operatorname{Var}(\varepsilon)=\tau_D^2+\sigma_{\mathrm{new}}^2\). Thus the success probability for a cutoff \(c\) is
This derivation depends on the normal model and independence assumption. It is not a universal formula for every trial design. For example, if the estimate has a skewed distribution, if its standard error depends materially on \(\theta\), or if success requires several dependent conditions, the predictive probability should be calculated from a model that represents those features.
Worked Example: Averaging Across Discrete Effect Scenarios
Suppose a hypothetical trial has a one-sided test with a prespecified significance level of 0.05 and a planned standard error of 0.20 points on the estimand scale. Under a normal approximation, the trial succeeds when its estimated favorable effect exceeds \(1.645\) standard errors, or approximately \(c=0.329\) points. Current evidence assigns probabilities 0.25, 0.50, and 0.25 to true effects of 0.10, 0.30, and 0.50 points, respectively. Assume the future estimate is unbiased and normally distributed with standard deviation 0.20 points at each effect value.
For \(\theta=0.10\), the standardized cutoff is \((0.329-0.10)/0.20=1.145\). The conditional probability of success is \(1-\Phi(1.145)\approx0.1261\).
For \(\theta=0.30\), it is \(1-\Phi((0.329-0.30)/0.20)=1-\Phi(0.145)\approx0.4424\). For \(\theta=0.50\), it is \(1-\Phi((0.329-0.50)/0.20)=1-\Phi(-0.855)=\Phi(0.855)\approx0.8037\).
Weight the conditional probabilities by the scenario probabilities. Using the unrounded normal probabilities gives \(0.25(0.1261)+0.50(0.4424)+0.25(0.8037)\approx0.4536\), or approximately 0.454. Thus, under this hypothetical scenario distribution and sampling model, the predictive probability that the trial meets its criterion is about 45%. It is not the probability that any one of the three effect values is true.
Continuous Uncertainty and a Prespecified Cutoff
The same calculation applies when current evidence is summarized by a continuous distribution for \(\theta\). The next example keeps the cutoff separate from the evidence model: the cutoff is part of the hypothetical trial’s prespecified success rule, while the mean and variance describe current uncertainty about the effect.
Worked Example: A Normal Distribution for the Effect
Suppose a hypothetical future trial succeeds if its estimate exceeds a prespecified cutoff of 0.294 points. Current evidence gives \(\theta\mid D\sim N(0.1662,0.010176)\), and the planned future estimate has standard error 0.200 points, with \(\hat{\theta}_{\mathrm{new}}\mid\theta\sim N(\theta,0.200^2)\). Assume an unbiased estimator, the stated normal distributions, and independent future sampling error.
The predictive standard deviation is \(\sqrt{0.010176+0.200^2}=\sqrt{0.050176}\approx0.2240\) points. The standardized distance from the predictive mean to the success cutoff is \((0.294-0.1662)/0.2240\approx0.5705\).
Therefore, \(P(S\mid D)=1-\Phi(0.5705)\approx0.284\). Given the model and current evidence, the chance that the future estimate exceeds 0.294 points is approximately 28.4%. This probability is lower than one-half because the cutoff exceeds the predictive mean. It is not a 28.4% probability that the true effect is positive.
More Than One Success Condition
Some protocols define success using multiple conditions, such as meeting an efficacy criterion while keeping an adverse-event estimate below a specified limit. The event \(S\) must then represent their joint occurrence. Multiplying separate probabilities is justified only when the relevant predictive outcomes are independent; otherwise, their joint distribution must be modeled.
Worked Example: Combining Efficacy and Safety Conditions
Suppose a hypothetical trial succeeds only if its efficacy estimate exceeds 0.329 points and its estimate of an adverse-event risk difference is below 0.08. Assume the current evidence and future sampling have already been combined to give predictive distributions: the efficacy estimate is normal with mean 0.45 and standard deviation 0.25 points, and the risk-difference estimate is normal with mean 0.02 and standard deviation 0.03. Assume these two predictive estimates are independent.
The predictive probability of meeting the efficacy condition is \(1-\Phi((0.329-0.45)/0.25)=\Phi(0.484)\approx0.686\). The predictive probability of meeting the safety condition is \(\Phi((0.08-0.02)/0.03)=\Phi(2)\approx0.9772\).
Under the stated independence assumption, the joint probability is \(0.686(0.9772)\approx0.670\). The resulting predictive probability of satisfying both conditions is about 67.0%. If efficacy and safety outcomes are dependent, multiplying these probabilities is generally incorrect; the joint predictive distribution, including the dependence, is needed.
What Predictive Probability Is—and Is Not
Predictive probability is easily confused with related quantities. Their answers can differ because they condition on different information or represent different uncertainty.
| Quantity | What is held fixed or averaged over? | Interpretation |
|---|---|---|
| Predictive probability \(P(S\mid D)\) | Averages over uncertainty in \(\theta\mid D\) and future trial variability | Probability the planned future trial meets its rule, given current evidence and the model |
| Conditional power | Typically conditions on interim data and uses a specified assumed effect for the unobserved future data | Probability of eventual success under that conditional-power setup and its effect assumption |
| Observed response | Uses data already observed | An estimate or outcome, not a probability of a future trial meeting its rule |
| Posterior probability of a parameter condition | Averages over \(\theta\mid D\), without future trial sampling | For example, the probability that \(\theta\) exceeds a clinical threshold—not necessarily the future trial’s success probability |
Conditional power is not a single uniquely specified calculation: its assumptions about the effect in the remaining data matter. A common form conditions on interim observations and assumes a fixed treatment effect for future observations. A Bayesian predictive calculation instead averages over a distribution of effects given the available evidence, while also representing future sampling variability. The methods may use overlapping data and related likelihood models, but they answer different questions unless their conditioning and assumptions are deliberately aligned.
Likewise, an observed interim response of, for example, 0.30 points is not a 30% probability of eventual success. It is an estimate on an effect scale. To turn interim evidence into a probability of future success requires a model for uncertainty about the effect, a model for future data, and the prespecified success rule.
Diagnostics, Limitations, and Common Pitfalls
A numerical answer is only as useful as its assumptions. Before interpreting a predictive probability, check that the endpoint, estimand, analysis population, missing-data strategy, and decision rule match the planned trial. Confirm that the estimated future standard error corresponds to the intended sample size and design. If the design is adaptive or includes interim decisions, calculate the probability under that design rather than treating it as a fixed trial with an unchanged analysis.
- Check the effect distribution. Inspect whether the distribution given current evidence is adequately represented by the chosen normal model or scenario set. Skewness, multimodality, or substantial prior sensitivity can make a single normal approximation misleading.
- Check the sampling model. Validate the assumed standard error, bias, and distribution of the future estimator. A normal approximation may be poor for small samples, rare events, or constrained estimates.
- Check the success event. Make sure every threshold, test direction, multiplicity adjustment, and required endpoint is included. Relaxing or changing the rule after seeing current results changes the question.
- Check dependence. For joint efficacy, safety, or co-primary criteria, use their joint predictive distribution unless independence is justified.
- Check sensitivity. Recalculate under plausible alternative effect distributions, future standard errors, and model assumptions. A decision that changes across reasonable assumptions should not be presented as resting on a precise probability.
A frequent mistake is to set \(\theta\) equal to its current estimate and report the resulting conditional success probability as the predictive probability. That is wrong when current uncertainty about \(\theta\) is material: it substitutes one effect value for the full distribution \(p(\theta\mid D)\). The correction is to integrate the conditional success probability over that distribution, analytically when possible or by numerical integration or simulation.
Simulation is a practical alternative when the integral is not available in closed form. Draw plausible parameter values from their distribution given current evidence; for each draw, simulate future trial data under the planned design; apply the success rule; and estimate the probability by the proportion of simulated trials that succeed. The simulation must preserve relevant dependence and design features. Its estimate also has Monte Carlo error, so enough draws should be used and that error assessed.
Finally, a predictive probability is not itself a go/no-go rule. Whether a probability warrants continuing depends on the consequences of decisions, costs, and alternatives. The probability summarizes one important uncertainty for decision-making; it does not remove uncertainty about model adequacy or determine the decision without context.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- A future estimate has distribution \(N(0.25,0.10^2)\) given a fixed effect value, and success requires the estimate to exceed 0.20. What is the conditional probability of success at that effect value?
- Current evidence gives \(\theta\mid D\sim N(0.20,0.10^2)\), and the future estimate has standard error 0.15. For a cutoff of 0.30, calculate the predictive probability of success.
- Why is an observed interim estimate not itself a predictive probability of success?
- A trial requires both efficacy and safety conditions. When may their predictive probabilities be multiplied, and what is needed otherwise?
Solutions. For question 1, standardize the cutoff: \((0.20-0.25)/0.10=-0.50\). Thus \(P(S\mid\theta)=1-\Phi(-0.50)=\Phi(0.50)\approx0.6915\), or 69.2%. For question 2, the predictive standard deviation is \(\sqrt{0.10^2+0.15^2}=\sqrt{0.0325}\approx0.1803\). The standardized cutoff is \((0.30-0.20)/0.1803\approx0.5547\), so \(P(S\mid D)=1-\Phi(0.5547)\approx0.290\), or about 29.0%. For question 3, the interim estimate is an observed estimate, not a probability; calculating the future success probability requires uncertainty about the effect, future sampling variability, and the success rule. For question 4, multiplication is valid if the two predictive success events are independent. Otherwise, use their joint predictive distribution and account for dependence.