Choosing Priors for Predictive Probability

Why Prior Choice Matters

Predictive probability asks how likely trial success is, given the data available now and a model for the treatment effect and future observations. In a Bayesian calculation, that model includes a prior distribution for the effect. The prior therefore affects the predictive probability even when the interim data, design, and success boundary remain unchanged.

This tutorial focuses on choosing and comparing three normal treatment-effect priors: weakly informative, skeptical, and enthusiastic. These labels describe how a prior is centered and how much spread it allows; they do not specify universal numerical values. A prior is reasonable only relative to the treatment-effect scale, plausible clinical effects, and the information in the trial. The introductory material on posterior predictive probability gives the general interpretation; here the emphasis is on how the prior enters the calculation.

A prior should be specified on the treatment-effect scale before using the unblinded interim treatment-effect estimate to calculate predictive probability. Choosing an enthusiastic prior after seeing an interim result simply because it produces a desired probability does not provide a neutral assessment. When prior knowledge is uncertain, comparing several defensible specifications makes that dependence visible.

Normal-Trial Model and Predictive Probability

Consider a trial estimating a treatment effect \(\theta\), measured in standardized outcome units. Assume a normal model with known variance, so that an estimate based on information \(I\) has sampling distribution \(\hat{\theta}_I\mid\theta\sim N(\theta,1/I)\). Information is measured on the inverse-squared-effect scale. At an interim analysis with information \(I_1\), let the observed estimate be \(\hat{\theta}_1\), and define the interim score \(S_1=I_1\hat{\theta}_1\). Suppose the trial continues to final information \(I_2>I_1\), with future information \(\Delta I=I_2-I_1\).

Under independent increments, the future score increment \(S_2-S_1\), conditional on \(\theta\), is normal with mean \(\Delta I\theta\) and variance \(\Delta I\). The final standardized statistic is \(Z_2=S_2/\sqrt{I_2}\). For this example, success means crossing a prespecified upper boundary \(c\) at the final analysis; take \(c=1.96\). Assume there is no earlier efficacy or futility stopping that would alter the event being predicted.

Use a normal prior \(\theta\sim N(m_0,\tau_0^2)\). The interim estimate has likelihood \(\hat{\theta}_1\mid\theta\sim N(\theta,1/I_1)\). Combining these distributions gives the posterior variance \(v_1\) and mean \(m_1\) below. Conditional on the interim data, integrating over this posterior gives the predictive distribution of the final score and hence the probability of success.

The second argument of \(N(\cdot,\cdot)\) is a variance. The predictive variance includes two sources: sampling variation in future data, \(\Delta I\), and uncertainty about \(\theta\) after the interim, \((\Delta I)^2v_1\). Omitting the latter would fail to average over posterior uncertainty in the effect.

Deriving the Posterior and Predictive Distribution

The posterior formulas follow by multiplying the prior density by the interim likelihood. Terms in the exponent that depend on \(\theta\) are \(-\tfrac12\{(\theta-m_0)^2/\tau_0^2+I_1(\hat{\theta}_1-\theta)^2\}\). The coefficient of \(\theta^2\) is the posterior precision, \(1/\tau_0^2+I_1\), and the coefficient of the linear term is \(m_0/\tau_0^2+I_1\hat{\theta}_1\). Completing the square therefore gives the stated \(v_1\) and \(m_1\).

For the predictive result, conditional on \(\theta\), write \(S_2=S_1+U\), where \(U\mid\theta\sim N(\Delta I\theta,\Delta I)\). The posterior mean of \(U\) is \(\Delta I m_1\). By the law of total variance, its posterior predictive variance is \[ E\{\operatorname{Var}(U\mid\theta)\mid\text{interim data}\} +\operatorname{Var}\{E(U\mid\theta)\mid\text{interim data}\} =\Delta I+(\Delta I)^2v_1. \] A normal mixture with this normal posterior is normal, so \(S_2\) has the displayed predictive distribution. Success requires \(S_2>c\sqrt{I_2}\); standardizing that normal variable yields the predictive-probability formula.

Comparing Three Priors on the Same Data

The next comparison keeps the hypothetical trial data and design fixed and changes only the prior. For illustration, take the weakly informative prior to be \(N(0,0.20^2)\), the skeptical prior to be \(N(0,0.05^2)\), and the enthusiastic prior to be \(N(0.20,0.05^2)\). All effects and prior standard deviations are in standardized outcome units. These values illustrate the calculation; they are not recommendations for another trial.

A narrow prior has more influence than a broad prior because its precision contributes more to the posterior precision. In this comparison, the skeptical and enthusiastic priors each have precision 400, four times the interim information of 100. Their posterior means are therefore pulled strongly toward their prior centers. A broad prior contributes less precision, so the interim estimate has greater influence.

Choosing and Checking a Prior

A defensible prior choice begins with the effect scale and the range of effects that would matter clinically. State the prior mean and spread in those units, and check what probability the prior assigns to effects that domain experts regard as negligible, plausible, or implausibly large. “Weakly informative” should not mean arbitrarily diffuse: an extremely broad prior can place substantial mass on unrealistic effects and can make the predictive calculation depend on those values.

Also check compatibility between the prior and the interim estimate. One check is the prior predictive distribution of the interim estimate: under the model above, \(\hat{\theta}_1\) before seeing the interim data has distribution \(N(m_0,\tau_0^2+1/I_1)\). An interim estimate far into the tail of this distribution may indicate conflict between prior assumptions and observed data, or a problem with the model. It is a prompt for investigation, not automatic proof that either source is wrong.

The prior comparison is useful because it separates a data-driven conclusion from a conclusion that depends heavily on assumptions. Report the prior specifications and the resulting probabilities, rather than presenting one probability as if it were determined by the interim data alone. A Bayesian predictive probability is conditional on the data and the specified model and prior; it is not the probability that the null hypothesis is true.

Common Mistakes, Alternatives, and Limitations

• Calling a prior “skeptical” without specifying it. The correction is to report its distribution, center, and spread on the treatment-effect scale. Different skeptical priors can yield materially different predictive probabilities.
• Using the prior mean as if it were the true effect. The predictive calculation averages over the full posterior distribution. Substituting a single effect value removes posterior uncertainty and changes the method.
• Ignoring posterior uncertainty in the future score. Its predictive variance is \(\Delta I+(\Delta I)^2v_1\), not just \(\Delta I\). Omitting the second term generally understates predictive uncertainty.
• Adjusting the prior after seeing an unblinded interim estimate to obtain a preferred result. Specify and justify priors in advance, or clearly label a post hoc calculation as exploratory. A sensitivity analysis should compare plausible assumptions rather than select a favorable one.
• Using the formula when the model does not match the design. The calculation assumes a normal estimate with known variance, fixed information increments, independent increments, and the stated final success event. Early stopping, changing information, estimated nuisance parameters, or non-normal outcomes may require a different predictive calculation.

The main alternative for a quick operational calculation is plug-in conditional power: specify one effect value, often the interim estimate or a design effect, and calculate the chance of success conditional on that value. Predictive probability instead averages conditional success over a posterior distribution for the effect. Neither is assumption-free. Plug-in conditional power depends on the selected effect value; predictive probability depends on the prior and likelihood. The formula here is also limited to one remaining final analysis with a single upper boundary; it does not calculate success across multiple future looks or account for uncertainty in nuisance parameters.