From Interim Evidence to a Continuation Rule
An interim analysis can inform whether a development program should continue, but an interim estimate is not the final trial result. It is therefore useful to ask a forward-looking question: given the information available now and the trial that remains, how likely is final success? A predictive-probability rule converts that question into a decision by specifying how large that probability must be to justify continuing.
The rule is not universal. A high continuation threshold may conserve resources by stopping trials unlikely to succeed, but it can also stop trials with a genuine treatment effect. A low threshold preserves more opportunities for eventual success but continues more trials that will fail. The threshold should reflect the development decision, the consequences of stopping or continuing, and the reliability of the predictive model.
This probability is conditional on the model and design assumptions being used. It is not the probability that a hypothesis is true, nor is it the probability that the interim result itself is statistically significant. The final success criterion must be defined before applying the rule—for example, that the final estimated treatment effect exceeds a clinically relevant boundary.
Constructing the Rule
Let \(Y_I\) be the interim estimate of the treatment effect, measured in effect units, and let \(Y_F\) denote the final statistic used to determine success. Write \(c\) for the prespecified success boundary. A simple normal predictive model represents the conditional distribution of the final statistic as
Here \(m_F(y)\) is the predicted conditional mean of the final statistic and \(s_F\) is its predictive standard deviation, including the remaining sampling uncertainty represented by the model. If success means \(Y_F>c\), then
where \(\Phi\) is the standard normal cumulative distribution function. This expression follows by standardizing \(Y_F\): the event \(Y_F>c\) is equivalent to \((Y_F-m_F(y))/s_F>(c-m_F(y))/s_F\), and symmetry of the standard normal distribution gives the displayed upper-tail probability.
If the predictive mean is an increasing linear function, \(m_F(y)=a y+b\), with \(a>0\), then the rule \(\mathrm{PP}(y)\ge q\) is equivalent to a threshold on the interim estimate:
The equivalence follows because \(\Phi\) is increasing: \(\mathrm{PP}(y)\ge q\) exactly when \((a y+b-c)/s_F\ge z_q\). A threshold on a probability is therefore also an interim-data boundary in this normal, monotone case. If the predictive relationship is nonlinear or non-monotone, the continuation region must instead be found directly from the model.
Worked Example: Predictive Probability at an Interim Estimate
Suppose a hypothetical trial uses an effect scale in which final success means \(Y_F>0.1960\). The model-based predictive distribution is \(Y_F\mid Y_I=y\sim N(0.7692y,\,0.1074^2)\). These conditional model quantities are stipulated for this illustration. At an interim estimate of \(y=0.3500\), calculate the predictive probability of final success.
Substitution into the normal tail expression gives
The predictive probability is about 75.23%. If the prespecified rule requires at least 60% predictive probability to continue, this interim result meets the rule. The calculation does not mean there is a 75.23% probability that the treatment effect is above zero; it is the model-based probability of meeting the stated final-trial criterion, conditional on this interim estimate and the assumed remaining design.
Choose the Threshold, Then Evaluate Its Consequences
A predictive probability alone does not determine the decision. The continuation threshold \(q\) is a decision input. For the model above, \(a=0.7692\), \(b=0\), \(c=0.1960\), and \(s_F=0.1074\). If the threshold is \(q=0.60\), then \(z_q\approx0.2533\), giving the interim boundary
Thus the rule continues when the interim estimate is at least 0.2902 effect units. To understand how often this rule continues under different treatment effects, make a separate sampling assumption about the interim estimate. Suppose, for this hypothetical operating-characteristic calculation, that under a fixed true effect \(\theta\),
This assumes an unbiased normally distributed interim estimate with known standard error, a fixed information amount, and no unmodeled changes in the estimand or data-generating process. Under those assumptions, the probability of continuing is
Worked Example: Operating Consequences at Three Treatment Effects
Use the 60% predictive-probability rule, with \(y_{\mathrm{crit}}=0.2902\) and interim standard error \(1/\sqrt{40}\). Calculate the chance of continuation under three hypothetical true effects.
True effect \(\theta=0.000\). The standardized boundary is \(0.2902\sqrt{40}\approx1.8354\). Therefore
Under this scenario, the rule continues about 3.322% of trials. Equivalently, it stops about 96.68%, since the continuation and stopping probabilities sum to one (ignoring equality at the boundary, which has probability zero under a continuous distribution).
True effect \(\theta=0.300\). The standardized boundary is \((0.2902-0.3000)\sqrt{40}\approx-0.0620\). Hence
A true effect of 0.300 effect units is just above the interim boundary, but sampling variation means continuation occurs in only about 52.47% of such trials.
True effect \(\theta=0.400\). The standardized boundary is \((0.2902-0.4000)\sqrt{40}\approx-0.6944\). Thus
The rule continues about 75.63% of trials under this effect. This is an operating characteristic of the decision rule under the stated sampling model; it is not the predictive probability calculated for any particular observed interim estimate.
Interpreting the Rule and Checking Its Assumptions
A useful evaluation examines the rule across effects that matter to the development decision, not just one favorable scenario. In particular, the continuation probability when the true effect is negligible describes how often the program would proceed despite little or no effect under the assumed model. The probability of continuing at a clinically worthwhile effect describes how often the interim rule preserves a potentially valuable program. These rates expose the trade-off created by the chosen threshold; they do not, by themselves, identify the best threshold.
The predictive calculation and the operating-characteristic calculation use related but distinct probabilities. The predictive probability conditions on the observed interim estimate and uses the model for final outcomes. The operating characteristic instead averages the decision over possible interim estimates under a specified true effect. Keeping these questions separate prevents a common interpretive error.
Before applying the rule, verify that the model and the trial plan represent what will actually happen. Important checks include whether the endpoint, estimand, analysis population, variance assumptions, and remaining sample size match the planned final analysis; whether the interim estimator has the assumed sampling distribution; and whether missing data, delayed outcomes, or operational failures could change the information available. Predictive probabilities based on a misspecified model can be precisely calculated and still be misleading.
For a simulation-based predictive model, inspect calibration over simulated scenarios: among cases assigned predictive probability near \(r\), final success should occur at a frequency reasonably close to \(r\) under the simulation assumptions. Also check Monte Carlo precision near the decision threshold, because small numerical error can change a continue/stop classification when the predictive probability is close to \(q\). Recalculate the operating curve over a range of effects and plausible design conditions rather than reporting only a single rate.
How This Rule Differs From Alternatives
| Approach | Decision quantity | Key distinction |
|---|---|---|
| Predictive-probability rule | Probability of meeting the final success criterion, conditional on interim data and the model | Uses the planned final design and a predictive distribution for final outcomes |
| Conditional-power rule | Probability of final success conditional on interim data, under a specified assumed treatment effect or test statistic model | Often evaluates a particular effect assumption; the assumption must be explicit |
| Interim significance boundary | Interim test statistic or p-value | Assesses interim evidence against a null reference; it does not directly express the probability of final success |
| Fixed effect boundary | Interim estimate compared with a prespecified effect value | Can be simple to implement but may not reflect remaining sample size or uncertainty unless calibrated for the design |
A predictive-probability rule is attractive when the final criterion and remaining design can be represented credibly. A conditional-power rule can be useful when decision makers want to see results under a specified assumed effect, but it can be sensitive to that assumption. An interim p-value or estimate boundary may be operationally simpler, though simplicity does not make it equivalent to a probability of final success. No approach removes the need to justify the decision threshold and the assumptions behind its calculations.
Common Pitfall and Limitations
- Wrong reasoning: “The predictive probability is 60%, so the treatment has a 60% chance of being effective.” Correction: It is the conditional probability of meeting the specified final success criterion under the stated predictive model and design.
- Wrong reasoning: “The continuation rate at \(\theta=0\) is the predictive probability at the interim.” Correction: The continuation rate averages the decision over interim estimates generated under a fixed true effect; the predictive probability conditions on one observed interim estimate.
- Wrong reasoning: “A threshold of 60% is automatically appropriate.” Correction: The threshold is a policy choice with consequences for both unnecessary continuation and premature stopping. Evaluate those consequences in context.
- Limitation: Normal approximations, fixed variance, and a single treatment-effect parameter may be inadequate for sparse outcomes, complex endpoints, or substantial uncertainty in the model. Simulation or a richer predictive model may be needed, with its assumptions made explicit.
A decision rule should therefore be reported with its success criterion, predictive model, threshold, interim boundary, and operating consequences. The introductory material on probability of success provides the basic decision context; here the key extension is to make the interim conditioning and the future-trial design explicit.
Check Your Understanding
Answer each question using the definitions and assumptions above. Short solutions are included for checking.
- If \(Y_F\mid y\sim N(m_F(y),s_F^2)\) and success means \(Y_F>c\), write the predictive probability. Solution: \(\mathrm{PP}(y)=\Phi((m_F(y)-c)/s_F)\), by standardizing the normal variable.
- For a 60% threshold, why is the interim boundary \(0.2902\) in the worked example? Solution: \(z_{0.60}\approx0.2533\), so \(y_{\mathrm{crit}}=[0.1960+(0.2533)(0.1074)]/0.7692=0.2902\), to four significant figures.
- Under the stated interim sampling model, calculate continuation probability at \(\theta=0\). Solution: \(1-\Phi(0.2902\sqrt{40})=1-\Phi(1.8354)\approx0.03322\).
- Does a predictive probability of 0.75 mean there is a 75% probability the treatment effect is positive? Solution: No. It is the conditional model-based probability that the final-trial success criterion is met, which may differ from the probability that the treatment effect is positive.
- If the predictive-probability threshold rises while the predictive model remains fixed and increases with the interim estimate, what happens to the continuation boundary? Solution: It rises because \(z_q=\Phi^{-1}(q)\) increases with \(q\); consequently, fewer interim estimates meet the continuation rule.