Predictive Probability of Trial Success in Group Sequential Trials

Why Average Over Both Effect Uncertainty and Future Outcomes?

At an interim analysis, a trial that has not stopped may still have a substantial amount of information to collect. The probability of eventual success depends both on what future participants will contribute and on the treatment effect that is still uncertain. Predictive probability combines these two sources of uncertainty: it averages the chance of meeting the success criterion over a posterior distribution for the treatment effect.

This quantity is useful when the decision concerns the trial as it currently stands. A conditional-power calculation asks what would happen if a specified treatment effect were true. A predictive calculation instead uses the current posterior uncertainty about that effect, as well as the randomness in the remaining data. The result is conditional on the information available at the interim analysis and on the plan for continuing the trial.

Definition: The predictive probability of success is the posterior predictive probability that the trial will meet a prespecified success criterion at a future analysis, conditional on the interim data and on the stated continuation plan. If $\theta$ is the treatment effect, $D_1$ is the interim data, and $S$ is the future success event, then $$ \operatorname{PP}(S\mid D_1)=\int P(S\mid D_1,\theta)\,\pi(\theta\mid D_1)\,d\theta. $$ Here $\pi(\theta\mid D_1)$ is the posterior distribution of the treatment effect. The inner probability averages over future trial outcomes at a fixed $\theta$; the outer average accounts for uncertainty about $\theta$.

A predictive probability is not the probability that the null hypothesis is true, nor is it a guarantee about the trial’s outcome. It answers a conditional planning question: given the interim data, the posterior distribution used, the assumed future information, and the success rule, what is the probability of success after averaging over both parameter uncertainty and future sampling variability?

One Remaining Final Analysis

For a transparent calculation, consider a trial that has one final analysis remaining. Use the canonical normal model in which the standardized statistic at information $I$, conditional on treatment effect $\theta$, has mean $\theta\sqrt{I}$ and variance 1. Assume independent normal increments, known variance (or a setting where the canonical normal model is an adequate approximation), and fixed information at the final analysis. Let $I_1$ be the information at the interim analysis, $I_F$ the planned final information, and $Z_1=z_1$ the observed interim statistic. The trial is assumed to continue to $I_F$, and success is determined by the stated final boundary.

The following conditional distribution follows from the independent-increment model. It is given here to make the predictive calculation explicit; the key step for prediction will be averaging its success probability over the posterior distribution of $\theta$.

Z_F\mid Z_1=z_1,\theta\sim N\left(\sqrt{t}\,z_1+\theta\frac{I_F-I_1}{\sqrt{I_F}},\,1-t\right), \qquad t=\frac{I_1}{I_F}.

To see the mean and variance, write the unstandardized accumulated score as the interim score plus the independent future increment. The interim score is $\sqrt{I_1}z_1$; the future increment has mean $\theta(I_F-I_1)$ and variance $I_F-I_1$. Dividing their sum by $\sqrt{I_F}$ gives the displayed conditional mean and variance. In particular, the treatment-effect contribution from the future increment is $\theta(I_F-I_1)/\sqrt{I_F}$, not $\theta\sqrt{I_F-I_1}$.

Result: For a one-sided final success rule $Z_F\ge c$, the fixed-effect success probability at a specified $\theta$ is $$ P(Z_F\ge c\mid Z_1=z_1,\theta) = 1-\Phi\left( \frac{c-\sqrt{t}\,z_1-\theta(I_F-I_1)/\sqrt{I_F}}{\sqrt{1-t}} \right), $$ where $\Phi$ is the standard normal cumulative distribution function. The predictive probability is the posterior average of this expression over $\pi(\theta\mid D_1)$.

The posterior may be continuous or discrete. With a discrete posterior that assigns probability $w_j$ to effect value $\theta_j$, the integral becomes a weighted sum:

\operatorname{PP}(S\mid D_1) = \sum_j w_j P(S\mid D_1,\theta_j), \qquad \sum_j w_j=1.

The weights must represent the posterior distribution after the interim data have been observed. In the examples below, posterior weights are supplied for illustration; they are hypothetical inputs, not claims that a particular prior or data set necessarily produces those weights.

Worked Example: One-Sided Success With Posterior Effect Uncertainty

Suppose a hypothetical trial has $I_1=25$, $I_F=100$, and interim statistic $z_1=1.25$. It will declare success at the final analysis if $Z_F\ge 2.00$. Assume the posterior assigns probability 0.60 to $\theta=0$ and 0.40 to $\theta=0.40$. Calculate the predictive probability.

First, $t=25/100=0.25$, so the conditional standard deviation is $\sqrt{1-t}=\sqrt{0.75}=0.8660$. The conditional mean at effect $\theta$ is $$ \sqrt{0.25}(1.25)+\theta\frac{100-25}{\sqrt{100}} =0.625+7.5\theta. $$ For $\theta=0$, the mean is $0.625$, and the probability of crossing the boundary is $$ P(Z_F\ge2.00\mid z_1,\theta=0) = 1-\Phi\left(\frac{2.00-0.625}{0.8660}\right) = 1-\Phi(1.588) \approx 0.0561. $$ For $\theta=0.40$, the mean is $0.625+0.75(0.40)=0.925$ only if the future effect term were $0.75\theta$; here the information ratio must be checked: $(I_F-I_1)/\sqrt{I_F}=75/10=7.5$. Thus the correct mean is $0.625+7.5(0.40)=3.625$. The corresponding success probability is $$ 1-\Phi\left(\frac{2.00-3.625}{0.8660}\right) = 1-\Phi(-1.876) \approx 0.9697. $$

The predictive probability is the posterior-weighted average: $$ 0.60(0.0561)+0.40(0.9697)=0.4215. $$ Thus, under the stipulated posterior and continuation plan, the hypothetical trial has about a 42.15% posterior predictive probability of crossing the final one-sided boundary. This is appreciably different from either fixed-effect probability because the posterior places weight on two different effect values.

Two-Sided Success and a Second Calculation

For a two-sided success rule, success means crossing either boundary: $Z_F\ge c$ or $Z_F\le -c$. These events are disjoint when $c>0$, so their probabilities add. At each fixed $\theta$, the relevant probability is

P(|Z_F|\ge c\mid z_1,\theta) = \Phi\left(\frac{-c-m(\theta)}{s_F}\right) + 1-\Phi\left(\frac{c-m(\theta)}{s_F}\right),

where $m(\theta)=\sqrt{t}\,z_1+\theta(I_F-I_1)/\sqrt{I_F}$ and $s_F=\sqrt{1-t}$. The predictive probability again averages this two-tail probability over the posterior. The two-sided rule should not be replaced by a one-sided calculation unless the protocol’s success criterion is in fact one-sided.

Worked Example: Two-Sided Final Boundary

Suppose a hypothetical trial has $I_1=25$, $I_F=100$, $z_1=-0.50$, and final success rule $|Z_F|\ge2.00$. For illustration, let the posterior assign probability 0.90 to $\theta=0$ and 0.10 to $\theta=0.40$. The conditional standard deviation is again $0.8660$, and the conditional mean is $$ \sqrt{0.25}(-0.50)+\theta\frac{75}{10}=-0.2500+7.5\theta. $$

At $\theta=0$, the mean is $-0.2500$. The lower-tail probability is $ \Phi((-2.00+0.2500)/0.8660)=\Phi(-2.021)\approx0.02164$; the upper-tail probability is $1-\Phi((2.00+0.2500)/0.8660)=1-\Phi(2.598)\approx0.0047$. The two-sided success probability is therefore approximately $0.02634$.

At $\theta=0.40$, the mean is $-0.2500+7.5(0.40)=2.7500$. The lower-tail probability is essentially zero, since $\Phi((-2.00-2.7500)/0.8660)=\Phi(-5.485)$, while the upper-tail probability is $1-\Phi((2.00-2.7500)/0.8660)=1-\Phi(-0.8660)\approx0.8068$. The posterior predictive probability is $$ 0.90(0.02634)+0.10(0.8068)=0.1044. $$

The large positive-effect component contributes substantial upper-boundary success probability, but it receives only 10% posterior weight. The final predictive probability is about 10.44%, not the probability calculated under either effect value alone.

What the Calculation Assumes, and How to Check It

A numerical predictive probability is meaningful only relative to its model and continuation plan. For the calculations above, the essential assumptions are:

The interim and future score increments follow the stated independent-increment normal model, with the specified information values.
The interim statistic and treatment-effect scale use the same canonical model, $E(Z(I)\mid\theta)=\theta\sqrt I$.
The posterior distribution is based on the interim data and the chosen prior, and its weights or density are properly normalized.
The future information and success boundary are specified, and the trial continues according to the plan used in the calculation.
Any nuisance parameters that materially affect the distribution are known, adequately estimated, or otherwise accounted for. Treating them as fixed when they are uncertain can make the predictive probability too concentrated.

Practical checks include confirming $I_1<I_F$, checking that the conditional variance $1-I_1/I_F$ is positive, and verifying that posterior weights sum to one. Also check the scale of the effect contribution: it is $\theta(I_F-I_1)/\sqrt{I_F}$. A calculation using $\theta\sqrt{I_F-I_1}$ is generally inconsistent with the canonical model. Finally, compute the fixed-effect success probabilities at representative posterior values before averaging; this often exposes a sign, boundary, or standard-error error.

The predictive probability is conditional on the assumed model. It can be misleading if the posterior assigns implausible weight to important treatment effects, if the information schedule changes, or if the trial’s actual stopping and adaptation plan differs from the one used. It can also be sensitive to the prior, especially with limited interim information. A single reported number should therefore be accompanied by the success definition, information assumptions, posterior specification, and any important modeling qualifications.

How It Differs From Other Interim Quantities

Quantity	What is averaged or fixed?	Question answered
Conditional power at a specified effect	Treatment effect is fixed at a chosen value; future outcomes vary.	If this effect were true, what is the chance of success?
Plug-in conditional power	Treatment effect is set to an interim estimate; future outcomes vary.	What chance follows if the interim estimate is treated as the effect?
Predictive probability	Future outcomes vary, and success probabilities are averaged over the posterior effect distribution.	Given the interim data and posterior, what is the chance of success?

These are different summaries, not competing estimates of one fixed probability. Predictive probability incorporates the posterior as specified, but it does not remove dependence on modeling choices. Nor does it automatically describe the probability of success under every possible future adaptation: it describes the particular future plan used to define the event.

Common Mistakes and Limitations

Using a single effect value and calling the result predictive. That gives a fixed-effect conditional probability. Predictive probability requires averaging over the posterior distribution.
Adding a parameter-uncertainty adjustment to the variance without justification. The definition is an average of conditional success probabilities. Its calculation is not generally obtained by simply inflating the conditional standard deviation.
Using the wrong future-effect term. Under the canonical model the term is $\theta(I_F-I_1)/\sqrt{I_F}$. Check it against the assumed mean $E(Z(I)\mid\theta)=\theta\sqrt I$.
Ignoring the actual success rule. A two-sided criterion requires both tails; a one-sided boundary uses only the prespecified direction.
Interpreting the result as unconditional or as a probability that the treatment works. It is conditional on the observed interim information, posterior, model, and continuation plan.

The single-final-look calculation does not by itself handle success at any of several future analyses. With multiple remaining looks, the success event involves the joint distribution of the future statistics and the sequential boundaries; calculating it requires a multivariate or sequential probability calculation rather than the one-dimensional final-statistic formula used here.

Key takeaway: Predictive probability is the posterior average of future success probabilities. It includes both uncertainty in future outcomes and uncertainty in the treatment effect, and its interpretation is tied to the stated posterior, model, success rule, and continuation plan.

Check Your Understanding

Answer each question, then compare with the brief solutions.

In one sentence, distinguish predictive probability from conditional power at a fixed treatment effect.
With $I_1=36$ and $I_F=100$, what is the conditional standard deviation of $Z_F$ given the interim statistic and $\theta$, under the canonical model?
Suppose a posterior puts probability 0.25 on an effect whose conditional success probability is 0.20 and probability 0.75 on an effect whose conditional success probability is 0.80. What is the predictive probability?
Why does a two-sided final success rule require two tail probabilities?
State two assumptions or qualifications that should accompany a reported predictive probability.

Solutions. (1) Conditional power fixes the treatment effect at a stated value, whereas predictive probability averages conditional success probabilities over the posterior distribution of the effect. (2) $t=36/100=0.36$, so the standard deviation is $\sqrt{1-0.36}=\sqrt{0.64}=0.8000$. (3) $0.25(0.20)+0.75(0.80)=0.05+0.60=0.6500$, or 65.00%. (4) Success can occur by crossing either the lower or upper boundary, and the disjoint tail probabilities must both be included. (5) Examples include the independent-increment normal model, the posterior and prior used, the planned final information, the success boundary, continuation assumptions, and how nuisance-parameter uncertainty is treated.