Predictive Probability of Final Trial Success

What Predictive Probability Answers

An interim posterior probability describes uncertainty about treatment effect given the data observed so far. Predictive probability of final success asks a different question: given those interim data, how likely is it that the planned final data will lead to a specified posterior efficacy decision? It averages over both uncertainty about the treatment effect and the yet-unobserved data.

The final decision rule must be specified before calculating this probability. For example, an arm might be declared efficacious if its final posterior probability of exceeding a clinically meaningful effect $\delta$ is at least $q$. Predictive probability then concerns whether that rule will be met after the remaining planned observations are collected. It is not the current posterior probability that the arm is efficacious, and it is not a probability that the null hypothesis is true.

Definition: Let $D_I$ denote the interim data, $D_A$ the data planned after interim, and $S(D_I,D_A)$ the event that the specified final posterior efficacy criterion is met. The predictive probability of final success is $$ \operatorname{PP}_{\mathrm{success}}=P\{S(D_I,D_A)\mid D_I\}, $$ where the probability averages over the posterior uncertainty about the model parameters and the sampling distribution of the future data under the planned design.

The calculation presumes a specified likelihood, prior, interim dataset, remaining sample size, and final decision rule. The planned future observations must be conditionally independent of the interim observations given the model parameters, or their dependence must be represented in the joint model. The probability is conditional on the design and model as specified; changing the prior, endpoint model, remaining sample size, or final threshold can change it.

Normal-Outcome Derivation

Consider one treatment contrast $\theta$, such as a treatment-minus-control difference in mean response, measured in the outcome's units. Suppose the interim posterior is normal, $\theta\mid D_I\sim N(m_I,v_I)$, and the remaining data can be summarized by an independent estimate $Y$ with $Y\mid\theta\sim N(\theta,s_A^2)$. Here $s_A^2$ is the sampling variance of the estimate from the additional planned data. This formulation assumes a normal likelihood with known sampling variance and a normal interim posterior; the latter can arise from a normal prior and normal likelihood.

After observing $Y$, conjugacy gives a final posterior variance $v_F$ and mean $M_F$. Since the interim posterior acts as the prior distribution for the remaining data, the final posterior quantities are

v_F=\left(\frac{1}{v_I}+\frac{1}{s_A^2}\right)^{-1}, \qquad M_F=v_F\left(\frac{m_I}{v_I}+\frac{Y}{s_A^2}\right).

Suppose final success requires $P(\theta>\delta\mid D_I,Y)\ge q$, with $0<q<1$. Under the final normal posterior this probability is $\Phi((M_F-\delta)/\sqrt{v_F})$. Since $\Phi$ is increasing, the success criterion is equivalent to a threshold on the final posterior mean:

M_F\ge \delta+z_q\sqrt{v_F}, \qquad z_q=\Phi^{-1}(q).

To find the chance of crossing this threshold, calculate the distribution of $M_F$ before observing $Y$. Integrating the sampling model over the interim posterior gives $Y\mid D_I\sim N(m_I,v_I+s_A^2)$. Substitution into the expression for $M_F$ shows that its predictive mean is $m_I$, and its predictive variance is $v_I-v_F$. Therefore,

M_F\mid D_I\sim N(m_I,v_I-v_F), \qquad \operatorname{PP}_{\mathrm{success}} =1-\Phi\left( \frac{\delta+z_q\sqrt{v_F}-m_I}{\sqrt{v_I-v_F}} \right).

This derivation also explains the two sources of uncertainty. The final posterior variance $v_F$ determines how much evidence is required to meet the posterior criterion. The predictive variance $v_I-v_F$ describes how much the final posterior mean can move as the remaining data arrive. If there are no additional observations, $v_F=v_I$, so there is no future-data variation: the probability is 1 if the current posterior already meets the criterion and 0 otherwise.

Worked Example: Normal Outcome With a Planned Final Sample

Suppose a hypothetical treatment contrast is measured in units of symptom improvement. At interim, the posterior is $\theta\mid D_I\sim N(1.2,1.0)$, so $m_I=1.2$ units and $v_I=1.0$ square units. The remaining planned data provide an independent estimate with sampling variance $s_A^2=0.75$ square units. Final success requires posterior probability at least $0.95$ that the improvement exceeds $\delta=0.2$ units. The normal model, known variances, and independent future-data assumption are taken as appropriate for this hypothetical example.

First calculate the final posterior variance: $v_F=(1/1.0+1/0.75)^{-1}=0.4286$ square units, with $\sqrt{v_F}=0.6547$ units. The final posterior mean must satisfy $M_F\ge0.2+1.6449(0.6547)=1.2768$ units.

The predictive standard deviation of $M_F$ is $\sqrt{v_I-v_F}=\sqrt{1.0-0.4286}=0.7559$ units. Thus $$ \operatorname{PP}_{\mathrm{success}} =1-\Phi\left(\frac{1.2768-1.2}{0.7559}\right) =1-\Phi(0.1016) \approx 0.460. $$

Conditional on the interim data and these model and design assumptions, the chance that the final posterior probability of exceeding 0.2 units will be at least 0.95 is about 46.0%. This is not a 46.0% posterior probability that the treatment truly exceeds 0.2 units; that is a different probability about $\theta$ given the data.

Binary Outcomes and Exact Predictive Calculation

For a binary endpoint, a conjugate beta-binomial model provides a direct finite-sample calculation. Let $p$ be the event probability for one arm, with interim posterior $p\mid D_I\sim\operatorname{Beta}(a,b)$. If $n_A$ additional participants are planned and $R$ is their number of events, then $R\mid D_I$ has a beta-binomial predictive distribution. After observing $R=r$, the final posterior is $\operatorname{Beta}(a+r,b+n_A-r)$.

For a rule requiring $P(p>p_0\mid D_I,R)\ge q$, first identify which possible counts $r$ meet the final rule. Then sum their interim-conditional predictive probabilities. The predictive mass for $r$ is

P(R=r\mid D_I) ={n_A\choose r} \frac{B(a+r,b+n_A-r)}{B(a,b)}, \qquad r=0,\ldots,n_A,

where $B$ is the beta function. The rule and calculation concern an event probability $p$; if efficacy is defined by a difference between treatment and control probabilities, the posterior event must instead be defined for that contrast and its joint posterior distribution.

Worked Example: Beta-Binomial Predictive Probability

For a hypothetical binary outcome, suppose the interim posterior for an arm's event probability is $\operatorname{Beta}(3,2)$. Two further participants are planned. Final success means that the posterior probability $P(p>0.5\mid\text{all data})$ is at least $0.80$. Assume the beta-binomial model is appropriate and the two future outcomes are conditionally independent given $p$.

The predictive probabilities for $R=0,1,2$ events are, respectively, ${2\choose r}B(3+r,2+2-r)/B(3,2)$. Using $B(3,2)=1/12$, these evaluate to $0.200$, $0.400$, and $0.400$; they sum to 1.

The final posteriors for the three possible counts are $\operatorname{Beta}(3,4)$, $\operatorname{Beta}(4,3)$, and $\operatorname{Beta}(5,2)$. Their probabilities of $p>0.5$ are $0.34375$, $0.65625$, and $0.890625$, respectively. For example, for integer beta parameters, the beta probability at 0.5 can be evaluated from a binomial tail; for $\operatorname{Beta}(5,2)$, the probability above 0.5 is $57/64=0.890625$.

Only $R=2$ meets the 0.80 final threshold. Therefore, $$ \operatorname{PP}_{\mathrm{success}} =P(R=2\mid D_I)=0.400. $$

Under this hypothetical model and two-person continuation, the conditional probability of meeting the specified final posterior criterion is 40.0%. The calculation uses the predictive distribution of future outcomes, not just the current posterior probability $P(p>0.5\mid D_I)$.

Multiple Arms, Checks, and Interpretation

In a multi-arm trial, calculate the event that matches the actual final rule. If success means that at least one of several arms meets its own posterior criterion, the event is a union of arm-specific success events. If success requires every specified arm to meet its criterion, it is an intersection. These are not interchangeable: the probability of at least one success generally differs from the probability of joint success. With independent arm-specific posterior predictive distributions, a union probability can be calculated from the complements; with shared controls, hierarchical priors, or other dependence, the joint predictive distribution must retain that dependence.

A useful calculation sequence is:

Write the final rule.
State the clinically meaningful threshold, posterior probability cutoff, and whether the event concerns one arm, at least one arm, or all specified arms.

Specify the interim posterior and future-data model.
Check that the likelihood, prior, planned remaining sample, and any dependence between arms are represented.

Evaluate the predictive event.
Use the analytic normal formula when its assumptions apply, enumerate possible future counts in a conjugate discrete model, or calculate the corresponding predictive event under the specified joint model.

Check and report the result.
Verify that component probabilities are in $[0,1]$, that discrete predictive probabilities sum to 1, and that the reported probability is explicitly conditional on interim data and model assumptions.

Several diagnostics help reveal errors. In a normal calculation, verify $v_F\le v_I$, so $v_I-v_F$ is nonnegative, and confirm that all terms in a threshold have compatible units. In a discrete calculation, check that every possible future count is included and that its predictive mass sums to 1. As a direction check, increasing the remaining sample size often makes the eventual decision more predictable, but it does not necessarily increase the probability of success: added data can clarify that an arm is ineffective as well as support efficacy. Also check that the final rule is applied to the final posterior, not mistakenly to the interim posterior.

A common mistake is to report the current posterior probability of efficacy as the predictive probability of final success. For instance, $P(\theta>\delta\mid D_I)$ concerns the unknown effect given interim data; it does not account for how the planned future observations may change the final posterior or whether that posterior will cross its decision threshold. The correction is to define the final decision event first, then average its indicator over the predictive distribution of the future data conditional on $D_I$.

Predictive probability also differs from a prior-predictive assurance calculation, which averages over uncertainty before observing the interim data. Here the conditioning is specifically on the observed interim data. Nor does a high predictive probability guarantee that the model is well calibrated or that a design has desirable long-run operating characteristics. It can be sensitive to the prior, likelihood, assumed remaining sample, and decision threshold; those choices should be checked and reported. For a more complicated model, the same target remains the conditional probability of the final event, even when the calculation requires numerical integration or simulation.

Key takeaway: Predictive probability of final success is the interim-conditional probability that future planned data will make the prespecified final posterior rule true. Derive the final rule, model future data conditional on current information, and evaluate that event without confusing it with current posterior efficacy.

Check Your Understanding

Answer each question, then compare with the solutions below.

In the normal model, what is the distinction between $v_F$ and $v_I-v_F$?
If the final rule is $P(\theta>\delta\mid\text{final data})\ge q$, write its equivalent threshold for $M_F$.
In the beta-binomial example, what predictive probability would result if only $R=1$ and $R=2$ met the final rule?
Why can a larger remaining sample make the final decision more predictable without necessarily increasing predictive probability of success?

Solutions. (1) $v_F$ is the final posterior variance of the effect after future data are observed; $v_I-v_F$ is the predictive variance of the final posterior mean before those data are observed. (2) $M_F\ge\delta+z_q\sqrt{v_F}$, where $z_q=\Phi^{-1}(q)$. (3) Add the predictive masses: $0.400+0.400=0.800$. (4) More data can make the posterior decision clearer in either direction, supporting efficacy when the effect is favorable and failing the efficacy rule when it is not.

Predictive Probability of Final Trial Success

What You'll Learn