Predictive Probability Across Multiple Future Looks

Why Multiple Future Looks Require a Joint Probability

After an interim analysis, a trial may have several planned opportunities to stop for efficacy. The quantity of interest here is the posterior predictive probability that the trial will cross an efficacy boundary at at least one of those future looks. It is not the sum of separate look-specific probabilities: future test statistics are correlated, and adding their marginal crossing probabilities generally counts some paths more than once.

The calculation has two sources of uncertainty. Even if the treatment effect were fixed, future data would be random. In addition, the treatment effect itself is uncertain given the interim data. The posterior predictive probability averages the future crossing probability over that posterior uncertainty, using the joint distribution of all remaining statistics.

Definition: Let $Z_1,\ldots,Z_K$ be the standardized test statistics at the $K$ future analyses, and let $c_1,\ldots,c_K$ be their one-sided efficacy boundaries. Given the interim data $D_0$, the posterior predictive probability of efficacy crossing is $$ P\!\left(\bigcup_{j=1}^{K}\{Z_j\ge c_j\}\mid D_0\right). $$ This is the posterior probability, conditional on the interim data, of a future data path that crosses at least one specified boundary.

Use the usual canonical normal group-sequential model. Let $I_0$ be the information at the interim analysis, $z_0$ the observed interim statistic, and $I_1<\cdots<I_K$ the future information times, all greater than $I_0$. Let $\theta$ denote the treatment effect on the scale for which the mean score at information $I$ is $\theta I$. The model assumes independent normal score increments with variance equal to the corresponding increment in information. Equivalently, the cumulative score at information $I$ has mean $\theta I$ and variance $I$. The future information times and efficacy boundaries are treated as fixed, and posterior uncertainty about $\theta$ is represented by $\pi(\theta\mid D_0)$.

These assumptions give the canonical normal model; they are not automatic for every endpoint or design. They require that the score-scale normal approximation is adequate, that information is correctly specified, and that future increments are independent of the observed interim data conditional on $\theta$. If those conditions are poor, the resulting predictive probability may not describe the actual trial path well.

Deriving the Joint Predictive Distribution

Conditional on the observed interim statistic and a fixed value of $\theta$, the future cumulative score at information $I_j$ consists of the interim score plus an independent future increment. Its mean is $z_0\sqrt{I_0}+\theta(I_j-I_0)$, and its future-increment variance is $I_j-I_0$. Dividing by $\sqrt{I_j}$ gives the conditional mean and covariance of the standardized statistics.

E(Z_j\mid z_0,\theta) =\sqrt{\frac{I_0}{I_j}}z_0 +\theta\frac{I_j-I_0}{\sqrt{I_j}}.

For $j\le k$, the future score increments through $I_j$ are shared by the statistics at $I_j$ and $I_k$. Their covariance before standardization is $I_j-I_0$. Standardizing each statistic therefore gives the conditional covariance below. In particular, setting $j=k$ gives the conditional variance $(I_j-I_0)/I_j$, as required.

\operatorname{Cov}(Z_j,Z_k\mid z_0,\theta) =\frac{I_j-I_0}{\sqrt{I_jI_k}},\qquad j\le k.

Now suppose the posterior for the treatment effect has mean $m$ and variance $v$. Define $a_j=(I_j-I_0)/\sqrt{I_j}$. Averaging the conditional mean over the posterior replaces $\theta$ by $m$. The predictive covariance has two parts: the covariance of future score increments and the additional covariance induced by uncertainty about the shared treatment effect.

E(Z_j\mid D_0)=\sqrt{\frac{I_0}{I_j}}z_0+ma_j, \qquad \operatorname{Cov}(Z_j,Z_k\mid D_0) =\frac{I_j-I_0}{\sqrt{I_jI_k}}+v a_ja_k,\quad j\le k.

Thus, under a normal posterior for $\theta$, the vector of future statistics has a multivariate normal posterior predictive distribution. More generally, for any posterior distribution, the predictive crossing probability is the integral of the conditional crossing probability over that posterior:

P\!\left(\bigcup_{j=1}^{K}\{Z_j\ge c_j\}\mid D_0\right) =\int P\!\left(\bigcup_{j=1}^{K}\{Z_j\ge c_j\}\mid z_0,\theta\right) \pi(\theta\mid D_0)\,d\theta.

For a normal posterior, the complement of the crossing event is the joint event $Z_j<c_j$ for every future look. Consequently, the desired probability is one minus a multivariate normal probability over that joint region. This complement formulation makes clear why the covariance among looks must be included.

Worked Example: One Future Look With Posterior Uncertainty

Suppose $I_0=1$, $z_0=0.9$, and there is one future look at $I_1=2$, with efficacy boundary $c_1=2$. Assume the interim posterior is $\theta\mid D_0\sim N(0,0.1)$. This is a hypothetical normal-model calculation with fixed future information and a posterior distribution supplied for the example.

The predictive mean is $\sqrt{1/2}(0.9)+0(1/\sqrt{2})=0.6364$. The conditional future-increment variance is $(2-1)/2=0.5$, and posterior uncertainty adds $0.1(1/\sqrt{2})^2=0.05$. Thus the predictive variance is $0.55$, and the standard deviation is $\sqrt{0.55}=0.7416$.

The standardized boundary is $$ \frac{2-0.6364}{0.7416}\approx 1.839. $$ Therefore the posterior predictive probability of crossing is $1-\Phi(1.839)\approx 0.03298$, or about $3.30\%$. In this hypothetical trial, given the interim data and posterior model, the chance of crossing the single future efficacy boundary is about $3.30\%$.

Worked Example: Crossing at Either of Two Future Looks

Consider $I_0=1$, $z_0=1.1213$, and future information times $I_1=2$ and $I_2=4$. Let both efficacy boundaries be 1, and suppose $\theta\mid D_0\sim N(0.2929,0.1)$. These hypothetical values are chosen so that each future predictive mean is 1.

For the first look, $a_1=1/\sqrt{2}=0.7071$, so its predictive mean is $1.1213/\sqrt{2}+0.2929(0.7071)\approx1.000$. For the second look, $a_2=3/2=1.5$, and its mean is $1.1213/2+0.2929(1.5)\approx1.000$.

The predictive variances and covariance are $$ \begin{aligned} \operatorname{Var}(Z_1\mid D_0)&=\frac{1}{2}+0.1(0.7071)^2=0.5500,\\ \operatorname{Var}(Z_2\mid D_0)&=\frac{3}{4}+0.1(1.5)^2=0.9750,\\ \operatorname{Cov}(Z_1,Z_2\mid D_0)&=\frac{1}{\sqrt{8}}+0.1(0.7071)(1.5)=0.4596. \end{aligned} $$ The predictive correlation is $0.4596/\sqrt{0.5500(0.9750)}\approx0.6276$. Since each mean equals its boundary, the two standardized boundary cutoffs are zero. For a centered bivariate normal pair with correlation $\rho$, the probability both coordinates are below zero is $1/4+\arcsin(\rho)/(2\pi)$. Therefore, $$ P(Z_1\ge1\text{ or }Z_2\ge1\mid D_0) =1-\left\{\frac14+\frac{\arcsin(0.6276)}{2\pi}\right\} \approx0.642. $$ The chance of crossing at least one of the two boundaries is about $64.2\%$. This calculation includes both the dependence from shared future score increments and the additional dependence induced by uncertainty about $\theta$.

Mixture Posteriors and Calculation Checks

If the posterior for $\theta$ is a mixture rather than a single normal distribution, calculate a crossing probability under each posterior component and average those probabilities using the posterior component weights. For multiple looks, each component has its own joint predictive distribution. Averaging component means and variances first and then treating the result as one normal distribution is generally not equivalent: a mixture of normal distributions need not itself be normal.

Worked Example: Averaging Across Two Posterior Components

Suppose there is one future look with $I_0=1$, $z_0=0$, $I_1=2$, and boundary $c_1=1.5$. Assume the posterior assigns weight 0.7 to $N(0,0.04)$ and weight 0.3 to $N(0.8,0.04)$. The future score-increment variance is $1/2$, and each component contributes posterior variance $0.04(1/\sqrt{2})^2=0.02$. Both components therefore have predictive variance $0.52$, with standard deviation $0.7211$.

Under the first component the predictive mean is 0, giving standardized boundary $1.5/0.7211\approx2.080$ and crossing probability about 0.0188. Under the second component the predictive mean is $0.8/\sqrt{2}=0.5657$, giving standardized boundary $(1.5-0.5657)/0.7211\approx1.296$ and crossing probability about 0.0975. The mixture probability is $$ 0.7(0.0188)+0.3(0.0975)\approx0.0424. $$ Thus the posterior predictive crossing probability is about $4.24\%$. The weighting is by posterior component probabilities, not by the prior weights unless those weights remain unchanged after observing the interim data.

A useful diagnostic is to check the predictive covariance matrix before calculating the boundary probability. Its diagonal entries must be positive, its entries must be symmetric, and it must be positive semidefinite. Under the stated model, each diagonal includes the future-increment variance and the nonnegative posterior-uncertainty contribution. Also check that all future information times exceed $I_0$, and that increasing posterior variance does not accidentally reduce a displayed covariance contribution $v a_ja_k$ when the $a_j$ are positive.

For computation, the complement event is often convenient: evaluate the joint probability that every future statistic lies below its boundary, then subtract from 1. For a non-normal posterior, the defining posterior integral remains the target; it can be evaluated by suitable numerical methods. The important requirement is that the calculation respect the joint distribution at all remaining looks, not replace it with unrelated marginal probabilities.

Interpretation, Alternatives, and Limitations

This probability is conditional on the interim data, the posterior model, the assumed future information schedule, and the specified boundaries. It describes uncertainty about future boundary crossing after integrating over uncertainty in the treatment effect. It is not the probability that $H_0$ is true, nor is it the long-run type I error rate of the trial. The design's frequentist error properties are defined over repeated sampling under the hypotheses; a posterior predictive probability answers a different, model-based question.

A plug-in conditional-power calculation evaluates future success at a selected effect value, often a point estimate or a prespecified design effect. The posterior predictive approach instead averages across the posterior distribution of effects. The former can be simpler to interpret as a scenario calculation; the latter reflects parameter uncertainty but depends on the prior and likelihood model. Neither choice removes the need to use the correct dependence among future looks.

The method has limitations. It relies on the canonical normal approximation and on the posterior being appropriate for the interim data and analysis plan. If information is estimated with substantial uncertainty, the future statistic model may require additional treatment of that uncertainty. If the planned boundaries, information times, or analysis rules change, the event being calculated must be updated accordingly. A misspecified posterior or an inaccurate normal score model can produce a precise-looking probability that is nevertheless poorly calibrated to the actual trial.

Common Mistakes

Adding marginal crossing probabilities. This double-counts paths that cross at more than one look. Use the joint probability of the union, or one minus the joint probability of staying below every boundary.
Leaving out dependence. Future statistics share score increments, and posterior uncertainty contributes further covariance. Use the covariance matrix derived for the joint predictive distribution.
Using the wrong conditional covariance. Given the interim statistic and fixed $\theta$, the covariance for $j\le k$ is $(I_j-I_0)/\sqrt{I_jI_k}$. It is not this quantity plus an interim-data covariance term.
Calling predictive probability a type I error rate. It is conditional on interim data and a posterior model; it is not the repeated-sampling probability of a false positive under $H_0$.
Collapsing a mixture to a normal without justification. Average component-specific event probabilities using posterior weights unless the resulting distribution is known to have the form assumed by the calculation.

Key takeaway: Calculate the posterior predictive probability of crossing by forming the joint future-statistic distribution conditional on the interim data, incorporating posterior uncertainty about $\theta$, and evaluating the probability of crossing at least one boundary. Dependence across looks is essential.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

For future looks $j\le k$, what is the conditional covariance given $z_0$ and fixed $\theta$, and what additional term appears in the predictive covariance under a normal posterior with variance $v$?
Why is the probability of crossing at least one of two future boundaries not generally the sum of the two marginal crossing probabilities?
Suppose the posterior for $\theta$ has two components. How should component-specific crossing probabilities be combined?
Does a posterior predictive crossing probability equal the trial's type I error rate? Explain briefly.

Solutions. (1) The conditional covariance is $(I_j-I_0)/\sqrt{I_jI_k}$; the posterior contribution is $v a_ja_k$, where $a_j=(I_j-I_0)/\sqrt{I_j}$. (2) The future statistics are dependent, so paths crossing both boundaries occur in both marginal events; use the joint union probability or its complement. (3) Weight the component-specific probabilities by their posterior component probabilities and sum. (4) No. It is conditional on interim data and a posterior model, whereas type I error is a repeated-sampling error probability under $H_0$.