Comparing Interim Scenarios with Conditional and Predictive Power

Why Compare Interim Scenarios?

An interim result does not determine the chance of eventual success by itself. The calculation also depends on what effect is assumed to continue after the interim, and on whether uncertainty about that effect is included. Comparing scenarios makes those choices visible: the same final success rule can produce different probabilities because the interim evidence, assumed effect, or prior distribution differs.

This tutorial uses a one-sided final success rule \(Z_F \ge c\), where \(Z_F\) is the standardized statistic at the final analysis. For the numerical examples, take \(c=1.645\). This is a convenient common boundary for illustration; in an actual group sequential design, the final boundary must be the one specified by the design. It should not be replaced by 1.645 merely because that value is familiar.

Conditional-Normal Model and the Three Calculations

Assume a normally distributed test statistic with known variance, a fixed planned final information level, and canonical independent increments. Let \(t\) be the fraction of planned information available at the interim analysis, \(Z_t=z\) the observed interim statistic, and \(\theta\) the canonical treatment effect. Under this model, \(Z_t\mid\theta\sim N(\sqrt{t}\theta,1)\). The future increment is independent of the interim increment, and its standardized version has distribution \(N(\sqrt{1-t}\theta,1)\).

Combining the observed and future increments gives the conditional distribution of the final statistic:

The first term in the conditional mean is the contribution already observed; the second is the expected contribution from the remaining information under the specified effect. The variance \(1-t\) represents the uncertainty in that future increment. Consequently, conditional power at assumed effect \(\theta\) is

The plug-in calculation substitutes the interim maximum-likelihood estimate \(\hat\theta=z/\sqrt{t}\) for \(\theta\). The design-effect calculation instead substitutes a prespecified effect, commonly the effect used in the design. Neither calculation averages over uncertainty about \(\theta\); they answer different conditional questions because they use different effect assumptions.

For a posterior predictive calculation, specify a prior distribution and update it using the interim data. For example, with prior \(\theta\sim N(m_0,v_0)\), the normal likelihood above gives

Averaging the conditional final distribution over this posterior gives a normal predictive distribution. Its mean adds the future contribution under the posterior mean; its variance includes both future sampling variation and uncertainty about the effect:

Thus posterior predictive probability is the upper-tail probability of this distribution above \(c\). It is not conditional power evaluated at one “best” effect: posterior uncertainty adds the term \((1-t)^2V_t\) to the predictive variance.

Worked Comparisons

Justification, Diagnostics, and Limitations

The conditional-power formula follows directly from independent increments. Write the final statistic as the contribution already observed plus the independent future contribution:

Conditioning on \(Z_t=z\) fixes the first term. The second term then has mean \((1-t)\theta\) and variance \(1-t\). Adding the fixed contribution gives the stated conditional normal distribution, and standardizing \(Z_F-c\) gives its upper-tail probability. This also explains why the future standard deviation is \(\sqrt{1-t}\), not 1.

Several checks help identify errors. As \(t\) approaches 1, little information remains, so the conditional variance should approach zero; at \(t=1\), the final statistic is already observed and the remaining-success probability is an indicator, not a formula with a zero denominator. For fixed \(t\), \(z\), and \(c\), conditional power must increase as the assumed effect increases. Predictive variance should be at least \(1-t\), since parameter uncertainty adds a nonnegative term. Numerical calculations should use the same final boundary and information fraction across methods when the goal is a direct comparison.

The main alternatives answer distinct questions. Plug-in conditional power gives a single-effect forecast based on the interim estimate, which can be unstable when the interim estimate is noisy. Design-effect conditional power uses a prespecified effect and is interpretable as a scenario calculation, but can be optimistic or pessimistic if that effect is implausible. Posterior predictive probability incorporates effect uncertainty, but depends on the prior as well as the likelihood. These are not interchangeable estimates of one quantity; they are complementary summaries conditional on different assumptions.

A common mistake is to call a low conditional power “evidence that the null hypothesis is true,” or to treat a higher predictive probability as proof that success is likely in any objective sense. Neither follows. Each number is a probability of a future event under a specified statistical model and conditioning information. A related computational pitfall is using an unadjusted fixed boundary when the group sequential design specifies a different final boundary. Even perfectly calculated power is misleading if \(c\) does not match the planned success rule.

The canonical normal calculation is an approximation or working model in many applications. It relies on correctly specified information, a stable effect scale, and the assumed distribution of the test statistic. Nuisance-parameter uncertainty, non-normal outcomes, changing estimands, or changes to the planned information can require additional modeling. In particular, results computed for a fixed final information level should not automatically be carried over to a changed sample size or analysis plan.