Interpreting Conditional Power Under Null and Alternative Effects

What Conditional Power Is Conditional On

Conditional power is a probability of meeting a trial’s success criterion in the future, given the data observed so far and a specified assumption about the treatment effect. It is not a single, assumption-free property of an interim result. In particular, the probability can differ substantially depending on whether the assumed effect is zero, the effect used to design the trial, or an effect estimated from interim data.

This distinction matters when an interim report describes conditional power as “low.” The number answers a conditional, forward-looking question under its stated assumptions. It does not, by itself, establish that the treatment has no benefit, that the null hypothesis is true, or that continuing the trial is pointless.

For a precise calculation, suppose there is one interim analysis and one final analysis, with a two-sided final success criterion \(|Z_2|\ge c\). Here \(Z_1=z_1\) is the observed interim standardized statistic, and \(Z_2\) is the final standardized statistic. Let \(I_1\) and \(I_2\) be the interim and final information, with \(0<I_1<I_2\), and let \(t=I_1/I_2\) be the information fraction. Write \(\Delta\) for the assumed final noncentrality parameter: under the assumed effect, the unconditional final statistic has mean \(\Delta\).

The calculation below uses the canonical normal model: information is correctly specified; the statistic has independent normal increments on the information scale; the variance is treated as known (or the normal approximation is adequate); and the interim statistic and final statistic use a consistent direction and analysis. The final boundary \(c\) is fixed for this illustration. For a single final look using a two-sided level 0.05 normal test, \(c=1.96\); an actual group sequential design may use a different boundary.

Conditional Distribution and Two-Sided Calculation

Under the canonical model, the future increment from \(I_1\) to \(I_2\) is independent of the observed interim increment. Conditional on \(Z_1=z_1\), the final statistic has distribution

The second argument of \(N(\cdot,\cdot)\) is the variance. Thus the conditional standard deviation is \(\sqrt{1-t}\). The conditional mean combines the contribution carried forward from the observed interim statistic, \(\sqrt{t}\,z_1\), and the assumed drift in the information still to be collected, \((1-t)\Delta\).

Both tails appear because success is defined in either direction. If the trial has a prespecified one-sided success criterion, use that criterion’s tail instead; do not report the two-sided expression as though it were one-sided power.

Three Effect Assumptions, Three Questions

The formula makes clear why the effect assumption changes the interpretation. Under the null, \(\Delta=0\), so the calculation asks about the chance of eventual rejection if there is no treatment effect. Under the design effect, \(\Delta=\Delta_{\text{design}}\), it asks about eventual success if the effect used for planning is the true effect. A plug-in calculation instead substitutes an effect estimate from the interim data. These are distinct conditional probabilities, not competing estimates of one uniquely defined probability.

A useful justification for the conditional distribution is to express the final statistic as the scaled interim statistic plus the standardized future increment. Under the assumed effect, that increment has mean \(\Delta\sqrt{1-t}\) and variance 1 before scaling by \(\sqrt{1-t}\). Equivalently,

where \(\varepsilon\) is independent of \(Z_1\). Conditioning on \(Z_1=z_1\) leaves the first two terms fixed and the last term random. Its variance is \(1-t\), giving the stated conditional normal distribution. At \(t=1\), no information remains to be collected and the conditional variance is zero; the formula is intended for \(t<1\).

Interim Plug-In Effects and Their Limits

A plug-in effect calculation estimates the treatment effect from interim data and uses that estimate as though it were the effect governing future data. In the simple known-variance canonical model, if \(Z_1\) estimates a constant drift and the interim estimate has the usual information scaling, the estimated final noncentrality is

More generally, use the interim effect estimate \(\widehat{\theta}\) and final information \(I_2\) to form \(\widehat{\Delta}_{\text{plug-in}}=\widehat{\theta}\sqrt{I_2}\), provided that this matches the trial’s effect and information parameterization. The plug-in calculation is not a posterior predictive probability: it does not average future success probabilities over uncertainty about the treatment effect.

Why Low Conditional Power Is Not a Futility Conclusion

Conditional power is useful for describing what may happen next under an explicit effect assumption. It is not, on its own, a decision rule. A low value under the design effect says that success is unlikely conditional on the current data and that design effect. It does not prove that the treatment effect is zero or clinically unimportant. The interim statistic may be noisy, the assumed design effect may not match a smaller but worthwhile effect, or the design’s final boundary may be difficult to cross from the observed position.

A stopping decision requires a prespecified or otherwise justified policy, consideration of the consequences of stopping, and attention to the criterion used to calculate the probability. In stochastic curtailment, for example, conditional power may be compared with a specified threshold as part of a stopping principle. That does not change the probability’s meaning into evidence that a parameter has a particular value. Predictive probability asks a different question by averaging over uncertainty in the effect using a specified distribution; conditional power fixes the effect assumption for the calculation.

Diagnostics and Scope

Before interpreting a reported value, check that its calculation matches the design and analysis:

• Verify the effect scale. Confirm that \(\Delta\) is the final noncentrality parameter in the same direction and scale as \(Z_1\), not an unconverted treatment-effect estimate.
• Check the information fraction. The conditional variance is \(1-t\); entering a sample-size fraction when it is not the information fraction can distort the result.
• Use the actual success boundaries. The displayed formula is for a single remaining final look with fixed two-sided boundary \(\pm c\). With several remaining group sequential looks, conditional power is the probability of crossing the applicable boundary at any remaining look, calculated from the joint conditional distribution of the future statistics. A terminal-only calculation can then give the wrong probability.
• Check model adequacy. Estimated variance, non-normal outcomes, changing allocation, missing data, or a mismatch between the interim and final analysis may require a design-specific calculation rather than the simple canonical normal formula.

The main alternatives answer different questions. Null conditional power describes rejection probability conditional on the observed history if the null is true. Design-effect conditional power describes future success conditional on a particular planning effect. Plug-in conditional power substitutes an interim estimate. Predictive probability integrates over effect uncertainty rather than fixing one effect, while a futility rule adds a decision threshold and policy. These methods should not be treated as interchangeable merely because they all produce probabilities between zero and one.

A frequent mistake is to see low conditional power under the design effect and conclude, “The treatment cannot work, so the trial should stop.” The correction is to state the conditioning assumptions: the calculation says success is unlikely if the design effect is true and the current interim data are fixed. It neither assigns a probability to the treatment effect nor rules out a smaller beneficial effect. Whether to stop depends on a decision rule and the scientific and operational context, not on that number alone.