Conditional Power Under Competing Effect Assumptions

Why the Effect Assumption Matters

At an interim analysis, the trial has produced some evidence, but future outcomes remain unknown. Conditional power quantifies the probability that a specified final test will reject its null hypothesis, given the interim data and an assumption about the treatment effect in the remaining participants. It is not a single number determined by the interim data alone: the assumed future effect matters.

Three assumptions are especially useful to compare. The observed-effect assumption treats the interim estimate as the effect that will continue. The originally planned-effect assumption uses the effect value underlying the initial design's power calculation. The clinically relevant target assumption uses an effect selected for its clinical meaning, whether or not it matches either the interim estimate or the design assumption. These answer different questions. A large conditional power under the observed effect does not establish that the treatment effect is large or clinically important.

The calculations below use a one-sided normal test. Let $\delta$ denote the treatment effect, with positive values favoring treatment. The hypotheses are $H_0:\delta\leq 0$ and $H_a:\delta>0$. At significance level $\alpha$, the final test rejects when its standardized statistic exceeds $c_\alpha=z_{1-\alpha}$, the $1-\alpha$ quantile of the standard normal distribution.

Definition: Conditional power under an assumed future effect $\delta_a$ is the probability, conditional on the observed interim data, that the prespecified final test rejects $H_0$, when the remaining outcomes are generated according to the model with effect $\delta_a$. The subscript $a$ emphasizes that this is an assumption, not necessarily the true effect.

Normal-Theory Calculation

Suppose the interim and future stages provide independent estimates $Y_1$ and $Y_2$ of the same treatment effect $\delta$, with known variances $v_1$ and $v_2$. Assume $Y_j\sim N(\delta,v_j)$, and write $I_j=1/v_j$ for the information in stage $j$. These assumptions describe a simple normal model; in an actual trial, the estimates and variances must correspond to the endpoint, allocation, and analysis specified in its design.

The inverse-variance weighted estimate from both stages is $(I_1Y_1+I_2Y_2)/(I_1+I_2)$. Its standard error is $1/\sqrt{I_1+I_2}$. Thus the final test statistic is

Z_{\mathrm{final}}=\frac{I_1Y_1+I_2Y_2}{\sqrt{I_1+I_2}}.

Once the interim estimate $Y_1=y_1$ is observed, it is fixed for the conditional calculation. Under the assumed future effect $\delta_a$, the future estimate has distribution $Y_2\sim N(\delta_a,v_2)$. Independence means that the only remaining randomness in the final statistic comes from $Y_2$.

Conditional power formula: Under the assumptions above, $Z_{\mathrm{final}}$, conditional on $Y_1=y_1$ and assuming future effect $\delta_a$, is normal with mean $(I_1y_1+I_2\delta_a)/\sqrt{I_1+I_2}$ and variance $I_2/(I_1+I_2)$. Therefore, $$ CP(\delta_a\mid y_1) = 1-\Phi\left( \frac{c_\alpha-\dfrac{I_1y_1+I_2\delta_a}{\sqrt{I_1+I_2}}} {\sqrt{I_2/(I_1+I_2)}} \right), $$ where $\Phi$ is the standard normal cumulative distribution function.

The variance in this formula is conditional variance, not the unconditional variance of the final statistic. Conditional on the interim data, the first-stage contribution no longer varies. The final test still uses its prespecified cutoff $c_\alpha$; the calculation does not itself change the test or its significance level.

Worked Comparison Under Three Effect Assumptions

Worked Example: Observed, Planned, and Clinically Relevant Effects

Hypothetical setup. Suppose independent stage estimates have $Y_1\sim N(\delta,4)$ and $Y_2\sim N(\delta,2)$, so $I_1=0.25$ and $I_2=0.5$. The interim estimate is $y_1=6$ outcome units. The original design used $\delta_{\mathrm{plan}}=4$ units, and the prespecified clinically relevant target is $\delta_{\mathrm{clin}}=3$ units. Use a one-sided test with $\alpha=0.025$, so $c_\alpha=1.960$. Assume the effect is constant across stages, the stage estimates are independent and normal, and the variances are known as specified.

Here $I_1+I_2=0.75$. The conditional mean and standard deviation of the final statistic, for an assumed effect $\delta_a$, are

\mu_{\mathrm{cond}}(\delta_a) =\frac{0.25(6)+0.5\delta_a}{\sqrt{0.75}} =\frac{50(6)+100\delta_a}{\sqrt{30000}}, \qquad \sigma_{\mathrm{cond}}=\sqrt{\frac{0.5}{0.75}}=0.8165.

The final test rejects when $Z_{\mathrm{final}}>1.960$. For the observed-effect assumption, use $\delta_a=6$, the interim estimate. Then

\mu_{\mathrm{cond}}(6)=\frac{50(6)+100(6)}{173.205}=5.196, \qquad CP(6)=\Phi\left(\frac{5.196-1.960}{0.8165}\right) =\Phi(3.963)\approx 0.99996.

For the originally planned effect, $\delta_a=4$:

\mu_{\mathrm{cond}}(4)=\frac{50(6)+100(4)}{173.205}=4.041, \qquad CP(4)=\Phi\left(\frac{4.041-1.960}{0.8165}\right) =\Phi(2.549)\approx 0.9946.

For the clinically relevant target, $\delta_a=3$:

\mu_{\mathrm{cond}}(3)=\frac{50(6)+100(3)}{173.205}=3.464, \qquad CP(3)=\Phi\left(\frac{3.464-1.960}{0.8165}\right) =\Phi(1.842)\approx 0.9673.

All three results use the same interim data, future variance, and rejection threshold. They differ because they posit different effects for the future stage. The observed-effect calculation is close to one because it assumes that the large interim estimate, 6 units, will persist. The target-effect calculation is lower, but still high in this hypothetical setup. None of these values is the probability that a particular effect assumption is true.

What the Three Calculations Answer

The observed-effect calculation asks: If the interim estimate were a good forecast of the future effect, how likely is rejection at the final analysis? It is responsive to the observed data, but it can be optimistic when the estimate is noisy. In particular, an interim estimate selected because it is unusually large tends, on average, to be less extreme on repetition. Conditional power that plugs it in without adjustment does not account for that tendency.

The planned-effect calculation asks whether the trial could still succeed if the effect assumed during the original design were to continue in the future. It helps assess progress against the design's benchmark. It does not make that benchmark clinically important merely because it was used to plan the trial.

The clinically relevant target calculation asks how likely the final test would be to reject if the future effect equaled a specified meaningful value. This can make the operational question easier to interpret: the target is stated in outcome units and chosen for clinical reasons. It is still a conditional probability under a model, not evidence that the target is the actual effect.

A useful comparison keeps the interim data, future information, significance level, and test boundary fixed while changing only $\delta_a$. That isolates the role of the effect assumption. In practice, analyses may also compare candidate future sample sizes or information levels, but those comparisons must recalculate the variance and conditional distribution rather than hold them fixed incorrectly.

Derivation and Assumptions

The formula follows directly from the stage-two sampling distribution. Given $Y_1=y_1$,

Z_{\mathrm{final}} =\frac{I_1y_1+I_2Y_2}{\sqrt{I_1+I_2}}, \qquad Y_2\sim N(\delta_a,1/I_2).

A linear transformation of a normal random variable is normal. Its conditional mean is obtained by replacing $Y_2$ with its mean $\delta_a$. Its conditional variance is the squared coefficient of $Y_2$, multiplied by $1/I_2$:

E(Z_{\mathrm{final}}\mid Y_1=y_1,\delta_a) =\frac{I_1y_1+I_2\delta_a}{\sqrt{I_1+I_2}}, \qquad \operatorname{Var}(Z_{\mathrm{final}}\mid Y_1=y_1,\delta_a) =\frac{I_2^2(1/I_2)}{I_1+I_2} =\frac{I_2}{I_1+I_2}.

For a normal variable with mean $\mu_{\mathrm{cond}}$ and standard deviation $\sigma_{\mathrm{cond}}$, the probability of exceeding $c_\alpha$ is $1-\Phi((c_\alpha-\mu_{\mathrm{cond}})/\sigma_{\mathrm{cond}})$. This gives the stated formula. Equivalently, it is $\Phi((\mu_{\mathrm{cond}}-c_\alpha)/\sigma_{\mathrm{cond}})$.

The derivation relies on independent stage estimates, a correctly specified normal model, and known variances. It also assumes that the effect can be represented by the same $\delta_a$ in the future stage and that the final analysis and rejection boundary are specified. If the endpoint is binary, time-to-event, clustered, or otherwise analyzed with a different model, a normal approximation may still be useful, but its validity and information calculation require justification for that setting.

Remaining Information Can Change the Answer

Worked Example: More Future Information at the Same Target Effect

Continue the hypothetical normal model, but suppose the future-stage estimate has variance $v_2=1$, rather than 2, while the interim estimate remains $y_1=6$ with $v_1=4$. Thus $I_1=0.25$, $I_2=1$, and assume the clinically relevant future effect is $\delta_a=3$ units. The stages are still assumed independent, with known variances and the same one-sided final cutoff $1.960$.

The conditional mean and standard deviation are

\mu_{\mathrm{cond}}=\frac{0.25(6)+1(3)}{\sqrt{1.25}} =\frac{4.5}{1.1180}=4.025, \qquad \sigma_{\mathrm{cond}}=\sqrt{\frac{1}{1.25}}=0.8944.

Therefore,

CP(3\mid y_1=6) =\Phi\left(\frac{4.025-1.960}{0.8944}\right) =\Phi(2.309)\approx 0.9895.

This is higher than the 0.9673 target-effect conditional power in the first example. The reason is not a change in the assumed effect or interim result: the future estimate now has lower variance and therefore contributes more information. For a real sample size update, the relationship between sample size and variance must follow the endpoint's model and allocation. Simply substituting a larger information value is justified only when that information calculation is appropriate.

Diagnostics, Alternatives, and Limitations

Before interpreting a conditional power comparison, check the inputs and the model. Confirm that the effect assumptions are expressed in the same units as the interim estimate; that $I_1$ and $I_2$ correspond to the actual and proposed information; and that the direction of the alternative and the final critical value match the planned test. A common arithmetic check is that increasing $\delta_a$ increases conditional power in this one-sided, positive-effect setup, while increasing future information generally gives the future assumption more influence.

Warning signs include unstable interim estimates, substantial missing outcomes, treatment-effect changes over time, or a variance estimate that is highly uncertain. A plug-in normal calculation can look precise even when these inputs are not. If the variance is estimated, uncertainty in that estimate is not represented by treating it as known. If the future effect may differ from the interim effect, the observed-effect assumption may be particularly unconvincing. Sensitivity analysis across plausible effects is more informative than presenting only the largest conditional power.

Conditional power is distinct from two related quantities. Unconditional design power is calculated before observing interim data, averaging over the possible interim and future data under a specified true effect. Predictive power averages future success probabilities over a distribution representing uncertainty about the effect, rather than fixing one plug-in future effect. Predictive power therefore requires an explicit uncertainty model or distribution. These approaches answer different questions; none removes the need to specify the final test and its error-control properties.

The conditional-power calculation alone does not determine whether an unblinded sample size adaptation preserves the trial's type I error. That depends on the adaptation rule and the final inferential procedure, not merely on how accurately conditional power was computed. Also, conditional power is conditional on the observed interim data: it is not the unconditional probability of success for the trial from its start.

Common Mistake

Common mistake: “The observed-effect conditional power is 0.99996, so the trial has a 99.996% chance of success.” This treats an assumption-based calculation as an unconditional forecast and ignores that it assumes the interim estimate of 6 units will persist. The correction is to state the conditioning and assumption: under this model, given the interim estimate of 6 units and assuming a future effect of 6 units, the probability of crossing the specified final boundary is approximately 0.99996. Report the planned-effect and clinically relevant target results as well when they address the decision at hand.

Practical Comparison

Effect assumption	Question answered	Typical caution
Observed interim effect	What if the interim estimate continues?	Can be optimistic when the estimate is noisy or extreme.
Originally planned effect	What if the design effect continues?	The design assumption need not be clinically meaningful or realistic.
Clinically relevant target	What if the future effect equals a meaningful target?	The target is a scenario, not a claim that this is the true effect.
Predictive approach	What if uncertainty about the effect is averaged over?	Requires a distribution or other explicit uncertainty model.

A sound report names the effect assumption, interim data, test boundary, remaining information, model, and resulting probability. If the three conditional powers differ substantially, the disagreement is itself useful: it shows that the conclusion depends on assumptions about the future effect. It is not a reason to select whichever value is most favorable.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

In the first worked example, what does the conditional power of approximately 0.9673 mean under the clinically relevant target assumption?
Why is the conditional standard deviation of the final statistic less than 1 in the first worked example?
If the interim estimate is unexpectedly large, which effect assumption is most likely to produce an optimistic conditional power, and why?
How does conditional power differ from unconditional design power?

Solutions. (1) Given the interim estimate of 6 units and the stated model, if the future effect is 3 units, the probability that the one-sided final statistic exceeds 1.960 is approximately 0.9673. It is not a probability that the effect is 3 units. (2) Once the interim data are conditioned on, the first-stage contribution is fixed; only the independent stage-two estimate varies. Its contribution has conditional variance $I_2/(I_1+I_2)=0.5/0.75=2/3$, so its standard deviation is $\sqrt{2/3}=0.8165$. (3) The observed-effect assumption is most likely to be optimistic because it treats the noisy, potentially extreme interim estimate as the future effect rather than accounting for uncertainty or regression toward less extreme values. (4) Conditional power fixes the interim data and calculates success probability under a specified future-effect assumption. Unconditional design power averages over possible interim as well as future data under a specified effect, before the interim data are observed.