Conditional Power Under a Prespecified Design Effect

Why Specify the Effect Used for Conditional Power?

After an interim analysis, conditional power asks how likely the trial is to meet its success criterion at the planned final analysis, given the information observed so far. The answer depends on what is assumed about the treatment effect during the remaining part of the trial. One choice is to use the effect estimate from the interim data. Another is to use a prespecified design effect: the effect value selected in advance for planning or interpretation.

These choices can give different probabilities even when they use the same interim statistic and final boundary. The design-effect calculation answers a conditional question under a fixed assumed effect; it does not update that effect using the interim estimate. The plug-in calculation instead treats the interim estimate as the effect that will continue. Neither probability is a probability that the assumed effect is true.

Definition: In a normal trial with one remaining final analysis, conditional power under a prespecified design effect is the probability of satisfying the final success criterion, conditional on the observed interim statistic, when the future treatment-effect drift is set to a prespecified value \(\theta_d\). For a one-sided final rule \(Z_2\ge c\), it is \(P(Z_2\ge c\mid Z_1=z_1;\theta=\theta_d)\).

Model and Conditional Distribution

Use the canonical normal model with information \(I_1\) at the interim analysis and planned final information \(I_2\), where \(0<I_1<I_2\). Let \(Z_1\) and \(Z_2\) be the standardized statistics at these analyses. Under a fixed effect \(\theta\), the score accumulated over an information increment \(\Delta I\) is normally distributed with mean \(\theta\Delta I\) and variance \(\Delta I\). Increments from disjoint information intervals are independent. The information schedule and final boundary are treated as fixed, and the variance is known or the canonical normal approximation is taken as valid.

For this tutorial, success is one-sided: the final statistic must be at least \(c\). The boundary \(c\) is the boundary specified by the design for the final analysis; it need not be the ordinary single-analysis critical value if the design uses an adjusted boundary. There is one remaining final look, not several future opportunities to cross a boundary.

Write the total score at the final analysis as the interim score plus the future increment. Given \(Z_1=z_1\), the interim score is \(z_1\sqrt{I_1}\). The future increment, under the design effect \(\theta_d\), has mean \(\theta_d(I_2-I_1)\) and variance \(I_2-I_1\). Dividing their sum by \(\sqrt{I_2}\) gives the conditional distribution:

Z_2\mid Z_1=z_1;\theta=\theta_d \sim N\left( \sqrt{\frac{I_1}{I_2}}z_1+ \frac{\theta_d(I_2-I_1)}{\sqrt{I_2}}, \frac{I_2-I_1}{I_2} \right).

The second argument of \(N\) is the variance. Define the conditional mean \(m_d\) and conditional standard deviation \(s_c\) by

m_d=\sqrt{\frac{I_1}{I_2}}z_1+ \frac{\theta_d(I_2-I_1)}{\sqrt{I_2}}, \qquad s_c=\sqrt{\frac{I_2-I_1}{I_2}}.

Therefore the design-effect conditional power is

CP_{\mathrm{design}} =P(Z_2\ge c\mid Z_1=z_1;\theta=\theta_d) =1-\Phi\left(\frac{c-m_d}{s_c}\right) =\Phi\left(\frac{m_d-c}{s_c}\right),

where \(\Phi\) is the standard normal cumulative distribution function. This formula makes the roles of the inputs explicit: the observed interim statistic contributes to the conditional mean, the design effect determines the assumed future drift, and the remaining information determines the conditional variance.

Worked Calculations

Worked Example: Design Effect Versus Interim Plug-In

Suppose a hypothetical normal trial has interim information \(I_1=100\), planned final information \(I_2=250\), and observed interim statistic \(z_1=1.50\). The prespecified design effect is \(\theta_d=0.20\), and the one-sided final boundary is \(c=1.96\). Calculate design-effect conditional power, then compare it with plug-in conditional power.

The conditional standard deviation is

s_c=\sqrt{\frac{250-100}{250}}=\sqrt{0.60}=0.7745967.

Under the design effect, the conditional mean is

m_d=\sqrt{\frac{100}{250}}(1.50)+ \frac{0.20(150)}{\sqrt{250}} =0.9486833+1.8973666 =2.8460499.

Using the unrounded mean in the standardization,

CP_{\mathrm{design}} =\Phi\left(\frac{2.8460499-1.96}{0.7745967}\right) =\Phi(1.1439) \approx 0.8736.

Thus, under the assumed effect of \(0.20\), the conditional probability of crossing the final boundary is about 87.36%.

For the plug-in calculation, the interim effect estimate is \(\hat{\theta}=z_1/\sqrt{I_1}=1.50/10=0.15\). Substituting this estimate for the design effect gives a conditional mean of \(z_1\sqrt{I_2/I_1}=1.50\sqrt{2.5}=2.371708\). Hence

CP_{\mathrm{plug\text{-}in}} =\Phi\left(\frac{2.371708-1.96}{0.7745967}\right) =\Phi(0.5315) \approx 0.7024.

The design-effect probability is higher because the assumed future effect, \(0.20\), exceeds the interim estimate, \(0.15\). The two values answer different conditional questions; the difference is not an arithmetic inconsistency.

Worked Example: Finding the Design Effect for a Target Conditional Power

Consider a hypothetical trial with \(I_1=100\), \(I_2=200\), \(z_1=0.50\), and final boundary \(c=1.96\). Find the design effect that gives conditional power \(0.80\), using the normal model.

The conditional standard deviation is \(s_c=\sqrt{100/200}=0.7071068\), and the interim contribution to the conditional mean is \(\sqrt{100/200}(0.50)=0.3535534\). A standard normal value with cumulative probability \(0.80\) is approximately \(0.841621\). To obtain conditional power \(0.80\), the conditional mean must satisfy

\frac{m_d-1.96}{0.7071068}=0.841621, \qquad m_d=1.96+(0.841621)(0.7071068)\approx2.5551.

Solve the conditional-mean equation for \(\theta_d\):

\theta_d= \frac{\left(2.5551-0.3535534\right)\sqrt{200}}{100} \approx 0.3113.

Checking with the rounded target mean gives

CP_{\mathrm{design}} =\Phi\left(\frac{2.5551-1.96}{0.7071068}\right) =\Phi(0.8416) \approx0.8000.

The required design effect is about \(0.3113\) in the model's effect units. This calculation is an algebraic calibration under the assumed normal model; it does not show that this effect is plausible or clinically important.

Worked Example: A More Modest Conditional Probability

Suppose a hypothetical trial has \(I_1=60\), \(I_2=100\), \(z_1=1.20\), a prespecified standardized design effect \(\theta_d=0.15\), and a one-sided final boundary \(c=2.00\). The interim result is below the boundary, so continuing to the planned final analysis is consistent with this stated example. Calculate both conditional powers.

The conditional standard deviation is \(\sqrt{40/100}=0.6324555\). The design-effect conditional mean is

m_d=\sqrt{0.60}(1.20)+\frac{0.15(40)}{\sqrt{100}} =0.9295160+0.6000000 =1.5295160.

Therefore

CP_{\mathrm{design}} =\Phi\left(\frac{1.5295160-2.00}{0.6324555}\right) =\Phi(-0.7439) \approx0.2285.

The interim effect estimate is \(1.20/\sqrt{60}\approx0.1549\), slightly greater than the design effect. Plugging it in gives conditional mean \(1.20\sqrt{100/60}\approx1.5492\), and

CP_{\mathrm{plug\text{-}in}} =\Phi\left(\frac{1.549193-2.00}{0.6324555}\right) =\Phi(-0.7128) \approx0.2381.

Both probabilities are low because the conditional means are below the final boundary relative to the remaining standard deviation. The plug-in value is a little higher because its assumed effect is a little larger.

Interpreting, Checking, and Comparing the Result

Conditional power is conditional on the observed interim result and on the assumed future effect. Under \(\theta_d\), it is the model-based probability of crossing the specified final boundary if the trial continues as planned. It is not the probability that the null hypothesis is true, the probability that the treatment effect equals \(\theta_d\), or a guarantee about the eventual result. Nor is it the unconditional probability of success before the interim data are observed.

The main alternative within this calculation is plug-in conditional power. It uses \(\hat{\theta}=z_1/\sqrt{I_1}\) as the future effect, whereas design-effect conditional power uses the prespecified \(\theta_d\). Neither choice is automatically preferable for every purpose. A design effect is useful for asking a clearly defined “if this effect holds” question and can avoid treating a noisy interim estimate as the best forecast of the future effect. Plug-in power directly reflects the observed estimate, but may be especially variable when interim information is limited. A report should identify which effect was used rather than presenting a conditional-power number without its assumption.

Before interpreting a result, check that \(I_2>I_1\), that the final boundary \(c\) matches the planned success rule, and that the information scale and effect units are compatible. Check the arithmetic by verifying that the conditional variance is positive and less than one when there is a positive but incomplete information increment. Also inspect sensitivity: because \(m_d\) increases with \(\theta_d\), design-effect conditional power must increase as the assumed effect increases. A result that violates this direction is a signal to check the formula or computation.

This calculation has important limits. It relies on a normal independent-increments model, a fixed information plan, a specified effect, and one remaining final look. It does not account for uncertainty about the effect or variance, changes to the analysis plan, or a success rule involving multiple future looks. If these assumptions do not describe the trial, the displayed probability need not represent its chance of success. Conditional power is a planning or monitoring summary, not by itself a rule for changing sample size or stopping a trial.

Common Mistake

Wrong reasoning: “The interim estimate is \(0.15\), so conditional power under the design effect must use \(0.15\).” That substitutes the plug-in effect for the prespecified effect and answers a different question. Correction: retain \(\theta_d\) in the future-drift term when calculating \(CP_{\mathrm{design}}\); calculate plug-in conditional power separately if it is useful for comparison. Always label the assumed effect and the resulting probability.

Key takeaway: Under a one-sided normal final rule, design-effect conditional power is \(\Phi((m_d-c)/s_c)\), with the observed interim statistic in \(m_d\) and the prespecified effect governing the assumed future drift. Changing that effect changes the question as well as the answer.

Check Your Understanding

Try each question, then compare your reasoning with the solutions below.

With \(I_1=80\), \(I_2=160\), \(z_1=1.00\), and \(c=2.00\), write the conditional mean and standard deviation under \(\theta_d=0.10\).
For fixed \(I_1\), \(I_2\), \(z_1\), and \(c\), explain why increasing \(\theta_d\) cannot decrease design-effect conditional power.
In the first worked example, why is the plug-in conditional power lower than design-effect conditional power?
Does a design-effect conditional power of \(0.80\) mean there is an 80% probability that the design effect is true? Explain.

Solutions. (1) \(s_c=\sqrt{80/160}=0.7071\), and \(m_d=\sqrt{80/160}(1.00)+0.10(80)/\sqrt{160}=0.7071+0.6325=1.3396\). (2) The conditional mean increases by \(\Delta\theta_d(I_2-I_1)/\sqrt{I_2}>0\) for any increase \(\Delta\theta_d>0\), while \(s_c\) stays fixed; thus \(\Phi((m_d-c)/s_c)\) increases. (3) The interim estimate is \(0.15\), below the prespecified design effect \(0.20\), so the plug-in calculation assumes less future drift. (4) No. It is the probability of final success conditional on the interim result under the assumption that \(\theta=\theta_d\); it assigns no probability to whether that assumption is true.