Plug-In Conditional Power from the Interim Effect Estimate

Why Plug In the Interim Estimate?

At an interim analysis, the treatment effect is not known. Conditional power asks how likely the planned final success criterion is to be met, given the data already observed. To calculate it, one must specify what effect to assume for the future observations. The plug-in approach uses the interim treatment-effect estimate for that purpose.

This choice is intuitive but consequential: an interim estimate above zero can produce appreciable conditional power even when it is imprecise, while a modest estimate can produce low conditional power despite a design that had high power under its original target effect. The calculation is a model-based probability conditional on the interim result and on the plug-in assumption; it is not a probability that the treatment effect is true or that the trial will succeed.

Definition: Plug-in conditional power is the conditional probability of meeting a specified future success criterion, given the interim data, when the unknown treatment effect in the future-data distribution is set equal to its estimate from the interim data.

Model and Assumptions

Use a canonical normal model on the information scale. Normalize the planned final information to 1, and let \(t\), with \(0<t<1\), be the interim information fraction. Let \(U(t)\) be the cumulative score statistic at information fraction \(t\), and let \(Z_1=U(t)/\sqrt{t}\) be its standardized interim value. Under a constant standardized treatment effect \(\theta\), assume

U(t)\sim N(\theta t,t),

and assume independent increments: \(U(1)-U(t)\) is independent of the interim data and has distribution \(N(\theta(1-t),1-t)\). Thus, the variance of the future increment equals its information increment. The treatment effect is assumed constant over the remaining information, and the information schedule and final criterion are treated as fixed.

For the examples below, success means that the final statistic crosses a prespecified one-sided upper threshold \(c\): \(U(1)\ge c\). The numerical choice \(c=1.96\) is an illustrative cutoff, corresponding to a one-sided upper-tail probability of about 0.025 under a standard normal null distribution. In an actual group sequential design, the final boundary should be the boundary specified by that design.

These assumptions make the conditional distribution normal with a known variance. They are appropriate for a canonical normal score model, such as one based on a known variance or a valid large-sample normal approximation. They need reconsideration if the treatment effect changes over time, the planned information is not achieved as assumed, the statistic is not well approximated by the model, or relevant nuisance quantities are uncertain.

Deriving the Plug-In Formula

Given the interim result \(U(t)=\sqrt{t}Z_1\), write the final statistic as its observed interim part plus the future increment. Conditional on the interim data, the first part is fixed and only the second part varies:

U(1)=U(t)+\{U(1)-U(t)\}, \qquad U(1)\mid U(t)\sim N\bigl(\sqrt{t}Z_1+\theta(1-t),\,1-t\bigr).

If the future effect were specified as a fixed value \(\theta\), the conditional probability of crossing \(c\) would be the upper-tail probability under this normal distribution. For plug-in conditional power, estimate the effect from the interim data. Since \(E\{U(t)\}=\theta t\), the interim maximum-likelihood estimate in this model is \(\hat{\theta}=U(t)/t=Z_1/\sqrt{t}\). Substituting it gives the calculation.

\begin{aligned} \operatorname{CP}_{\mathrm{plug}} &=P\{U(1)\ge c\mid U(t)\} \text{ with }\theta=\hat{\theta}\\ &=\Phi\!\left( \frac{\sqrt{t}Z_1+\hat{\theta}(1-t)-c}{\sqrt{1-t}} \right)\\ &=\Phi\!\left( \frac{Z_1/\sqrt{t}-c}{\sqrt{1-t}} \right). \end{aligned}

Here \(\Phi\) is the cumulative distribution function of \(N(0,1)\). The last equality follows because \(\sqrt{t}Z_1+(Z_1/\sqrt{t})(1-t)=Z_1/\sqrt{t}\). In this normalized model, \(Z_1/\sqrt{t}\) is the interim estimate of the drift parameter, not the interim standardized statistic itself.

Result: For a single future final look with the one-sided criterion \(U(1)\ge c\), plug-in conditional power is \(\Phi\{(Z_1/\sqrt{t}-c)/\sqrt{1-t}\}\). The result depends on both the observed interim estimate and the remaining information \(1-t\).

Worked Calculations

Worked Example: Interim Estimate Near the Final Threshold

Suppose \(t=0.400\), the observed interim statistic is \(Z_1=1.200\), and the final upper threshold is \(c=1.960\). The normal model and fixed information assumptions above are taken to hold. Calculate plug-in conditional power.

First, the plug-in effect estimate is \(Z_1/\sqrt{t}\). The standard deviation of the future increment is \(\sqrt{1-t}\). Therefore,

\begin{aligned} \operatorname{CP}_{\mathrm{plug}} &=\Phi\!\left( \frac{1.200/\sqrt{0.400}-1.960}{\sqrt{0.600}} \right)\\ &=\Phi\!\left( \frac{1.8973666-1.960}{0.7745967} \right)\\ &=\Phi(-0.0808594)\\ &\approx 0.468. \end{aligned}

Under the plug-in assumption, the conditional probability of crossing the final threshold is about 46.8%. This is not a 46.8% probability that the treatment works; it is the model-based chance of final crossing if the future effect equals the observed interim estimate.

Worked Example: A Smaller Interim Statistic

Now suppose \(t=0.500\), \(Z_1=1.000\), and \(c=1.960\), under the same model. The plug-in estimate is \(1.000/\sqrt{0.500}\), and half of the planned information remains.

\begin{aligned} \operatorname{CP}_{\mathrm{plug}} &=\Phi\!\left( \frac{1.000/\sqrt{0.500}-1.960}{\sqrt{0.500}} \right)\\ &=\Phi(-0.7718586)\\ &\approx 0.220. \end{aligned}

The plug-in conditional power is about 22.0%. A design may have had high power at its target effect, yet this observed interim estimate implies a substantially smaller conditional probability under the plug-in assumption.

Worked Example: Sensitivity to the Observed Estimate

Compare two interim results at the same information fraction, \(t=0.500\), with the same final threshold \(c=1.960\). First take \(Z_1=0.800\); then take \(Z_1=1.200\). The study conditions and remaining information are unchanged, so only the observed estimate changes.

\begin{aligned} Z_1=0.800:\quad \operatorname{CP}_{\mathrm{plug}} &=\Phi\!\left( \frac{0.800/\sqrt{0.500}-1.960}{\sqrt{0.500}} \right)\\ &=\Phi(-1.17185)\approx 0.121,\\[4pt] Z_1=1.200:\quad \operatorname{CP}_{\mathrm{plug}} &=\Phi\!\left( \frac{1.200/\sqrt{0.500}-1.960}{\sqrt{0.500}} \right)\\ &=\Phi(-0.37186)\approx 0.355. \end{aligned}

The higher interim statistic raises plug-in conditional power from about 12.1% to 35.5%. This is not a change in the design or final rule; it is the consequence of substituting a larger estimated effect while holding those features fixed.

How to Interpret and Check the Result

At fixed \(t\) and \(c\), plug-in conditional power increases with \(Z_1\), because \(\Phi\) is increasing and the coefficient of \(Z_1\) in the normal argument is positive. This monotonicity is a useful check: a calculation that gives lower plug-in conditional power for a larger interim statistic, all else equal, has likely used a sign or substitution incorrectly.

The information fraction matters as well. It determines both the estimate \(Z_1/\sqrt{t}\) and the remaining standard deviation \(\sqrt{1-t}\). Consequently, conditional power is not determined by \(Z_1\) alone. The same standardized interim statistic at different information fractions corresponds to different estimated effects and different amounts of future information.

Several edge checks help catch mistakes:

If the estimated effect \(Z_1/\sqrt{t}\) equals the threshold \(c\), the normal argument is zero and the plug-in conditional power is 0.5.
If the estimated effect is below \(c\), the plug-in power is below 0.5; if it is above \(c\), it is above 0.5.
As \(t\) approaches 1, very little information remains. The formula is defined only for \(t<1\), and its limiting behavior reflects an increasingly small amount of future randomness. At \(t=1\), there is no future increment to model.
A computed value outside \([0,1]\) is impossible for a probability and indicates an implementation error. The normal argument can be any real number, but applying \(\Phi\) returns a value between 0 and 1.

The formula concerns one final look and one upper-tail criterion. It does not calculate the chance of crossing any of several remaining boundaries, and it does not replace a design-specific sequential success rule. For those settings, the event whose probability is required must be represented appropriately.

What the Plug-In Assumption Does—and Does Not—Say

A conditional-power calculation with a prespecified effect uses that chosen effect in the future increment distribution. The plug-in version instead uses \(\hat{\theta}\), so the assumed future effect changes with the observed interim data. The latter is data-responsive, but it does not account for uncertainty about the effect by averaging over a distribution of plausible effects. It simply evaluates the conditional probability at one estimated value.

That distinction matters when interim estimates are noisy. A high estimate may reflect both a real effect and random variation, and substituting it as if it were the future effect can make the resulting probability optimistic. Conversely, a low estimate can yield low plug-in conditional power even when a larger effect remains plausible. Plug-in conditional power should therefore be reported as an assumption-dependent summary, not as a forecast that incorporates all uncertainty.

The earlier material on conditional power establishes the general idea of conditioning on interim data. The practical advantage of this plug-in form is that it is direct and easy to compute. Its limitation is equally direct: it inherits the instability and possible selection bias of the interim estimate, especially when the estimate is based on little information or is observed after repeated monitoring. A sensitivity analysis over reasonable effect assumptions can show whether a conclusion depends strongly on the plug-in choice.

Common Mistakes

Using \(Z_1\) as the effect estimate. In this normalized information model, the estimate is \(Z_1/\sqrt{t}\). Substituting \(Z_1\) directly omits the information scaling.
Using the interim information fraction as the future variance. The future increment has variance \(1-t\), so its standard deviation is \(\sqrt{1-t}\).
Calling the result the probability that the null hypothesis is true. It is a conditional probability of a specified future data event under a plug-in effect assumption; it is not a posterior probability for \(H_0\).
Interpreting the calculation as a guarantee of success. A conditional power of 0.8 still allows a 0.2 probability of not crossing the final threshold under the assumed model and effect.

Key takeaway: For one final upper-threshold look, plug-in conditional power is obtained by inserting the interim estimate \(Z_1/\sqrt{t}\) into the future-data distribution. Its value is conditional on the observed interim result and on treating that estimate as the future effect.

Check Your Understanding

Answer each question, then compare with the solutions below.

For \(t=0.400\), \(Z_1=1.200\), and \(c=1.960\), what is the plug-in effect estimate, and which standard deviation appears in the denominator?
At fixed \(t\) and \(c\), explain why increasing \(Z_1\) cannot decrease plug-in conditional power.
For \(t=0.250\), \(Z_1=1.200\), and \(c=1.960\), calculate plug-in conditional power.
Does plug-in conditional power include uncertainty about the treatment effect by averaging across plausible effects? Explain.

Solutions. (1) The estimate is \(1.200/\sqrt{0.400}=1.8974\); the future standard deviation is \(\sqrt{1-0.400}=\sqrt{0.600}=0.7746\). (2) The normal argument is \((Z_1/\sqrt{t}-c)/\sqrt{1-t}\), which increases with \(Z_1\); since \(\Phi\) is increasing, so does the probability. (3) The argument is \((1.200/\sqrt{0.250}-1.960)/\sqrt{0.750}=(2.400-1.960)/0.8660=0.5081\), giving \(\Phi(0.5081)\approx0.694\). (4) No. It evaluates the conditional probability at the single estimated effect; it does not average over effect uncertainty.