Conditional Error And Conditional Power

Why These Two Conditional Probabilities Differ

At an interim analysis, the observed data affect the distribution of the final test statistic. Two useful probabilities can then be calculated conditional on those interim data. Conditional error asks how likely the current design is to reject the null hypothesis, given the interim data, if the null is true. Conditional power asks how likely it is to reject, given those same data, under a specified alternative. They condition on the same interim information but use different assumptions about the treatment effect.

This distinction matters when a trial is adapted. Conditional error is a measure of the remaining type I error associated with the original design, conditional on what has been observed. Conditional power describes the chance of eventual rejection under an alternative model. A design choice that targets a desired conditional power does not, by that fact alone, preserve the original conditional error.

Definition: At an interim history \(Y_1=y_1\), the conditional error is the probability, under \(H_0\), that the specified trial design will reject \(H_0\) using the future data, conditional on \(Y_1=y_1\). The conditional power at a specified alternative parameter value is the probability, under that alternative, that the design will reject \(H_0\) using the future data, conditional on the same interim history.

Both quantities are conditional probabilities of a future rejection, but they answer different questions because they are calculated under different distributions. Conditional error is tied to the null and the rejection rule; conditional power depends on the alternative value selected for planning or assessment. Neither is the probability that a hypothesis is true.

A Conditional Normal Model

For a transparent calculation, suppose a one-sided trial uses a final statistic \(Z\) and rejects \(H_0\) if \(Z\geq c\). Take \(c=1.96\), corresponding approximately to a one-sided significance level of 0.025 under the standard normal null distribution. Let \(t\) be the fraction of total information observed at interim, with \(0<t<1\), and let \(Z_1=z_1\) be the observed interim statistic.

Assume the canonical normal model with independent standardized increments. Under a standardized treatment effect \(\delta\), write the first and remaining increments as \(U_1\) and \(U_2\), where \(U_1\sim N(\delta\sqrt{t},1)\), \(U_2\sim N(\delta\sqrt{1-t},1)\), and the increments are independent. The interim statistic is \(Z_1=U_1\), while the final statistic is \(Z=\sqrt{t}U_1+\sqrt{1-t}U_2\). This model assumes the information fractions are correctly specified, the increments are independent and normally distributed, and the effect is represented by the same \(\delta\) over the two information periods.

Z\mid Z_1=z_1,\delta \sim N\left(\sqrt{t}\,z_1+\delta(1-t),\,1-t\right).

To obtain this result, condition on \(U_1=z_1\). The first contribution to the final statistic is then fixed at \(\sqrt{t}z_1\). The remaining contribution is \(\sqrt{1-t}U_2\); it has mean \(\sqrt{1-t}\,\delta\sqrt{1-t}=\delta(1-t)\) and variance \(1-t\). Adding the fixed first contribution gives the displayed conditional mean and variance.

Under \(H_0\), \(\delta=0\), so the conditional error for the current boundary is

\operatorname{CE}(z_1) =P_0(Z\geq c\mid Z_1=z_1) =1-\Phi\left(\frac{c-\sqrt{t}\,z_1}{\sqrt{1-t}}\right).

At a specified alternative \(\delta\), conditional power is

\operatorname{CP}_{\delta}(z_1) =P_{\delta}(Z\geq c\mid Z_1=z_1) =1-\Phi\left(\frac{c-\sqrt{t}\,z_1-\delta(1-t)}{\sqrt{1-t}}\right).

The only change in these expressions is the alternative mean contribution \(\delta(1-t)\). That change is consequential: using conditional power where conditional error is required amounts to calculating a null rejection probability under the wrong distribution.

Worked Example: Both Quantities at the Same Interim Result

Suppose the trial has observed half its total information, so \(t=0.50\), and the interim statistic is \(z_1=1.50\). The final one-sided rejection boundary is \(c=1.96\). Calculate the conditional error and conditional power at the alternative \(\delta=1.50\).

First, the conditional null mean is \(\sqrt{0.50}(1.50)=1.0607\), and the conditional standard deviation is \(\sqrt{0.50}=0.7071\). Therefore,

\operatorname{CE}(1.50) =1-\Phi\left(\frac{1.96-1.0607}{0.7071}\right) =1-\Phi(1.2718) \approx 0.1017.

Thus, given this interim statistic, the original design has about a 10.17% probability of rejecting under the null. This is a conditional probability for this interim history, not the trial's unconditional type I error rate.

Under \(\delta=1.50\), the future information adds conditional mean \(\delta(1-t)=1.50(0.50)=0.7500\). The standardized boundary is

\frac{1.96-1.0607-0.7500}{0.7071} \approx 0.2112, \qquad \operatorname{CP}_{1.50}(1.50) =1-\Phi(0.2112) \approx 0.4164.

The conditional power is about 41.64% under the specified alternative. It is not the conditional error: the first calculation uses the null distribution of future data, while the second uses the distribution under \(\delta=1.50\). The observed interim statistic and the rejection boundary are identical in both calculations.

How Conditional Power Depends on the Alternative

Conditional error for a fixed design and interim history is calculated under \(H_0\); it does not change merely because an analyst chooses a different alternative for a power calculation. Conditional power, by contrast, is indexed by the alternative. It is therefore incomplete to report a conditional power without stating the assumed effect and the model used to calculate it.

Worked Example: Varying the Assumed Effect

Keep the interim information fraction \(t=0.50\), the observed statistic \(z_1=1.50\), and the final boundary \(c=1.96\). Calculate conditional power under \(\delta=0.50\) and \(\delta=2.00\), using the conditional error from the preceding example for comparison.

For \(\delta=0.50\), the alternative adds conditional mean \(0.50(1-0.50)=0.2500\). Hence,

\operatorname{CP}_{0.50}(1.50) =1-\Phi\left(\frac{1.96-1.0607-0.2500}{0.7071}\right) =1-\Phi(0.9183) \approx 0.1793.

For \(\delta=2.00\), the added conditional mean is \(2.00(0.50)=1.000\), giving

\operatorname{CP}_{2.00}(1.50) =1-\Phi\left(\frac{1.96-1.0607-1.000}{0.7071}\right) =1-\Phi(-0.1424) \approx 0.5566.

The conditional error remains approximately 0.1017 because the null model and the original boundary have not changed. Conditional power rises from about 17.93% to 55.66% as the assumed effect increases. This is not a contradiction: the conditional rejection probability under the null and the conditional rejection probability under an alternative need not be equal.

Why a Power Target Does Not Control Conditional Error

A useful way to see the practical difference is to choose a boundary to achieve a conditional power target and then calculate the conditional error that boundary implies. This reverses the question: rather than evaluating the original rule, we ask what null rejection probability would result from a rule designed for power.

Worked Example: A Boundary Chosen for 80% Conditional Power

Retain \(t=0.50\), \(z_1=1.50\), and the alternative \(\delta=1.50\). Find the continuation boundary that gives 80% conditional power, then calculate its conditional error under \(H_0\).

Under the alternative, the conditional mean is \(\sqrt{0.50}(1.50)+1.50(0.50)=1.8107\), and the standard deviation is \(0.7071\). For 80% power, the standardized boundary must be the 20th percentile of the standard normal distribution, approximately \(-0.8416\). Thus,

c_{80} =1.8107+0.7071(-0.8416) \approx 1.2155.

Now evaluate that same boundary under the null. The conditional null mean is \(1.0607\), so

\operatorname{CE}_{c_{80}}(1.50) =1-\Phi\left(\frac{1.2155-1.0607}{0.7071}\right) =1-\Phi(0.2190) \approx 0.4133.

The boundary yields 80% conditional power at \(\delta=1.50\), but its conditional error is about 41.33% at this interim history, not the original design's 10.17%. This calculation does not endorse that boundary as an adaptation; it demonstrates that a power target alone does not preserve the conditional error.

Assumptions, Diagnostics, and Common Pitfalls

The numerical formulas rely on the stated canonical model and the specified rule \(Z\geq c\). In an actual design, the conditional distribution must reflect the test statistic, information accrued, dependence between interim and future data, and any stopping or continuation rule. If the statistic or correlation structure changes, the conditional distribution must be recalculated under the appropriate model; it is not safe simply to reuse the expressions above. Earlier material on changing the interim statistic and correlation structure addresses why that specification matters.

Several checks help identify a calculation that may be going wrong:

Check the conditioning information. The observed \(z_1\) and information fraction \(t\) must correspond to the interim analysis being evaluated.
Check which distribution is used. Conditional error sets \(\delta=0\); conditional power uses the stated alternative. The extra mean term in this model is \(\delta(1-t)\), not \(\delta\) or \(\delta\sqrt{1-t}\).
Check the tail direction. For an upper-boundary rejection rule \(Z\geq c\), use the upper tail \(1-\Phi(\cdot)\). A lower-boundary rule requires a different rejection event.
Check the result's scale. Conditional error and conditional power are probabilities between 0 and 1. Under a positive alternative, conditional power will generally exceed conditional error in this setting, but that is not a universal diagnostic rule for every model or alternative.
Check whether the model captures adaptation. If the design has additional stopping rules or data-dependent boundaries, the rejection event must include them. A calculation for a single final boundary cannot silently represent a more complicated rule.

Common mistake: “The conditional power is 0.4164, so the remaining type I error is 0.4164.” This substitutes a probability under \(\delta=1.50\) for the rejection probability under \(H_0\). The correction is to calculate the null conditional tail probability separately; here it is approximately 0.1017 for the original boundary.

Conditional error is useful when assessing whether a revised continuation design can be calibrated to retain the error allowance associated with the original design. Conditional power is useful for describing the chance of rejection under a specified effect, and can inform discussions of whether continuing is worthwhile. They are complementary, not competing, summaries. An unconditional type I error rate or power averages over possible interim data under the relevant model; a conditional quantity is evaluated at a particular interim history. Neither replaces the other.

Key takeaway: Always name the model used for the probability. Conditional error is a null rejection probability given interim data; conditional power is an alternative rejection probability given the same data. A power target is not an error-control condition.

Check Your Understanding

Try each question, then compare your reasoning with the solutions below.

In the canonical model, identify the conditional mean and variance of the final statistic given \(Z_1=z_1\) under \(H_0\).
For \(t=0.50\), \(z_1=0\), and \(c=1.96\), calculate the conditional error.
Using the same values as question 2 and \(\delta=1.50\), calculate the conditional power.
Explain why a conditional power value must be accompanied by a specified alternative.
A proposed boundary has 80% conditional power at a chosen effect. What additional quantity must be checked before claiming that the change preserves conditional error?

Solutions. (1) Under \(H_0\), the conditional mean is \(\sqrt{t}z_1\) and the variance is \(1-t\). (2) The conditional mean is zero and the standard deviation is \(0.7071\), so \(\operatorname{CE}(0)=1-\Phi(1.96/0.7071)=1-\Phi(2.7719)\approx0.0028\). (3) The alternative conditional mean is \(1.50(0.50)=0.7500\); hence \(\operatorname{CP}_{1.50}(0)=1-\Phi((1.96-0.7500)/0.7071)=1-\Phi(1.7112)\approx0.0435\). (4) The alternative changes the distribution of future data and therefore changes the conditional rejection probability; without the assumed effect, the probability is not specified. (5) Calculate the conditional error of the proposed rule under \(H_0\), conditional on the interim data, and compare it with the allowable conditional error from the design being retained.