Conditional Power Under Alternative Variance Assumptions

Why the Variance Assumption Matters

A conditional-power calculation asks how likely the planned final test is to reject \(H_0\), given the interim data and a specified effect for the future. For a continuous endpoint, the standard deviation determines the standard error of the final treatment-effect estimate. A larger assumed standard deviation therefore generally lowers the conditional power for the same observed interim difference, assumed future effect, and sample size.

The variance used in that calculation may come from different sources. It may be known from the model, retained at the value used for original planning, or estimated from interim data and used as a plug-in estimate. These are distinct assumptions, not interchangeable labels for the same quantity. The comparisons below isolate their effect on conditional power; they do not account for interim stopping boundaries or other boundary adjustments.

Model and notation: Consider a two-arm trial with equal allocation, a continuous endpoint, and independent normally distributed observations with a common standard deviation \(\sigma\). Let \(n_1\) be the number of participants per arm observed at interim, \(n_2\) the number per arm still to be enrolled, and \(N=n_1+n_2\) the final number per arm. Let \(D_1\) be the observed interim difference in sample means (treatment minus control), and let \(\delta^*\) be the assumed treatment difference for future participants. For an assumed standard deviation \(\sigma_a\), the calculation treats future observations as having standard deviation \(\sigma_a\). The final test is a one-sided, upper-tail normal test with critical value \(c=z_{1-\alpha}\). Conditional power here is the probability of crossing that fixed final boundary, conditional on the interim data.

Conditional Power Under a Specified Variance

At the end of the trial, the difference in means is the weighted average of the interim and future differences. Conditional on the observed interim data, the future difference has mean \(\delta^*\) and variance \(2\sigma_a^2/n_2\). The final difference therefore has conditional mean \((n_1D_1+n_2\delta^*)/N\). Standardizing by the final standard error \(\sigma_a\sqrt{2/N}\) gives a final \(Z\) statistic with conditional mean \(m_a\) and conditional standard deviation \(\sqrt{n_2/N}\):

m_a=\frac{n_1D_1+n_2\delta^*}{\sigma_a\sqrt{2N}}, \qquad Z_{\mathrm{final}}\mid\text{interim data} \sim N\left(m_a,\frac{n_2}{N}\right).

The second argument of \(N(\cdot,\cdot)\) is the variance. The conditional power is the chance that this final statistic exceeds \(c\):

CP(\sigma_a) =P(Z_{\mathrm{final}}>c\mid\text{interim data}) =\Phi\left( \frac{m_a-c}{\sqrt{n_2/N}} \right).

This expression makes the roles of the inputs explicit. The observed interim difference \(D_1\) is fixed when conditioning. The assumed future effect \(\delta^*\) supplies the mean for future data. The variance assumption affects the standardization \(m_a\); the fraction of information remaining determines the conditional standard deviation of the final \(Z\) statistic.

Three Ways to Specify the Variance

With a known variance, \(\sigma_a\) is the standard deviation specified by the assumed model and treated as known. The normal calculation is exact under the stated normal model and known common variance.

With a fixed-planning variance, \(\sigma_a=\sigma_{\mathrm{plan}}\), the value used in original planning. It is held fixed during re-estimation, even if interim data suggest a different spread. This is a design assumption; it does not mean that the true population variance is known to equal the planning value.

With an interim-estimated variance, \(\sigma_a=s_1\), where \(s_1\) is an interim estimate, such as a pooled within-arm standard deviation. This is a plug-in calculation: it treats \(s_1\) as if it were the relevant standard deviation for the future and final analysis. Because \(s_1\) is itself random, this plug-in conditional power is generally an approximation rather than an exact probability under an unknown-variance model.

Worked Example: Conditional Power With a Known Variance

Suppose a hypothetical equal-allocation trial has \(n_1=40\) participants per arm at interim, with observed difference \(D_1=3\) mmHg. The plan is for \(N=100\) per arm, so \(n_2=60\). Use the assumed future difference \(\delta^*=4\) mmHg, known standard deviation \(\sigma=10\) mmHg, and one-sided \(\alpha=0.025\), so \(c=z_{0.975}=1.960\). Assume independent normal outcomes with the same known variance in both arms and across trial stages.

First calculate the conditional mean of the final statistic:

m=\frac{40(3)+60(4)}{10\sqrt{2(100)}} =\frac{360}{141.421} =2.546.

The conditional standard deviation is \(\sqrt{60/100}=0.7746\). Thus

CP(10)=\Phi\left(\frac{2.546-1.960}{0.7746}\right) =\Phi(0.756) \approx 0.775.

Under these assumptions, the probability of crossing the specified final one-sided boundary is about 77.5%, conditional on the interim data and on a future difference of 4 mmHg. It is not a probability that the treatment effect is true or that the null hypothesis is false.

Comparing the Assumptions With the Same Interim Data

To isolate the variance assumption, retain the data and design from the example above, but compare three possible standard deviations: a known value of 10 mmHg, a fixed-planning value of 12 mmHg, and an interim estimate \(s_1=11\) mmHg. The values are hypothetical. In every case, \(D_1=3\) mmHg, \(\delta^*=4\) mmHg, \(n_1=40\), \(n_2=60\), \(N=100\), and \(c=1.960\). The model assumptions for the calculation are equal allocation, independent normal observations, a common variance, and the stated variance value applying to future data.

Worked Example: Comparing Three Variance Inputs

For all three calculations the numerator in \(m_a\) is \(40(3)+60(4)=360\), and the conditional standard deviation of the final statistic is \(\sqrt{0.6}=0.7746\).

Known variance, 10 mmHg: \(m_a=360/[10\sqrt{200}]=2.546\), so \(CP=\Phi[(2.546-1.960)/0.7746]\approx 0.775\).

Fixed-planning variance, 12 mmHg: \(m_a=360/[12\sqrt{200}]=2.121\), so \(CP=\Phi[(2.121-1.960)/0.7746]\approx 0.583\).

Interim-estimated variance, \(s_1=11\) mmHg: \(m_a=360/[11\sqrt{200}]=2.314\), so \(CP=\Phi[(2.314-1.960)/0.7746]\approx 0.676\).

The larger standard deviation lowers the standardized conditional mean and hence conditional power in this example. The values do not show that the 12 mmHg assumption is correct, or that the interim estimate is more reliable; they show the consequences of using each input in this calculation.

How the Variance Assumption Changes a Target-Based Sample Size

A target conditional power \(\gamma\) can be expressed as a requirement on the conditional mean. Since \(\Phi\) is increasing, the condition \(CP(\sigma_a)\geq\gamma\) is equivalent to

\frac{n_1D_1+(N-n_1)\delta^*}{\sigma_a\sqrt{2N}} \geq c+z_{\gamma}\sqrt{\frac{N-n_1}{N}}.

Here \(N\) appears on both sides, so a sample size is ordinarily found by checking candidate values of \(N\), subject to the design's permitted sample sizes and any maximum. The variance assumption changes the left side while the interim data, future effect assumption, and target remain fixed.

Worked Example: Sample Size Under Alternative Variances

Use \(n_1=40\), \(D_1=3\) mmHg, \(\delta^*=4\) mmHg, one-sided \(c=1.960\), and target conditional power \(\gamma=0.80\), for which \(z_\gamma=0.8416\). Compare \(\sigma_a=10\), 11, and 12 mmHg. Assume the same equal-allocation normal model as above, and hold the interim observations and assumed future difference fixed. For each variance, search integer final sample sizes per arm \(N>40\).

For \(\sigma_a=10\) mmHg, \(N=104\) gives \(m_a=376/[10\sqrt{208}]=2.607\) and conditional standard deviation \(\sqrt{64/104}=0.7845\). Thus \(CP=\Phi[(2.607-1.960)/0.7845]\approx0.795\), below target. At \(N=105\), \(m_a=380/[10\sqrt{210}]=2.622\) and the conditional standard deviation is \(\sqrt{65/105}=0.7868\). The resulting conditional power is approximately \(\Phi(0.841)=0.800\). The first qualifying integer is about 105 per arm.

For \(\sigma_a=11\) mmHg, at \(N=125\), \(m_a=460/[11\sqrt{250}]=2.644\), with conditional standard deviation \(\sqrt{85/125}=0.8246\), giving conditional power about 0.797. At \(N=126\), \(m_a=464/[11\sqrt{252}]=2.658\), with standard deviation \(\sqrt{86/126}=0.8262\), giving about 0.801. The first qualifying integer is about 126 per arm.

For \(\sigma_a=12\) mmHg, at \(N=148\), \(m_a=552/[12\sqrt{296}]=2.673\), with standard deviation \(\sqrt{108/148}=0.8542\), giving about 0.798. At \(N=149\), \(m_a=556/[12\sqrt{298}]=2.684\), with standard deviation \(\sqrt{109/149}=0.8555\), giving about 0.801. The first qualifying integer is about 149 per arm.

The search illustrates how a higher assumed standard deviation can require more participants to reach the same conditional-power target. These are conditional calculations under each variance input, not guarantees of unconditional final power.

Justification, Diagnostics, and Limitations

The conditional distribution used above follows directly from the independent-increments structure of the normal model. Given the interim data, \(D_1\) is fixed. The future difference \(D_2\) has distribution \(N(\delta^*,2\sigma_a^2/n_2)\). The final difference is \((n_1D_1+n_2D_2)/N\). Multiplying its conditional mean by the inverse final standard error yields \(m_a\); multiplying its conditional standard deviation by that same inverse standard error yields \(\sqrt{n_2/N}\). Therefore the standardized final statistic has the conditional normal distribution stated above. The independence assumption is essential: if future observations are not independent of interim observations under the model, the conditional variance needs a covariance term.

Several checks help identify a calculation that may be going wrong:

Check units: \(D_1\), \(\delta^*\), and \(\sigma_a\) must use the same endpoint units, so \(m_a\) is dimensionless.
Check allocation and counts: these formulas use equal numbers per arm at both stages. Do not insert total participants across both arms in place of \(N\) or \(n_1\).
Check the remaining-information term: if \(n_2\) is small, the conditional standard deviation \(\sqrt{n_2/N}\) is small, but the assumed future effect has less weight in \(m_a\) as well.
Check the variance source and timing: an interim estimate should be identified explicitly, including how it was calculated. A noisy estimate can change the re-estimated sample size substantially.
Check the test boundary: \(c=1.960\) is the one-sided 0.025 normal boundary used here. A different test or boundary requires a corresponding conditional-power calculation.

The known-variance calculation is appropriate when the variance really is specified as known under the model. A fixed-planning value is simple and avoids reacting to a noisy interim estimate, but may be poorly calibrated if the actual variability differs materially from that value. An interim plug-in estimate can adapt to observed variability, but may be unstable when based on limited data and does not automatically account for uncertainty in the variance estimate. In particular, replacing an unknown variance by \(s_1\) does not make the resulting normal calculation an exact \(t\)-based conditional probability.

All three approaches also depend on the assumed future treatment difference \(\delta^*\), the normal model, and the specified final test. This tutorial's expression uses a single fixed final boundary. A group sequential design with additional stopping boundaries or a boundary-adjusted final rule requires a boundary-aware calculation rather than treating this expression as its complete operating characteristic.

Key takeaway: Conditional power is conditional on both the interim data and the assumptions used for future data. State the variance input, the assumed future effect, and the final boundary whenever reporting it; a conditional-power number without those assumptions is not reproducible.

Check Your Understanding

Answer each question, then compare with the solutions below.

If the variance input increases while all other quantities in the formula remain fixed and the conditional mean is positive, what happens to the standardized conditional mean?
In a two-arm trial, the interim difference is 2 units, \(n_1=30\) per arm, \(n_2=70\), \(\delta^*=3\) units, and \(\sigma_a=5\) units. Find \(m_a\) for \(N=100\).
Why is an interim-estimated variance plug-in calculation not generally an exact conditional probability for an unknown-variance model?
What is the distinction between a fixed-planning variance and a known variance?

Solutions. (1) It decreases, because \(\sigma_a\) is in the denominator of \(m_a\); the resulting conditional power decreases for a fixed boundary and positive conditional mean. (2) \(m_a=[30(2)+70(3)]/[5\sqrt{200}]=270/70.711=3.818\). The calculation uses the stated units consistently, and assumes equal allocation and the common-variance normal model. (3) The interim estimate is random, and treating it as the true future standard deviation ignores uncertainty in that estimate; the plug-in normal calculation is an approximation. (4) A known variance is treated as a true, specified model quantity. A fixed-planning variance is a value retained from original planning, whether or not it equals the true population variance.