Comparing Continuous-Outcome Power

Why Calculate Continuous-Outcome Power?

For a continuous endpoint, the treatment effect is often expressed as a difference between two population means: for example, a change of 5 points on a symptom scale or 2 mmHg in mean blood pressure. A power calculation asks how often a planned test would reject the null hypothesis if a specified difference were truly present. Comparing equal and unequal fixed allocation makes the precision cost of an allocation imbalance explicit.

The comparison is meaningful only when the designs are evaluated under the same assumptions: the same total sample size, treatment effect, outcome variance, significance level, test, and analysis. Changing any of these while comparing allocation ratios can make it unclear which design feature caused a power difference.

Model and Power Formula

Consider a two-arm trial with \(n_1\) participants assigned to treatment 1 and \(n_0\) assigned to treatment 0, with fixed sample sizes \(n_1\) and \(n_0\). The total sample size is \(N=n_1+n_0\). Let \(X_{ij}\) be the continuous outcome for participant \(i\) in arm \(j\), where \(j\) is 0 or 1.

Under these assumptions, the difference in sample means, \(\bar{x}_1-\bar{x}_0\), has mean \(\Delta\) and standard error \(SE=\sigma\sqrt{1/n_1+1/n_0}\). For a two-sided test at significance level \(\alpha\), reject \(H_0\) when the absolute value of the standardized statistic exceeds \(z_{1-\alpha/2}\), the \(1-\alpha/2\) quantile of the standard normal distribution.

Under a specified true effect \(\Delta\), this statistic has a normal distribution with variance 1 and mean \(\lambda=\Delta/SE\). The quantity \(\lambda\) is the signed noncentrality parameter: it expresses the true difference in standard-error units. Power is the probability, under that specified alternative, that the test statistic falls outside the null rejection region.

Here \(\Phi\) is the standard normal cumulative distribution function. The formula includes both tails, including when \(\Delta\) is negative. For a positive effect large relative to its standard error, the lower-tail probability is often very small, but it is still part of the two-sided power.

Deriving the Standard Error and Power

The standard error follows from the variance of a difference of independent sample means. Under the model, each sample mean has variance \(\sigma^2/n_j\). Independence means the covariance between the sample means is zero, so their difference has variance equal to the sum of those variances. Dividing by its standard deviation gives the stated standardized statistic.

To obtain power, standardize the difference in means using this standard error. Under the alternative its expectation is \(\Delta/SE=\lambda\), and its variance is 1. Thus \(Z\) has distribution \(N(\lambda,1)\) under the alternative. The probability of exceeding \(c\) is \(1-\Phi(c-\lambda)\); the probability of falling below \(-c\) is \(\Phi(-c-\lambda)\). Adding these disjoint rejection-tail probabilities gives the power formula.

This calculation makes the inputs visible. A larger absolute effect increases \(|\lambda|\); a larger \(\sigma\) or larger standard error reduces it. Increasing the total sample size generally reduces the standard error. A smaller \(\alpha\) makes the critical value \(c\) larger, so rejection is harder and power is lower when the other quantities stay fixed.

Comparing Equal and Unequal Allocation

At fixed total sample size, allocation determines the standard error through \(1/n_1+1/n_0\). To see why equal allocation minimizes that quantity, use \(n_1+n_0=N\):

For a fixed sum \(N\), the product \(n_1n_0\) is largest when the two arm sizes are equal (or as close to equal as integers allow). Consequently, equal allocation gives the smallest standard error in this common-variance model when the total sample size is held fixed. With a nonzero effect and the same two-sided test, a smaller standard error generally gives greater power. Unequal allocation can still be chosen for reasons beyond this particular power comparison, but its power should be calculated rather than assumed to match equal allocation.

Checking a Power Comparison

Before interpreting the result, check that the calculation matches the planned design and analysis. In particular, confirm the following:

• Endpoint and effect: State the outcome scale and the target mean difference \(\Delta\) in its units. A power value without a specified effect is incomplete.
• Variance: Specify the assumed common standard deviation \(\sigma\), not just a variance estimate from an unrelated outcome or scale.
• Allocation and total: Verify \(n_1+n_0=N\) and use the actual arm sizes in \(SE\). Do not substitute \(N\) into an equal-allocation formula when the planned allocation is unequal.
• Test and significance level: Match the critical value to the planned one-sided or two-sided test and to \(\alpha\). This tutorial's main formula is two-sided.
• Model: Consider whether independence, approximate normality of the sample means, and the common-variance assumption are defensible for the planned analysis.

The calculations above use a z test with known \(\sigma\). In practice, \(\sigma\) is commonly estimated and the analysis may use a t test; that procedure has a different reference distribution, especially when samples are small. The next tutorial on small-sample power comparisons addresses that setting. If arm variances differ, the common-variance standard error used here is not the appropriate general calculation. Likewise, if allocation is changed in response to accumulating outcomes, fixed arm sizes do not describe the design, so this fixed-allocation calculation alone does not characterize its power.

Common Mistakes and Design Interpretation

A second error is to report a power number without its assumptions. For example, “power is 80%” is not a complete design statement unless the assumed effect, variance, allocation, total sample size, and test are identified. Power is a property of the specified model and testing procedure under a specified alternative, not a permanent property of a treatment or endpoint.

The main comparison here is between fixed equal and fixed unequal allocation at matched conditions. Equal allocation minimizes the standard error for a difference in means under the common-variance model at fixed \(N\). Unequal allocation raises that standard error, with the amount depending on the ratio. A fair comparison preserves the other inputs and reports both the power values and the assumptions that produced them.