Power Loss From Unequal Allocation

Why Unequal Allocation Can Reduce Power

A fixed total sample size does not guarantee equal precision for a treatment comparison. When outcome variances are the same in both arms, observations in the smaller arm contribute less information about the difference in means than the same number of observations would under balanced allocation. The practical consequence is a larger standard error and, for a specified treatment effect and significance level, lower power.

The comparison in this tutorial isolates that allocation effect: hold the total sample size, treatment effect, outcome variance, and test fixed, and change only the arm sizes. This is a benchmark for fixed allocation plans. It does not by itself describe the additional inferential issues that can arise when allocations depend on accumulating outcomes.

Model and Power Formula

Definition: Suppose there are \(n_1\) independent outcomes in treatment arm 1 and \(n_0\) independent outcomes in control arm 0, with fixed total \(N=n_1+n_0\). Assume outcomes are normally distributed with common known variance \(\sigma^2\), and let the population means be \(\mu_1\) and \(\mu_0\). The estimated treatment difference is \(\bar{x}_1-\bar{x}_0\), with standard error \(SE=\sigma\sqrt{1/n_1+1/n_0}\). For the two-sided test \(H_0:\mu_1-\mu_0=0\) against \(H_a:\mu_1-\mu_0\ne0\), the normal test rejects when the absolute standardized statistic exceeds \(z_{1-\alpha/2}\).

Under a specified alternative, write \(\delta=\mu_1-\mu_0\). The standardized statistic has a normal distribution with variance 1 and mean \(\lambda=\delta/SE\), called its noncentrality parameter. The power is the probability of rejection when that alternative is true. For a positive \(\delta\), it is

\text{Power} =P(|Z|>c\mid \delta) =1-\Phi(c-\lambda)+\Phi(-c-\lambda) =\Phi(\lambda-c)+\Phi(-\lambda-c), \qquad c=z_{1-\alpha/2}.

Here \(Z\) denotes the standardized test statistic under the alternative and \(\Phi\) is the standard normal cumulative distribution function. The second tail is included because the test is two-sided. For a negative treatment difference the power is the same with \(|\lambda|\) in place of \(\lambda\).

These formulas are exact under the stated normal-outcome, known-variance model. In practice, \(\sigma\) is usually estimated. A pooled two-sample \(t\) test then has a noncentral-\(t\) power calculation under normal outcomes and equal variances; the normal formula is a useful approximation, especially when sample sizes are not small. The numerical examples below use the known-\(\sigma\) model so the allocation effect is transparent.

Why Balanced Allocation Minimizes the Standard Error

For a fixed \(N\), the standard error depends on the allocation through \(1/n_1+1/n_0\). The following algebra shows why that quantity is smallest when the arm sizes are equal.

\frac{1}{n_1}+\frac{1}{n_0} =\frac{n_1+n_0}{n_1n_0} =\frac{N}{n_1n_0}, \qquad n_1n_0\leq \left(\frac{n_1+n_0}{2}\right)^2 =\frac{N^2}{4}.

The inequality follows from \((n_1-n_0)^2\geq0\), which implies \(4n_1n_0\leq(n_1+n_0)^2\). Therefore \(N/(n_1n_0)\geq4/N\), with equality when \(n_1=n_0=N/2\). At fixed \(N\), the balanced allocation gives the smallest variance and standard error for the estimated difference.

The power comparison follows as well. For \(\lambda>0\), the two-sided normal power is \(\Phi(\lambda-c)+\Phi(-\lambda-c)\), whose derivative with respect to \(\lambda\) is \(\phi(c-\lambda)-\phi(c+\lambda)>0\). The inequality holds because \(|c-\lambda|<c+\lambda\) and the standard normal density decreases as the absolute value of its argument increases. Thus, for a nonzero specified effect, the smaller standard error under balance produces a larger \(|\lambda|\) and higher power.

Worked Comparisons at Fixed Total Sample Size

Worked Example: Comparing 50:50 and 70:30 Allocation

Consider a hypothetical normally distributed outcome with common known standard deviation \(\sigma=10\) units. Suppose the true difference is \(\delta=4\) units, the total sample size is \(N=100\), and the two-sided significance level is \(\alpha=0.05\). The assumptions for the formula are that observations are independent, the outcome variance is the same in both arms, and the sample sizes are fixed as specified.

With a 50:50 allocation, \(n_1=n_0=50\). The standard error and noncentrality parameter are

SE_{50:50}=10\sqrt{\frac{1}{50}+\frac{1}{50}} =10\sqrt{0.04}=2, \qquad \lambda_{50:50}=\frac{4}{2}=2.

The critical value is \(c=z_{0.975}\approx1.95996\). Hence

\text{Power}_{50:50} =1-\Phi(1.95996-2)+\Phi(-1.95996-2) \approx0.5160.

Now allocate 70 participants to treatment and 30 to control. The total remains 100, and the same outcome assumptions and treatment difference apply:

SE_{70:30} =10\sqrt{\frac{1}{70}+\frac{1}{30}} =10\sqrt{0.047619} \approx2.182, \qquad \lambda_{70:30}=\frac{4}{2.182}\approx1.83303.

Using this value without rounding it prematurely,

\text{Power}_{70:30} =1-\Phi(1.95996-1.83303)+\Phi(-1.95996-1.83303) \approx0.4496.

The power reduction is approximately \(0.5160-0.4496=0.0664\), or 6.64 percentage points, using the unrounded powers for the subtraction. In this hypothetical comparison, both plans enroll 100 participants and test the same effect at the same significance level; the unequal plan has lower power because its standard error is larger.

Worked Example: A More Unequal 80:20 Allocation

Keep the hypothetical outcome parameters from the preceding example: \(N=100\), \(\delta=4\) units, \(\sigma=10\) units, and \(\alpha=0.05\). For an 80:20 allocation, \(n_1=80\) and \(n_0=20\). The model assumptions remain independent normal outcomes with equal known variances and fixed arm sizes.

The standard error and noncentrality parameter are

SE_{80:20} =10\sqrt{\frac{1}{80}+\frac{1}{20}} =10\sqrt{0.0625}=2.5, \qquad \lambda_{80:20}=\frac{4}{2.5}=1.6.

The two-sided power is

\text{Power}_{80:20} =1-\Phi(1.95996-1.6)+\Phi(-1.95996-1.6) =1-\Phi(0.35996)+\Phi(-3.55996) \approx0.3596.

Relative to the balanced power of approximately 0.5160, this plan loses about \(0.1564\), or 15.64 percentage points. The calculation illustrates that a larger allocation imbalance can produce a larger penalty in this specified setting; it does not imply that every change in allocation ratio causes the same power reduction.

Worked Example: A 60:40 Allocation With a Different Outcome Scale

For a separate hypothetical comparison, suppose \(N=200\), \(\sigma=5\) outcome units, and \(\delta=2\) units, with a two-sided test at \(\alpha=0.05\). Outcomes are assumed independent and normal with the same known variance in both arms, and arm sizes are fixed. Compare 100:100 with 120:80.

For 100 participants per arm,

SE_{50:50}=5\sqrt{\frac{1}{100}+\frac{1}{100}} =5\sqrt{0.02}\approx0.7071, \qquad \lambda_{50:50}=\frac{2}{0.7071}\approx2.8284.

Thus the power is

\text{Power}_{50:50} =1-\Phi(1.95996-2.8284)+\Phi(-1.95996-2.8284) \approx0.8074.

For the 120:80 allocation,

SE_{60:40} =5\sqrt{\frac{1}{120}+\frac{1}{80}} =5\sqrt{0.0208333} \approx0.7217, \qquad \lambda_{60:40}=\frac{2}{0.7217}\approx2.7713.

The resulting power is

\text{Power}_{60:40} =1-\Phi(1.95996-2.7713)+\Phi(-1.95996-2.7713) \approx0.7914.

The reduction is about \(0.0160\), or 1.60 percentage points. It is smaller than in the 70:30 and 80:20 comparisons, but the relevant lesson here is the calculation: the power difference depends on the specified effect, variance, significance level, total sample size, and allocation ratio.

Reading the Comparison Carefully

A power penalty is a difference between powers for two explicitly defined designs, not a fixed property of an allocation ratio by itself. A report should state the total sample size, arm sizes, variance assumption, target treatment difference, test, and significance level. If any of these change between designs, the resulting power difference no longer isolates allocation.

Feature	Balanced allocation	Unequal allocation
Total sample size	Same fixed \(N\) in a matched comparison	Same fixed \(N\) in a matched comparison
Standard error under equal variances	Smallest when arm sizes are equal	Larger than the balanced value
Power for the same nonzero effect and test	At least as high under the stated model	Lower than balanced under the stated model
Practical interpretation	More even information in the two means	The smaller arm contributes less precision

The table describes the comparison under equal variances. If the two arms have different variances, the variance of the difference is \(\sigma_1^2/n_1+\sigma_0^2/n_0\), so the allocation minimizing that expression need not be balanced. Likewise, the formulas above do not automatically apply to binary outcomes, clustered observations, covariate-adjusted analyses, or outcome-dependent allocation. Those settings require their own variance and power calculations.

Diagnostics and Common Pitfalls

Before interpreting a numerical power loss, check that the comparison actually holds the intended conditions fixed. Verify that the arm sizes sum to the same \(N\), that the same \(\delta\), \(\sigma\), \(\alpha\), and sidedness are used in both calculations, and that the standard error is computed from the arm-specific sample sizes. Recalculate \(\lambda=\delta/SE\) from the displayed standard error; this catches many arithmetic or transcription errors.

Using the wrong total. A 70:30 plan at \(N=100\) is 70 and 30, not 70 and 70. Confirm that the counts sum to \(N\).
Dropping a tail. A two-sided test has two rejection tails. The term \(\Phi(-\lambda-c)\) may be tiny for a large positive effect, but it belongs in the formula.
Comparing different effects or tests. A power difference does not isolate allocation if the effect size, significance level, or one-sided versus two-sided test also changes.
Treating a planning approximation as guaranteed trial power. The calculation is conditional on the assumed effect and variance. If these are misspecified, achieved power can differ.
Applying equal-variance results without checking variance structure. Under unequal variances, use the appropriate variance formula rather than claiming balance must be optimal.

Common mistake: “The total sample size is unchanged, so power is unchanged.” The correction is that power depends on the standard error, not only on \(N\). Under equal variances, \(SE=\sigma\sqrt{1/n_1+1/n_0}\), which increases as the fixed total is divided more unevenly.

A further limitation is that the fixed-allocation calculation treats \(n_1\) and \(n_0\) as specified counts. In a response-adaptive trial, the final counts may be random and related to observed outcomes. A fixed-count power calculation can serve as a benchmark, but by itself it does not account for allocation uncertainty or establish the operating characteristics of the adaptive procedure.

Check Your Understanding

Use the equal-variance, two-sided normal model unless a question asks you to explain a limitation. Full solutions follow each question.

For fixed \(N=80\), which allocation has the smaller standard error under equal variances: 40:40 or 48:32? Explain using the arm-size product.
With \(n_1=60\), \(n_0=40\), and \(\sigma=3\) units, calculate the standard error for the difference in means.
If the standard error rises while \(\delta\) and \(\alpha\) remain fixed, what happens to \(|\lambda|\) and power under the model?
A colleague compares a balanced plan tested two-sided at \(\alpha=0.05\) with an unequal plan tested one-sided at \(\alpha=0.05\). Can their power difference be attributed solely to allocation? Why?

Solutions.

1. The products are \(40(40)=1600\) and \(48(32)=1536\). Since the total is fixed, the standard error is minimized by the larger product, so 40:40 has the smaller standard error.

2. Substitute the arm sizes and standard deviation:

SE=3\sqrt{\frac{1}{60}+\frac{1}{40}} =3\sqrt{0.041667} \approx0.6124\text{ units}.

4. No. The sidedness differs, so the critical values and rejection regions differ as well. To isolate the effect of allocation, both plans must use the same test and significance level, in addition to the same total sample size, effect, and variance assumptions.