Why Adaptive Designs Can Lose Power

Why More Benefit Need Not Mean More Power

Response-adaptive randomization can assign a larger share of participants to a treatment that appears more promising. If that treatment truly has better outcomes, this allocation can improve the average outcome among trial participants. But the trial also needs information to estimate and test a treatment contrast. With a fixed total sample size, assigning fewer participants to one arm can make that contrast less precise.

The distinction is between two objectives: the outcomes experienced by participants during the trial, and the strength of evidence about a treatment difference. Neither objective automatically determines the other. The calculations below isolate one important mechanism: under a common-variance, two-arm model with a fixed total sample size, imbalance increases the variance of the difference in sample means. For clarity, the power calculations use a known variance and a two-sided normal test. They are benchmarks, not a claim that every response-adaptive trial has this exact conditional distribution.

Why Balance Minimizes the Contrast Variance

Under the model, the sample means are independent, with variances \(\sigma^2/n_1\) and \(\sigma^2/n_2\). Therefore the variance of their difference is

For fixed \(N\), minimizing this variance is equivalent to maximizing \(n_1n_2\). The identity

follows from \((n_1-n_2)^2\geq 0\). Thus the variance is at least \(4\sigma^2/N\). If \(N\) is even, equal allocation attains this bound. If \(N\) is odd, arm sizes are integers, so the minimum variance occurs when the sizes differ by one; the bound \(4\sigma^2/N\) is then strict. For instance, with \(N=5\), the closest allocation is 2 and 3, giving variance \(\sigma^2(1/2+1/3)=5\sigma^2/6\), which exceeds \(4\sigma^2/5\).

This is a result about information for the specified contrast under the stated assumptions. It does not say that equal allocation is best for every trial objective, nor that imbalance always makes a design inferior overall.

From a Larger Standard Error to Lower Power

Let the true mean difference be \(\delta=\mu_1-\mu_2>0\), and let \(SE=\sigma\sqrt{1/n_1+1/n_2}\). Under the alternative, the standardized statistic \(Z=(\bar{x}_1-\bar{x}_2)/SE\) has a normal distribution with variance 1 and mean \(\lambda=\delta/SE\). For the two-sided known-variance test, the rejection rule is \(|Z|>z_{1-\alpha/2}\), where \(z_{1-\alpha/2}\) is the corresponding standard normal quantile. Its power is therefore

where \(\Phi\) is the standard normal cumulative distribution function. For \(\lambda>0\) and \(c=z_{1-\alpha/2}>0\), the derivative of this power with respect to \(\lambda\) is \(\phi(\lambda-c)-\phi(\lambda+c)>0\), because \(|\lambda-c|<\lambda+c\) and the standard normal density \(\phi\) decreases as the absolute value of its argument increases. A larger standard error reduces \(\lambda\), so it reduces power in this model.

Random Arm Sizes and Unconditional Power

In an adaptive trial, final arm sizes can themselves be random. Power for a random-allocation design is the probability of rejection over all sources of randomness in the trial, including the random allocation path and outcomes. A useful benchmark is to calculate conditional power for each possible realized allocation, then average those powers weighted by the probabilities of those allocations. In an actual response-adaptive design, allocation decisions depend on accrued outcomes, so this averaging is not generally exact if one simply inserts realized counts into a fixed-allocation normal formula. The calculation is exact for the simple mixture below, where the allocation pattern is selected independently of the outcomes, and it illustrates how allocation variability can cost power.

Participant Outcomes and Inferential Precision Can Diverge

Suppose, hypothetically, arm 1 has mean outcome \(0.5\) units and arm 2 has mean outcome \(0\), with higher outcomes preferred. A 50/50 design has expected mean outcome across enrolled participants of \(0.5(0.5)+0.5(0)=0.25\) units. A 70/30 allocation favoring arm 1 has expected mean outcome \(0.7(0.5)+0.3(0)=0.35\) units, an increase of \(0.10\) units per participant under these assumed means. Yet, as the preceding example showed for the same effect and variance, the more imbalanced allocation has lower power for the arm contrast.

This arithmetic does not prove that a particular adaptive rule will improve participant outcomes. In practice, the rule must learn which arm is better, and early estimates can be wrong. It does show why the objectives can pull in different directions: assigning more participants to a genuinely better arm can raise expected outcomes, while leaving fewer observations in the other arm can reduce precision for estimating the difference.

Assumptions, Diagnostics, and Alternatives

The variance comparison is a benchmark, not a complete analysis of response-adaptive randomization. Its direct use assumes a fixed total, two arms, independent normal outcomes, a common known variance, and the stated contrast estimator. In a response-adaptive design, allocation probabilities change with observed data; counts and outcomes can consequently be dependent. Conditioning only on the final counts may not recover the fixed-allocation sampling distribution. Valid power assessment should reflect the actual allocation rule and the analysis planned for the trial, including any weighting or variance estimation that the analysis requires.

Useful diagnostics for design evaluation include the distribution of final arm sizes across simulated trials, the frequency of very small arms, and the distribution of the estimated contrast's standard error. Evaluate rejection rates under the null as well as under clinically relevant alternatives: a power estimate is interpretable only alongside an appropriate type I error rate. Also check that the simulated allocation rule, outcome-generating model, stopping rules, and analysis procedure match the intended design. A single calculation at the average arm sizes can hide a substantial tail of trials with severe imbalance.

The main alternative for the precision objective is fixed equal allocation, which maximizes the contrast information under the common-variance assumptions when \(N\) is even. A prespecified fixed unequal allocation is another option when the trial has a reason to favor a particular arm, but under this model it gives less precision for the contrast than the most balanced integer allocation at the same \(N\). Response-adaptive allocation makes a different compromise: the allocation evolves using accruing data and may prioritize participant outcomes as well as inference. Comparing these approaches requires stating the objective and assumptions, rather than ranking designs by power alone.

A practical conclusion is to report both performance dimensions rather than treating one as a proxy for the other. Power describes the probability of rejecting a specified null under a specified alternative and analysis. Expected participant outcome describes the average outcome under the allocation rule and the assumed treatment effects. An adaptive design can improve the latter while reducing the former; whether that tradeoff is worthwhile depends on the trial's goals and operating characteristics.