Power Under Response-Adaptive Randomization

Power Is a Property of the Full Design

In a response-adaptive trial, the number assigned to each arm is random: assignments depend on information accumulated during the trial. Power therefore cannot generally be read off from a single fixed sample-size formula. It is the probability that the prespecified analysis rejects the null hypothesis when a specified alternative is true, with that probability taken over the trial’s random assignments and outcomes.

This definition matters because an adaptive design includes both an allocation rule and an analysis rule. To evaluate power, one must use the same allocation rule, stopping rule (if any), estimator, and rejection rule that the trial will use. A calculation conditional on a particular allocation is useful, but it answers a narrower question than the unconditional power of the design.

For a two-arm comparison, let \(N_1\) and \(N_0\) denote the final numbers assigned to arms 1 and 0, with \(N_1+N_0=N\) when the total sample size is fixed. Let the estimand be the mean contrast \(\delta=\mu_1-\mu_0\). Under a two-sided test of \(H_0:\delta=0\), power depends on how often the test statistic crosses either rejection boundary when the true contrast is \(\delta\). In an adaptive design, that probability averages over the possible allocation histories, not just over outcome noise for one chosen pair of sample sizes.

How Allocation Counts Enter Contrast Variance

Start with the familiar two-arm sample-mean contrast \(\hat{\delta}=\bar{X}_1-\bar{X}_0\). For fixed arm counts and independent outcomes with arm-specific variances \(\sigma_1^2\) and \(\sigma_0^2\), its variance is the sum of the two arm-mean variances. If the counts are random, this expression becomes a function of the realized counts.

This conditional formula requires care in an adaptive trial. It holds when, conditional on the counts being considered, the observations used in the contrast have the stated independent outcome variances and the conditional mean of the contrast is \(\delta\). Those conditions are natural for fixed allocation, and can be reasonable for a working approximation or a count mechanism independent of the outcome residuals. They do not automatically hold when the allocation rule uses earlier trial outcomes: conditioning on the final counts can then select histories with different outcome patterns.

Under the stated conditions, if the contrast has conditional mean \(\delta\) for every count pair, the law of total variance gives a direct result. Since \(E(\hat{\delta}\mid N_1,N_0)=\delta\), the variance of this conditional expectation is zero. Thus the unconditional variance is the expected conditional variance:

The reciprocal counts, rather than the reciprocals of the average counts, appear in this expression. When the conditional mean is not constant, the general decomposition also includes variation in that mean:

For outcome-adaptive assignment, even this count-based decomposition may not capture all relevant dependence. The allocation history can matter beyond the final counts. The decomposition remains true, but the conditional variance and conditional mean must describe the actual adaptive data-generating process; they cannot simply be assumed to equal the fixed-count expressions.

From Contrast Variance to Conditional Power

Suppose for illustration that the outcome variances are known and that, conditional on the counts, the contrast is normally distributed with mean \(\delta\) and variance \(\sigma_1^2/N_1+\sigma_0^2/N_0\). A two-sided \(z\)-test at significance level \(\alpha\) rejects when the absolute standardized contrast exceeds \(z_{1-\alpha/2}\). Conditional on the counts, the standardized statistic has mean \(\lambda=\delta/SE\) and variance 1, where \(SE\) is the conditional standard error.

The first tail is rejection in the lower direction and the second is rejection in the upper direction. For a design with several possible allocation outcomes, unconditional power is the average of the conditional rejection probabilities over those outcomes only when those probabilities correctly condition on the relevant allocation history. For outcome-dependent allocation, simulation of the complete trial is often the clearest way to obtain that average.

Averaging Over Random Counts

Even when conditional normal calculations are justified, averaging the conditional powers is not generally equivalent to calculating power from an average variance. Power is a nonlinear function of the standard error. The next example isolates this feature in a count-mixture benchmark. It assumes the count outcome is independent of outcome residuals, so it illustrates the mathematics of random counts rather than claiming that outcome-dependent response adaptation satisfies this assumption.

Evaluating Power for the Actual Adaptive Procedure

When assignment probabilities depend on observed responses, a defensible power evaluation must reproduce that dependence. For each simulated trial, generate outcomes under the chosen alternative, apply the allocation rule using only the information available at each assignment, complete the trial, and apply the prespecified analysis. The estimated power is the fraction of simulated trials in which the analysis rejects \(H_0\). The outcome model, adaptation rule, sample-size rule, and test must all match the proposed design.

Diagnostics, Alternatives, and Limitations

A useful power analysis reports more than one number. Check the distribution of \(N_1\) and \(N_0\), including how often either arm receives few observations, and inspect whether allocation histories with different counts also have different outcome patterns. For simulation, verify that assignment probabilities are updated at the intended times, that every replicate reaches the specified sample size or stopping rule, and that the final test is implemented exactly as planned. Monte Carlo uncertainty should be small enough for the design decision; increase \(M\) if the estimated power is too imprecise.

The main alternatives answer different questions. A fixed-count normal calculation is quick and interpretable for a particular allocation. A count-mixture calculation averages such conditional results when the conditional model is valid and the count distribution is available. Full-design simulation can account for response-dependent allocation and a complicated test, but its accuracy depends on the outcome model and the correctness of the code. These methods are complementary: the simple formula helps explain the role of counts, while simulation evaluates the specified adaptive procedure.

A count-only variance calculation has important limitations. It can miss dependence between allocation and observed outcomes, and an average standard error does not determine the average rejection probability. Moreover, a power estimate is specific to the alternative and analysis used; it does not establish power uniformly over all possible effects. The earlier material on type I error and confidence interval coverage concerns other operating characteristics: high power under an alternative does not by itself establish correct error control or coverage.

Common Mistake

Wrong reasoning: “The adaptive trial is expected to assign 40 participants to arm 1 and 60 to arm 0, so I can calculate power as though those counts were fixed.” That replaces random counts by their averages and ignores both the nonlinear power calculation and the possibility that outcome-dependent allocation changes the distribution of the estimator.

Correction: A fixed-count calculation is conditional on those counts. For random counts, average valid conditional rejection probabilities over the allocation outcomes. When assignments respond to trial outcomes, evaluate the allocation and analysis together—often by simulating complete trials—and report the alternative and assumptions used.