Sensitivity to Unequal Outcome Variances

Why Unequal Variances Matter

A response-adaptive rule may assign more patients to the arm with the higher response rate. That can be attractive when the goal is to give more patients access to an arm that appears more beneficial. But the response-rate target does not necessarily minimize the uncertainty of an estimated treatment contrast. Precision also depends on how variable the outcomes are within each arm. If the more variable arm receives too few patients, its contribution to the variance of the contrast can be large.

To isolate this issue, first consider a setting where the rule targets arm-specific response probabilities, while the primary endpoint is a quantitative outcome. The response probabilities determine the intended allocation; the quantitative endpoint has arm-specific means and variances. These quantities need not be the same. A second example considers a binary endpoint that is itself the response used by the allocation rule, where response probabilities and outcome variances are linked.

A realized adaptive allocation is random, so \(q\) need not equal its target exactly. The comparisons below first treat the arm counts as fixed at their planned or realized values. They describe the variance conditional on those counts, under independent observations within and across arms, finite arm-specific variances, and an estimator for which the usual arm-mean variance calculation is appropriate. For an adaptive trial, valid inference must also account for the assignment process where required; these calculations isolate the effect of unequal outcome variances rather than replacing the earlier material on adaptive inference.

Variance of the Treatment Contrast

For a difference in sample means, \(\bar{Y}_1-\bar{Y}_0\), independent observations give the conditional variance

Thus, allocation balance alone does not determine precision. A smaller arm count is more costly when that arm also has a large outcome variance. With equal variances, equal allocation minimizes the variance. With unequal variances, the minimizing allocation generally gives more patients to the arm with the larger standard deviation.

The result assumes that both arms receive a positive fraction of the sample. In practice, allocation constraints may prevent attaining this fraction exactly. It is a precision benchmark, not a claim that variance minimization should override patient benefit, ethical constraints, or the design’s scientific goals.

Why the Variance-Optimal Fraction Has This Form

For fixed \(N\), minimizing the contrast variance is equivalent to minimizing \(g(q)=\sigma_1^2/q+\sigma_0^2/(1-q)\) over \(0<q<1\). Differentiating gives \(g'(q)=-\sigma_1^2/q^2+\sigma_0^2/(1-q)^2\). At an interior minimum, setting the derivative to zero and using positive standard deviations yields \(\sigma_1/q=\sigma_0/(1-q)\). Rearranging gives \(q=\sigma_1/(\sigma_0+\sigma_1)\). The second derivative is \(g''(q)=2\sigma_1^2/q^3+2\sigma_0^2/(1-q)^3>0\), so this stationary point is the unique minimum.

Consequently, a response-rate-proportional target is variance-optimal for this contrast only when

assuming positive response probabilities and standard deviations. If those ratios differ, the response-rate target and the precision target differ. This is not automatically a design error: they optimize different objectives.

When the Targets Do Not Agree

A mismatch can leave the high-variance arm relatively undersampled. The resulting increase in standard error reduces the separation between the expected test statistic under a nonzero effect and its rejection threshold. Under a normal approximation for a two-sided level-\(\alpha\) test, the noncentrality parameter is approximately \(\lvert\mu_1-\mu_0\rvert/SE\). A larger standard error therefore generally means lower power at the same effect size and sample size. This is an approximation; finite-sample power depends on the test and its reference distribution.

Binary Outcomes: Variance Can Be Tied to Response

If the endpoint is a binary response \(Y\), then within arm \(a\), \(\mu_a=p_a\) and \(\sigma_a^2=p_a(1-p_a)\). The response rate and variance are no longer separate inputs. However, targeting allocation in proportion to \(p_a\) still does not generally minimize the variance of the difference in response proportions. The variance-optimal fraction is proportional to the standard deviations \(\sqrt{p_a(1-p_a)}\), not to \(p_a\). For example, the response rate can be high while its Bernoulli variance is modest; variance is largest at a response probability of one half.

What to Compare and What to Check

Unequal variances should be evaluated alongside allocation, not treated as a standalone property. A useful analysis reports the arm counts, the estimated arm-specific variances, the resulting standard error, and the intended allocation objective. For a difference in means, a heteroscedastic standard error uses the separate arm variances, \(s_1^2/N_1+s_0^2/N_0\); it does not pool the variances as if they were equal. A Welch-type test can use this standard error with an approximate degrees-of-freedom adjustment. In an adaptive trial, the design and analysis must still address the adaptive assignment mechanism.

Several diagnostics help identify a consequential mismatch:

• Compare \(q_{\text{resp}}\) with \(q_{\text{var}}\), rather than judging imbalance from the arm counts alone.
• Calculate the contrast variance at the response target and at relevant alternatives, keeping the total sample size fixed.
• Check whether a small arm is also the arm with the larger endpoint variance; this combination can inflate uncertainty.
• Assess whether variance estimates are stable enough to support the comparison. Highly uncertain estimates make a plug-in variance-optimal target unreliable.
• When actual adaptive counts vary, examine operating characteristics across possible allocation paths rather than relying only on one expected allocation fraction.

A common mistake is to say, “The response rate is higher in arm 1, so allocating more patients there must give the most powerful test.” This confuses a potential patient-benefit objective with a precision objective. The correction is to calculate the contrast variance using both arm-specific variances and counts, then compare the resulting power under the scientific assumptions of interest.

Another mistake is to assume that unequal variances automatically invalidate a comparison. They do not. They change the appropriate standard error and can make a particular allocation inefficient; inference can still be appropriate when the analysis accommodates heteroscedasticity and the adaptive design. Conversely, merely reporting a variance-robust test does not remove the precision cost of an allocation that leaves a noisy arm small.