Selection Rules For Unequal Arm Sizes

Why unequal arm sizes change interim comparisons

At an interim analysis, candidate treatments may have accrued different numbers of participants. Their estimated outcomes can still be compared on the same clinical scale, but the estimates do not generally have the same precision. A mean based on a smaller arm tends to vary more from one sample to another than a mean based on a larger arm, even when both arms have the same underlying outcome variability.

This creates two distinct questions. Which treatment has the largest estimated benefit? And which has the strongest evidence of benefit, given the uncertainty in its estimate? A rule that answers the first question by ranking raw estimates is not interchangeable with a rule that answers the second by incorporating standard errors. The intended selection objective should determine the rule.

For clarity, the comparisons here concern independently sampled candidate arms with no shared control observations. That restriction makes the variance calculations direct; shared-control comparisons require accounting for the dependence induced by the common group. The examples use hypothetical data only.

Model and standard errors with unequal sizes

Suppose outcomes in arm \(i\) are independent observations with mean \(\mu_i\) and variance \(\sigma_i^2\). Then \(\bar{x}_i\) estimates \(\mu_i\), and \( \operatorname{Var}(\bar{x}_i)=\sigma_i^2/n_i \). If the arms are independent, the estimated contrast between arms \(i\) and \(j\), \(\bar{x}_i-\bar{x}_j\), has variance \( \sigma_i^2/n_i+\sigma_j^2/n_j \). The corresponding standard error is the square root of that variance. These formulas require independent observations within and between arms, and an appropriate variance model. If all arms share a common variance \(\sigma^2\), the contrast variance simplifies to \(\sigma^2(1/n_i+1/n_j)\).

In practice, \(\sigma_i\) is usually unknown. For a single arm mean, a usual estimated standard error is \(s_i/\sqrt{n_i}\), where \(s_i\) is the arm-specific sample standard deviation. For a contrast with potentially unequal variances, use \( \sqrt{s_i^2/n_i+s_j^2/n_j} \). If a common-variance model is justified, a pooled variance estimate can be used instead. The choice is part of the model: unequal sample sizes do not by themselves imply equal or unequal outcome variances.

Choosing what the selection rule should reward

The raw-effect rule ranks the observed treatment means directly. It is appropriate when the interim objective is to retain the arm with the largest observed outcome on the clinical scale. It does not adjust for unequal precision, however: a smaller arm's mean can move around more, so it can reach the top of the ranking by chance more readily.

A precision-standardized score compares an estimate with a benchmark in units of its standard error. For a common benchmark \(\mu_0\), one possible score is \( (\bar{x}_i-\mu_0)/SE_i \). This score measures how strongly the estimate exceeds the benchmark relative to its uncertainty. It does not generally rank arms by their estimated clinical benefit. An arm with a smaller mean but a much smaller standard error can have the larger standardized score.

Another option is a lower-bound score. For a chosen multiplier \(c>0\), define \(L_i=\bar{x}_i-cSE_i\). This penalizes imprecision on the outcome scale and can be used when the objective is a conservative estimate of benefit, rather than the highest point estimate. A common choice \(c=1.645\) corresponds to the one-sided 95% normal quantile under a normal approximation. As a selection score, \(L_i\) is not by itself a confirmatory confidence statement or a guarantee of a particular error rate.

Allocation and precision: a short derivation

For two independent arms with a common variance \(\sigma^2\) and fixed total sample size \(N=n_1+n_2\), the variance of the difference in means is \( \sigma^2(1/n_1+1/n_2) \). The allocation that minimizes this variance is the one that maximizes \(n_1n_2\), since \(1/n_1+1/n_2=N/(n_1n_2)\). With continuous allocations, the identity \((n_1-n_2)^2\geq 0\) implies \(n_1n_2\leq N^2/4\), with equality when both sizes are \(N/2\). If sample sizes must be integers, the variance is minimized by the most balanced feasible allocation: the two sizes differ by at most one. Equal allocation is not literally possible when \(N\) is odd.

Binary outcomes, diagnostics, and limitations

For an independent binary outcome, each participant's outcome can be modeled as Bernoulli with event probability \(p_i\) in arm \(i\). The observed proportion \(\hat{p}_i\) estimates \(p_i\), and its model-based variance is \(p_i(1-p_i)/n_i\). Replacing the unknown \(p_i\) by \(\hat{p}_i\) gives the usual estimated standard error. The normal approximation is most credible when the arm has enough events and non-events; very small counts or proportions near zero or one can make a normal-based precision score unstable or misleading.

At an interim review, check whether the actual arm sizes and variance estimates used in the rule match the data available at that review. Investigate unexpectedly small or large standard errors, sparse binary outcomes, substantial missingness, and departures from the assumed independence or variance structure. If outcome variability differs across arms, a common-variance standard error may misstate precision. If sample sizes differ because of informative withdrawal or other selection processes, the simple formulas do not correct the resulting bias.

The main alternatives are not merely different formulas; they encode different aims. Raw means target the largest observed outcome. Standardized scores target strength of evidence relative to a specified benchmark. Lower-bound scores target a conservative precision-adjusted value on the outcome scale. Allocation changes can improve precision, but unequal sample sizes may still be justified by recruitment, safety, cost, or scientific priorities. No precision adjustment removes the need to prespecify the rule, the variance model, and the outcome direction.

Common pitfall

A common mistake is to say, “The arm with the highest interim mean is best,” even when its estimate is much less precise, or to replace that statement with “The highest standardized score identifies the largest treatment effect.” Neither conclusion follows. A standardized score is measured relative to its standard error and benchmark, not in the original outcome units. State the rule's objective explicitly, calculate the arm-specific precision using the actual arm sizes, and report what the resulting ranking does and does not mean. A second pitfall is claiming that equal allocation always means exactly \(N/2\) participants per arm; with odd integer \(N\), the closest feasible allocation has sizes differing by one.