Unequal Allocation and Post-Selection Estimation

Why Allocation Ratios Matter After Selection

Unequal randomization is often chosen for practical or scientific reasons: a control arm may be smaller, or an experimental arm may receive more participants to improve its precision. But allocation affects more than the standard error of each treatment estimate. It also changes how often an arm ranks first and how far its estimate tends to be pushed upward when it is selected for having a large observed effect. Consequently, a selection adjustment calculated as if the arms had equal sample sizes can be inappropriate.

The calculations here use a normal model at a specified analysis stage. They isolate how unequal allocation affects precision, ranking, and conditional bias. A real MAMS design may select or stop arms across multiple stages; the selection event and covariance structure must then reflect that actual design.

These results use known outcome variances to make the selection calculations explicit. In practice, variances are generally estimated; the displayed standard errors then use the corresponding estimates, and the normal calculations are approximations whose quality should be checked. The allocation ratio alone is not enough to determine precision: outcome variability and the absolute sample sizes also matter.

Standard Errors and the Shared Control

For a treatment-versus-control difference, independent arm sample means give

Increasing \(n_i\) reduces the active-arm contribution to variance; increasing \(n_0\) reduces the control contribution for every active-versus-control comparison. With two active arms using the same control, the estimates are not independent: both contain \(-\bar{X}_0\). In particular,

The shared-control uncertainty cancels when the two estimated treatment effects are subtracted. This matters for ranking: the probability that arm 1 beats arm 2 depends on the variability of their active-arm means, not on the control mean’s sampling variance. Unequal allocation can therefore change both the uncertainty of each treatment-versus-control estimate and the uncertainty of their ranking.

How Allocation Changes the Chance an Arm Ranks First

With two arms, select arm 1 if \(D_1>D_2\). Since the difference is normal under the stated model, the probability of selecting arm 1 is

where \(\Phi\) is the standard normal cumulative distribution function. If the true effects are equal, this probability is \(0.5\), even when the estimates have different variances: their difference is then a mean-zero normal variable. If one effect is larger, allocating more participants to its arm generally reduces the noise in the ranking and can increase its probability of being selected. It does not guarantee selection in any particular trial.

Conditional Bias When an Arm Is Selected

The arm selected for having the larger estimated effect tends to have a positive estimation error. A useful two-arm result quantifies that tendency under the normal model. Write \(e_i=D_i-\theta_i\), \(W=D_1-D_2\), \(s=\sqrt{\operatorname{Var}(W)}\), and \(a=(\theta_1-\theta_2)/s\). Arm 1 is selected when \(W>0\), an event with probability \(\Phi(a)\).

Because \(e_1\) and \(W\) are jointly normal, the conditional mean of \(e_1\) given \(W\) is \(\operatorname{Cov}(e_1,W)(W-(\theta_1-\theta_2))/s^2\). Averaging over the truncated normal distribution \(W>0\) gives the conditional bias:

where \(\phi\) is the standard normal density. For the last equality, \(\operatorname{Cov}(e_1,W)=\operatorname{Var}(e_1)-\operatorname{Cov}(e_1,e_2)=v_1-c\). The ratio \(\phi(a)/\Phi(a)\) is the inverse Mills ratio for selection of arm 1. The formula has units of the outcome: \((v_1-c)/s\) has outcome units, while the ratio is dimensionless.

A first-principles justification uses the standard normal variable \(Z=(W-(\theta_1-\theta_2))/s\). Selection of arm 1 is \(Z>-a\), and direct integration of the normal density yields \(E(Z\mid Z>-a)=\phi(a)/\Phi(a)\). The conditional regression of \(e_1\) on \(W\), followed by this truncated-normal mean, gives the displayed bias. This is a conditional mean error among trials selecting arm 1; subtracting it is a simple conditional bias adjustment, not a guarantee that an individual observed estimate equals the true effect.

Using These Calculations in a MAMS Design

The formulas clarify which quantities must be recalculated when randomization ratios change. For each arm, use its actual sample size and outcome variance in the treatment-effect variance. For ranking, use the variance of the relevant difference and include covariance terms; shared-control estimates are not independent. For a conditional bias calculation, use both the selection probability and the covariance between the selected estimate’s error and the statistic that triggered selection.

In a multi-arm setting, the chance of being selected cannot generally be obtained by applying a two-arm probability independently to every pair. The events “arm 1 beats arm 2” and “arm 1 beats arm 3” are dependent, particularly when estimates share a control. With three or more arms, selection probabilities require the joint distribution and the exact rule, such as choosing the largest estimate or retaining all arms above a threshold. Unequal allocation changes that joint distribution.

At multiple stages, the data used at interim and final analyses are also dependent because the interim observations contribute to later estimates. The relevant covariance must reflect the stagewise sample sizes and the rule that selects or drops arms. A single-stage calculation can be a useful diagnostic or component of an analysis, but it is not automatically the correction for a complete multi-stage selection process.

Diagnostics, Alternatives, and Limitations

Before using a normal-model adjustment, check that the assumptions and inputs describe the analysis actually performed. In particular:

• Verify sample sizes and ratios. Use the numbers randomized or analyzed at the relevant stage, not a nominal target that was not reached.
• Check the shared-control covariance. If arms use different control data or different analysis populations, \(c=\sigma_0^2/n_0\) may not apply.
• Check the selection statistic. The formulas above select by the largest estimated treatment effect. A rule based on a test statistic, a threshold, or several interim outcomes has a different selection event.
• Check the distributional approximation. Strong skewness, very small samples, or variance estimates that are unstable can make the normal calculation unreliable. For binary outcomes, the estimation model and variance differ; the normal-outcome formulas should not be transferred unchanged.

A common mistake is to calculate each treatment-versus-control standard error correctly and then treat the estimates as independent when finding the selection probability. With a shared control, this double-counts control uncertainty in the variance of \(D_1-D_2\). The correction is to include the covariance: \(\operatorname{Var}(D_1-D_2)=v_1+v_2-2c\). Another mistake is to subtract an unconditional bias from every selected estimate. The conditional bias depends on the actual selection event and the allocation-dependent covariance; a generic correction need not be valid.

More complete alternatives include selection-aware likelihood methods, test inversion, or resampling methods that reproduce the design’s selection rule. These approaches can account for more complicated selection pathways, but they require the actual trial design and its joint sampling structure. The simple normal adjustment is most useful when its assumptions are plausible and the selection event is exactly the two-arm ranking described above.