Allocation Uncertainty And Sampling Variance

Why Final Arm Sizes Matter

A trial can plan to assign participants equally to two arms without ending with exactly equal arm sizes. If each participant is independently assigned to treatment with probability one half, the final treatment count is random, and the control count is its complement. That variation matters because a mean calculated from a smaller arm is generally less precise than a mean calculated from a larger one.

The issue is separate from whether the treatments have different effects or whether adaptive allocation benefits participants. Here the question is narrower: how does uncertainty in the final arm sizes affect the sampling variance of a difference in means? We first describe the outcome variation for a given allocation, then average that variance over possible allocations.

Variance Given the Arm Sizes

For a simple model, suppose each participant’s outcome in arm \(a\) has mean \(\mu_a\) and variance \(\sigma_a^2\), for \(a=0,1\). Assume outcomes are independent within and between arms, and that the allocation mechanism is independent of the outcomes. Conditional on the arm sizes, the arm means have variances \(\sigma_1^2/n_1\) and \(\sigma_0^2/n_0\). Their independence gives

This is the conditional sampling variance: it describes outcome variation when the arm sizes are held fixed. If the sizes are themselves random, the variance relevant across repeated trials is the average of this expression over the allocation mechanism, provided the estimator’s conditional expectation does not vary with the allocation:

The final equality follows because \(E[\hat{\tau}\mid n_1,n_0]=\mu_1-\mu_0\) for every allocation with both arms represented. Thus the second term is zero under this model. Allocation uncertainty increases variance through the random denominators; it does not add a separate variance term for the arm counts themselves. The formula requires both arms to be nonempty. A design that can produce an empty arm needs separate handling because the difference in arm means is then undefined.

Why Variability in the Counts Usually Raises Variance

There is a simple general comparison. Suppose the possible allocations all have positive arm sizes, and let \(E[n_1]=m_1\) and \(E[n_0]=m_0\). The function \(f(x)=1/x\) is convex for positive \(x\), since \(f''(x)=2/x^3>0\). Jensen’s inequality therefore gives

Multiplying by the nonnegative outcome variances and adding yields

The right side is the variance for fixed arm sizes equal to the expected sizes, interpreted as a benchmark when those sizes are feasible. The inequality is strict if a random arm size with positive variance affects a term with positive outcome variance. This explains why simply substituting expected counts into the fixed-allocation formula tends to understate the average variance. It is a comparison under the stated model, not a claim that every realized variable allocation has larger variance than every fixed allocation.

Using the Realized Allocation in a Trial

Once a trial is complete, the arm sizes are known. For estimating precision from the observed data, replace the population variances in the conditional formula with the arm-specific sample variances \(s_1^2\) and \(s_0^2\). The resulting estimated standard error describes uncertainty conditional on the realized allocation. It is useful for reporting the estimate, but it does not by itself describe how much that standard error could have varied across repetitions of the allocation procedure.

Assumptions, Diagnostics, and Alternatives

The calculation is most directly justified when the total sample size is fixed, both arms are represented, assignments are independent of outcomes, and the outcome model has independent observations with stable arm-specific means and variances. If assignments are made with fixed arm sizes, there is no final-count uncertainty: use the conditional variance at those specified sizes. If assignments are independent with a constant treatment probability, the count varies, and the expectation of the conditional variance is the relevant quantity under this model.

In practice, check the observed arm sizes and the variance estimates in each arm. Very small arms make the reciprocal-count terms large and the sample variances unstable; an empty arm makes the difference in means unavailable. For a specified allocation rule, calculate or simulate the distribution of the arm counts and average \(\sigma_1^2/n_1+\sigma_0^2/n_0\) over allocations where the estimator is defined. Do not approximate this average merely by plugging expected arm sizes into the formula when the counts vary substantially.

This calculation also has limits. It does not account for correlations between participants, changes in outcome distributions over time, or complications from an allocation rule that depends on accumulated outcomes. In those settings, the simple independence model may not describe the estimator’s repeated-sampling variance. The formula isolates the contribution of arm-size variation; it is not a complete analysis of every response-adaptive design.

The main alternatives answer different design questions. Fixed allocation gives a direct variance at specified arm sizes and avoids uncertainty in the final counts. A random-allocation design can instead be assessed by averaging its conditional variance across possible counts. When the actual trial has finished, a conditional standard error uses the realized counts. These quantities should not be conflated: the first two compare design-level precision, while the last describes estimated precision for the observed allocation.