Allocation Imbalance and Effective Sample Size

Why Allocation Counts Affect Contrast Precision

In a two-arm trial, the treatment contrast is typically estimated by subtracting the mean outcome in one arm from the mean outcome in the other. Even when the total number of participants is fixed, the split between arms matters: a mean based on fewer observations is generally less precise. Effective sample size provides a compact way to express the precision cost of an unequal split in units of an equivalent equal allocation.

The measure developed here isolates allocation imbalance under a specific benchmark: both arms have the same outcome variance, the arm sizes are fixed, and the contrast is the difference of unweighted arm means. It is a useful design diagnostic, but it is not automatically the effective sample size of every estimator used in an adaptive trial.

Definition: Let $n_1$ and $n_0$ be the numbers assigned to arms 1 and 0, with fixed total $N=n_1+n_0$ and both arm sizes positive. Under a common outcome variance, the total effective sample size for allocation imbalance is $$ N_{\mathrm{eff}}=\frac{4n_1n_0}{n_1+n_0}. $$ It is the total size of an equally allocated two-arm design whose variance for the difference of arm means matches the variance for the observed allocation, under the equal-variance model.

The subscript $0$ here labels an arm; it is not a reference to a null hypothesis. If either arm has no participants, the contrast cannot be estimated from both arm means and the formula gives zero. In this tutorial, “effective sample size” refers to the total-size equivalent just defined. Other settings may use different effective-sample-size definitions, so the comparison being made should always be stated.

Deriving the Variance Connection

Suppose outcomes in each arm are independent, have the same variance $\sigma^2$, and the arm counts are fixed. The variance of the unweighted difference in sample means is the sum of the two mean variances. For a balanced allocation with the same total $N$, each arm would have $N/2$ participants.

\operatorname{Var}(\bar{X}_1-\bar{X}_0) =\frac{\sigma^2}{n_1}+\frac{\sigma^2}{n_0} =\sigma^2\frac{N}{n_1n_0}.

An equal allocation of total size $m$ has $m/2$ observations per arm, so its contrast variance under this model is $4\sigma^2/m$. Equating that variance to the variance at the observed allocation gives the effective size:

\frac{4\sigma^2}{m}=\sigma^2\frac{N}{n_1n_0} \quad\Longrightarrow\quad m=\frac{4n_1n_0}{N} =N_{\mathrm{eff}}.

This comparison also gives a direct measure of the variance penalty relative to a balanced allocation of the same total size. The standard error ratio is the square root of that variance ratio.

\frac{\operatorname{Var}(\bar{X}_1-\bar{X}_0)} {\operatorname{Var}_{\mathrm{balanced}}(\bar{X}_1-\bar{X}_0)} =\frac{N}{N_{\mathrm{eff}}} =\frac{N^2}{4n_1n_0}, \qquad \frac{SE_{\mathrm{observed}}}{SE_{\mathrm{balanced}}} =\sqrt{\frac{N}{N_{\mathrm{eff}}}}.

Writing the allocation fraction in arm 1 as $q=n_1/N$ makes the relationship especially clear: $N_{\mathrm{eff}}/N=4q(1-q)$. Since $(n_1-n_0)^2\geq 0$, we have $4n_1n_0\leq(n_1+n_0)^2$; thus $N_{\mathrm{eff}}\leq N$, with equality only for equal allocation. The effective size falls as the split becomes more unequal, and reaches zero at the boundary where one arm has no observations.

Calculating the Effective Size

Worked Example: An 80-to-20 Allocation

Consider a hypothetical trial with 80 participants in arm 1 and 20 in arm 0. The total is $N=80+20=100$. Calculate the effective size and the variance penalty relative to a balanced allocation of 100.

Substitute the counts into the definition:

N_{\mathrm{eff}} =\frac{4(80)(20)}{80+20} =\frac{6{,}400}{100} =64.

Thus, under the equal-variance model, this 100-person allocation has the same contrast variance as an equally allocated design with 64 participants. It does not mean that 36 participants’ outcomes are discarded; it is a variance-equivalence statement.

The variance ratio is $100/64=1.5625$, so the contrast variance is 1.5625 times the variance under a balanced allocation of 100. The standard error ratio is $\sqrt{1.5625}=1.25$. In other words, under the stated model, the standard error is 25% larger than in the balanced comparison.

The allocation ratio alone is not enough to give the effective size: the total also matters. But for a fixed total, the allocation fraction determines the proportion of nominal sample size retained as effective size. This permits comparisons across designs with the same total or, with care, across different totals.

Worked Example: Comparing Two Allocation Fractions

Two hypothetical designs each plan to enroll 120 participants. Design A allocates 60 to each arm; Design B allocates 90 to arm 1 and 30 to arm 0. Compute the effective size and compare their contrast variances under a common outcome variance $\sigma^2$.

For Design A:

N_{\mathrm{eff,A}} =\frac{4(60)(60)}{120} =120, \qquad \operatorname{Var}_A(\bar{X}_1-\bar{X}_0) =\sigma^2\left(\frac{1}{60}+\frac{1}{60}\right) =\frac{\sigma^2}{30}.

For Design B:

N_{\mathrm{eff,B}} =\frac{4(90)(30)}{120} =90, \qquad \operatorname{Var}_B(\bar{X}_1-\bar{X}_0) =\sigma^2\left(\frac{1}{90}+\frac{1}{30}\right) =\frac{4\sigma^2}{90} =\frac{2\sigma^2}{45}.

The variance ratio is $(2\sigma^2/45)/(\sigma^2/30)=4/3$, which agrees with $120/90=4/3$. The standard error in Design B is therefore $\sqrt{4/3}$, or about 1.155, times the standard error in Design A. These calculations compare precision under the specified common-variance model; they do not say which design is preferable on patient-benefit or other grounds.

Random Allocation Counts in an Adaptive Trial

In response-adaptive randomization, arm counts may be random because assignments change as accumulating information changes the allocation probabilities. A realized effective size can be calculated from the realized counts, but it is not generally interchangeable with an effective size calculated from average counts. The reciprocal arm counts appear in the variance formula, so averaging counts first can conceal variability in the allocation split.

The following calculation isolates this distinction using a hypothetical random allocation mechanism independent of outcomes. Suppose the total is always 100, with counts 50 and 50 with probability one-half, or 70 and 30 with probability one-half. Conditional on the counts, assume independent outcomes with a common variance $\sigma^2$. These assumptions let us use the fixed-count variance formula within each allocation.

Worked Example: Averaging Across Random Splits

For a 50-to-50 split, the realized effective size is $4(50)(50)/100=100$, and the conditional contrast variance is $0.0400\sigma^2$. For a 70-to-30 split, the effective size is $4(70)(30)/100=84$, and the conditional variance is:

\sigma^2\left(\frac{1}{70}+\frac{1}{30}\right) =\sigma^2\frac{100}{2{,}100} =0.04762\sigma^2.

The average of the two realized effective sizes is $(100+84)/2=92$. But the average conditional variance is:

\frac{1}{2}(0.0400\sigma^2+0.04762\sigma^2) =0.04381\sigma^2.

The single equal-allocation total with this same variance has variance $4\sigma^2/m$. Equating it to the average conditional variance gives $m=4/0.04381$, approximately 91.30. Thus the equal-allocation equivalent of the average conditional variance is about 91.30, not the average realized effective size of 92. The difference arises because the variance depends on the reciprocal counts, not linearly on the effective size.

This example is a conditional-variance illustration, not a general formula for an adaptive estimator’s unconditional variance. If allocation probabilities depend on earlier outcomes, the counts and observed outcomes can be dependent. The variance of an estimator may then involve features of the adaptive assignment and estimation procedure that are not captured by the counts alone. The earlier material on variance estimation for adaptive treatment contrasts is relevant to that broader inferential problem.

When Equal-Variance Effective Size Is Not Enough

If the outcome variances differ between arms, the contrast variance for fixed counts is instead:

\operatorname{Var}(\bar{X}_1-\bar{X}_0) =\frac{\sigma_1^2}{n_1}+\frac{\sigma_0^2}{n_0}.

The count-based $N_{\mathrm{eff}}$ still describes how unequal the allocation is, but it no longer determines the contrast variance by itself. For example, allocating fewer participants to the arm with the larger outcome variance can be more costly than the count-only measure suggests. Conversely, a split that is unequal by count may be less damaging when the smaller arm has lower outcome variance. Any variance-equivalence calculation must therefore state its reference model and variance assumptions.

There is also a difference between an allocation diagnostic and an estimator-specific effective sample size. The formula above is derived for the unweighted difference in arm means under fixed counts and a common variance. In an adaptive trial, inverse-probability weighted or stabilized estimators use weights as well as counts. Their variance depends on the weights and assignment probabilities, not solely on $n_1$ and $n_0$. A count-based effective size should not be presented as the exact precision of such an estimator without a separate justification.

Diagnostics, Alternatives, and Limitations

A useful diagnostic is to report the actual arm counts, the allocation fraction, and $N_{\mathrm{eff}}/N$ together. These values reveal whether a small effective size comes from a highly unequal split or simply a small total. In an adaptive design, examine these quantities across simulated trials or repeated design scenarios rather than relying only on a single average split. Also check whether either arm is sometimes very small or empty; a summary based on average counts can obscure those failures.

The main alternative is to report the contrast variance or standard error directly using the variance model and estimator actually used. That is more informative when arm variances differ or the estimator is weighted, but it requires more modeling or estimation choices. For the simple equal-variance, unweighted comparison, effective size is a compact, interpretable summary. It is not a replacement for uncertainty estimates, and it says nothing by itself about bias, interval coverage, or the patient outcomes produced by the allocation rule.

A further limitation is that matching variances does not make two designs equivalent in every respect. They can differ in the number of observations per arm, in the reliability of arm-specific variance estimates, and in how often extreme allocations occur. Effective size compresses the allocation into one number and necessarily loses those details.

Common pitfall: “The trial enrolled 100 participants, so it has an effective sample size of 100.” This is false for an unequal allocation under the equal-variance contrast benchmark. For 80 and 20 participants, the effective size is 64, because $4(80)(20)/100=64$. The correction is to calculate the size from both arm counts, then state the model under which the equal-allocation variance comparison applies.

Key takeaway: For fixed, positive arm counts and a common outcome variance, $N_{\mathrm{eff}}=4n_1n_0/N$ converts allocation imbalance into an equal-allocation total-size equivalent. Its variance interpretation is exact for the stated contrast and assumptions; adaptive dependence, unequal variances, and weighted estimators require additional analysis.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

A hypothetical fixed-allocation trial has 45 participants in arm 1 and 15 in arm 0. Find $N_{\mathrm{eff}}$ and the variance ratio relative to a balanced allocation of 60, assuming a common variance.
For $N=100$, what allocation fraction $q$ gives $N_{\mathrm{eff}}=75$? Give both possible arm splits.
Why is the average of realized effective sizes not necessarily the equal-allocation equivalent of the average conditional variance across random counts?
A researcher reports $N_{\mathrm{eff}}=96$ for a 60-to-40 allocation. Is that value correct under the definition in this tutorial? Explain.

Solutions. (1) $N_{\mathrm{eff}}=4(45)(15)/60=45$. The variance ratio is $60/45=1.333$, so the variance is one-third larger than for a balanced allocation of 60. (2) Solve $4q(1-q)=75/100=0.75$. Then $q(1-q)=0.1875$, giving $q=0.25$ or $q=0.75$; the splits are 25-to-75 and 75-to-25. (3) Conditional variance depends on $1/n_1+1/n_0$, which is nonlinear in the counts. Averaging counts or realized effective sizes therefore need not preserve the average variance. (4) Yes. $4(60)(40)/(60+40)=9{,}600/100=96$. This is the total equal-allocation equivalent under the common-variance benchmark, not a universal effective size for every estimator.