Diagnosing Allocation Imbalance

Why Imbalance Must Be Judged Against a Target

A trial can finish with noticeably different numbers of participants in its arms even when every assignment followed the randomization rule. Conversely, counts that look balanced may be inappropriate if the design intended unequal allocation. The relevant question is not simply whether the arm counts are equal, but whether the realized counts are unusual relative to the allocation probabilities the design actually used.

This distinction matters in adaptive and platform trials, where target probabilities can be unequal, change over time, or be restricted by operational bounds. A diagnosis should identify the probability rule, calculate the allocation it implies, and then compare the observed counts with the variability expected under that rule. Counts alone do not establish that the randomization procedure failed.

Definition: For \(k\) arms, let \(N_j\) be the number assigned to arm \(j\), with \(\sum_{j=1}^k N_j=n\). If the design assigns each participant to arm \(j\) with fixed probability \(\pi_j\), where \(\sum_{j=1}^k \pi_j=1\), the expected count is \(n\pi_j\). The realized allocation proportion is \(\hat{\pi}_j=N_j/n\), and the count deviation from target is \(D_j=N_j-n\pi_j\).

The deviations are constrained by \(\sum_jD_j=0\): an excess in one or more arms must be balanced by deficits elsewhere. That dependence means the arm-specific deviations should not be treated as independent measurements. When the allocation probabilities are fixed and each assignment is an independent random draw from those probabilities, the count vector has a multinomial distribution. This model assumes a fixed total \(n\), a fixed probability vector, and assignments that follow the stated randomization rule independently. If probabilities change with trial history, this fixed-probability model does not describe the whole run.

Measuring Deviations Under Fixed Probabilities

A first diagnostic is to report both the observed counts and proportions alongside their expected values. For each arm, a marginal standardized deviation is

Z_j=\frac{N_j-n\pi_j}{\sqrt{n\pi_j(1-\pi_j)}}.

The denominator is the standard deviation of \(N_j\) under the fixed-probability model. Thus, \(Z_j\) describes the deviation in units of that arm's expected count standard deviation. It is a descriptive diagnostic, not an independent standard normal statistic for each arm: the counts are dependent, and small expected counts can make a normal approximation poor.

A joint summary for a fixed probability vector is the Pearson allocation discrepancy:

X^2=\sum_{j=1}^k \frac{(N_j-n\pi_j)^2}{n\pi_j}.

This measures squared deviations relative to expected counts. Under the multinomial model, its exact finite-sample distribution is discrete. When expected counts are sufficiently large, its distribution is often approximated by a chi-square distribution with \(k-1\) degrees of freedom. For sparse counts or a small trial, exact multinomial calculations or simulation under the specified randomization rule are more dependable than that approximation.

Why the Joint Discrepancy Has \(k-1\) Degrees of Freedom

The Pearson summary has \(k-1\), not \(k\), independent count deviations because the total number randomized is fixed. A short derivation also shows why its expected value under the model is \(k-1\). For a multinomial count, \(E(N_j)=n\pi_j\) and \(\operatorname{Var}(N_j)=n\pi_j(1-\pi_j)\). Therefore,

E(X^2) =\sum_{j=1}^k \frac{E\{(N_j-n\pi_j)^2\}}{n\pi_j} =\sum_{j=1}^k \frac{n\pi_j(1-\pi_j)}{n\pi_j} =\sum_{j=1}^k(1-\pi_j) =k-1.

The final equality uses \(\sum_j\pi_j=1\). This expectation is a useful calibration, not a rule that every run with \(X^2\) above \(k-1\) is suspicious. A single valid randomization can produce a large discrepancy by chance; the relevant reference is the distribution of the statistic under the actual design.

Worked Example: A Two-Arm Trial With Equal Allocation

Hypothetical data. A trial assigns \(n=12\) participants to two arms with fixed probabilities \(\pi_A=\pi_B=0.5\). Suppose the realized counts are \(N_A=8\) and \(N_B=4\). Is the run imbalanced relative to its allocation rule?

The expected counts are \(12(0.5)=6\) in each arm. The deviations are \(D_A=8-6=2\) and \(D_B=4-6=-2\). The observed proportions are \(8/12=0.667\) and \(4/12=0.333\), so the observed difference in proportions is \(0.334\) (using the displayed rounded values).

For either arm, the count standard deviation is \(\sqrt{12(0.5)(0.5)}=\sqrt{3}=1.732\). Thus \(Z_A=2/1.732=1.155\), and \(Z_B=-1.155\). The Pearson discrepancy is

X^2=\frac{(8-6)^2}{6}+\frac{(4-6)^2}{6} =\frac{4}{6}+\frac{4}{6}=1.333.

This is a visible departure from a 6-to-6 split, but it is only 12 participants and the count deviations are about 1.16 marginal standard deviations. The calculation does not show a randomization failure. To judge how often a difference this large occurs, use the exact binomial distribution for \(N_A\sim\operatorname{Binomial}(12,0.5)\), or simulate the allocation rule.

Worked Example: Unequal Allocation Across Three Arms

Hypothetical data. A three-arm trial plans fixed probabilities \((\pi_1,\pi_2,\pi_3)=(0.50,0.30,0.20)\) and randomizes \(n=20\) participants. A run yields counts \((13,4,3)\). Assess the deviation without incorrectly treating equal counts as the target.

The expected counts are \(20(0.50)=10\), \(20(0.30)=6\), and \(20(0.20)=4\). The observed proportions are \(0.650\), \(0.200\), and \(0.150\); the count deviations are \((3,-2,-1)\), which sum to zero.

The arm-specific standard deviations are \(\sqrt{20(0.50)(0.50)}=2.236\), \(\sqrt{20(0.30)(0.70)}=2.049\), and \(\sqrt{20(0.20)(0.80)}=1.789\). The corresponding standardized deviations are \(3/2.236=1.342\), \(-2/2.049=-0.976\), and \(-1/1.789=-0.559\).

The joint discrepancy is

X^2=\frac{3^2}{10}+\frac{(-2)^2}{6}+\frac{(-1)^2}{4} =0.900+0.667+0.250=1.817.

The counts do not match their expected values exactly, but the deviations are modest relative to their marginal standard deviations. In particular, it would be misleading to call the 13-to-4-to-3 allocation a failure merely because it is far from equal allocation: equal allocation was not the design target. With these small expected counts, use the exact multinomial distribution or design-based simulation rather than relying on a chi-square approximation.

Small Samples and Probability Constraints

In a small trial, random variation can make realized proportions depart substantially from their targets. A difference of one or two participants can be a large fraction of the total, and the count distribution is discrete. This is why proportion differences should be read together with expected counts and the allocation rule, rather than interpreted against a universal percentage threshold.

Probability constraints require a second comparison: distinguish the nominal target from the probabilities the implemented rule can actually use. A lower or upper bound can prevent the randomizer from attaining the nominal probability. The constraint changes the allocation mechanism; it does not guarantee that the final counts will equal either the nominal or constrained expected allocation.

Worked Example: A Probability Cap Changes the Effective Target

Hypothetical data. An adaptive rule would like to assign \(90\%\) of participants to arm A and \(10\%\) to arm B, but the randomization system caps either arm's assignment probability at \(0.75\) and requires at least \(0.25\). When the rule requests \(0.90\) for arm A, the implemented probabilities are therefore \(0.75\) and \(0.25\). Suppose \(n=8\) assignments produce counts \(N_A=7\), \(N_B=1\).

Relative to the nominal \(0.90/0.10\) target, expected counts would be \(7.2\) and \(0.8\). Relative to the probabilities actually used, expected counts are \(8(0.75)=6\) and \(8(0.25)=2\). The latter are the appropriate expectations for diagnosing random variation under the implemented rule. The observed proportions, \(0.875\) and \(0.125\), are close to the nominal target but show one more assignment to A than the constrained rule's expected count.

The probability of at least seven assignments to A under the actual fixed probability \(0.75\) is

P(N_A\geq 7) ={8\choose7}(0.75)^7(0.25)+(0.75)^8 =0.3671.

Thus, a count of seven or more in A is not rare under this constrained rule. The example also illustrates why an audit should record both the requested target and the probability actually applied at each assignment. A discrepancy from the nominal target may reflect a constraint, while a discrepancy from the applied probabilities is allocation variability.

Practical Diagnostics Across Trial Runs

When reviewing multiple simulated or completed runs, use the same summary for each run and compare its distribution with the distribution generated by the intended randomization algorithm. Useful summaries include arm counts, allocation proportions, count deviations, and the Pearson discrepancy. For a design with fixed probabilities, exact calculations may be feasible; otherwise, simulate many runs using the same sample size and allocation rule. Report how often a prespecified imbalance criterion is exceeded, rather than selecting a threshold after seeing the results.

If assignment probabilities change in response to trial history, a single fixed multinomial reference is generally inappropriate. The relevant expected count for arm \(j\) is the sum across assignments of the expected assignment probabilities, \(E(N_j)=\sum_{i=1}^n E(q_{ij})\), where \(q_{ij}\) is the probability of assignment to arm \(j\) at assignment \(i\). But this expectation alone does not specify the distribution of the final count. Simulate complete trial runs under the specified rule, preserving its probability updates and constraints, and compare the observed run with that design-based distribution.

A useful diagnostic workflow is to verify the probabilities that were actually applied, compute counts and deviations against the matching target, then compare the resulting summaries with exact or simulated reference runs. If discrepancies recur, inspect whether the cause is small expected counts, a mismatch between nominal and implemented probabilities, or an implementation error. For adaptive rules, examine the assignment records and algorithm outputs; a count-level summary cannot reveal which of these explanations applies by itself.

Identify the applicable target.
Write down the allocation probabilities used by the randomizer, distinguishing fixed probabilities, nominal targets, and constrained probabilities.

Calculate the realized discrepancy.
Report \(N_j\), \(N_j/n\), \(n\pi_j\) when probabilities are fixed, and a joint summary such as \(X^2\) where appropriate.

Use a matching reference distribution.
Choose exact calculation or simulation for small samples, sparse expected counts, or history-dependent probabilities; do not apply the fixed-probability approximation without its assumptions.

Investigate patterns, not just one count.
Across runs, look for systematic departures from the design-based reference and verify the randomization records before attributing a cause.

Alternatives, Limitations, and Common Pitfalls

Raw count differences are easiest to communicate but are sensitive to sample size and do not account for unequal targets. Proportion differences put arms on a common scale but still need a reference distribution. Standardized deviations account for each arm's marginal variance, while the Pearson discrepancy summarizes the joint departures. Exact binomial or multinomial calculations retain finite-sample discreteness under fixed probabilities. Simulation is more flexible and can reproduce probability bounds or adaptive updating, provided it faithfully implements the full randomization algorithm.

None of these summaries establishes that an arm is clinically better, that an adaptive rule made appropriate decisions, or that treatment-effect estimates are unbiased. They diagnose the allocation pattern relative to a specified mechanism. A single unusual run may occur under a correctly implemented design, while repeated departures across runs can signal a coding, data-capture, or specification problem that deserves investigation.

Mistake: treating equal counts as the default benchmark. That is wrong when the design specifies unequal probabilities. Compare each count with \(n\pi_j\), not with \(n/k\).
Mistake: comparing counts only with a nominal target. If a probability cap or floor changed the probability used, the constrained probability is needed to assess random variation; report the nominal target separately to describe the constraint's effect.
Mistake: treating a large deviation in one run as proof of failure. Random allocation produces variation. Compare with an exact or design-based reference and check the assignment records before concluding that the procedure malfunctioned.
Mistake: using the fixed multinomial model after probabilities adapt. When assignment probabilities depend on trial history, simulate the evolving rule rather than treating all assignments as independent draws from one fixed vector.

Key takeaway: Allocation imbalance is a discrepancy from a specified probability rule, not simply unequal arm counts. Interpret that discrepancy using the probabilities actually applied, the sample size, and a reference distribution that matches the design.

Check Your Understanding

Try each question, then compare your reasoning with the solutions below.

A fixed two-arm rule assigns with probabilities \(0.60\) and \(0.40\), and \(n=10\). What are the expected counts?
For \(n=10\) and \(\pi_A=0.60\), a hypothetical run has \(N_A=8\). Calculate \(D_A\) and \(Z_A\), rounding \(Z_A\) to three significant figures.
Why is an equal-count benchmark inappropriate for fixed three-arm probabilities \((0.50,0.30,0.20)\)?
A history-dependent rule changes assignment probabilities during the trial. Why is a single fixed multinomial reference generally unsuitable?

Solutions. (1) The expected counts are \(10(0.60)=6\) and \(10(0.40)=4\). (2) \(D_A=8-6=2\). The standard deviation is \(\sqrt{10(0.60)(0.40)}=\sqrt{2.4}=1.549\), so \(Z_A=2/1.549=1.29\). (3) The target counts are unequal: for a fixed total \(n\), their expectations are \(0.50n\), \(0.30n\), and \(0.20n\), not \(n/3\) each. (4) Assignment probabilities are not constant across assignments, so the count distribution is not generally multinomial with one fixed probability vector; simulate the complete adaptive rule or use a suitable sequential reference.