Lagrange Multiplier Derivation for Multiple Arms

Why a Lagrange Multiplier Helps

With several treatment arms, the allocation problem is not simply to divide a fixed sample equally. If outcome variability differs by arm, assigning more participants to a more variable arm can reduce the uncertainty of the arm means being estimated. The key question is how to make that division optimally under a fixed total sample size.

The multiarm Neyman allocation gives a concise answer: when the objective is to minimize the sum of the variances of the arm-specific sample means, and there are no unequal recruitment costs or other constraints, the sample size in each arm is proportional to that arm’s outcome standard deviation. A Lagrange multiplier derives this result directly and shows why the standard deviation, rather than the variance, determines the allocation ratio.

Model and Assumptions

Consider \(K\) treatment arms. Arm \(i\) has sample size \(n_i\), population outcome standard deviation \(\sigma_i\), and sample mean \(\bar{x}_i\), for \(i=1,\ldots,K\). The total sample size is fixed at \(N\). For the derivation, treat each \(\sigma_i\) as known and strictly positive.

Assume participants’ outcomes are independent within and across arms, and that observations within arm \(i\) come from a population with finite variance \(\sigma_i^2\). Then the variance of the arm mean is \(\operatorname{Var}(\bar{x}_i)=\sigma_i^2/n_i\). The allocation problem studied here minimizes the sum of these arm-mean variances:

V(n_1,\ldots,n_K)=\sum_{i=1}^{K}\frac{\sigma_i^2}{n_i}, \qquad \sum_{i=1}^{K}n_i=N.

This objective also yields the same allocation as minimizing the variance of an equally weighted average of the \(K\) arm means, because that variance is \(V/K^2\), a constant multiple of \(V\). The objective is not automatically the right one for every scientific question: it represents equal emphasis on the arm-mean variances. A particular contrast or a different set of constraints can require a different objective.

Definition: Under a fixed total sample size and strictly positive arm standard deviations, the multiarm Neyman allocation is the continuous allocation that minimizes \(\sum_{i=1}^{K}\sigma_i^2/n_i\), subject to \(\sum_{i=1}^{K}n_i=N\). Its target for arm \(i\) is \(n_i=N\sigma_i/\sum_{j=1}^{K}\sigma_j\).

Lagrange Multiplier Derivation

The constraint couples the arm sample sizes: increasing one requires reducing at least one other. A Lagrange multiplier incorporates this fixed-total constraint into the objective. Write the Lagrangian as

\mathcal{L}(n_1,\ldots,n_K,\lambda) = \sum_{i=1}^{K}\frac{\sigma_i^2}{n_i} +\lambda\left(\sum_{i=1}^{K}n_i-N\right).

At an interior optimum, the partial derivative with respect to each \(n_i\) is zero. For arm \(i\), this gives

\frac{\partial\mathcal{L}}{\partial n_i} = -\frac{\sigma_i^2}{n_i^2}+\lambda=0.

Thus \(\lambda=\sigma_i^2/n_i^2\), so \(n_i=\sigma_i/\sqrt{\lambda}\), taking the positive root because sample sizes are positive. Summing over the arms and imposing the total \(N\) gives

N=\sum_{i=1}^{K}n_i =\frac{1}{\sqrt{\lambda}}\sum_{i=1}^{K}\sigma_i, \qquad \sqrt{\lambda}=\frac{\sum_{j=1}^{K}\sigma_j}{N}.

Substitution yields the optimal target:

n_i = \frac{N\sigma_i}{\sum_{j=1}^{K}\sigma_j}, \qquad i=1,\ldots,K.

The ratio for any two arms is therefore \(n_i/n_j=\sigma_i/\sigma_j\). The allocation is proportional to standard deviations, not variances: the variance \(\sigma_i^2\) enters the objective, but the derivative with respect to \(n_i\) depends on \(\sigma_i^2/n_i^2\). Equating those derivatives across arms makes sample size scale with \(\sigma_i\).

Why This Stationary Point Is the Minimum

The first-order equations identify a stationary allocation, but a minimum must also be justified. For each arm, the second derivative of its contribution to the objective is

\frac{d^2}{dn_i^2}\left(\frac{\sigma_i^2}{n_i}\right) = \frac{2\sigma_i^2}{n_i^3}>0 \qquad\text{for }n_i>0.

Each term is strictly convex, so their sum is strictly convex over positive sample sizes. Also, if any \(n_i\) approaches zero while \(\sigma_i>0\), then \(\sigma_i^2/n_i\) grows without bound. The optimum therefore lies in the interior, and strict convexity means the stationary allocation is the unique global minimum for the continuous problem.

The multiplier has a useful interpretation. At the optimum, \(\sigma_i^2/n_i^2=\lambda\) for every arm. Since \(\sigma_i^2/n_i^2\) is the magnitude of the reduction in the objective from a small increase in \(n_i\), the allocation equalizes marginal variance reductions across arms. If one arm had a larger marginal reduction than another, shifting a small amount of sample size toward it would lower the total objective.

Worked Example: Three Arms With Different Standard Deviations

Suppose a hypothetical trial plans to enroll \(N=120\) participants across three arms. For planning, assume the outcome standard deviations are \(\sigma_1=4\), \(\sigma_2=6\), and \(\sigma_3=8\) outcome units. Assume independent outcomes, no unequal costs, and the sum-of-arm-mean-variances objective above.

The sum of the standard deviations is \(4+6+8=18\). The continuous targets are

n_1=120\left(\frac{4}{18}\right)=26.67,\qquad n_2=120\left(\frac{6}{18}\right)=40.00,\qquad n_3=120\left(\frac{8}{18}\right)=53.33.

The targets sum to \(26.67+40.00+53.33=120.00\), and their ratio is \(4:6:8\), or \(2:3:4\). The objective at these continuous targets is approximately

V_{\text{Neyman}} = \frac{4^2}{26.67}+\frac{6^2}{40.00}+\frac{8^2}{53.33} \approx 0.600+0.900+1.200 =2.700

For comparison, equal allocation gives \(n_i=40\) in every arm and \(V_{\text{equal}}=16/40+36/40+64/40=2.900\). Under the stated objective and assumptions, the Neyman targets have the lower sum of arm-mean variances. The target values are continuous; implementing integer sample sizes is a separate practical step.

Special Case: Equal Variability

The derivation also explains when the multiarm targets reduce to equal allocation. If every arm has the same positive standard deviation \(\sigma\), then the denominator is \(K\sigma\), and each target is \(N\sigma/(K\sigma)=N/K\). Equal allocation is therefore optimal for this objective when arm variances are equal, not simply because there are multiple arms.

Worked Example: Equal Standard Deviations Give Equal Targets

A hypothetical three-arm study has a fixed total of \(N=90\) participants. Suppose the outcome standard deviation is \(5\) units in each arm, with independent outcomes and no unequal costs. The sum of the standard deviations is \(5+5+5=15\), so each target is

n_i=90\left(\frac{5}{15}\right)=30,\qquad i=1,2,3.

The resulting objective is \(V=25/30+25/30+25/30=2.500\). To see why imbalance is not helpful here, consider the alternative allocation \((20,30,40)\). It also totals 90, but its objective is \(25/20+25/30+25/40\approx1.250+0.8333+0.6250=2.708\), which is larger. With equal standard deviations, the optimal proportions are equal.

Checking a Candidate Allocation

For a proposed allocation under this model, three checks are especially useful: the sample sizes must sum to \(N\); their proportions must match the standard-deviation proportions; and the values \(\sigma_i^2/n_i^2\) should be equal across arms at the continuous optimum. These checks expose common algebra mistakes, such as using \(\sigma_i^2\) rather than \(\sigma_i\) in the allocation formula.

Worked Example: Verify the Marginal-Balance Condition

Suppose a hypothetical three-arm study has \(N=140\) and planning standard deviations \(2\), \(5\), and \(7\) outcome units. Assume the model conditions for the derivation hold. Their sum is \(14\), so the targets are

n_1=140\left(\frac{2}{14}\right)=20,\qquad n_2=140\left(\frac{5}{14}\right)=50,\qquad n_3=140\left(\frac{7}{14}\right)=70.

They sum to 140, and the target ratio \(20:50:70\) reduces to \(2:5:7\). At these targets, the marginal quantities are

\frac{\sigma_1^2}{n_1^2}=\frac{4}{400}=0.01000,\qquad \frac{\sigma_2^2}{n_2^2}=\frac{25}{2500}=0.01000,\qquad \frac{\sigma_3^2}{n_3^2}=\frac{49}{4900}=0.01000.

The equality confirms the first-order condition. The objective is \(4/20+25/50+49/70=0.200+0.500+0.700=1.400\). An alternative allocation \((35,50,55)\) totals 140 but gives \(4/35+25/50+49/55\approx0.1143+0.500+0.8909=1.505\), a larger value.

Assumptions, Alternatives, and Limitations

The derivation is a planning result, not a guarantee that a realized trial will attain the target variance. It assumes that the arm standard deviations used in the calculation represent the relevant outcome distributions, observations are independent, and the objective gives equal emphasis to each arm-mean variance. If observations are dependent, the variance formula may need additional terms. If arms have unequal per-participant costs, minimizing variance for a fixed budget changes the allocation; the cost-adjusted Neyman derivation addresses that setting.

Equal allocation is a simple alternative and remains optimal for this objective when all arm standard deviations are equal. When they differ, it generally does not minimize the stated objective, though other scientific or operational goals may still favor equal allocation. Likewise, a target based on assumed standard deviations is only as appropriate as those assumptions. Check that the standard deviations are on the same outcome scale, that each is positive, and that the stated objective matches the estimand or precision goal.

A frequent mistake is to reason, “the variance is \(\sigma_i^2\), so allocate in proportion to \(\sigma_i^2\).” That skips the fixed-total optimization. The correct first-order condition is \(\sigma_i^2/n_i^2=\lambda\); solving it gives \(n_i\) proportional to \(\sigma_i\). Another mistake is to treat continuous targets as already feasible assignments. The derivation establishes the ideal continuous proportions; actual sample sizes must be whole numbers and must obey any design constraints.

Key takeaway: With independent arm outcomes, positive known standard deviations, a fixed total sample size, and the objective \(\sum_i \sigma_i^2/n_i\), the unique continuous optimum is \(n_i=N\sigma_i/\sum_j\sigma_j\). It allocates more participants to arms with greater outcome variability because doing so balances marginal reductions in variance.

Check Your Understanding

Try each question, then compare your reasoning with the concise solutions.

In one sentence, state the objective and constraint used to derive the multiarm Neyman allocation.
Three arms have standard deviations \(3\), \(4\), and \(5\), and the total sample size is \(120\). Find the continuous targets.
Why does the allocation use standard deviations rather than variances?
For equal positive standard deviations across \(K\) arms, what allocation follows from the formula?
Name two conditions under which the derived allocation should not be used without changing the model or objective.

Solutions. (1) Minimize \(\sum_i\sigma_i^2/n_i\) subject to \(\sum_i n_i=N\). (2) The standard deviations sum to 12, so the targets are \(120(3/12)=30\), \(120(4/12)=40\), and \(120(5/12)=50\). (3) The first-order condition is \(\sigma_i^2/n_i^2=\lambda\); solving for positive \(n_i\) gives \(n_i=\sigma_i/\sqrt{\lambda}\), hence proportionality to \(\sigma_i\). (4) Each arm receives \(N/K\). (5) Examples include unequal recruitment costs, dependence among outcomes that changes the variance formula, or an objective that does not give equal emphasis to arm-mean variances.