Plug-In Neyman Targets From Pilot Variances

From Pilot Variability to an Allocation Target

Neyman allocation minimizes the variance of a two-arm difference in sample means when the arm-specific outcome variances are known, the total sample size is fixed, and the allocation is otherwise unconstrained. In practice, those variances are rarely known before a trial begins. A pilot study provides estimates of outcome variability, which can be substituted into the allocation formula to obtain a plug-in Neyman target.

The target is a planning quantity, not a promise about the realized sample. In a response-adaptive trial, it can be expressed as the desired probability of assigning an eligible participant to each arm. If assignment is randomized independently with those fixed probabilities, the eventual counts will vary around their expected values. Adaptive rules may also use the target as a reference allocation, but the target alone does not specify such a rule.

Definition: For two arms, let \(N=n_0+n_1\) be the planned total sample size, and let \(\sigma_0\) and \(\sigma_1\) be the outcome standard deviations in arms 0 and 1. When both standard deviations are strictly positive, the unconstrained Neyman allocation has ratio \(n_1/n_0=\sigma_1/\sigma_0\), or arm-1 share \(\pi_1=\sigma_1/(\sigma_0+\sigma_1)\). If exactly one standard deviation is zero, there is no interior minimum: over allocations with positive sample sizes in both arms, the variance approaches its infimum as the zero-variance arm's allocation approaches zero. If each arm must meet a positive minimum size, the variance-minimizing allocation assigns the minimum to the zero-variance arm. If both standard deviations are zero, every allocation with positive sample sizes in both arms has variance zero. The plug-in Neyman target replaces the unknown standard deviations with pilot estimates \(s_0\) and \(s_1\): \(\hat{\pi}_1=s_1/(s_0+s_1)\) and \(\hat{\pi}_0=s_0/(s_0+s_1)\), when both estimates are positive; zero estimates require the corresponding boundary or tie-case treatment.

Assumptions Behind the Plug-In Calculation

The target is appropriate for minimizing the variance of the usual difference between independent arm sample means when the following conditions are reasonable:

The outcome is measured on a common scale in both arms, and the estimand is the difference in arm means.
Outcomes within each arm are independent, with finite arm-specific variances. The variance of the sample mean in arm \(a\) is then \(\sigma_a^2/n_a\).
The total sample size \(N\) is fixed, each participant has the same allocation cost, and there are no binding minimum-size or other allocation constraints.
Pilot participants are sufficiently comparable to trial participants that the pilot standard deviations estimate the variability expected in the trial.

If the pilot contains \(m_a\) observations in arm \(a\), the usual estimate is the sample standard deviation \(s_a\), computed using deviations from that arm’s pilot mean and divisor \(m_a-1\). Each arm needs at least two observations to calculate this estimate. The pilot arm sizes \(m_0\) and \(m_1\) determine how much information was available to estimate variability; they are not automatically the trial allocation targets.

Why Substituting Standard Deviations Gives the Target

For a fixed total \(N\), the variance of the difference in sample means is \(\sigma_0^2/n_0+\sigma_1^2/n_1\), where \(n_0=N-n_1\). Minimizing this expression yields the Neyman ratio. The plug-in procedure applies the same calculation with the pilot standard deviations in place of the unknown population values.

V(n_1)=\frac{\sigma_0^2}{N-n_1}+\frac{\sigma_1^2}{n_1}, \qquad 0<n_1<N.

When both standard deviations are strictly positive, differentiating with respect to \(n_1\), setting the derivative to zero, and rearranging gives the ratio. In this case, \(V''(n_1)=2\sigma_0^2/(N-n_1)^3+2\sigma_1^2/n_1^3\) is positive throughout the feasible interval, so the stationary point is a minimum.

\frac{dV}{dn_1} = \frac{\sigma_0^2}{(N-n_1)^2} - \frac{\sigma_1^2}{n_1^2} =0 \quad\Longrightarrow\quad \frac{\sigma_0}{n_0}=\frac{\sigma_1}{n_1} \quad\Longrightarrow\quad \frac{n_1}{n_0}=\frac{\sigma_1}{\sigma_0}.

Since \(n_0+n_1=N\), this ratio implies \(n_1=N\sigma_1/(\sigma_0+\sigma_1)\). Replacing the standard deviations by \(s_0\) and \(s_1\) gives continuous target counts \(N\hat{\pi}_0\) and \(N\hat{\pi}_1\). Integer counts must be chosen if the design requires exact sample sizes; rounding can make the final allocation slightly different from the continuous target.

Worked Example: Constructing a Target From Pilot Summaries

Hypothetical data. A pilot study reports an outcome standard deviation of 10 points in arm 0 and 15 points in arm 1. A subsequent trial plans to enroll \(N=120\) participants, with equal allocation costs and no minimum-size constraints. Find the plug-in target and its estimated variance.

First substitute the pilot standard deviations into the allocation proportions:

\hat{\pi}_1=\frac{15}{10+15}=0.6000, \qquad \hat{\pi}_0=\frac{10}{10+15}=0.4000.

The corresponding continuous sample-size targets are \(120(0.6000)=72\) in arm 1 and \(120(0.4000)=48\) in arm 0. These are already integers. Under the plug-in model, the estimated variance of the difference in sample means is

\widehat{\operatorname{Var}}(\bar{x}_1-\bar{x}_0) = \frac{s_0^2}{n_0}+\frac{s_1^2}{n_1} = \frac{10^2}{48}+\frac{15^2}{72} = 2.0833+3.125 = 5.2083\ \text{points}^2.

For comparison, equal allocation would give 60 participants per arm and plug-in variance \(100/60+225/60=5.4167\ \text{points}^2\). Thus, under these pilot estimates and stated assumptions, the plug-in target has a lower estimated variance than equal allocation. This comparison concerns the variance predicted by the pilot estimates, not a guarantee about the variance realized in the trial.

Worked Example: Calculating Pilot Standard Deviations Before Plugging In

Hypothetical data. A pilot records outcomes of 8, 10, 12, and 10 points in arm 0, and 9, 13, 15, and 11 points in arm 1. The planned trial has \(N=80\) participants. Calculate the sample standard deviations, continuous target counts, and rounded allocation.

The pilot means are 10 points in arm 0 and 12 points in arm 1. Using divisor \(m_a-1=3\), the sample variances and standard deviations are

s_0^2=\frac{(8-10)^2+(10-10)^2+(12-10)^2+(10-10)^2}{3} =\frac{8}{3}, \qquad s_0=\sqrt{\frac{8}{3}}=1.633\ \text{points},

s_1^2=\frac{(9-12)^2+(13-12)^2+(15-12)^2+(11-12)^2}{3} =\frac{20}{3}, \qquad s_1=\sqrt{\frac{20}{3}}=2.582\ \text{points}.

The plug-in share for arm 1 is therefore

\hat{\pi}_1 = \frac{\sqrt{20/3}}{\sqrt{8/3}+\sqrt{20/3}} \approx 0.6126, \qquad \hat{\pi}_0\approx 0.3874.

Multiplying by 80 gives continuous targets of approximately 49.01 participants in arm 1 and 30.99 in arm 0. Rounding to 49 and 31 gives a total of 80. Using the unrounded pilot variance estimates, the estimated variance at these integer counts is

\frac{8/3}{31}+\frac{20/3}{49} \approx 0.2221\ \text{points}^2.

The pilot had four participants in each arm, but the target assigns about 61% of the trial to arm 1 because its estimated outcome standard deviation is larger. The target follows estimated variability, not the pilot’s allocation ratio.

Using the Target in a Trial Design

A practical calculation should keep the estimated quantities, the target, and the implementation distinct. The following sequence makes those steps explicit.

Define the outcome and trial population.
Confirm that the pilot standard deviations describe the outcome, measurement scale, and participant population relevant to the planned comparison.

Estimate variability separately by arm.
Calculate \(s_0\) and \(s_1\) from the pilot observations, using the sample standard deviation within each arm.

Compute shares and counts.
Use \(\hat{\pi}_a=s_a/(s_0+s_1)\), then multiply each share by the planned total \(N\). Apply any required integer rounding or design constraints afterward.

Specify how randomization will use the target.
A target share can define a fixed assignment probability or inform a separately specified adaptive rule. State which interpretation is intended rather than treating the target as an entire randomization algorithm.

The plug-in target is most defensible when pilot measurements were collected and analyzed comparably to the planned trial. Check for arm-specific changes in measurement procedures, eligibility, follow-up, or outcome definition. Also check whether extreme observations or a very small pilot make a reported standard deviation an unstable summary of variability. These checks identify reasons to question the inputs; they do not turn the estimated target into a known-variance optimum.

Interpretation, Alternatives, and Limitations

The Neyman target responds to variability alone: it assigns more participants to the arm with the larger standard deviation because additional observations there reduce the contribution of that arm to the variance of the mean difference. It does not, by itself, account for treatment benefit, patient preference, unequal costs, minimum arm sizes, or other design goals. If such features matter, the unconstrained two-arm target derived here may not be the appropriate design solution.

Approach	What determines the allocation?	Practical interpretation
Equal allocation	The two arm shares are each one half.	Simple benchmark; it need not minimize variance when arm variances differ.
Known-variance Neyman allocation	The true arm-specific standard deviations.	The variance-minimizing allocation under the stated unconstrained model, if those values are known.
Plug-in Neyman target	Pilot estimates of the arm-specific standard deviations.	A practical estimated target; its performance depends on how informative and relevant the pilot estimates are.

A plug-in calculation treats \(s_0\) and \(s_1\) as inputs, but they are themselves estimated from finite pilot samples. Consequently, the calculated target is uncertain. It should be reported as an estimate based on the pilot, not as the allocation that is certain to minimize the trial’s actual variance. The calculation also assumes that the pilot’s variability carries over to the trial; a change in population or outcome measurement can undermine that premise.

Common Mistake

A frequent error is to allocate in proportion to the variances, using \(s_1^2/(s_0^2+s_1^2)\). That is not the two-arm Neyman share for a difference in means. The variance contribution in each arm is \(\sigma_a^2/n_a\); minimizing the sum makes the sample-size ratio proportional to standard deviations, not variances. For example, standard deviations 10 and 15 give an arm-1 target of \(15/(10+15)=0.6000\), not \(225/(100+225)=0.6923\). Another mistake is to call the plug-in target guaranteed optimal: it is optimal only for the variance values used in the calculation and under the model and constraints stated above.

Key takeaway: Estimate each arm’s standard deviation from relevant pilot data, substitute those estimates into the Neyman formula, and report the result as an estimated target. Keep the pilot evidence, continuous target, integer implementation, and actual randomization procedure conceptually separate.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions below.

A pilot estimates standard deviations of 5 points in arm 0 and 10 points in arm 1. What are the plug-in allocation shares?
For \(N=100\), pilot standard deviations are 8 and 12 points in arms 0 and 1. Find the continuous target counts and the plug-in variance at those counts.
Why should pilot arm sizes not be copied automatically as the trial target proportions?
Name two conditions that could make a pilot-based target inappropriate for a planned trial.

Solutions. (1) The shares are \(\hat{\pi}_0=5/(5+10)=1/3\) and \(\hat{\pi}_1=10/15=2/3\). (2) The arm-1 share is \(12/(8+12)=0.6000\), giving 40 participants in arm 0 and 60 in arm 1. The plug-in variance is \(8^2/40+12^2/60=1.6+2.4=4.000\ \text{points}^2\). (3) Pilot counts describe the pilot’s design and precision; the trial target is determined by the estimated arm-specific standard deviations and the trial’s total sample size. (4) Examples include a pilot population unlike the trial population, a changed outcome definition or measurement procedure, or a design with unequal costs or binding allocation constraints.