Selection With Unequal Randomization And Shared Control

Why allocation and a shared control matter

When a trial compares several experimental arms with one control, each treatment–control estimate includes the same control estimate. That shared component affects both the precision of each contrast and the dependence between contrasts. Unequal randomization matters too: an arm with fewer participants generally has a less precise contrast, even when its estimated treatment effect is large.

These features are consequential when deciding which arm to carry forward and when testing for efficacy. A selection rule based on raw estimated effects can choose a different arm from a rule based on standardized evidence. And a multiplicity adjustment that treats the treatment statistics as independent can misrepresent their joint distribution. The calculations below make these effects explicit.

Model and assumptions: Suppose there are \(K\) experimental arms and one control arm. For arm \(i\), outcomes are independent across participants and normally distributed with mean \(\mu_i\) and common variance \(\sigma^2\); the control outcomes have mean \(\mu_0\) and the same variance. The group sizes \(n_0,n_1,\ldots,n_K\) are fixed, and the groups contain distinct participants. Define the treatment effect as \(\Delta_i=\mu_i-\mu_0\), estimated by \(D_i=\bar{X}_i-\bar{X}_0\). The formulas below assume this model and allocation; they do not automatically apply if the allocation depends on accumulating outcomes.

Standard errors and dependence between contrasts

Because the treatment and control groups are independent, the variance of one estimated contrast is the sum of the variances of its two sample means. For two different treatment arms, the treatment means are independent of each other and of the control mean. The only random component shared by both contrasts is the control mean.

\operatorname{Var}(D_i)=\sigma^2\left(\frac{1}{n_i}+\frac{1}{n_0}\right), \qquad SE(D_i)=\sigma\sqrt{\frac{1}{n_i}+\frac{1}{n_0}}.

\begin{aligned} \operatorname{Cov}(D_i,D_j) &=\operatorname{Cov}(\bar{X}_i-\bar{X}_0,\bar{X}_j-\bar{X}_0)\\ &=\operatorname{Var}(\bar{X}_0) =\frac{\sigma^2}{n_0}, \qquad i\ne j,\\ \operatorname{Corr}(D_i,D_j) &=\frac{1/n_0} {\sqrt{(1/n_i+1/n_0)(1/n_j+1/n_0)}}. \end{aligned}

The positive covariance follows directly from the shared control term: a control mean that happens to be low makes every estimated treatment–control difference larger. The correlation is not necessarily large. It depends on the control size relative to each treatment size, so unequal allocation can produce different correlations for different pairs of arms.

Using an estimated variance: If \(\sigma\) is unknown, estimate it with the pooled within-group standard deviation \(s\), under the common-variance model. Then use \(s\sqrt{1/n_i+1/n_0}\) as the estimated standard error and a \(t\) statistic with \(df=\sum_{g=0}^{K}n_g-(K+1)\). Under independent normal outcomes and a common variance, the pooled variance estimate is independent of the group means. The resulting joint test statistics have a multivariate \(t\) distribution, rather than an exactly multivariate normal distribution.

Worked calculations

Worked Example: Two Unequally Sized Arms and One Control

Hypothetical data. A continuous outcome is analyzed under the common-variance model, with known \(\sigma=10\) outcome units. There are \(n_0=120\) control participants, \(n_1=30\) participants in arm 1, and \(n_2=15\) in arm 2. The sample means are 50, 55, and 57.5 units, respectively. Calculate the contrast standard errors, their correlation, and the one-sided Bonferroni-adjusted test results for \(H_{0i}:\Delta_i\leq 0\) versus \(H_{ai}:\Delta_i>0\), using familywise \(\alpha=0.05\).

For arm 1, the estimated effect is \(D_1=55-50=5\) units, and

SE(D_1)=10\sqrt{\frac{1}{30}+\frac{1}{120}} =\sqrt{4.1667}=2.041.

For arm 2, \(D_2=57.5-50=7.5\) units, and

SE(D_2)=10\sqrt{\frac{1}{15}+\frac{1}{120}} =\sqrt{7.500}=2.739.

The covariance is \(10^2/120=0.8333\) squared outcome units. Therefore the correlation, calculated from the unrounded variances and standard errors, is

\operatorname{Corr}(D_1,D_2) =\frac{100/120} {\sqrt{[100(1/30+1/120)][100(1/15+1/120)]}} =0.1491.

The known-variance test statistics are \(Z_1=5/2.041=2.449\) and \(Z_2=7.5/2.739=2.739\). Their one-sided unadjusted \(p\)-values are approximately 0.00715 and 0.00308. Bonferroni adjustment multiplies each by the number of candidate tests, two: the adjusted values are approximately 0.0143 and 0.00617. Both are below 0.05. This conclusion controls the familywise error rate for these two one-sided tests, whether or not arm 2 was first identified as the stronger observed result. The displayed effects and standard errors are in outcome units; the test statistics and \(p\)-values are unitless.

Worked Example: Selection by Effect Versus Selection by Evidence

Hypothetical data. Suppose the known outcome standard deviation is 12 units, with \(n_0=100\), \(n_1=25\), and \(n_2=100\). The observed control mean is 40 units; the two treatment means are 46 and 45 units. Compare selecting the arm with the largest estimated effect to selecting the arm with the largest standardized statistic.

The estimated effects are \(D_1=46-40=6\) units and \(D_2=45-40=5\) units. Thus, selection by raw effect chooses arm 1. But the unequal sample sizes give different standard errors:

SE(D_1)=12\sqrt{\frac{1}{25}+\frac{1}{100}} =12\sqrt{0.05}=2.683, \qquad SE(D_2)=12\sqrt{\frac{1}{100}+\frac{1}{100}} =12\sqrt{0.02}=1.697.

The corresponding statistics are \(Z_1=6/2.683=2.236\) and \(Z_2=5/1.697=2.946\). Selection by largest standardized statistic therefore chooses arm 2: its estimated effect is smaller, but it is more precisely estimated. The contrast covariance is \(12^2/100=1.440\) squared units, and the correlation is

\operatorname{Corr}(D_1,D_2) =\frac{0.01}{\sqrt{0.05(0.02)}}=0.3162.

Neither rule is universally correct; the rule must match the trial objective. Selecting for the largest observed effect targets a different quantity from selecting for the strongest standardized evidence. If the chosen arm is then tested, its test must still be interpreted in light of the selection rule and the original family of candidate tests. A valid simultaneous procedure or a suitable multiplicity adjustment accounts for that family; simply reporting the selected arm's unadjusted \(p\)-value does not.

Adjustment after selecting among arms

For a one-sided family of \(K\) tests, Bonferroni compares each unadjusted \(p\)-value with \(\alpha/K\), or reports \(p_i^{\mathrm{adj}}=\min(1,Kp_i)\). This control does not require independent test statistics. It is therefore a straightforward option when the shared-control correlation is difficult to model. Its potential cost is conservatism, especially when the tests are positively correlated.

A maxT procedure uses the joint distribution of the standardized treatment–control statistics. With known variance under the stated normal model, let \(Z_i=D_i/SE(D_i)\), and let \(c\) satisfy

P\left(\max_{1\leq i\leq K}Z_i\leq c\right)=1-\alpha

under the joint null distribution. Reject an individual one-sided null when its statistic exceeds \(c\). Equivalently, its adjusted \(p\)-value is the probability, under the joint null, that the maximum statistic is at least as large as its observed statistic. The correlation matrix for this calculation has diagonal entries 1 and off-diagonal entries given by the shared-control formula. With an estimated common variance, calibration should use the corresponding multivariate \(t\) distribution and degrees of freedom, not silently substitute independent normal tests.

A simultaneous maxT rule supports selection followed by claims about any arm that crosses the common threshold: the probability of one or more false rejections in the family is controlled at the chosen level under the model. Bonferroni offers a simpler conservative adjustment; maxT uses the dependence structure and can be more powerful. Neither adjustment repairs an incorrect standard error, a misspecified dependence structure, or a selection procedure that changes the inferential family without being accounted for.

Worked Example: Three Arms, Unequal Allocation, and a Familywise Rule

Hypothetical data. Take known \(\sigma=10\) outcome units and group sizes \(n_0=120\), \(n_1=30\), \(n_2=15\), and \(n_3=60\). Suppose the observed means are 50 units for control and 54, 56, and 53 units for arms 1, 2, and 3. Calculate the standard errors, the pairwise correlations, and the Bonferroni decisions for one-sided tests at familywise \(\alpha=0.05\).

The effects are \(D_1=4\), \(D_2=6\), and \(D_3=3\) units. Their standard errors are

\begin{aligned} SE(D_1)&=10\sqrt{1/30+1/120}=2.041,\\ SE(D_2)&=10\sqrt{1/15+1/120}=2.739,\\ SE(D_3)&=10\sqrt{1/60+1/120}=1.581. \end{aligned}

Every pair has covariance \(100/120=0.8333\) squared units. Applying the correlation formula gives \(\operatorname{Corr}(D_1,D_2)=0.1491\), \(\operatorname{Corr}(D_1,D_3)=0.2582\), and \(\operatorname{Corr}(D_2,D_3)=0.1925\). For example,

\operatorname{Corr}(D_1,D_3) =\frac{1/120}{\sqrt{(1/30+1/120)(1/60+1/120)}} =0.2582.

The test statistics are \(Z_1=4/2.041=1.960\), \(Z_2=6/2.739=2.191\), and \(Z_3=3/1.581=1.897\). The Bonferroni threshold is \(\alpha/K=0.05/3=0.01667\) per test, equivalent to a one-sided standard-normal critical value of about 2.128. Only arm 2 exceeds that threshold. Its unadjusted one-sided \(p\)-value is about 0.01423, so its Bonferroni-adjusted value is about 0.04269, below 0.05. The adjusted values for arms 1 and 3 are approximately 0.07507 and 0.08667, respectively, so they do not reject. The largest observed effect and the largest statistic both belong to arm 2 here; the adjustment still reflects testing all three candidates.

Derivation and practical checks

The covariance result can be justified by expanding the covariance and using independence across groups. For \(i\ne j\),

\begin{aligned} \operatorname{Cov}(\bar{X}_i-\bar{X}_0,\bar{X}_j-\bar{X}_0) ={}&\operatorname{Cov}(\bar{X}_i,\bar{X}_j) -\operatorname{Cov}(\bar{X}_i,\bar{X}_0)\\ &-\operatorname{Cov}(\bar{X}_0,\bar{X}_j) +\operatorname{Var}(\bar{X}_0). \end{aligned}

The first three terms are zero because the groups are independent. The remaining term is \(\operatorname{Var}(\bar{X}_0)=\sigma^2/n_0\), proving the stated covariance. Dividing by the two contrast standard deviations gives the correlation formula. This also explains why the shared-control dependence is positive and why increasing \(n_0\) reduces the shared component of each contrast variance.

Before applying an adjusted test, verify the ingredients rather than relying only on the formula:

Check the allocation counts. Use the actual analyzed group sizes, not the planned sizes, if exclusions or missing outcomes changed them.
Check the outcome model. Strong skew, outliers, unequal variances, clustering, or repeated measurements can invalidate the simple standard errors or joint distribution.
Check the selection rule. State whether selection used raw effects, standardized statistics, or another prespecified criterion. These rules can select different arms.
Check the multiplicity family. The adjustment must include every candidate whose test could support a confirmatory claim, including arms that were not ultimately selected.

For example, with arm-specific variances \(\sigma_i^2\) and control variance \(\sigma_0^2\), the contrast variance becomes \(\sigma_i^2/n_i+\sigma_0^2/n_0\), while the covariance between two contrasts remains \(\sigma_0^2/n_0\) when groups are independent. The equal-variance correlation formula must not be used unchanged. Likewise, if allocation changes in response to accumulating outcomes, the final sample sizes and treatment statistics may be dependent in ways not represented by the fixed-allocation model. In that situation, a design-specific analysis is needed rather than treating the realized allocation as though it had been fixed independently of outcomes.

Common mistake and limits

A common mistake is to calculate each treatment–control standard error correctly and then treat the resulting test statistics as independent. The marginal standard errors alone do not describe the joint distribution: each contrast contains the same control mean, so the statistics are correlated. The correction is to calculate the covariance and correlation as well, then use an adjustment that is valid for the resulting family. Bonferroni remains valid without independence; a maxT adjustment requires a correctly specified joint distribution.

A second mistake is to assume that a selected arm's unadjusted test is an ordinary single prespecified test. Choosing an arm because its observed result was favorable makes the selection rule part of the inferential procedure. A familywise adjustment over the original candidate arms protects the family of claims under the stated assumptions; it does not make the selected estimate unbiased, nor does it guarantee that a selected effect estimate is close to its true value. Also, the formulas here are for parallel, independent groups with a shared control. They do not directly cover paired outcomes, multiple control groups, or outcome-dependent allocation.

Check Your Understanding

Answer each question, then compare with the brief solutions below.

With known \(\sigma=8\), \(n_0=80\), and \(n_1=20\), what is \(SE(D_1)\)?
For two arms with \(n_0=100\), \(n_1=25\), \(n_2=100\), what is \(\operatorname{Cov}(D_1,D_2)\) if \(\sigma=10\)?
Why can selecting the arm with the largest raw effect differ from selecting the arm with the largest \(Z\) statistic?
Does Bonferroni require independent tests? What does maxT require that Bonferroni does not?

Solutions. (1) \(SE(D_1)=8\sqrt{1/20+1/80}=8\sqrt{0.0625}=2.000\) outcome units. (2) The covariance is \(\sigma^2/n_0=100/100=1.000\) squared outcome units. (3) The arms can have different standard errors because their sample sizes differ; standardizing divides each estimated effect by its own uncertainty. (4) Bonferroni does not require independence. maxT uses the joint distribution, including the correlations among the statistics, so that distribution must be correctly specified or calibrated.