Unequal Allocation and Heteroscedastic Arm Statistics

Why Allocation and Variance Both Matter

In a multi-arm, multi-stage trial, each treatment-control contrast uses observations from two arms. Unequal allocation changes how many observations contribute to each arm mean, while arm-specific variances determine how noisy those observations are. Both features affect the contrast's variance. In addition, contrasts for different treatments are correlated because they use the same control observations. Statistics at successive looks are correlated because cumulative analyses reuse earlier observations.

These covariances are not bookkeeping details: they determine the joint distribution used to calibrate stagewise boundaries. A boundary calculation based on an incorrect correlation matrix can give the wrong probability of at least one false rejection, even if each individual statistic has been standardized correctly.

Model and notation: At stage \(s\), let \(n_{a,s}\) be the cumulative number of participants in treatment arm \(a\), and \(n_{0,s}\) the cumulative number in the shared control arm. Let \(\bar{X}_{a,s}\) and \(\bar{X}_{0,s}\) be their sample means, and define the treatment-control contrast \(D_{a,s}=\bar{X}_{a,s}-\bar{X}_{0,s}\). Assume independent participants within and across arms, normally distributed outcomes with arm-specific means and variances \(\sigma_a^2\), and cumulative samples in which observations at an earlier look are included at later looks. For now, take the variances as known. These assumptions make the contrast vector jointly normal.

Variance and Covariance of the Contrasts

Independence of treatment and control observations gives the variance of a contrast as the sum of the two sample-mean variances. The sample sizes need not be equal, and neither do the arm variances.

\operatorname{Var}(D_{a,s}) =\frac{\sigma_a^2}{n_{a,s}}+\frac{\sigma_0^2}{n_{0,s}}.

For two distinct treatment arms at the same look, the only shared random component is the control mean. Their covariance is therefore the control mean's variance. Across looks, the overlap must be handled carefully. If the cumulative sample sizes in one arm are \(n_s\) and \(n_t\), the smaller sample is contained in the larger one. The covariance of the two sample means is \(\sigma^2/\max(n_s,n_t)\), not \(\sigma^2/\min(n_s,n_t)\).

\operatorname{Cov}(D_{a,s},D_{b,t}) = \mathbf{1}(a=b)\frac{\sigma_a^2}{\max(n_{a,s},n_{a,t})} + \frac{\sigma_0^2}{\max(n_{0,s},n_{0,t})},

where \(\mathbf{1}(a=b)\) is 1 when the two contrasts use the same treatment arm and 0 otherwise. The first term accounts for overlapping treatment observations only when the arm is the same; the second term accounts for the shared control observations in every pair of contrasts.

Why the Overlap Formula Has a Maximum

Suppose \(s\) is the earlier look and \(t\) the later look, so \(n_s\leq n_t\) in a particular arm. Write the centered sample means as sums of independent centered observations divided by their sample sizes. Exactly \(n_s\) observations appear in both sums, so the covariance is \(n_s\sigma^2/(n_s n_t)=\sigma^2/n_t\). Since \(n_t=\max(n_s,n_t)\), this gives the stated maximum formula. The same reasoning applies separately to the treatment and control parts of each contrast.

The standardized statistic is \(Z_{a,s}=D_{a,s}/\sqrt{\operatorname{Var}(D_{a,s})}\) when testing a zero contrast under the null. More generally, under a nonzero mean contrast, standardize \(D_{a,s}\) after centering by that mean. Correlations are obtained by dividing each contrast covariance by the square root of the corresponding two contrast variances. Standardization gives unit marginal variance; it does not make the statistics independent.

Worked Example: Two Contrasts Sharing a Control

Consider one look with known variances \(\sigma_0^2=100\), \(\sigma_1^2=144\), and \(\sigma_2^2=196\). Suppose there are \(n_0=120\) control participants, \(n_1=60\) participants in treatment 1, and \(n_2=40\) in treatment 2. Find the two contrast variances and their correlation.

For treatment 1, the contrast variance is \(\operatorname{Var}(D_{1})=144/60+100/120=2.4+0.8333=3.2333\). For treatment 2, it is \(\operatorname{Var}(D_{2})=196/40+100/120=4.9+0.8333=5.7333\).

The treatment arms are independent, so their contrast covariance comes only from the shared control mean: \(\operatorname{Cov}(D_1,D_2)=100/120=0.8333\). Thus

\operatorname{Corr}(D_1,D_2) =\frac{0.8333}{\sqrt{3.2333\times5.7333}} \approx \frac{0.8333}{4.3055} \approx 0.1935.

The positive correlation is due to the shared control. It is less than one because each contrast also contains independent observations from its own treatment arm. The unequal allocations and variances produce unequal contrast variances, even though both contrasts use the same control group.

Worked Example: Correlation Across Cumulative Looks

For one treatment-control contrast, suppose \(\sigma_1^2=144\) and \(\sigma_0^2=100\). At the first look, let \(n_{1,1}=30\) and \(n_{0,1}=60\); at the second look, let \(n_{1,2}=60\) and \(n_{0,2}=120\). The samples are cumulative. Calculate both variances and the correlation between the two contrasts.

At the first look, \(\operatorname{Var}(D_{1,1})=144/30+100/60=4.8+1.6667=6.4667\). At the second look, \(\operatorname{Var}(D_{1,2})=144/60+100/120=2.4+0.8333=3.2333\).

The covariance uses the larger cumulative sample size in each arm: \(\operatorname{Cov}(D_{1,1},D_{1,2})=144/60+100/120=3.2333\). Therefore,

\operatorname{Corr}(D_{1,1},D_{1,2}) =\frac{3.2333}{\sqrt{6.4667\times3.2333}} \approx 0.7071.

Here the second look has exactly twice the sample size in both arms, so its contrast variance is half the first-look variance. The correlation is consequently \(\sqrt{1/2}\), approximately 0.7071. It is not one: the later contrast includes new observations as well as the observations used earlier.

Using the Covariance Matrix in Boundary Calibration

Collect the stagewise standardized statistics into a vector \(Z\), and let \(R\) be their correlation matrix, calculated from the contrast covariance formulas. Under a joint null hypothesis and the model above, \(Z\) has a multivariate normal distribution with mean zero and correlation matrix \(R\). For upper one-sided boundaries \(u_j\), the probability of at least one crossing under that joint null is

P\!\left(\bigcup_{j=1}^{K}\{Z_j\geq u_j\}\right) =1-\Phi_K(u_1,\ldots,u_K;R),

where \(\Phi_K(u_1,\ldots,u_K;R)\) is the probability that all \(K\) components are below their respective boundaries. To calibrate a common boundary for a target joint-null error probability \(\alpha\), choose \(u\) to satisfy \(1-\Phi_K(u,\ldots,u;R)=\alpha\). For stage-specific boundaries, use the corresponding vector of thresholds. The multivariate-normal calibration and recursive crossing calculations introduced in the earlier material on those subjects use this same principle: the joint distribution, including its correlations, determines the crossing probability.

The covariance matrix also provides a direct check on the calculation. It must be symmetric, have ones on the diagonal after standardization, and be positive semidefinite. A non-positive-semidefinite matrix indicates an error in the covariance inputs or an incompatible model. Correlations outside \([-1,1]\) are an immediate warning, but correlations within that range alone do not guarantee that the whole matrix is valid.

Worked Example: Four Statistics and a Conservative Boundary

Suppose two treatment arms are compared with a shared control at two cumulative looks. Let \(\sigma_0^2=64\), \(\sigma_1^2=100\), and \(\sigma_2^2=144\). At look 1, take \(n_{0,1}=40\), \(n_{1,1}=20\), and \(n_{2,1}=30\); at look 2, all three sample sizes double. Calculate the contrast covariance matrix, then give a conservative common upper boundary for four tests at familywise level \(\alpha=0.025\).

The look-1 contrast variances are \(100/20+64/40=6.6\) for arm 1 and \(144/30+64/40=6.4\) for arm 2. At look 2 they are \(100/40+64/80=3.3\) and \(144/60+64/80=3.2\). The same-look covariance between the two arms is the control contribution: \(64/40=1.6\) at look 1 and \(64/80=0.8\) at look 2.

For the same treatment arm across looks, the covariance equals the later-look variance because both sample sizes double. For different treatment arms across looks, only the shared control contributes, giving \(64/80=0.8\). In the order \(D_{1,1},D_{2,1},D_{1,2},D_{2,2}\), the contrast covariance matrix is

\begin{pmatrix} 6.6 & 1.6 & 3.3 & 0.8\\ 1.6 & 6.4 & 0.8 & 3.2\\ 3.3 & 0.8 & 3.3 & 0.8\\ 0.8 & 3.2 & 0.8 & 3.2 \end{pmatrix}.

Divide each entry by the product of the corresponding standard deviations to obtain \(R\), the correlation matrix for the four standardized statistics. Exact joint-null calibration would solve \(1-\Phi_4(c,c,c,c;R)=0.025\), generally by numerical evaluation. A simple conservative alternative uses Bonferroni: require each of the four marginal upper-tail probabilities to be no greater than \(0.025/4=0.00625\). Since each standardized statistic is standard normal under its null, \(c=\Phi^{-1}(1-0.00625)=\Phi^{-1}(0.99375)\approx2.498\).

The union bound then gives a familywise error probability no greater than \(4(0.00625)=0.025\) under the joint null. This boundary is valid but does not exploit the dependence in \(R\); exact multivariate calibration can generally be less conservative. Calibration under the joint null alone should not automatically be taken as strong familywise control for every configuration of true and false hypotheses; the relevant testing structure and error-control argument must also support that conclusion.

Assumptions, Diagnostics, and Alternatives

The formulas rely on independent participants, independent treatment arms, and nested cumulative samples. The outcome variances must be appropriate for the arms and populations represented at each look. Normal outcomes with known variances give the stated multivariate-normal distribution exactly. If variances are estimated, the resulting standardized statistics generally do not have this exact joint-normal distribution in finite samples; normal calibration may be an approximation whose quality depends on sample size and the variance estimator.

Before using the matrix in a boundary calculation, check the allocation counts and their timing against the actual analysis plan. In particular, verify that sample sizes are cumulative, that the same control participants are indeed reused, and that no participants are counted twice within an arm. Check that each diagonal variance equals the sum of its treatment and control contributions, and that each off-diagonal term includes only genuinely shared observations. Large discrepancies between empirical and planned arm variances, substantial missingness, or changes in the analysis population can undermine the assumed covariance structure.

A common alternative is Bonferroni calibration, which controls the union probability without requiring correlations, but can be conservative when tests are positively correlated. A covariance-based multivariate-normal calibration uses the dependence structure and can be more efficient, but is only as trustworthy as the model and covariance matrix supplied. Neither choice repairs an incorrect variance model or establishes strong control without the appropriate testing framework.

Common Pitfalls

Using the smaller sample size across looks. This overstates the covariance of nested sample means. The overlap calculation gives \(\sigma^2/\max(n_s,n_t)\).
Ignoring the shared control. Contrasts from independent treatment arms are not independent when they subtract the same control mean.
Assuming equal allocation or a common variance. Those simplifications can produce incorrect standard errors and correlations when arm sizes or variances differ.
Equating marginal calibration with familywise calibration. Giving every test a nominal level does not by itself control the probability of any false crossing across arms and looks.
Treating estimated-variance statistics as exactly normal. The exact normal result here assumes known variances and the stated outcome model; estimated variances can require approximation or a different calibration model.

Key takeaway: Under unequal allocation and arm-specific variances, calculate each contrast variance from its two arm contributions, and calculate covariance from the observations actually shared. Standardize the resulting covariance matrix before using it for joint boundary calibration.

Check Your Understanding

Answer each question, then compare with the solutions below.

Two treatments share a control with variance \(\sigma_0^2=81\) and \(n_0=90\). What is the covariance of their contrasts at this look, assuming independent treatment arms?
A sample mean at an early look uses 25 observations and a later cumulative mean uses 50. If the individual outcome variance is 36, what is the covariance between the means?
Why does unequal allocation affect the correlation between two treatment-control statistics even when the control variance and control sample size are unchanged?
Four marginal tests each have upper-tail probability 0.01 under their respective nulls. What familywise upper bound follows from Bonferroni?

Solutions. (1) The shared control contribution is \(81/90=0.9\), so the contrast covariance is 0.9. (2) The covariance is \(36/\max(25,50)=36/50=0.72\). (3) Allocation changes the treatment contributions to the variances; because correlation divides covariance by the product of the contrast standard deviations, the correlation changes even if the shared-control covariance stays fixed. (4) The union bound is \(4(0.01)=0.04\), so the familywise probability is at most 0.04.