Selection Among Correlated Treatment-Control Contrasts

Why Correlation Matters When Choosing the Largest Contrast

When several treatment arms share one control group, their treatment-control estimates are not independent: each estimate contains the same control-group sample mean. If the arm with the largest estimated effect is selected, the estimate includes both treatment-specific noise and noise shared across the contrasts. A correction based on independent estimates treats that shared component as if it could make one contrast a winner over another. It cannot: a common shift moves all contrasts together without changing their ranking.

The distinction matters when quantifying the average overstatement caused by selection. The result depends not only on the marginal standard error of each contrast, but also on their joint distribution. The focus here is selection of the largest contrast at one analysis, with equal marginal standard errors. It is not a general correction for every selection process in a multi-stage design.

A useful model is \(\hat{\boldsymbol{\theta}}\sim N(\boldsymbol{\theta},\Sigma)\). For the results below, assume a common marginal standard error \(s\) and an equicorrelation structure: every pair of distinct estimates has correlation \(\rho\), with \(0\leq\rho<1\). This is an appropriate structure, for example, when treatment groups have equal sizes and outcomes have a common variance. The estimates are exactly jointly normal if the underlying group means are normal; with sufficiently large samples, the normal model may instead be an approximation. The calculations also assume a fixed set of arms, a single selection based on the largest estimate, and no additional selection rule or stopping event.

How a Shared Control Creates Equicorrelation

Suppose each treatment group and the common control group contains independent observations with outcome variance \(\sigma^2\). For treatment group \(i\), the estimated contrast is \(\hat{\theta}_i=\bar{X}_i-\bar{X}_0\), where \(\bar{X}_0\) is the common control mean. If treatment group \(i\) has \(n_{Ti}\) participants and the control group has \(n_C\), then

The covariance is positive because both contrasts subtract the same random control mean. Under equal treatment-group sizes \(n_T\), the common marginal variance is \(s^2=\sigma^2/n_T+\sigma^2/n_C\), so

This correlation is not a nuisance that can be ignored when calculating the expected maximum. It describes which part of the variation can change the ranking. To see that directly, write the estimation errors as

where \(Z_0,Z_1,\ldots,Z_K\) are independent standard normal random variables. Each error has variance \(s^2\), and any two errors have covariance \(s^2\rho\), as required. The shared term \(s\sqrt{\rho}Z_0\) is identical for all arms. It affects the level of every estimate but not which one is largest; only the arm-specific terms \(s\sqrt{1-\rho}Z_i\) can reorder contrasts when their true effects are equal.

Equal True Effects: The Shared Component Cancels From the Ranking

If all \(K\) true effects equal \(\theta\), the selected estimate’s bias is \(E(\max_i\hat{\theta}_i)-\theta\). Using the representation above, the common normal term has mean zero and adds equally to all arms. Therefore,

This is the independent-arm expected-maximum factor multiplied by \(\sqrt{1-\rho}\). It is smaller than the independent-arm result whenever \(\rho>0\). The independent calculation uses the full marginal standard error \(s\) as if all of it could distinguish the arms, even though the shared component cannot affect their order. The factor is not a change to the marginal standard error; each contrast still has standard error \(s\).

For two arms, the factor has a simple closed form. The identity \(\max(a,b)=(a+b)/2+|a-b|/2\) gives, for equal true effects,

Here the difference has mean zero and variance \(2s^2(1-\rho)\), and a zero-mean normal variable with standard deviation \(\tau\) has expected absolute value \(\tau\sqrt{2/\pi}\). For three arms, \(c_3=E(\max(Z_1,Z_2,Z_3))\approx0.846284\); in general it can be evaluated from the standard normal distribution or numerically.

Unequal True Effects: Selection Depends on the Effect Pattern

When true effects differ, the selected arm is not simply the arm with the largest noise. The identity of the winner depends jointly on the true-effect pattern and the estimation errors. The equal-effects formula \(s\sqrt{1-\rho}\,c_K\) no longer applies as a general correction. For two arms with common marginal standard error \(s\), correlation \(\rho\), and true-effect difference \(\delta=\theta_2-\theta_1\), define \(U=\hat{\theta}_2-\hat{\theta}_1-\delta\). Then \(U\) is normal with mean zero and standard deviation \(\tau=s\sqrt{2(1-\rho)}\). Arm 2 wins when \(U>-\delta\). Since its error is the first arm’s error plus \(U\), the average selected-estimate bias is

where \(\phi\) is the standard normal density. The first equality follows by writing the selected error as the error for arm 1 plus \(U\) when arm 2 wins; the arm 1 error has expectation zero. The last equality follows by integrating the normal density over \(U>-\delta\). This expression is positive for finite \(\delta\), but its size depends on both the difference in true effects and the covariance.

Practical Checks, Alternatives, and Limitations

Before using a joint-normal selection calculation, verify that its covariance model matches how the contrasts were estimated. In a simple shared-control trial, the covariance formula follows from independent treatment and control samples and the reuse of the control mean. If group sizes differ, variances differ, outcomes are not well approximated by normal distributions, or the selection rule uses more than the largest estimate, the equal-standard-error equicorrelation result may not apply. A covariance matrix must also be mathematically valid: it must be symmetric and positive semidefinite.

A useful diagnostic is to construct \(\Sigma\) from the design and compare its diagonal and off-diagonal entries with the marginal variances and shared-control covariances implied by the estimator. For a normal-model check, simulate many vectors from \(N(\boldsymbol{\theta},\Sigma)\), select the largest estimate in each vector, and compare the simulated average \(\hat{\theta}_J-\theta_J\) with the formula where one is available. Agreement within Monte Carlo error supports the calculation, but does not establish that the assumed model describes the real data. A mismatch can indicate an incorrect covariance matrix, a coding error in the selection rule, or a model that omits relevant features of the design.

The independent-arm calculation is a useful benchmark when the estimates really are independent. With equal true effects and positive equicorrelation, it gives too large an expected bias because it attributes shared variation to arm-specific ranking noise. For unequal effects, however, do not infer a universal direction or size of error from that simple comparison: the true-effect pattern interacts with the correlation in determining who wins. A direct joint-normal calculation or simulation using the full covariance matrix is more appropriate than applying an independent-arm factor.

This expected-bias calculation is not a confidence interval, does not control a type I error rate, and does not by itself provide an adjusted estimate. For equal effects, it describes an average over repeated selections; it does not say how much a particular realized selected estimate is biased. The model also says nothing by itself about selection across multiple stages, selection by a rule other than the largest contrast, or uncertainty in the variance and covariance estimates. Those features require methods that represent the corresponding design and selection event.