Shared-Control Covariance in Selected Effect Estimates

Why a Shared Control Matters After Selection

In a multi-arm trial, several treatment effects may be estimated against the same control group. Each estimate has uncertainty, but the estimates are not independent: a control sample mean that happens to be low raises every treatment-minus-control estimate, while a high control mean lowers them all. This dependence matters especially when an arm is selected using the estimated effects. The estimates being compared, and the selection event itself, are linked.

This tutorial derives that dependence under a normal sampling model and follows its consequences for a comparison of two arms. The focus is the covariance created by sharing a control, not a general treatment of selection among arbitrary correlated contrasts.

Model and Covariance Derivation

Suppose independent samples are observed in control, treatment arm 1 and treatment arm 2. Let their sample means be \(\bar{X}_0,\bar{X}_1,\bar{X}_2\), and let their population means be \(\theta_0,\theta_1,\theta_2\). Assume outcomes within each group are independent and normally distributed, the groups are mutually independent, and the relevant population variances are known. Thus \(\bar{X}_i\sim N(\theta_i,v_i)\), where \(v_i=\sigma_i^2/n_i\) is the variance of the sample mean in group \(i\). These assumptions make the contrasts jointly normal. In applications, the variances may instead be estimated; the exact normal calculations below then serve as a model-based approximation.

Define the treatment-effect estimates as differences in sample means, with positive values favoring treatment:

Their expectations are \(\delta_1=\theta_1-\theta_0\) and \(\delta_2=\theta_2-\theta_0\). Each contrast includes the same random control mean. By linearity of covariance and independence of the three group means,

The individual variances and the correlation therefore are

The positive covariance does not mean the treatment estimates are identical or that every difference between them is small. It means their random movements include a common component. In particular, subtracting one contrast from the other cancels that component:

The control contributes to the marginal variance of each estimate, but not to the variance of their difference. This distinction is central when selection compares the two effects.

How Dependence Enters a Selection Event

Suppose the arm with the larger estimated treatment effect is selected. Arm 1 wins when \(Q>0\), where \(Q=\hat{\delta}_1-\hat{\delta}_2\). Under the stated model, \(Q\sim N(\mu,s^2)\), with \(\mu=\delta_1-\delta_2\) and \(s=\sqrt{v_1+v_2}\). The probability that arm 1 wins is \(P(Q>0)=\Phi(\mu/s)\), where \(\Phi\) is the standard normal cumulative distribution function.

To understand the selected estimate, note that the covariance between \(\hat{\delta}_1\) and the comparison \(Q\) is

Thus the shared control raises the covariance between the two effects, but cancels from the covariance between arm 1’s estimate and the difference used to select it. For jointly normal variables, the conditional mean of \(\hat{\delta}_1\) given \(Q\) is its mean plus \(\operatorname{Cov}(\hat{\delta}_1,Q)/s^2\) times \(Q-\mu\). Averaging this expression over \(Q>0\) gives the conditional mean after arm 1 wins.

For clarity, set \(a=-\mu/s\) and \(\lambda(a)=\phi(a)/[1-\Phi(a)]\), where \(\phi\) is the standard normal density. Truncating a normal variable below at zero gives \(E(Q\mid Q>0)=\mu+s\lambda(a)\). Therefore,

The added term is positive: among repetitions in which arm 1 wins, its estimate tends to be higher than its unconditional mean. The amount depends on the treatment-arm sampling variance \(v_1\), the variability \(s\) of the comparison, and how difficult it is for arm 1 to win, as encoded by \(a\). The control variance \(v_0\) does not appear in this two-arm conditional-mean expression because the same control contribution cancels in the selection comparison.

Conditional Variability and a Comparison

Selection changes variability as well as the mean. Under the joint normal model, the variance of \(\hat{\delta}_1\) conditional on \(Q>0\) can be obtained by separating its component predictable from \(Q\) from its residual component. The residual variance is \(\operatorname{Var}(\hat{\delta}_1)-v_1^2/s^2\), and the predictable component has conditional variance \((v_1/s)^2\operatorname{Var}(Z\mid Z>a)\), where \(Z=(Q-\mu)/s\) is standard normal. Since \(\operatorname{Var}(Z\mid Z>a)=1+a\lambda(a)-\lambda(a)^2\), combination and simplification yield

This is a variance, in squared outcome units. It is generally not the unconditional variance: the selection condition truncates the comparison, and the estimate contains a component associated with that comparison. The conditional distribution of the estimate is not generally normal, even though the unconditioned estimates are jointly normal.

Diagnostics, Alternatives, and Limitations

A practical check is to compare the covariance structure implied by the design with the structure used in calculations or software. If all contrasts use one control group, their pairwise covariance should include that control mean’s variance. The following checks help identify common problems:

• Check the design. Confirm whether control observations are genuinely reused across contrasts. If control samples are disjoint, the shared-control covariance formula does not apply.
• Check the variance scale. \(v_i\) is a variance of a sample mean, not a standard deviation. The covariance \(v_0\) has squared outcome units.
• Check the comparison. When the decision is based on \(\hat{\delta}_1-\hat{\delta}_2\), verify that the shared control cancels and that its variance is \(v_1+v_2\).
• Check the selection rule. The formulas condition on choosing the larger of these two estimates. A different rule or additional selection stages changes the event being conditioned on.

An alternative design or analysis might use separate controls, model group means jointly, or account for the covariance directly when evaluating selection. Treating the contrasts as independent is not a harmless simplification: it changes the variance of their difference and hence the selection probability and conditional distribution. Conversely, using the shared-control covariance alone does not solve every post-selection estimation problem. The calculations here assume known variances, normal sampling, independent groups, and a single comparison between two arms. Unequal variances are allowed through \(v_0,v_1,v_2\), but departures from the model, small-sample variance estimation, multiple selection stages, and more complex rules require additional analysis.