Covariance Between Arm-to-Arm Contrasts

Why the Pair of Arms Matters

A contrast compares two arm means, so its uncertainty depends on the uncertainty in both means. When two contrasts include the same arm, their estimates can vary together even if the individual observations in different arms are independent. To calculate that covariance correctly, identify the shared arm and the sign with which its mean enters each contrast.

This tutorial focuses on contrasts formed from differences between two arm means. It includes treatment-control contrasts and treatment-treatment contrasts. The result is a covariance in squared outcome units; it is not itself a correlation, which is unitless and also depends on the two contrast variances.

Model and Notation

Let arm \(g\) contain \(n_g\) observations \(Y_{g1},\ldots,Y_{gn_g}\), with mean \(\mu_g\) and variance \(\sigma_g^2\). Assume observations are independent within and between arms, and that observations within arm \(g\) have the same mean and variance. The arm sample mean is \(\bar{x}_g\), and its variance is \(\operatorname{Var}(\bar{x}_g)=\sigma_g^2/n_g\). Write \(v_g=\sigma_g^2/n_g\) for this variance. Normality is not required for these covariance results; finite variances and the stated independence are sufficient.

For a control arm labelled 0 and treatment arms labelled 1, 2, and so on, define a treatment-control contrast as \(C_i=\bar{x}_i-\bar{x}_0\). Define a treatment-treatment contrast as \(T_{ij}=\bar{x}_i-\bar{x}_j\). A minus sign is part of the definition: reversing the order of the arms reverses the contrast.

Deriving the Pairwise Rules

The basic calculation follows from expanding the contrasts and using independence. For distinct treatment arms \(i\) and \(j\), both \(C_i\) and \(C_j\) contain the same control mean with coefficient \(-1\). All other arm means in these two contrasts are independent of one another and of the control mean. Consequently, only the control term remains:

The plus sign in the last line comes from multiplying the two control coefficients, \((-1)(-1)=1\). The control means are the same random quantity, not two independent copies. This is why the covariance is \(v_0\), rather than zero.

For a treatment-control contrast and a treatment-treatment contrast, consider \(C_i\) and \(T_{jk}\), where \(j\ne k\). If \(i=j\), the shared treatment mean has coefficient \(+1\) in both contrasts, giving covariance \(v_i\). If \(i=k\), that treatment mean has coefficients \(+1\) and \(-1\), giving covariance \(-v_i\). If \(i\) is neither \(j\) nor \(k\), the contrasts have no arm mean in common, so their covariance is zero under the model. For two treatment-treatment contrasts, the same reasoning applies to any arm they share: same-sign coefficients give a positive contribution, opposite-sign coefficients a negative one. If they share no arm, their covariance is zero.

These rules are consequences of the model, not a general guarantee that unrelated-looking estimates are independent in every study. Shared participants, clustering, or other dependence between arms would invalidate the assumption that distinct arm means have zero covariance.

Worked Pairwise Calculations

Checks, Alternatives, and Limitations

A reliable calculation can be organized as a short sequence. Write each contrast explicitly as a difference of arm means, identify any shared arms, and multiply the two coefficients for each shared arm by that arm mean’s variance. Add the contributions if more than one arm is shared. Under independent arms, pairs with no shared arm have covariance zero.

The direct expansion of covariance is a useful alternative when the shared-arm pattern is not immediately clear. It makes the assumptions visible and helps catch a sign error. The shared-arm lookup is faster for routine pairwise calculations. A correlation calculation is useful when the goal is a unitless measure of association between two contrast estimates, but it requires both contrast variances as well as their covariance. It cannot replace the covariance when a covariance matrix is needed.

A separate control group for each treatment changes which random means are shared; it does not justify carrying over the shared-control covariance. Likewise, if outcomes in different arms are dependent, the zero-covariance contributions assumed above may no longer be zero. These formulas also describe the covariance of the contrasts under the specified variance model; they do not by themselves establish a causal treatment effect or address multiplicity.

Common warning signs include a covariance with outcome units rather than squared outcome units, a nonzero result for contrasts with no shared arm under the independence assumption, or a positive sign when the sole shared arm appears with opposite signs. If the arm variances are estimated rather than specified, substituting estimates gives an estimated covariance, not an exact known value. The calculations above use specified variances and do not account for uncertainty in variance estimates.