Covariance Formula for General Contrasts

Why a General Formula Helps

When two comparisons use some of the same groups, their estimates can vary together. The direction and size of that covariance depend on which group means they share, the coefficients attached to those means, and the variances of the means. A formula for arbitrary contrasts makes those contributions explicit, rather than requiring a separate argument for each pair of comparisons.

The earlier material on how contrast signs determine dependence gives useful intuition for shared groups. Here the goal is a general result: start with any two sets of contrast coefficients and calculate their covariance directly. The result does not require equal allocation, equal group variances, or normally distributed outcomes.

Definition: For \(G\) groups with means \(\mu_1,\ldots,\mu_G\), a linear contrast is a linear combination of the group means, \(L_a=\sum_{g=1}^{G}a_g\mu_g\), whose coefficients satisfy \(\sum_{g=1}^{G}a_g=0\). Its sample estimate is \(\hat L_a=\sum_{g=1}^{G}a_g\bar{x}_g\), where \(\bar{x}_g\) is the sample mean in group \(g\). A linear combination whose coefficients do not sum to zero can still be analyzed by the covariance formula below, though it is not a contrast under this definition.

Model and Covariance Formula

Suppose group \(g\) contributes \(n_g\) independent observations, with population mean \(\mu_g\) and finite variance \(\sigma_g^2\). Assume that observations are independent across groups and, within each group, have the same mean and variance. Then the random group mean \(\bar{X}_g\) has mean \(\mu_g\) and variance \(\operatorname{Var}(\bar{X}_g)=\sigma_g^2/n_g\). The group means are independent. Define \(v_g=\sigma_g^2/n_g\) as the variance of the group mean. Normality is not needed for these mean and covariance calculations.

For two coefficient sets \(a_1,\ldots,a_G\) and \(b_1,\ldots,b_G\), define the random linear combinations \(L_a=\sum_{g=1}^{G}a_g\bar{X}_g\) and \(L_b=\sum_{g=1}^{G}b_g\bar{X}_g\). If both coefficient sets sum to zero, these are estimates of linear contrasts; the covariance result itself holds whether or not the coefficients sum to zero.

Theorem: If the group means \(\bar{X}_1,\ldots,\bar{X}_G\) are independent and have finite variances \(v_1,\ldots,v_G\), then

\operatorname{Cov}(L_a,L_b) = \sum_{g=1}^{G}a_gb_gv_g = \sum_{g=1}^{G}a_gb_g\frac{\sigma_g^2}{n_g}.

The coefficient product \(a_gb_g\) determines the sign of group \(g\)'s contribution, while \(v_g\) determines its scale. A group contributes nothing when either contrast has coefficient zero for that group. Contributions from different groups can add or cancel.

Derivation

Covariance is linear in each argument when the coefficients are fixed. Expanding both linear combinations therefore gives a double sum. Independence makes every covariance between different group means zero, leaving only terms that use the same group mean.

\begin{aligned} \operatorname{Cov}(L_a,L_b) &=\operatorname{Cov}\left(\sum_{g=1}^{G}a_g\bar{X}_g,\, \sum_{h=1}^{G}b_h\bar{X}_h\right)\\ &=\sum_{g=1}^{G}\sum_{h=1}^{G} a_gb_h\operatorname{Cov}(\bar{X}_g,\bar{X}_h)\\ &=\sum_{g=1}^{G}a_gb_g\operatorname{Var}(\bar{X}_g)\\ &=\sum_{g=1}^{G}a_gb_gv_g. \end{aligned}

For the third line, terms with \(g\ne h\) vanish because independent random variables have covariance zero. For \(g=h\), the covariance is the variance \(v_g\). Substituting \(v_g=\sigma_g^2/n_g\) yields the second form of the theorem. This argument also shows why equality of group variances or sample sizes is unnecessary: each group simply supplies its own group-mean variance.

Worked Example: Two Comparisons Sharing a Control

Consider a hypothetical study with a control group \(C\) and treatment groups \(A\) and \(B\). Suppose the independent group means have variances \(v_C=4\), \(v_A=9\), and \(v_B=16\), in squared outcome units. Let the first contrast be treatment \(A\) minus control, and the second be treatment \(B\) minus control. In the order \(C,A,B\), their coefficient vectors are \(a=(-1,1,0)\) and \(b=(-1,0,1)\); each sums to zero.

The covariance is the sum of the coefficient products times the corresponding group-mean variances:

\begin{aligned} \operatorname{Cov}(L_a,L_b) &=(-1)(-1)(4)+(1)(0)(9)+(0)(1)(16)\\ &=4+0+0=4. \end{aligned}

The covariance is \(4\) squared outcome units. Only the control mean contributes, because it is the only group with a nonzero coefficient in both contrasts. Both comparisons subtract that same mean, so their control coefficients have a positive product.

As a check, the contrast variances are \(\operatorname{Var}(L_a)=(-1)^2(4)+(1)^2(9)=13\) and \(\operatorname{Var}(L_b)=(-1)^2(4)+(1)^2(16)=20\). The covariance is positive and smaller than either variance, as is consistent with these values.

Coefficient Products Can Cancel

Sharing groups does not guarantee a nonzero covariance. A group with a positive coefficient product contributes positively; a group with a negative product contributes negatively. If the weighted contributions balance exactly, the covariance is zero even when both contrasts involve the same group means. Checking every product, rather than just counting shared groups, is essential.

Worked Example: Shared Groups With Zero Covariance

Suppose four independent group means, in the order \(C,A,B,D\), have variances \(4,4,9,16\) squared outcome units. Consider contrasts with coefficient vectors \(a=(1,-1,0,0)\) and \(b=(1,1,-2,0)\). Both coefficient sets sum to zero. The first compares \(C\) with \(A\); the second combines all three of \(C,A,B\).

Applying the formula gives

\begin{aligned} \operatorname{Cov}(L_a,L_b) &=(1)(1)(4)+(-1)(1)(4)+(0)(-2)(9)+(0)(0)(16)\\ &=4-4+0+0=0. \end{aligned}

The \(C\) contribution is \(4\), while the \(A\) contribution is \(-4\); they cancel. The contrasts share group means, but their covariance is zero under this model. This establishes zero covariance, not a general claim of independence. Covariance alone does not establish independence unless additional distributional conditions are available.

Using Sample Sizes and Group Variances

In applications, group-mean variances are often calculated from the sampling model as \(\sigma_g^2/n_g\). This makes the role of allocation explicit: increasing a group’s sample size reduces that group’s contribution to the covariance, provided its population variance and the contrast coefficients remain fixed. The coefficients themselves do not change merely because sample sizes change.

Worked Example: Disjoint Group Comparisons With Unequal Allocation

Suppose a hypothetical four-group study has independent observations within and across groups. The group sample sizes and population standard deviations are \(n_C=25,\sigma_C=10\); \(n_A=16,\sigma_A=8\); \(n_B=20,\sigma_B=10\); and \(n_D=25,\sigma_D=5\), with outcome measured in points. The group means are observed as \(\bar{x}_C=10,\bar{x}_A=12,\bar{x}_B=13,\bar{x}_D=14\). Compare \(A-C\) with \(D-B\).

The estimated contrasts are \(12-10=2\) points and \(14-13=1\) point. Their coefficient vectors in the order \(C,A,B,D\) are \(a=(-1,1,0,0)\) and \(b=(0,0,-1,1)\). Each is a valid contrast, and they involve disjoint pairs of groups.

First calculate each group-mean variance:

v_C=\frac{10^2}{25}=4,\quad v_A=\frac{8^2}{16}=4,\quad v_B=\frac{10^2}{20}=5,\quad v_D=\frac{5^2}{25}=1.

Then substitute the coefficients and variances:

\begin{aligned} \operatorname{Cov}(L_a,L_b) &=(-1)(0)(4)+(1)(0)(4)+(0)(-1)(5)+(0)(1)(1)\\ &=0. \end{aligned}

The covariance is zero squared points. Each product is zero because no group mean appears in both contrasts. The assumed independence of the group means justifies this conclusion; the unequal sample sizes and variances do not change it.

Assumptions, Checks, and Alternatives

The central assumption for the displayed formula is independence of the group means. Independent observations across groups usually provide this in a parallel-group trial. If group means are correlated—for example, if the same individuals contribute to multiple means—the off-diagonal covariances need not vanish, and the formula as stated is not valid. In that case, the full expansion in the derivation still applies, but the nonzero cross-group terms must be included.

A practical calculation should be checked in three ways. First, align each coefficient with the correct group and its variance; a reordering error can change the answer. Second, calculate each product \(a_gb_gv_g\) separately and retain its sign. Third, check units: if the outcome is measured in points, a covariance has units of squared points. A covariance is not a correlation; it is not unit-free or restricted to the interval from \(-1\) to \(1\).

There are two convenient ways to calculate the result. The general coefficient formula is systematic and works for any number of groups. A pairwise approach identifies groups shared by both contrasts and sums only their contributions; it can be quicker when most coefficients are zero. These are not different statistical methods: the pairwise shortcut is simply the same formula with zero terms omitted.

Common pitfall: “The contrasts share a group, so their covariance must be positive.” This is wrong because the shared group’s contribution is \(a_gb_gv_g\), whose sign depends on the product \(a_gb_g\). Different shared groups can also contribute with opposite signs and cancel. Calculate the signed terms before interpreting the result.

The formula is limited by the variance inputs and independence assumption. If the \(\sigma_g^2\) values are unknown and replaced by estimates, the resulting covariance is an estimated quantity rather than the exact model covariance. If the group means are dependent, the omitted cross-group terms matter. Finally, a zero covariance is a statement about linear co-variation; by itself, it does not rule out other forms of dependence.

Check Your Understanding

For each question, use the independent-group-means formula. Solutions are given below the questions.

Two contrasts have coefficient vectors \(a=(1,-1,0)\) and \(b=(0,-1,1)\), in that group order. If the group-mean variances are \(2,5,7\), find their covariance.
With the same group-mean variances \(2,5,7\), use \(a=(-1,1,0)\) and \(b=(-1,0,1)\). Find the covariance and identify its source.
Why does \(\sum_g a_g=0\) define a contrast, while not being necessary for the covariance formula?
If two contrasts have zero covariance, what additional conclusion about independence can be made without further assumptions?

Solutions. (1) The coefficient products are \(0,(-1)(-1)=1,(0)(1)=0\), so the covariance is \(0(2)+1(5)+0(7)=5\) squared outcome units. (2) The products are \(1,0,0\), so the covariance is \(1(2)+0(5)+0(7)=2\) squared outcome units. It comes from the first group mean, which both contrasts subtract. (3) Coefficients summing to zero make the combination compare group means without changing if the same constant is added to every group mean. The covariance expansion only uses linearity and the covariances of the group means, so it also applies to linear combinations that do not sum to zero. (4) None in general: zero covariance alone does not imply independence.)