Why Contrast Direction Matters
In a multi-arm study, several comparisons can reuse the same group means. A contrast might compare a treatment with control, while another compares control with a treatment or compares two treatments. The group means involved determine how much uncertainty the comparisons share; the signs of their coefficients determine whether that shared uncertainty contributes positively or negatively to their covariance.
The direction of a comparison is part of its definition. For example, treatment minus control and control minus treatment have opposite estimates, but the same variance. When several contrasts are assembled into a covariance matrix, changing one direction changes the signs of its covariances with the other contrasts as well.
For a treatment-control comparison oriented as treatment minus control, the treatment receives coefficient \(+1\) and control receives coefficient \(-1\). Reversing the comparison reverses both signs. Other contrasts, such as a weighted comparison among several treatments, can use coefficients beyond \(+1\) and \(-1\), provided they sum to zero.
Assumptions and the Covariance Formula
Suppose there are \(G\) groups, indexed by \(g\), and the group means are calculated from independent samples. Let \(v_g=\operatorname{Var}(\bar{x}_g)\). Under independent observations within group \(g\), with population variance \(\sigma_g^2\) and sample size \(n_g\), \(v_g=\sigma_g^2/n_g\). The covariance calculation below requires independent group means with finite variances and fixed contrast coefficients. It does not require normally distributed observations.
If the group variances are unknown, estimate \(v_g\) by \(s_g^2/n_g\), where \(s_g\) is the sample standard deviation in group \(g\). Substituting these estimated variances gives an estimated contrast covariance matrix. This plug-in estimate is distinct from the population covariance matrix, which depends on the unknown \(\sigma_g^2\).
For two contrasts, the covariance is a sum of contributions from the group means they share. A group contributes only if both contrasts have a nonzero coefficient for that group. A positive product of coefficients contributes positively; a negative product contributes negatively.
To arrange several contrasts together, place their coefficient vectors in the rows of a matrix \(C\), and place the group-mean variances on the diagonal of \(V=\operatorname{diag}(v_0,\ldots,v_{G-1})\). Then the contrast covariance matrix is
The diagonal entries are contrast variances; the off-diagonal entries are covariances between pairs of contrasts. The standard error of contrast \(i\) is the square root of its variance. A correlation is obtained by dividing a covariance by the product of the two standard errors; unlike a covariance, it has no measurement units.
Worked Example: Three Contrasts With Mixed Directions
Worked Example: Three Contrasts With Mixed Directions
Hypothetical data. Consider one control group, indexed by 0, and three treatment groups, indexed by 1, 2, and 3. Their sample means are \(10,12,8,\) and \(11\) points, respectively. Suppose \(n_0=20,n_1=16,n_2=25,n_3=16\), and the sample standard deviations are \(s_0=4,s_1=4,s_2=5,s_3=4\) points. Assume independent samples across groups.
Using \(s_g^2/n_g\), the estimated variances of the group means are \(v_0=16/20=0.8\), \(v_1=16/16=1\), \(v_2=25/25=1\), and \(v_3=16/16=1\), in squared points. Define three contrasts: treatment 1 minus control, control minus treatment 2, and treatment 3 minus treatment 1. The second contrast deliberately points in the reverse direction from treatment minus control.
Each row sums to zero, as required for a contrast. The estimates, in the same order, are \(12-10=2\) points, \(10-8=2\) points, and \(11-12=-1\) point. In particular, control minus treatment 2 is \(+2\), not \(-2\).
The first variance is \(v_0+v_1=0.8+1=1.8\) squared points. The second variance is \(v_0+v_2=0.8+1=1.8\), and the third is \(v_1+v_3=1+1=2\). For the first and second contrasts, the shared control coefficients have opposite signs, so their covariance is \((-1)(1)(0.8)=-0.8\) squared points. The first and third contrasts share group 1 with coefficients \(1\) and \(-1\), giving covariance \((1)(-1)(1)=-1\). The second and third contrasts have no group with nonzero coefficients in both rows, so their covariance is zero.
The estimated standard errors are \(\sqrt{1.8}=1.342\), \(\sqrt{1.8}=1.342\), and \(\sqrt{2}=1.414\) points. The estimated correlation between contrasts 1 and 2 is \(-0.8/\sqrt{1.8(1.8)}=-0.4444\); between contrasts 1 and 3 it is \(-1/\sqrt{1.8(2)}=-0.5270\); and between contrasts 2 and 3 it is zero. Thus the first two estimates tend to move in opposite directions under repeated sampling, as do the first and third, under this independent-group model. The zero covariance for the second and third follows from their using disjoint group means.
Deriving the Matrix Formula
The scalar formula follows by expanding each contrast as a sum. Covariance is linear in each argument, so the covariance of two weighted sums is the sum of all pairwise coefficient products times the corresponding group-mean covariances. Independence makes the covariance between means from different groups zero. Only terms involving the same group remain.
The \((i,j)\) entry of \(CVC^\mathsf{T}\) is precisely \(\sum_g c_{ig}v_gc_{jg}\), which proves that the matrix expression collects those pairwise formulas. If the independent-sampling assumption does not hold, the cross-group covariances need not be zero, and this diagonal-\(V\) calculation is not appropriate.
Worked Example: Reversing One Contrast
Worked Example: Reversing One Contrast
Use the same hypothetical groups and variances as in the preceding example, but replace the second contrast, control minus treatment 2, with treatment 2 minus control. Its coefficient row is now \((-1,0,1,0)\), and its estimate is \(8-10=-2\) points. Reversing the direction multiplies the entire contrast by \(-1\), so its variance remains \(1.8\) squared points.
The covariance with treatment 1 minus control is now \( (-1)(-1)(0.8)=0.8\) squared points: both contrasts have coefficient \(-1\) for control. Its covariance with treatment 3 minus treatment 1 remains zero because the two contrasts have no common group with nonzero coefficients. The covariance matrix becomes
The correlation between the first two contrasts changes from \(-0.4444\) to \(+0.4444\), while their standard errors do not change. This is a change in parameterization and direction, not a change in the underlying group means or in the amount of uncertainty for that individual comparison.
Worked Example: A Treatment-Control Contrast and a Treatment-Treatment Contrast
Worked Example: A Treatment-Control Contrast and a Treatment-Treatment Contrast
Again use the hypothetical means and estimated group-mean variances above. Compare treatment 1 with control, \(\hat{\theta}_A=\bar{x}_1-\bar{x}_0=12-10=2\) points, and compare treatment 2 with treatment 1, \(\hat{\theta}_B=\bar{x}_2-\bar{x}_1=8-12=-4\) points. Their coefficient rows in group order \(0,1,2,3\) are \((-1,1,0,0)\) and \((0,-1,1,0)\); both rows sum to zero.
The variances are \(0.8+1=1.8\) and \(1+1=2\) squared points. They share treatment group 1, with coefficients \(+1\) and \(-1\), so their covariance is \((1)(-1)(1)=-1\) squared points. Their estimated correlation is \(-1/\sqrt{1.8(2)}=-0.5270\). Sharing a treatment group can therefore create dependence just as sharing control can; the sign depends on the product of the two coefficients for that shared group.
Reading, Checking, and Using the Matrix
A useful interpretation starts with the diagonal, then examines the shared groups behind each off-diagonal entry. For each contrast, the diagonal gives the estimated variance and hence the standard error in the outcome’s units. For each pair, identify groups appearing in both contrasts, multiply their coefficients, weight by the corresponding group-mean variance, and add the contributions. This short calculation explains why an entry is positive, negative, or zero.
A covariance is expressed in squared outcome units. For example, \(-0.8\) squared points is not a correlation of \(-0.8\). The correlation rescales that covariance by both standard errors and is unitless. When presenting results, keep these quantities distinct and label the units of the contrasts and variances.
The matrix formulation and the pairwise formula are equivalent, but useful in different ways. The pairwise formula makes the role of shared groups and signs transparent for one pair. The matrix formula is compact and convenient when many contrasts are considered together. For a small calculation, checking a few entries directly from the coefficient rows is a practical way to catch sign or indexing errors.
The calculation relies on independent group means. If groups are paired, clustered, or otherwise dependent, their cross-group covariances may be nonzero and must be included; simply putting \(v_g\) on a diagonal would omit them. Also, sample-variance substitution estimates the covariance matrix but does not make the unknown population variances known. These calculations describe covariance among estimates; they do not by themselves specify a multiple-testing procedure or an interim decision rule.
Common Mistakes
- Calling any linear combination a contrast. A contrast must have coefficients summing to zero. Check each coefficient row before applying the contrast terminology.
- Ignoring direction. Treatment minus control and control minus treatment have equal variances but opposite estimates, and their covariances with other contrasts reverse sign.
- Assuming shared control always means positive covariance. Its contribution is the product of the control coefficients times the control mean variance. Opposite-signed coefficients produce a negative contribution.
- Confusing covariance with correlation. Covariance carries squared outcome units; correlation is unitless and standardizes by both standard errors.
- Using independent-group formulas for dependent means. When group means are dependent, cross-group covariance terms cannot be discarded.
Check Your Understanding
Try each question, then compare your work with the solutions below.
- For groups 0, 1, and 2, a contrast has coefficients \((1,-1,0)\). Does it satisfy the contrast condition? If the sample means are \(5,7,\) and \(4\), what is its estimate?
- Suppose the independent group-mean variances for groups 0, 1, and 2 are \(2,3,\) and \(5\). Calculate the covariance of contrasts \(\bar{x}_0-\bar{x}_2\) and \(\bar{x}_1-\bar{x}_2\), and give their variances.
- What happens to a contrast’s variance and its covariance with every other contrast when its direction is reversed?
- Two contrasts use no group mean with nonzero coefficients in both. Under the independent-group model, what is their covariance, and why?
Solutions. (1) The coefficients sum to \(1-1+0=0\), so this is a contrast; its estimate is \(5-7=-2\) units. (2) The coefficient rows are \((1,0,-1)\) and \((0,1,-1)\). Their variances are \(2+5=7\) and \(3+5=8\); their covariance is \((1)(0)(2)+(0)(1)(3)+(-1)(-1)(5)=5\). (3) Multiplying a contrast by \(-1\) leaves its variance unchanged, because \((-1)^2=1\), and reverses the sign of its covariance with every other contrast. (4) The covariance is zero: every same-group coefficient product is zero, and different-group means have zero covariance under the stated independence assumption.