Covariance Propagation Under Within-Subject Contrasts

Why Covariance Changes Under Contrasts

A within-subject contrast is a linear combination of measurements from the same subject. Transforming a subject’s measurement vector into several contrasts changes not only the mean coordinates but also their covariance. In particular, two contrasts can be correlated even when their coefficient vectors are orthogonal in the usual Euclidean inner product. The covariance after transformation depends on both the contrast coefficients and the covariance among the original measurements.

This propagation matters because multivariate analyses of repeated measurements use the covariance in the coordinates actually analyzed. The covariance matrix of the original measurements cannot simply be carried over unchanged: it must be transformed along with the responses.

Definition: Let $Y=(Y_1,\ldots,Y_p)'$ be a random vector of $p$ repeated measurements, with finite second moments, mean vector $\mu=E[Y]$, and covariance matrix $\Sigma=\operatorname{Cov}(Y)$. Let $C$ be a fixed $p\times q$ matrix whose columns specify $q$ linear contrasts. The transformed response is $Z=C'Y$. Its covariance matrix is the propagated covariance of the contrasts.

The $j$th transformed response is $Z_j=c_j'Y$, where $c_j$ is the $j$th column of $C$. Thus each entry of the transformed covariance matrix describes a pair of contrasts: its $j$th diagonal entry is the variance of contrast $j$, and its $(j,k)$ entry is the covariance between contrasts $j$ and $k$.

Theorem: For any fixed $p\times q$ matrix $C$ and any random vector $Y$ with finite second moments, $$ E[Z]=C'\mu,\qquad \operatorname{Cov}(Z)=C'\Sigma C,\quad\text{where }Z=C'Y. $$ In particular, the $(j,k)$ entry of $C'\Sigma C$ is $c_j'\Sigma c_k=\operatorname{Cov}(c_j'Y,c_k'Y)$.

Deriving the Propagation Formula

The formula follows directly from the definition of covariance. The assumptions needed are that $C$ is nonrandom and fixed, and that the components of $Y$ have finite second moments so the displayed expectations and covariances exist.

Transform the mean.
Linearity of expectation gives $E[Z]=E[C'Y]=C'E[Y]=C'\mu$.

Center the transformed vector.
Since $Z-E[Z]=C'(Y-\mu)$, the same fixed linear map acts on the centered measurements.

Take the expected outer product.
Using the definition $\operatorname{Cov}(Z)=E[(Z-E[Z])(Z-E[Z])']$, pull the fixed matrices outside the expectation.

\begin{aligned} \operatorname{Cov}(Z) &=E\!\left[C'(Y-\mu)(Y-\mu)'C\right]\\ &=C'E\!\left[(Y-\mu)(Y-\mu)'\right]C\\ &=C'\Sigma C. \end{aligned}

For individual entries, the same argument gives $\operatorname{Cov}(Z_j,Z_k)=\operatorname{Cov}(c_j'Y,c_k'Y)=c_j'\Sigma c_k$. This entrywise expression is useful for checking matrix calculations and for interpreting which original measurement covariances contribute to a contrast covariance.

Worked Covariance Calculations

Worked Example: Two Adjacent Differences With Correlated Measurements

Suppose a subject has three repeated measurements with covariance matrix \[ \Sigma=\begin{pmatrix}4&2&1\\2&4&2\\1&2&4\end{pmatrix}. \] Consider the adjacent differences $Z_1=Y_1-Y_2$ and $Z_2=Y_2-Y_3$, so \[ C=\begin{pmatrix}1&0\\-1&1\\0&-1\end{pmatrix}. \] Calculate the full covariance matrix of the differences.

The contrast coefficient vectors are $c_1=(1,-1,0)'$ and $c_2=(0,1,-1)'$. First calculate the two products with $\Sigma$:

\Sigma c_1= \begin{pmatrix}2\\-2\\-1\end{pmatrix}, \qquad \Sigma c_2= \begin{pmatrix}1\\2\\-2\end{pmatrix}.

The variances are $c_1'\Sigma c_1=2+2=4$ and $c_2'\Sigma c_2=2-(-2)=4$. The covariance is $c_1'\Sigma c_2=1-2=-1$. Therefore,

C'\Sigma C= \begin{pmatrix}4&-1\\-1&4\end{pmatrix}.

The adjacent differences have equal variance but are negatively correlated. In particular, the shared measurement $Y_2$ enters with opposite signs in the two differences; its contribution is part of why the covariance is not zero. The correlation is $-1/\sqrt{4\cdot4}=-1/4$.

Worked Example: Orthogonal Contrasts Under a Symmetric Covariance

Take three measurements with \[ \Sigma=\begin{pmatrix}5&2&2\\2&5&2\\2&2&5\end{pmatrix}, \] and use the contrasts $c_1=(1,-1,0)'$ and $c_2=(1,1,-2)'$. These are orthogonal because $c_1'c_2=1-1+0=0$. Find their covariance matrix.

Write $\Sigma=3I+2\mathbf{1}\mathbf{1}'$, where $I$ is the $3\times3$ identity matrix and $\mathbf{1}=(1,1,1)'$. Both contrast vectors sum to zero, so $\mathbf{1}'c_1=\mathbf{1}'c_2=0$. Consequently, $\mathbf{1}\mathbf{1}'c_j=0$ for each $j$, and $\Sigma c_j=3c_j$.

The diagonal entries are $c_1'\Sigma c_1=3c_1'c_1=3(2)=6$ and $c_2'\Sigma c_2=3c_2'c_2=3(6)=18$. The off-diagonal entry is $c_1'\Sigma c_2=3c_1'c_2=0$. Thus,

C'\Sigma C=\begin{pmatrix}6&0\\0&18\end{pmatrix}.

Here the contrasts are uncorrelated, but the conclusion uses the particular covariance structure as well as their orthogonality. The result does not follow from $c_1'c_2=0$ alone.

Worked Example: Orthogonal Contrasts That Are Still Correlated

Keep $c_1=(1,-1,0)'$ and $c_2=(1,1,-2)'$, which satisfy $c_1'c_2=0$, but now let \[ \Sigma=\begin{pmatrix}1&0&0\\0&4&0\\0&0&9\end{pmatrix}. \] Calculate the propagated covariance and check whether the contrasts are uncorrelated.

Multiplication gives $\Sigma c_1=(1,-4,0)'$ and $\Sigma c_2=(1,4,-18)'$. Therefore, $c_1'\Sigma c_1=1+4=5$, $c_2'\Sigma c_2=1+4+36=41$, and $c_1'\Sigma c_2=1-4=-3$. The transformed covariance is

C'\Sigma C=\begin{pmatrix}5&-3\\-3&41\end{pmatrix}.

The off-diagonal entry is nonzero, so these orthogonal contrasts are correlated. The unequal measurement variances mean that $\Sigma$ does not act as a common scalar multiple of the identity on the contrast vectors.

Positive Semidefiniteness and Nonsingularity

Covariance matrices must be positive semidefinite: every linear combination of their coordinates has nonnegative variance. The propagation formula preserves this property, whether or not the original covariance or contrast matrix is invertible.

Theorem: If $\Sigma$ is positive semidefinite, then $C'\Sigma C$ is positive semidefinite. If $\Sigma$ is positive definite, then $C'\Sigma C$ is positive definite exactly when the columns of $C$ are linearly independent.

To prove the first claim, take any $a\in\mathbb{R}^q$. Then

a'C'\Sigma Ca=(Ca)'\Sigma(Ca)\geq 0,

because $\Sigma$ is positive semidefinite. This is precisely the defining quadratic-form condition for $C'\Sigma C$ to be positive semidefinite.

Now suppose $\Sigma$ is positive definite. If the columns of $C$ are linearly independent, then $Ca\ne0$ for every nonzero $a$. Positive definiteness of $\Sigma$ implies $(Ca)'\Sigma(Ca)>0$ for every such $a$, so $C'\Sigma C$ is positive definite. Conversely, if the columns of $C$ are linearly dependent, some nonzero $a$ satisfies $Ca=0$. For that $a$, $a'C'\Sigma Ca=0$, so $C'\Sigma C$ is not positive definite. This proves both directions. $\square$

This result provides a useful check: a computed contrast covariance cannot have a negative variance or a negative quadratic form. Also, even with a positive definite $\Sigma$, redundant contrast columns make the transformed covariance singular.

Interpretation and a Common Pitfall

The contrast covariance is the covariance relevant to analyses carried out on the transformed responses. Its off-diagonal entries quantify dependence between contrast scores across subjects; its diagonal entries quantify their marginal variances. The transformed mean is $C'\mu$, so the same matrix $C$ determines which mean combinations are represented and how their variability is measured.

A common mistake is to argue that orthogonal contrast vectors must produce uncorrelated contrast scores. Orthogonality means $c_j'c_k=0$, which concerns the coefficient vectors under the ordinary Euclidean inner product. The actual covariance is $c_j'\Sigma c_k$. These quantities agree up to a common scalar only in special cases, such as $\Sigma=\sigma^2 I$. As the diagonal-covariance example shows, unequal measurement variances can make $c_j'\Sigma c_k\ne0$ despite $c_j'c_k=0$.

Another useful check is the scale of a contrast. Multiplying a contrast vector by a constant $a$ multiplies its variance by $a^2$; multiplying one of two contrasts by $a$ multiplies their covariance by $a$. Therefore, differently normalized versions of the same contrast have different covariance entries, even though they encode the same direction of comparison.

Key takeaway: For a fixed contrast matrix $C$, transform the within-subject covariance using $C'\Sigma C$. Interpret its diagonal as contrast variances and its off-diagonal entries as covariances; do not infer zero covariance from Euclidean orthogonality alone.

Check Your Understanding

For each question, use the contrast covariance formula and state what the result means.

If $Z=C'Y$, what assumptions are needed for $\operatorname{Cov}(Z)=C'\Sigma C$, and how does the entry for contrasts $j$ and $k$ read?
Why does $c_j'c_k=0$ not, in general, imply $\operatorname{Cov}(c_j'Y,c_k'Y)=0$?
If a contrast is multiplied by $-2$, by what factor does its variance change? By what factor does its covariance with a different unchanged contrast change?
When $\Sigma$ is positive definite, what condition on $C$ ensures that $C'\Sigma C$ is positive definite?
In the three-measurement adjacent-difference example, what does the negative off-diagonal entry say about the two contrast scores?