Orthogonal Contrast Bases and Invariance

Why the Choice of Contrast Basis Matters

A within-subject effect is described by differences among measurement occasions, not by a common shift in all occasions. A contrast basis supplies coordinates for those differences. Different bases can describe the same space of differences: one basis may be convenient for interpretation, while another may make the coordinates orthogonal. The central issue is whether a change of coordinates merely re-expresses the same effect or discards part of it.

Let there be $k$ occasions, and write a mean profile as $\mu\in\mathbb{R}^k$. A contrast vector assigns weights summing to zero, so that its inner product with $\mu$ is unaffected by adding the same constant to every occasion. The collection of all such vectors is the contrast space. The results below show how an orthogonal basis decomposes this space and why any two full bases of it give the same omnibus null hypothesis.

Definition: A vector $c\in\mathbb{R}^k$ is a contrast vector if $c'\mathbf{1}_k=0$, where $\mathbf{1}_k$ is the vector of $k$ ones. A collection of nonzero contrast vectors $c_1,\ldots,c_q$ is orthogonal if $c_i'c_j=0$ whenever $i\ne j$. It is an orthogonal contrast basis for a subspace $\mathcal{S}$ if its vectors are pairwise orthogonal and span $\mathcal{S}$. If each vector has length one, the basis is orthonormal.

The contrast space for all $k$ occasions is $\mathcal{C}=\{c\in\mathbb{R}^k:c'\mathbf{1}_k=0\}$. It has dimension $k-1$: it is the orthogonal complement of the one-dimensional space spanned by $\mathbf{1}_k$. Thus a full contrast basis has $k-1$ linearly independent columns. A smaller contrast subspace may be the target when the scientific question concerns only a particular set of differences.

Put the basis vectors into the columns of a $k\times q$ matrix $C$. Then $C'\mathbf{1}_k=0$, and full column rank means the columns form a basis for their span. If $C'C=I_q$, the columns are orthonormal. Orthogonality is a geometric property of the chosen coordinates; it does not, by itself, assert that random contrast scores have zero covariance.

Orthogonal Coordinates Decompose the Contrast Component

For an orthonormal basis $C$ of a contrast subspace $\mathcal{S}$, the coordinates of a profile $x$ in that subspace are $C'x$. Its component in the subspace is $CC'x$, because $C'C=I_q$. If $C$ is a full orthonormal basis of the entire contrast space, this component is exactly the profile after removing its grand mean.

Result: If $C$ is a $k\times(k-1)$ orthonormal contrast basis, then $$ CC'=I_k-\frac{1}{k}\mathbf{1}_k\mathbf{1}_k'. $$ Consequently, for every $x\in\mathbb{R}^k$, $CC'x=x-\bar{x}\mathbf{1}_k$, where $\bar{x}=k^{-1}\mathbf{1}_k'x$, and $\|x-\bar{x}\mathbf{1}_k\|^2=\|C'x\|^2$.

Proof. The columns of $C$ are an orthonormal basis of $\mathcal{C}$, so $CC'$ is the orthogonal projection onto $\mathcal{C}$. The matrix $P=I_k-k^{-1}\mathbf{1}_k\mathbf{1}_k'$ is also the orthogonal projection onto $\mathcal{C}$: it is symmetric, $P\mathbf{1}_k=0$, and for every $c\in\mathcal{C}$, $Pc=c$. Orthogonal projection onto a fixed subspace is unique, so $CC'=P$. Multiplying by $x$ gives $CC'x=x-k^{-1}\mathbf{1}_k\mathbf{1}_k'x=x-\bar{x}\mathbf{1}_k$. Finally, $\|C'x\|^2=x'CC'x=\|x-\bar{x}\mathbf{1}_k\|^2$, since $P$ is a symmetric idempotent matrix. $\square$

Worked Example: Decomposing a Four-Occasion Profile

Consider the orthonormal contrast vectors $c_1=(1,-1,0,0)'/\sqrt{2}$, $c_2=(1,1,-2,0)'/\sqrt{6}$, and $c_3=(1,1,1,-3)'/\sqrt{12}$. Each is a contrast because its entries sum to zero. Their pairwise inner products are $c_1'c_2=(1-1)/\sqrt{12}=0$, $c_1'c_3=(1-1)/\sqrt{24}=0$, and $c_2'c_3=(1+1-2)/\sqrt{72}=0$. Their squared lengths are respectively $2/2=1$, $6/6=1$, and $12/12=1$. They therefore form an orthonormal basis of the three-dimensional contrast space for four occasions.

Take $x=(2,4,7,3)'$, whose grand mean is $4$. The centered profile is $x-4\mathbf{1}_4=(-2,0,3,-1)'$. The three contrast coordinates are $c_1'x=-2/\sqrt{2}=-\sqrt{2}$, $c_2'x=(-2+0-6)/\sqrt{6}=-8/\sqrt{6}$, and $c_3'x=(-2+0+3+3)/\sqrt{12}=4/\sqrt{12}=2/\sqrt{3}$.

The squared length of the centered profile is $(-2)^2+0^2+3^2+(-1)^2=14$. The sum of squared contrast coordinates is $(-\sqrt{2})^2+(-8/\sqrt{6})^2+(2/\sqrt{3})^2 =2+64/6+4/3=14$. Thus the coordinates recover the entire variation around the grand mean, with no component omitted.

Changing a Full Basis Preserves the Hypothesis

Suppose $C$ and $C_*$ are $k\times q$ full-column-rank matrices whose columns span the same contrast subspace. There is then a nonsingular $q\times q$ matrix $A$ such that $C_*=CA$. If both bases are orthonormal, $A$ is orthogonal; if they are not normalized, $A$ need only be invertible. The coordinates change, but the subspace does not.

Theorem: Let $C_*=CA$, where $A$ is nonsingular. For any mean profile $\mu$, the full contrast null hypotheses are equivalent: $$ C'\mu=0\quad\Longleftrightarrow\quad C_*'\mu=0. $$ More generally, $C'\mu$ and $C_*'\mu$ are coordinate descriptions of the same contrast effect, related by $C_*'\mu=A'C'\mu$.

Proof. By $C_*=CA$, we have $C_*'\mu=A'C'\mu$. If $C'\mu=0$, multiplication by $A'$ gives $C_*'\mu=0$. Conversely, if $C_*'\mu=0$, then $A'C'\mu=0$. Since $A$ is nonsingular, $A'$ is nonsingular, so multiplying by $(A')^{-1}$ gives $C'\mu=0$. Hence the two null hypotheses are equivalent. $\square$

Worked Example: Rotating a Two-Dimensional Contrast Basis

For three occasions, take the orthonormal contrast basis $c_1=(1,-1,0)'/\sqrt{2}$ and $c_2=(1,1,-2)'/\sqrt{6}$. Rotate its coordinates using $A=2^{-1/2}\begin{pmatrix}1&-1\\1&1\end{pmatrix}$, which is nonsingular and satisfies $A'A=I_2$. The new basis $C_*=CA$ spans exactly the same contrast space.

For $\mu=(5,2,1)'$, the original coordinates are $C'\mu=(3/\sqrt{2},5/\sqrt{6})'$, since $c_1'\mu=(5-2)/\sqrt{2}$ and $c_2'\mu=(5+2-2)/\sqrt{6}$. The rotated coordinates are $$ C_*'\mu=A'C'\mu =\frac{1}{\sqrt{2}} \begin{pmatrix}1&1\\-1&1\end{pmatrix} \begin{pmatrix}3/\sqrt{2}\\5/\sqrt{6}\end{pmatrix} = \begin{pmatrix}3/2+5/(2\sqrt{3})\\-3/2+5/(2\sqrt{3})\end{pmatrix}. $$

The numerical coordinates differ, as expected after a rotation. But $C'\mu\ne0$, and $C_*'\mu\ne0$; more importantly, one coordinate vector is zero if and only if the other is zero. Testing the full two-dimensional contrast effect therefore gives the same null hypothesis in either basis.

Invariance of the Multivariate Test

The same coordinate change applies when the contrast scores are used as multivariate responses. Let $Z=YC$ denote the data matrix after applying the within-subject contrasts to the measurement columns of a data matrix $Y$. Replacing $C$ with $C_*=CA$ gives $Z_*=ZA$. In a multivariate linear model, the hypothesis and error sum-of-products matrices for these responses transform as $H_*=A'HA$ and $E_*=A'EA$. These formulas follow because each residual or fitted response row is multiplied on the right by $A$, and its outer product is therefore transformed by congruence.

Assume $E$ is positive definite, so the generalized roots of $H$ relative to $E$ are the eigenvalues of $E^{-1}H$. These roots determine the usual MANOVA criteria. A nonsingular change of basis preserves them:

Theorem: If $A$ is nonsingular and $E$ is positive definite, then $E_*=A'EA$ is positive definite and $$ E_*^{-1}H_*=A^{-1}(E^{-1}H)A. $$ Thus $E_*^{-1}H_*$ and $E^{-1}H$ are similar and have the same eigenvalues, including multiplicities.

Proof. For any nonzero $v$, $v'E_*v=(Av)'E(Av)>0$, because $Av\ne0$ and $E$ is positive definite. Hence $E_*$ is positive definite. Also, $(A'EA)^{-1}=A^{-1}E^{-1}(A')^{-1}$, so $(A'EA)^{-1}(A'HA)=A^{-1}E^{-1}HA=A^{-1}(E^{-1}H)A$. Matrices related by this similarity transformation have the same characteristic polynomial: $\det(\lambda I-A^{-1}(E^{-1}H)A)=\det(A^{-1}(\lambda I-E^{-1}H)A)=\det(\lambda I-E^{-1}H)$. Their eigenvalues therefore agree, as claimed. $\square$

The invariance is also visible in the common criteria. For example, $\det(E_*)/\det(E_*+H_*)=\det(E)/\det(E+H)$, because the numerator and denominator determinants each acquire the same factor $\det(A)^2$. The traces of $E_*^{-1}H_*$ and $(E_*+H_*)^{-1}H_*$ are unchanged because the corresponding matrices are similar to their untransformed versions. Thus Wilks' lambda, the Hotelling–Lawley trace, Pillai's trace, and Roy's largest root are unchanged when the full response coordinates are changed nonsingularly.

Worked Example: Checking Generalized-Root Invariance

Suppose $E=\begin{pmatrix}2&0\\0&1\end{pmatrix}$ and $H=\begin{pmatrix}4&0\\0&0\end{pmatrix}$. Then $E^{-1}H=\begin{pmatrix}2&0\\0&0\end{pmatrix}$, whose roots are $2$ and $0$. Change coordinates using the nonsingular matrix $A=\begin{pmatrix}1&1\\0&1\end{pmatrix}$.

The transformed matrices are $E_*=A'EA=\begin{pmatrix}2&2\\2&3\end{pmatrix}$ and $H_*=A'HA=\begin{pmatrix}4&4\\4&4\end{pmatrix}$. Their generalized roots solve $\det(H_*-\lambda E_*)=0$: $$ \det\begin{pmatrix}4-2\lambda&4-2\lambda\\4-2\lambda&4-3\lambda\end{pmatrix} =(4-2\lambda)(-\lambda)=0. $$ The roots are $0$ and $2$, exactly as before. The contrast coordinates have been changed by a non-orthogonal basis transformation, but the full multivariate test has not changed.

What Orthogonality Does—and Does Not—Guarantee

Orthogonal bases are useful because they separate a contrast space into perpendicular directions. If $\mathcal{S}$ is the orthogonal direct sum of subspaces $\mathcal{S}_1,\ldots,\mathcal{S}_r$, then the projection onto $\mathcal{S}$ is the sum of the projections onto the $\mathcal{S}_j$. This gives a geometric decomposition of the effect. An omnibus null for the full space is equivalent to all of the component nulls holding together.

However, orthogonal contrast vectors need not produce statistically independent random scores. For a random measurement vector $X$ with covariance matrix $\Sigma$, the covariance of its contrast coordinates $C'X$ is $C'\Sigma C$. Even when $C'C=I$, this covariance need not be diagonal. Statistical uncorrelatedness follows only under additional conditions on $\Sigma$; independence requires still stronger distributional conditions in general.

Worked Example: Why Dropping a Contrast Changes the Question

With three occasions, the full contrast space has dimension two. Use the orthonormal basis from the rotation example and consider the profile $\mu=(5,2,1)'$. Its first contrast is $3/\sqrt{2}$, which is nonzero, and its second contrast is $5/\sqrt{6}$, also nonzero. The full null $C'\mu=0$ is false.

If an analysis retains only $c_1$, it tests the single condition $c_1'\mu=0$, which asks whether the first and second occasion means are equal. It does not test whether all three occasion means are equal: the profile $(2,2,7)'$, for example, has $c_1'\mu=0$ but $c_2'\mu=(2+2-14)/\sqrt{6}=-10/\sqrt{6}\ne0$. The one-dimensional test misses a genuine difference in the omitted direction.

Thus choosing another basis for the same two-dimensional space preserves the omnibus question, but keeping only one vector replaces it with a narrower question. A partial basis is not a harmless reparameterization of a full basis.

Common Mistake

Wrong reasoning: “The contrast vectors are orthogonal, so their scores are independent, and I can drop one without changing the omnibus test.” Orthogonality concerns inner products between fixed vectors. It does not establish independence of random scores, and dropping a vector reduces the tested subspace.
Correction: First check whether the new basis spans the same full contrast subspace. If it does, the basis change is nonsingular and preserves the zero-effect hypothesis and the MANOVA roots. If it does not, the tested hypothesis has changed. Treat any claim of independence separately, using assumptions about the covariance structure and distribution.

Key takeaway: An orthogonal basis provides clean geometric coordinates for within-subject contrasts. Any full-rank basis of the same subspace tests the same omnibus effect, and a nonsingular coordinate change preserves the generalized MANOVA roots. Orthogonality alone does not guarantee statistical independence, and deleting a basis direction changes the hypothesis.

Check Your Understanding

Answer each question by referring to the subspace and the coordinate transformation involved.

Why must a full contrast basis for $k$ occasions have $k-1$ vectors?
If two full-rank contrast matrices $C$ and $C_*$ span the same space, what condition relates them, and why does it preserve the zero-effect null?
What does $CC'x$ represent when $C$ is an orthonormal basis for a contrast subspace?
Why do the generalized roots remain unchanged under $H_*=A'HA$ and $E_*=A'EA$ for nonsingular $A$?
Give a reason orthogonal contrast scores need not be statistically independent.
Why can retaining only one vector from a two-dimensional contrast basis change the scientific question?