Orthogonal Contrast Sums of Squares

Why Orthogonal Contrasts Produce Separate Sums of Squares

A contrast summarizes a specified linear comparison among coordinates of a data vector. For instance, a contrast can compare two observations by subtracting one from the other, or compare the average of one collection with the average of another. When contrast vectors are orthogonal, their comparisons point in perpendicular directions. This geometric fact lets us express each comparison as a projection and, under a Gaussian model with spherical covariance, gives independent sums of squares.

The key is to keep three ideas distinct: a vector is a contrast when its coefficients sum to zero; two contrasts are orthogonal when their inner product is zero; and sums of squares from orthogonal contrasts are independent only with appropriate distributional assumptions. We develop these ideas for a fixed vector of observations, without assuming a particular layout for how those observations were collected.

The denominator adjusts for the length of the coefficient vector: multiplying \(c\) by a nonzero constant does not change \(SS(c)\). The formula also has a direct geometric interpretation. The projection of \(Y\) onto the line spanned by \(c\) is \(cc^\mathsf{T}Y/(c^\mathsf{T}c)\), and its squared length is \(SS(c)\).

Projection Matrices for Contrast Sums of Squares

For a nonzero contrast \(c\), define \[ P_c=\frac{cc^\mathsf{T}}{c^\mathsf{T}c}. \] This matrix is symmetric, since \(P_c^\mathsf{T}=P_c\), and idempotent: \[ P_c^2=\frac{cc^\mathsf{T}cc^\mathsf{T}}{(c^\mathsf{T}c)^2} =\frac{c(c^\mathsf{T}c)c^\mathsf{T}}{(c^\mathsf{T}c)^2} =P_c. \] It is therefore the orthogonal projection matrix onto the one-dimensional space spanned by \(c\). The contrast sum of squares is \(Y^\mathsf{T}P_cY\).

Now take \(q\) nonzero, mutually orthogonal contrast vectors \(c_1,\ldots,c_q\), and write \(P_j=P_{c_j}\). If \(i\ne j\), then \[ P_iP_j =\frac{c_i(c_i^\mathsf{T}c_j)c_j^\mathsf{T}} {(c_i^\mathsf{T}c_i)(c_j^\mathsf{T}c_j)} =0. \] Consequently \(P=\sum_{j=1}^q P_j\) is also symmetric and idempotent. Its rank is \(q\), because it projects onto the span of \(q\) nonzero, mutually orthogonal vectors. The combined sum of squares is \[ Y^\mathsf{T}PY=\sum_{j=1}^q SS(c_j). \]

Exact Distributions Under Gaussian Spherical Errors

A projection decomposition is a geometric identity and does not require a probability model. To obtain exact probability distributions and independence, suppose \[ Y\sim N_m(\mu,\sigma^2 I_m),\qquad \sigma^2>0, \] where \(I_m\) is the \(m\times m\) identity matrix. For each contrast define the unit vector \(d_j=c_j/\sqrt{c_j^\mathsf{T}c_j}\). The vectors \(d_1,\ldots,d_q\) are orthonormal, and \(SS(c_j)=(d_j^\mathsf{T}Y)^2\).

Proof. Extend the orthonormal vectors \(d_1,\ldots,d_q\) to an orthonormal basis of \(\mathbb{R}^m\), and let \(Q\) be the orthogonal matrix whose rows are the resulting basis vectors. The transformed vector \(QY\) is multivariate normal with mean \(Q\mu\) and covariance \[ Q(\sigma^2 I_m)Q^\mathsf{T}=\sigma^2 I_m. \] Its density factors into the product of its univariate normal coordinate densities: each coordinate has variance \(\sigma^2\), and the covariance matrix is diagonal. Thus the first \(q\) coordinates \(d_j^\mathsf{T}Y\) are independent, with distributions \(N(d_j^\mathsf{T}\mu,\sigma^2)\). Their squared values are exactly \(SS(c_j)\). Dividing each coordinate by \(\sigma\) gives an independent normal variable with variance one and mean \(d_j^\mathsf{T}\mu/\sigma\); its square has a noncentral chi-square distribution with one degree of freedom and noncentrality \((d_j^\mathsf{T}\mu/\sigma)^2=\lambda_j\). The sum of these independent squared normal variables has \(q\) degrees of freedom and noncentrality equal to the sum of their squared means. This proves both the individual and combined distribution statements. \(\square\)

Collections of Contrasts and Degrees of Freedom

Several orthogonal contrasts can be treated as a single component. If \(C\) is a \(q\times m\) matrix whose rows are mutually orthogonal nonzero contrast vectors, then \(CC^\mathsf{T}\) is diagonal with positive diagonal entries. The projection onto the row space of \(C\) is \[ P_C=C^\mathsf{T}(CC^\mathsf{T})^{-1}C, \] and the corresponding sum of squares is \(Y^\mathsf{T}P_CY\). Its degrees of freedom are \(\operatorname{rank}(P_C)=q\). For the separate orthogonal contrast components, the sum is the same quantity: \[ Y^\mathsf{T}P_CY=\sum_{j=1}^q\frac{(c_j^\mathsf{T}Y)^2}{c_j^\mathsf{T}c_j}. \]

This rank calculation is the general rule: the degrees of freedom of a projection sum of squares are the dimension of the subspace onto which it projects. Counting listed contrasts is valid here because nonzero orthogonal vectors are linearly independent. If vectors are duplicated or otherwise linearly dependent, their number is not the rank and cannot be used as the degrees of freedom.

Complete Contrast Bases and a Common Pitfall

The contrast subspace, consisting of all vectors perpendicular to \(\mathbf{1}\), has dimension \(m-1\). Any orthonormal basis \(d_1,\ldots,d_{m-1}\) for that subspace yields a complete contrast decomposition: \[ \sum_{j=1}^{m-1}d_jd_j^\mathsf{T} =I_m-\frac{\mathbf{1}\mathbf{1}^\mathsf{T}}{m}. \] The right side projects onto the contrast subspace. Therefore, for any observed \(Y\), the sum of the corresponding contrast sums of squares equals \[ Y^\mathsf{T}\left(I_m-\frac{\mathbf{1}\mathbf{1}^\mathsf{T}}{m}\right)Y =\sum_{i=1}^m(Y_i-\bar Y)^2, \qquad \bar Y=\frac{1}{m}\sum_{i=1}^mY_i. \] This identity is a geometric decomposition of corrected total variation into orthogonal directions.

Common mistake: “The contrast coefficients sum to zero, so two contrast sums of squares are independent.” This is incorrect. A zero coefficient sum says that each vector is a contrast; it says nothing about whether two contrast vectors are orthogonal. Even orthogonality, by itself, does not guarantee independence for a general non-Gaussian distribution. The correction is to check the inner product for geometric orthogonality, then check the probability model: independence of these components follows under the specified joint normal model with covariance \(\sigma^2I_m\).

Orthogonal contrast sums of squares are useful because they express variation as interpretable, non-overlapping directions and attach degrees of freedom directly to the dimensions of those directions. Their exact chi-square distributions provide the foundation for normal-theory inference about such components. In practical calculations, verify the contrast sums, vector inner products, and model assumptions separately; each supports a different part of the conclusion.