Cochran Decompositions for Normal Samples

Why Normal Sums of Squares Decompose

Many exact sampling results begin with a normal vector and divide its total squared length into pieces. In a grouped sample, for instance, one piece measures variation among group means and another measures variation within groups. The key is not merely that these pieces add up: under the right conditions, they are independent, and each has a chi-square distribution after scaling.

Orthogonal projections make those conditions precise. A projection selects a subspace of directions in the data vector; its quadratic form is the squared length of the projected vector. When the projection subspaces are mutually orthogonal and the normal vector has spherical covariance, the resulting pieces are independent. This is the projection form of Cochran's theorem, closely related to the earlier material on Gaussian quadratic forms.

Definition: A random variable has a noncentral chi-square distribution with $r$ degrees of freedom and noncentrality parameter $\lambda$, written $\chi^2_r(\lambda)$, if it can be represented as $$ \sum_{a=1}^{r}(Z_a+\delta_a)^2, $$ where $Z_1,\ldots,Z_r$ are independent $N(0,1)$ random variables and $\sum_{a=1}^{r}\delta_a^2=\lambda$. When $\lambda=0$, this is the central chi-square distribution $\chi^2_r$. The degrees of freedom $r$ count the squared normal coordinates; the noncentrality parameter records their squared mean lengths.

Orthogonal Projection Decomposition

Let $Y$ be an $n$-dimensional normal random vector with mean $\mu$ and covariance matrix $\sigma^2 I_n$, where $\sigma^2>0$. Suppose $P_1,\ldots,P_m$ are symmetric projection matrices satisfying $P_iP_j=0$ for $i\ne j$ and $\sum_{j=1}^{m}P_j=I_n$. Symmetry and idempotence, $P_j'=P_j$ and $P_j^2=P_j$, mean each $P_j$ projects onto a subspace. The conditions say those subspaces are orthogonal and together span the whole space.

Theorem (orthogonal projection form of Cochran's theorem): Under the conditions above, the random variables $$ Q_j=\frac{Y'P_jY}{\sigma^2}, \qquad j=1,\ldots,m, $$ are mutually independent, and $$ Q_j\sim \chi^2_{r_j}(\lambda_j),\qquad r_j=\operatorname{rank}(P_j),\qquad \lambda_j=\frac{\mu'P_j\mu}{\sigma^2}. $$ A rank-zero component is identically zero and can be omitted. In particular, $Q_j$ is central if and only if $P_j\mu=0$.

Proof. For each $j$, choose an orthonormal basis for the range of $P_j$. Because the ranges are mutually orthogonal and their dimensions sum to $n$, joining these bases gives an orthonormal basis of $\mathbb R^n$. Let $U$ be the orthogonal matrix whose columns are these basis vectors, ordered by subspace. In the coordinates $U'Y/\sigma$, the normal vector has covariance $I_n$, so its coordinates are independent unit-variance normal variables. Its mean is $U'\mu/\sigma$.

The quadratic form $Y'P_jY/\sigma^2$ is the sum of the squares of the coordinates belonging to the $j$th subspace: projection onto that subspace retains exactly those coordinates. Disjoint coordinate blocks of a vector with independent coordinates are independent. Thus the $Q_j$'s are independent. The squared means in the $j$th block sum to $\mu'P_j\mu/\sigma^2$, and the block has $r_j$ coordinates, giving the stated noncentral chi-square law. Finally, $\mu'P_j\mu=\|P_j\mu\|^2$, so this parameter is zero exactly when $P_j\mu=0$. $\square$

Since the projections sum to the identity, their unscaled quadratic forms also add:

Y'Y=\sum_{j=1}^{m}Y'P_jY.

The theorem therefore gives more than an algebraic identity. It specifies the distribution of every component, its degrees of freedom, its noncentrality, and its independence from the others.

Between-Group and Within-Group Components

Consider independent observations $Y_{ij}\sim N(\mu_j,\sigma^2)$, for groups $j=1,\ldots,k$ of sizes $n_j$. Write $n=\sum_{j=1}^k n_j$, $\bar Y_j=n_j^{-1}\sum_iY_{ij}$, and $\bar Y_{\cdot}=n^{-1}\sum_j n_j\bar Y_j$. Regard all $n$ observations as one vector $Y$.

The subspace of vectors that are constant within each group has dimension $k$. Projection onto it replaces each observation by its group average. Call this projection $P_G$. Projection onto the one-dimensional space of vectors constant over the whole sample is $P_0$. Since the global-constant subspace lies inside the group-constant subspace, $P_GP_0=P_0P_G=P_0$. The three relevant projections are $P_0$, $P_B=P_G-P_0$, and $P_W=I-P_G$: grand mean, between-group contrasts, and within-group residuals.

Theorem: These are mutually orthogonal projections with ranks $1$, $k-1$, and $n-k$, respectively. Their sums of squares are $$ \begin{aligned} Y'P_0Y&=n\bar Y_{\cdot}^{\,2},\\ Y'P_BY&=\sum_{j=1}^{k}n_j(\bar Y_j-\bar Y_{\cdot})^2,\\ Y'P_WY&=\sum_{j=1}^{k}\sum_{i=1}^{n_j}(Y_{ij}-\bar Y_j)^2. \end{aligned} $$ They add to $Y'Y$ and, under the stated normal model, are mutually independent after scaling by $\sigma^2$.

Justification. The group-constant space has one free value per group, so its projection has rank $k$; the global-constant space has rank $1$. Because $P_0$ is nested in $P_G$, $P_B=P_G-P_0$ is a projection onto the part of the group-constant space orthogonal to the global-constant space. Indeed, $(P_G-P_0)^2=P_G-2P_0+P_0=P_G-P_0$. Its rank is $k-1$. The residual projection $I-P_G$ has rank $n-k$ and is orthogonal to both of the other projections. Expanding each projected squared length gives the displayed sums of squares. The projection theorem then proves independence and gives the distributions below. $\square$

Let $\bar\mu_{\cdot}=\sum_j n_j\mu_j/n$. The corresponding scaled components have laws

\begin{aligned} \frac{Y'P_0Y}{\sigma^2} &\sim\chi^2_1\!\left(\frac{n\bar\mu_{\cdot}^{\,2}}{\sigma^2}\right),\\ \frac{Y'P_BY}{\sigma^2} &\sim\chi^2_{k-1}\!\left(\frac{\sum_j n_j(\mu_j-\bar\mu_{\cdot})^2}{\sigma^2}\right),\\ \frac{Y'P_WY}{\sigma^2} &\sim\chi^2_{n-k}. \end{aligned}

The within-group law is central even when the group means differ: the mean vector is constant within each group, so $P_W\mu=0$. The between-group law is central exactly when all group means are equal. The grand-mean component can be noncentral even under equal group means if their common value is not zero.

Worked Example: Three Unequal Group Means

Take three independent normal groups with sizes $2,3,2$, means $0,1,2$, respectively, and common variance $\sigma^2=2$. Here $n=7$ and the weighted grand mean is $\bar\mu_{\cdot}=(2(0)+3(1)+2(2))/7=1$.

The grand-mean noncentrality is $7(1)^2/2=3.5$. For the between-group component, the weighted squared deviations are $2(0-1)^2+3(1-1)^2+2(2-1)^2=4$, so its noncentrality is $4/2=2$. The within-group component has noncentrality zero because each group's mean is constant within that group. The ranks are $1$, $3-1=2$, and $7-3=4$. Therefore the three independent laws are

\frac{Y'P_0Y}{2}\sim\chi^2_1(3.5),\qquad \frac{Y'P_BY}{2}\sim\chi^2_2(2),\qquad \frac{Y'P_WY}{2}\sim\chi^2_4.

As a check, the total noncentrality is $\sum_j n_j\mu_j^2/\sigma^2=(2(0)^2+3(1)^2+2(2)^2)/2=11/2=5.5$, which equals $3.5+2+0$. The degrees of freedom also add: $1+2+4=7$.

Nested Groupings and Incremental Sums of Squares

The same reasoning applies when a grouping is refined in stages. If one projection space is contained in another, subtracting the smaller projection from the larger gives the projection onto the newly added directions. These increments are mutually orthogonal, so the associated normal sums of squares are independent. This is useful when a total variation is divided into successive, interpretable contrasts rather than just one between-group part.

Worked Example: Two Levels of Grouping

Suppose eight independent observations have distribution $N(1.5,1)$. Arrange them in four pairs, and arrange those pairs into two blocks of four. The subspaces of vectors constant over the whole sample, constant within each block, and constant within each pair have dimensions $1,2,4$. They are nested in that order.

The four orthogonal pieces are the grand-mean projection, the increment from one global mean to two block means, the increment from block means to four pair means, and the residual within pairs. Their ranks are $1$, $2-1=1$, $4-2=2$, and $8-4=4$. Since the population mean is constant across all observations, it lies entirely in the grand-mean subspace. Its noncentrality there is $8(1.5)^2/1=18$; its projections onto the other three subspaces are zero. Thus the four independent scaled sums of squares have laws

\chi^2_1(18),\qquad \chi^2_1,\qquad \chi^2_2,\qquad \chi^2_4.

Their degrees of freedom sum to $1+1+2+4=8$, and their noncentralities sum to $18$, the noncentrality of $Y'Y$. In unscaled form, the four sums are the grand-mean squared length, the between-block increment, the between-pair-within-block increment, and the within-pair residual sum of squares.

Worked Example: Equal Means Give a Central Between-Group Sum

Consider four independent groups with sizes $2,3,1,4$, common mean $5$, and common variance $9$. Then $n=10$, $k=4$, and the weighted grand mean is $5$. Every group mean equals the grand mean, so the between-group noncentrality is $\sum_j n_j(5-5)^2/9=0$. The within-group mean vector is also annihilated by its residual projection. Hence

\frac{Y'P_BY}{9}\sim\chi^2_3,\qquad \frac{Y'P_WY}{9}\sim\chi^2_6,

and these two sums are independent. Their expected unscaled values are $9(3)=27$ and $9(6)=54$, respectively, since a central chi-square variable with $r$ degrees of freedom has expectation $r$. The remaining grand-mean component has law $\chi^2_1(10(5)^2/9)=\chi^2_1(250/9)$ after division by $9$. The ranks $1,3,6$ account for all ten observations.

Common Pitfalls and Why the Conditions Matter

Assuming that sums add implies independence. The identity $Y'Y=\sum_jY'P_jY$ is algebraic. Independence requires the normal model and orthogonal projection subspaces; additivity alone does not establish it.
Using Euclidean orthogonality with a nonspherical covariance without checking. The proof relies on covariance $\sigma^2I_n$, which makes orthogonal coordinates independent. With a general covariance matrix, Euclidean-orthogonal projections need not produce independent quadratic forms.
Calling every component central. A component is central only when its projection annihilates the mean vector. In a one-way layout, the within-group component is central under arbitrary group means, but the between-group component is central only when all group means are equal.
Using $n-1$ degrees of freedom for every residual sum. In a one-way layout, estimating $k$ group means leaves $n-k$ within-group residual directions. The rank of the residual projection, not a memorized sample-variance convention, determines its degrees of freedom.

Key takeaway: To derive a normal sum-of-squares decomposition, identify the projection subspaces, verify that they are mutually orthogonal and span the data space, and compute each rank and projected mean. These steps determine independence, degrees of freedom, and central or noncentral chi-square laws.

Check Your Understanding

Use the projection viewpoint to answer these questions.

For a one-way layout with $n$ observations and $k$ nonempty groups, what are the ranks of the grand-mean, between-group, and within-group projections?
Why is the within-group sum of squares central when the group means differ?
Three groups have sizes $2,2,4$ and means $1,1,3$. Find the weighted grand mean and the between-group noncentrality when $\sigma^2=2$.
In the nested example, why are the increments between successive grouping spaces orthogonal?
Which condition in the projection theorem would need particular attention if $Y$ had covariance matrix $\Sigma\ne\sigma^2I_n$?