Cochran's Theorem for Gaussian Sum-of-Squares Partitions

Why a Gaussian Sum-of-Squares Partition Matters

Many exact results in normal-theory inference depend on splitting a total sum of squares into pieces that are not only easy to describe, but also independent. Orthogonal projections provide a matrix language for those pieces. Cochran’s theorem identifies when a collection of projected quadratic forms has independent chi-square distributions, and specifies their degrees of freedom and noncentrality.

The key hypotheses are structural: the matrices must be symmetric idempotent projections, and together they must sum to the identity. A sum of ranks equal to the sample-space dimension is an important check. Under the identity-partition hypothesis, that rank condition follows automatically, rather than needing to be imposed separately.

Definition: A matrix $A$ is an orthogonal projection matrix if $A'=A$ and $A^2=A$. For a vector $x$, the quadratic form $x'Ax$ is the squared length of its projection onto the column space of $A$.

Cochran’s Theorem

Let $X$ be an $n$-dimensional Gaussian random vector with mean $\mu$ and covariance matrix $\sigma^2 I_n$, where $\sigma^2>0$. Consider $k$ matrices $A_1,\ldots,A_k$, each an $n\times n$ symmetric idempotent matrix, and define $Q_i=X'A_iX$. The theorem concerns the case in which these projections form a complete partition of the identity.

Theorem (Cochran): Suppose $X\sim N_n(\mu,\sigma^2 I_n)$, and $A_1,\ldots,A_k$ are symmetric idempotent matrices satisfying $$ \sum_{i=1}^k A_i=I_n. $$ Let $r_i=\operatorname{rank}(A_i)$ and $Q_i=X'A_iX$. Then $$ \sum_{i=1}^k r_i=n, $$ and the quadratic forms $Q_1,\ldots,Q_k$ are independent, with $$ \frac{Q_i}{\sigma^2}\sim \chi^2_{r_i}(\lambda_i), \qquad \lambda_i=\frac{\mu'A_i\mu}{\sigma^2}. $$ Here $\chi^2_r(\lambda)$ denotes a noncentral chi-square distribution with $r$ degrees of freedom and noncentrality parameter $\lambda$. The component is central when $\lambda_i=0$, equivalently when $A_i\mu=0$. A rank-zero component is identically zero.

There are two related identities worth keeping visible. The quadratic forms sum to the original squared length, $ \sum_i Q_i=X'X$, and their noncentrality parameters sum to $ \sum_i\lambda_i=\mu'\mu/\sigma^2$. The theorem gives more than these algebraic decompositions: it asserts that the components are independent under the Gaussian model.

Why the Rank Condition Holds

For an orthogonal projection, the rank equals the trace. Indeed, a symmetric idempotent matrix has only 0 and 1 as eigenvalues, so its trace counts its eigenvalues equal to 1. Taking traces in the identity partition therefore gives

\sum_{i=1}^k r_i =\sum_{i=1}^k \operatorname{tr}(A_i) =\operatorname{tr}\left(\sum_{i=1}^k A_i\right) =\operatorname{tr}(I_n) =n.

The partition also forces the projection spaces to be mutually orthogonal. This is the matrix structure behind the decomposition.

Result: If symmetric idempotent matrices $A_1,\ldots,A_k$ sum to $I_n$, then $A_iA_j=0$ whenever $i\ne j$.

Proof. Fix $i$. Multiply the identity $\sum_j A_j=I_n$ on the left and right by $A_i$. Using $A_i^2=A_i$ gives

A_i=\sum_{j=1}^k A_iA_jA_i =A_i+\sum_{j\ne i} A_iA_jA_i,

so $\sum_{j\ne i} A_iA_jA_i=0$. Each term is positive semidefinite, since $$ x'A_iA_jA_ix=(A_jA_ix)'(A_jA_ix)\geq 0 $$ for every vector $x$; here symmetry and idempotence of $A_j$ were used. A sum of positive semidefinite matrices can equal zero only if every term is zero. Thus $A_iA_jA_i=0$ for every $j\ne i$. Since $A_iA_jA_i=(A_jA_i)'(A_jA_i)$, this implies $A_jA_i=0$, and transposing gives $A_iA_j=0$. $\square$

Where the Chi-Square Distributions Come From

The marginal distribution of each component can be derived directly. This also explains the noncentrality parameter in Cochran’s theorem. Fix one projection matrix $A_i$ with rank $r_i$. A symmetric idempotent matrix has an orthonormal eigenbasis, with $r_i$ eigenvalues equal to 1 and the others equal to 0. In that basis, $A_i$ is diagonal with $r_i$ ones followed by zeros.

Result: If $X\sim N_n(\mu,\sigma^2I_n)$ and $A$ is a symmetric idempotent matrix of rank $r$, then $$ \frac{X'AX}{\sigma^2}\sim\chi^2_r\left(\frac{\mu'A\mu}{\sigma^2}\right). $$

Derivation. Let $U$ be an orthogonal matrix whose columns are eigenvectors of $A$, ordered so the first $r$ eigenvalues are 1. The vector $U'X/\sigma$ has independent unit-variance normal coordinates: orthogonal transformation preserves the covariance $\sigma^2I_n$, and a Gaussian vector with diagonal covariance has independent coordinates. Write its coordinates as $Z_j+\delta_j$, where the $Z_j$ are independent $N(0,1)$ variables and $\delta=U'\mu/\sigma$. Then

\frac{X'AX}{\sigma^2} =\sum_{j=1}^r (Z_j+\delta_j)^2.

By definition, this is noncentral chi-square with $r$ degrees of freedom and noncentrality $\sum_{j=1}^r\delta_j^2$. In the same eigenbasis, $$ \sum_{j=1}^r\delta_j^2=\frac{\mu'A\mu}{\sigma^2}. $$ This proves the stated marginal distribution. In particular, the noncentrality is nonnegative. It is zero exactly when the projection of $\mu$ onto the column space of $A$ is zero, which is equivalent to $A\mu=0$. $\square$

Applying this derivation to each $A_i$ gives the individual chi-square laws in Cochran’s theorem. The theorem’s additional conclusion is their joint independence, which relies on the complete projection partition and the Gaussian assumption. Having chi-square marginals alone does not establish independence.

Worked Examples

Worked Example: Coordinate Projections in Three Dimensions

Let $X\sim N_3((0,2,0)',I_3)$, and let $A_1,A_2,A_3$ project onto the first, second, and third coordinate axes, respectively. Determine the distributions and independence of $Q_i=X'A_iX$.

The three matrices are diagonal with one 1 each, so each is symmetric idempotent of rank 1. Their sum is $I_3$, and the rank check is $1+1+1=3$. Cochran’s theorem therefore says that the three quadratic forms are independent.

The variance is $\sigma^2=1$. The noncentrality parameters are $$ \lambda_1=\mu'A_1\mu=0,\qquad \lambda_2=\mu'A_2\mu=4,\qquad \lambda_3=\mu'A_3\mu=0. $$ Thus $Q_1\sim\chi^2_1$, $Q_2\sim\chi^2_1(4)$, and $Q_3\sim\chi^2_1$, independently. The middle component is noncentral because the mean has a nonzero second coordinate; the other two are central.

Worked Example: A Partition Using Rotated Axes

Let $X\sim N_2((2,0)',4I_2)$. Consider $$ A_1=\frac12\begin{pmatrix}1&1\\1&1\end{pmatrix}, \qquad A_2=\frac12\begin{pmatrix}1&-1\\-1&1\end{pmatrix}. $$ Find the distributions of the two quadratic forms.

Each matrix is symmetric. Direct multiplication gives $A_1^2=A_1$ and $A_2^2=A_2$. Their sum is $I_2$, and each has rank 1, so the rank condition is $1+1=2$. The quadratic forms are independent.

For $\mu=(2,0)'$ and $\sigma^2=4$, calculation gives $$ \mu'A_1\mu=2,\qquad \mu'A_2\mu=2, \qquad \lambda_1=\lambda_2=\frac{2}{4}=\frac12. $$ Consequently, $$ \frac{Q_1}{4}\sim\chi^2_1\left(\frac12\right), \qquad \frac{Q_2}{4}\sim\chi^2_1\left(\frac12\right), $$ independently. Although the original mean lies along the first coordinate axis, each rotated projection receives a nonzero part of it.

Worked Example: A Rank Check That Is Not Enough

In two dimensions, take $A_1=A_2=\begin{pmatrix}1&0\\0&0\end{pmatrix}$. Each matrix is a symmetric idempotent projection and the sum of their ranks is $1+1=2$. Can Cochran’s theorem be used to claim that $Q_1$ and $Q_2$ are independent?

No. The matrices do not sum to $I_2$; their sum is $\begin{pmatrix}2&0\\0&0\end{pmatrix}$. In fact, $Q_1=X_1^2=Q_2$, so the two forms are identical, not independent (unless they are degenerate constants). Matching the total rank does not replace the identity-partition hypothesis. Here the total quadratic form is $2X_1^2$, not $X'X$.

What the Theorem Is Used For

Cochran’s theorem is useful whenever a Gaussian vector is split into projection-based sums of squares. It supplies exact degrees of freedom, identifies whether each piece is central or noncentral, and establishes independence when the projections partition the identity. Those facts underpin exact normal-theory procedures built from ratios or combinations of sums of squares.

The hypotheses matter as much as the distributional conclusion. In particular, independent chi-square components do not follow merely because each quadratic form has a chi-square marginal, nor does a sum-of-ranks check by itself prove independence. Verify the Gaussian spherical covariance, symmetry and idempotence of each matrix, and the identity sum before invoking the theorem.

Gaussian condition: $X$ must have covariance $\sigma^2I_n$, with $\sigma^2>0$.
Projection condition: every $A_i$ must be symmetric and idempotent.
Partition condition: the matrices must sum to $I_n$; this also forces pairwise orthogonality.
Rank check: the ranks must sum to $n$, as they necessarily do under the stated partition.

Key takeaway: For a Gaussian vector with covariance $\sigma^2I_n$, a complete partition of the identity by symmetric projection matrices yields independent scaled chi-square components. Each component’s degrees of freedom are its projection rank, and its noncentrality is $\mu'A_i\mu/\sigma^2$.

Check Your Understanding

Use the hypotheses as carefully as the distributional formulas.

Why does $\sum_i A_i=I_n$ imply $\sum_i \operatorname{rank}(A_i)=n$ for symmetric idempotent matrices?
For a component $Q_i=X'A_iX$, state its degrees of freedom and noncentrality parameter.
What condition on $\mu$ makes the $i$th component central?
Why does a collection of projections whose ranks sum to $n$ not necessarily satisfy Cochran’s theorem?
In the rotated-axes example, verify that the two noncentrality parameters sum to $\mu'\mu/\sigma^2$.