Cochran's Theorem for Gaussian Sums of Squares

From a Gaussian Squared Norm to Separate Components

A standard Gaussian vector has total squared length distributed as a chi-square variable. In many calculations, that total length is partitioned into several quadratic forms. The key question is when those pieces are themselves chi-square variables and, more strongly, when they are independent.

Cochran's theorem answers this question for a particular matrix decomposition. Its rank condition is not just bookkeeping: together with positive semidefiniteness, it forces each matrix to be a projection, and the projections to act on mutually orthogonal subspaces. Those geometric facts produce the independent chi-square pieces.

Definition: A real matrix \(A\) is positive semidefinite if it is symmetric and \(x^\mathsf{T}Ax\geq 0\) for every real vector \(x\) of the appropriate dimension. Its rank, \(\operatorname{rank}(A)\), is the dimension of its range. A symmetric matrix \(P\) is an orthogonal projection matrix if \(P^2=P\); it projects vectors onto its range.

Throughout, let \(Z\sim N_n(0,I_n)\), where \(I_n\) is the \(n\)-by-\(n\) identity matrix. For a symmetric matrix \(A_i\), the associated quadratic form is \(Q_i=Z^\mathsf{T}A_iZ\). A chi-square variable with \(r\) degrees of freedom, denoted \(\chi^2_r\), is the sum of squares of \(r\) independent \(N(0,1)\) variables; \(\chi^2_0\) denotes the constant zero.

Cochran's Theorem

Theorem (Cochran's theorem, identity-decomposition form): Suppose \(A_1,\ldots,A_k\) are fixed, symmetric, positive-semidefinite \(n\)-by-\(n\) matrices such that

\(\sum_{i=1}^k A_i=I_n\quad\text{and}\quad\sum_{i=1}^k\operatorname{rank}(A_i)=n.\)

If \(Z\sim N_n(0,I_n)\), then each \(Q_i=Z^\mathsf{T}A_iZ\) has distribution \(\chi^2_{r_i}\), where \(r_i=\operatorname{rank}(A_i)\), and \(Q_1,\ldots,Q_k\) are mutually independent. In addition, \(\sum_{i=1}^k Q_i=Z^\mathsf{T}Z\sim\chi^2_n\).

The assumptions are part of the result. The matrices must be positive semidefinite, not merely symmetric, and the ranks must sum to the ambient dimension. The proof first turns these numerical assumptions into a geometric decomposition.

Why the Rank Condition Forces Projections

Because the matrices are positive semidefinite and sum to \(I_n\), each is bounded above by \(I_n\) in the positive-semidefinite ordering. Indeed, \(I_n-A_i=\sum_{j\ne i}A_j\) is positive semidefinite. Consequently every eigenvalue of \(A_i\) lies between zero and one.

For a positive-semidefinite matrix, its trace is the sum of its nonnegative eigenvalues, while its rank is the number of nonzero eigenvalues. Since those eigenvalues are at most one, \(\operatorname{tr}(A_i)\leq \operatorname{rank}(A_i)\). But the matrix sum and rank-sum assumptions give

\sum_{i=1}^k\operatorname{tr}(A_i) =\operatorname{tr}(I_n)=n =\sum_{i=1}^k\operatorname{rank}(A_i).

Each difference \(\operatorname{rank}(A_i)-\operatorname{tr}(A_i)\) is nonnegative, and their sum is zero. Hence every difference is zero. The nonzero eigenvalues of each \(A_i\) are therefore all equal to one: they are at most one, and their sum equals their count. The remaining eigenvalues are zero. By the spectral theorem, \(A_i^2=A_i\), so each \(A_i\) is an orthogonal projection.

The ranges of these projections are mutually orthogonal. To see this, fix \(i\) and let \(x\) belong to the range of \(A_i\). Since \(A_i\) is a projection, \(A_ix=x\). The identity decomposition then implies \(\sum_{j\ne i}A_jx=0\). Taking the inner product with \(x\) gives

0=\sum_{j\ne i}x^\mathsf{T}A_jx.

Every term is nonnegative. Thus \(x^\mathsf{T}A_jx=0\) for each \(j\ne i\). For a positive-semidefinite matrix, this equality implies \(A_jx=0\): writing \(A_j\) in an orthonormal eigenbasis, the quadratic form is a sum of nonnegative eigenvalues times squared coordinates, so every coordinate in a positive-eigenvalue direction must vanish. Therefore \(A_j\) annihilates the range of \(A_i\). The ranges are pairwise orthogonal.

Proof of the Distribution and Independence Claims

Choose an orthonormal basis for the range of each \(A_i\). The ranges are pairwise orthogonal, and their dimensions \(r_i\) sum to \(n\). Concatenating these bases therefore gives an orthonormal basis of \(\mathbb{R}^n\). Let \(U\) be the orthogonal matrix whose columns are those basis vectors, ordered in groups by matrix \(A_i\).

The coordinates \(Y=U^\mathsf{T}Z\) are again independent \(N(0,1)\) variables. For example, this follows directly from the standard normal density: the change of variables \(z=Uy\) has absolute Jacobian determinant one and \(z^\mathsf{T}z=y^\mathsf{T}y\), so the transformed density is the product of \(n\) standard normal densities. In the chosen basis, \(A_i\) acts as the identity on its own range and as zero on the other ranges. Thus \(Q_i\) is the sum of squares of the \(r_i\) coordinates in its group. Those groups are disjoint, so the forms are independent, and each is \(\chi^2_{r_i}\). Finally, summing the forms gives \(Z^\mathsf{T}(\sum_i A_i)Z=Z^\mathsf{T}Z\), which is \(\chi^2_n\). \(\square\)

Worked Decompositions

Worked Example: Splitting Coordinates into Three Groups

Let \(Z=(Z_1,Z_2,Z_3,Z_4,Z_5)^\mathsf{T}\sim N_5(0,I_5)\), with independent standard normal coordinates. Define

A_1=\operatorname{diag}(1,1,0,0,0),\quad A_2=\operatorname{diag}(0,0,1,0,0),\quad A_3=\operatorname{diag}(0,0,0,1,1).

These matrices are symmetric and positive semidefinite, and their sum is \(I_5\). Their ranks are \(2,1,2\), which sum to \(5\). Cochran's theorem applies. The forms are

Q_1=Z_1^2+Z_2^2,\qquad Q_2=Z_3^2,\qquad Q_3=Z_4^2+Z_5^2.

Therefore \(Q_1\sim\chi^2_2\), \(Q_2\sim\chi^2_1\), and \(Q_3\sim\chi^2_2\), and the three are mutually independent. The total is \(Q_1+Q_2+Q_3=Z^\mathsf{T}Z\sim\chi^2_5\).

Worked Example: Projections onto Rotated Directions

Let \(Z=(Z_1,Z_2,Z_3)^\mathsf{T}\sim N_3(0,I_3)\). Define three unit vectors

u_1=\frac{1}{\sqrt{2}}(1,1,0)^\mathsf{T},\quad u_2=\frac{1}{\sqrt{2}}(1,-1,0)^\mathsf{T},\quad u_3=(0,0,1)^\mathsf{T}, \qquad A_i=u_iu_i^\mathsf{T}.

The vectors are orthonormal: each has squared length one, \(u_1^\mathsf{T}u_2=(1-1)/2=0\), and each of the first two has inner product zero with \(u_3\). Thus \(\sum_i A_i=I_3\), and each \(A_i\) has rank one. The ranks sum to three, so the theorem gives three independent \(\chi^2_1\) forms. In detail,

Q_1=\frac{(Z_1+Z_2)^2}{2},\qquad Q_2=\frac{(Z_1-Z_2)^2}{2},\qquad Q_3=Z_3^2.

Each displayed form is the square of a standard normal coordinate in the rotated orthonormal basis, and the theorem shows their independence. Their sum is \(Z_1^2+Z_2^2+Z_3^2\). The example illustrates why disjoint groups of the original coordinates are not required; orthogonal directions suffice.

Worked Example: What Can Go Wrong When Ranks Do Not Add to the Dimension

Let \(Z=(Z_1,Z_2)^\mathsf{T}\sim N_2(0,I_2)\), and set

A_1= \begin{pmatrix} 1&0\\ 0&1/2 \end{pmatrix}, \qquad A_2= \begin{pmatrix} 0&0\\ 0&1/2 \end{pmatrix}.

Both matrices are symmetric and positive semidefinite, and \(A_1+A_2=I_2\). But their ranks are two and one, so the sum of the ranks is three, not two. The quadratic forms are

Q_1=Z_1^2+\frac12 Z_2^2,\qquad Q_2=\frac12 Z_2^2.

They cannot have the chi-square laws with degrees of freedom equal to their ranks: \(E[Q_1]=1+1/2=3/2\), not \(2\), and \(E[Q_2]=1/2\), not \(1\). They are also dependent, since

\operatorname{Cov}(Q_1,Q_2) =\operatorname{Cov}\left(Z_1^2+\frac12Z_2^2,\frac12Z_2^2\right) =\frac14\operatorname{Var}(Z_2^2) =\frac12.

Here we used independence of \(Z_1\) and \(Z_2\), and \(\operatorname{Var}(Z_2^2)=2\). The rank condition cannot be dropped from the theorem.

Why the Result Matters

Cochran's theorem converts a matrix decomposition into an exact distributional statement. In applications, a total Gaussian squared norm can be split into components corresponding to orthogonal subspaces. The ranks state how many independent Gaussian coordinates contribute to each component, and therefore give the degrees of freedom of its chi-square distribution. Independence makes it possible to analyze the components separately while retaining the exact distribution of their sum.

The key logic is worth keeping distinct: the identity sum says the quadratic forms add to the total squared norm; positive semidefiniteness bounds the eigenvalues; and the rank-sum condition forces all nonzero eigenvalues to equal one. That last step rules out fractional weights and yields projections. Without it, an identity decomposition need not consist of projections and need not give independent chi-square forms.

Common Mistake

Wrong reasoning: “The matrices sum to the identity, so their quadratic forms must be independent chi-square variables.” A sum equal to the identity only guarantees that the forms add to \(Z^\mathsf{T}Z\). It does not, by itself, make the matrices projections or their ranges orthogonal.
Correction: Check symmetry, positive semidefiniteness, and the rank sum. Under all three conditions in the theorem, the matrices become mutually orthogonal projections, which justifies the chi-square distributions and independence.
Another useful check: For a projection, the only eigenvalues are zero and one. A positive-semidefinite matrix with an eigenvalue strictly between zero and one is not a projection, even if it appears in a collection whose sum is the identity.

Key takeaway: For positive-semidefinite matrices summing to \(I_n\), a total rank of \(n\) forces a decomposition into mutually orthogonal projections. Applied to a standard Gaussian vector, the corresponding quadratic forms are independent chi-square variables with degrees of freedom equal to their ranks.

Check Your Understanding

Use the theorem and its proof to answer the following questions.

Why does \(I_n-A_i\) being positive semidefinite imply that every eigenvalue of \(A_i\) is at most one?
In the theorem, what does the rank sum tell you about the dimensions of the projection ranges?
For independent standard normal coordinates \(Z_1,\ldots,Z_6\), consider the diagonal projections onto coordinates \(1,2\), onto \(3,4,5\), and onto \(6\). State the distribution and independence properties of the three quadratic forms.
Why is positive semidefiniteness needed when deducing that the ranges of distinct projections are orthogonal?
Can an identity decomposition with a matrix having a nonzero eigenvalue strictly between zero and one satisfy all the hypotheses of Cochran's theorem? Explain.