The Converse Direction in Cochran's Theorem

Why the Converse Matters

Cochran’s theorem gives conditions under which orthogonal Gaussian projections produce independent chi-square sums of squares. The converse asks what can be inferred in the other direction: if quadratic forms add exactly to the total Gaussian sum of squares and each form has a chi-square distribution, must their matrices really describe orthogonal projections? The answer is yes. The distribution of each form determines its matrix’s eigenvalues, and the sum identity then forces the projections to be mutually orthogonal.

This implication is useful because it turns distributional information into geometric structure. In particular, it identifies the degrees of freedom as matrix ranks rather than treating them as unrelated labels.

Setup and Assumptions

Definition: Let \(Z\sim N_n(0,I_n)\), where \(I_n\) is the \(n\times n\) identity matrix. Let \(A_1,\ldots,A_k\) be real symmetric \(n\times n\) matrices, and define \(Q_i=Z'A_iZ\). Suppose that

(1) \(\sum_{i=1}^k Q_i=Z'Z\) almost surely, and

(2) for each \(i\), \(Q_i\sim\chi^2_{\nu_i}\), where \(\nu_i\geq 0\). Here \(\chi^2_0\) denotes a point mass at zero; positive degrees of freedom may also be interpreted using the usual gamma-distribution extension.

Then each \(A_i\) is an orthogonal projection, \(\operatorname{rank}(A_i)=\nu_i\), and \(A_iA_j=0\) whenever \(i\ne j\). In particular, every \(\nu_i\) is an integer and \(\sum_i\nu_i=n\).

The standard Gaussian assumption is essential to the argument: after an orthogonal change of coordinates, a quadratic form becomes a weighted sum of independent squared standard normal variables. The matrices are taken symmetric because \(Z'AZ\) depends only on the symmetric part of \(A\).

First Consequence: The Matrices Sum to the Identity

Write \(B=\sum_{i=1}^k A_i-I_n\). The sum-of-squares condition says \(Z'BZ=0\) almost surely. Since \(B\) is symmetric and the standard Gaussian density is positive everywhere, this almost-sure equality implies \(B=0\). Here is the detail: if some vector \(z_0\) had \(z_0'Bz_0\ne 0\), continuity would give an open ball around \(z_0\) on which the quadratic form is nonzero. That ball has positive Gaussian probability, contradicting the almost-sure equality. Thus \(z'Bz=0\) for every \(z\). Taking \(z\) to be each coordinate vector gives \(B_{jj}=0\); taking \(z=e_j+e_\ell\) then gives \(2B_{j\ell}=0\) for \(j\ne\ell\). Therefore,

\sum_{i=1}^k A_i=I_n.

This step is important: equality of random quadratic forms has given a deterministic matrix identity, which can now be combined with the distributional assumptions.

How a Chi-Square Law Determines the Eigenvalues

Fix one index \(i\), and let \(\lambda_1,\ldots,\lambda_n\) be the eigenvalues of \(A_i\), including zeros. By the spectral theorem, an orthogonal change of coordinates transforms \(Z\) into another standard normal vector. Thus \(Q_i\) has the same distribution as \(\sum_{j=1}^n\lambda_jY_j^2\), where the \(Y_j\) are independent \(N(0,1)\) variables.

First, all \(\lambda_j\) must be nonnegative. If \(\lambda_j<0\), then there is positive probability that \(|Y_j|\) is sufficiently large while all the other coordinates are bounded. On that event the negative term \(\lambda_jY_j^2\) outweighs the other terms, so \(Q_i<0\). This contradicts the nonnegative support of a chi-square variable.

For \(t\) in a neighborhood of zero, the moment-generating function of \(Q_i\) is therefore

M_{Q_i}(t)=\prod_{j:\lambda_j>0}(1-2t\lambda_j)^{-1/2}.

The chi-square moment-generating function is \(M_{Q_i}(t)=(1-2t)^{-\nu_i/2}\). Taking the logarithmic derivative of the two expressions gives

\sum_{j:\lambda_j>0}\frac{\lambda_j}{1-2t\lambda_j} = \frac{\nu_i}{1-2t}.

This equality holds near zero and hence, after multiplying through by denominators, is an identity of rational functions. Each positive eigenvalue \(\lambda_j\) creates a pole at \(t=1/(2\lambda_j)\), with a nonzero coefficient. The right side has no pole except at \(t=1/2\) (and has no pole at all if \(\nu_i=0\)). Consequently, every positive eigenvalue must equal 1. With all positive eigenvalues equal to 1, the left side is \(m/(1-2t)\), where \(m\) is the number of such eigenvalues. Equality gives \(m=\nu_i\). When \(\nu_i=0\), there can be no positive eigenvalues, so the same conclusion holds with \(m=0\).

Thus the eigenvalues of \(A_i\) are all zero or one. By the spectral theorem, \(A_i^2=A_i\), so \(A_i\) is an orthogonal projection, and its rank is the number of eigenvalues equal to one. We have proved the individual matrix conclusion:

Result: If \(Z\sim N_n(0,I_n)\), \(A\) is real symmetric, and \(Z'AZ\sim\chi^2_\nu\), then \(A\) is an orthogonal projection and \(\operatorname{rank}(A)=\nu\). In particular, \(\nu\) must be an integer.

Worked Example: A Chi-Square Marginal Identifies a Projection

Let \(Z=(Z_1,Z_2,Z_3)'\sim N_3(0,I_3)\), and set \(A=\operatorname{diag}(1,1,0)\). Then

$$ Z'AZ=Z_1^2+Z_2^2. $$

Because \(Z_1\) and \(Z_2\) are independent standard normal variables, this has a \(\chi^2_2\) distribution. The eigenvalues of \(A\) are \(1,1,0\); directly, \(A^2=A\), and \(\operatorname{rank}(A)=2\). In this example the matrix structure and the chi-square degrees of freedom agree exactly.

The Sum Identity Forces Orthogonality

Each \(A_i\) is now known to be symmetric and idempotent. Fix \(i\), and use \(\sum_j A_j=I_n\) to write

A_i=A_iI_nA_i =A_i\left(\sum_{j=1}^k A_j\right)A_i =A_i+\sum_{j\ne i}A_iA_jA_i.

After cancelling \(A_i\), this gives \(\sum_{j\ne i}A_iA_jA_i=0\). Each summand is positive semidefinite: for every vector \(x\),

x'A_iA_jA_ix=(A_ix)'A_j(A_ix)\geq 0,

because \(A_j\) is an orthogonal projection. A sum of positive semidefinite matrices can equal zero only when each summand equals zero. More explicitly, for each \(j\ne i\), \(A_iA_jA_i=0\), and hence

\|A_jA_i x\|^2 =x'A_iA_j^2A_ix =x'A_iA_jA_ix =0

for every \(x\). Thus \(A_jA_i=0\); taking transposes gives \(A_iA_j=0\). The projections are pairwise orthogonal.

Theorem (Converse direction of Cochran’s theorem): Under the assumptions in the setup, the sum-of-squares identity and chi-square marginal laws imply that the matrices are pairwise orthogonal projections with \(\operatorname{rank}(A_i)=\nu_i\). Taking traces in \(\sum_iA_i=I_n\) gives

\(\sum_i\nu_i=\sum_i\operatorname{tr}(A_i)=\operatorname{tr}(I_n)=n.\)

The matrix conclusions do not require the \(Q_i\) to be assumed independent. For a standard Gaussian vector, the resulting orthogonality of the projection spaces also yields independence of the projected Gaussian components and hence of their squared lengths, as in the earlier material on Cochran’s theorem.

Worked Consequences and a Common Pitfall

Worked Example: An Orthogonal Decomposition in Two Coordinates

Let \(Z\sim N_2(0,I_2)\), and choose the perpendicular unit vectors \(u=(3/5,4/5)'\) and \(v=(-4/5,3/5)'\). Define \(A_1=uu'\) and \(A_2=vv'\). Their matrices are

A_1= \begin{pmatrix} 9/25&12/25\\ 12/25&16/25 \end{pmatrix}, \qquad A_2= \begin{pmatrix} 16/25&-12/25\\ -12/25&9/25 \end{pmatrix}.

Adding the entries gives \(A_1+A_2=I_2\). Each is a rank-one projection because \(u\) and \(v\) are unit vectors, and \(u'v=0\) gives \(A_1A_2=0\). The quadratic forms are \(Q_1=(u'Z)^2\) and \(Q_2=(v'Z)^2\). The coordinates \(u'Z\) and \(v'Z\) are independent standard normal variables, so both quadratic forms are \(\chi^2_1\), and \(Q_1+Q_2=Z'Z\). This is a non-coordinate example: the projection directions are rotated relative to the original axes.

Worked Example: Why a Matching Mean Is Not Enough

Let \(Z\sim N_3(0,I_3)\), \(A_1=\operatorname{diag}(2/3,1/3,0)\), and \(A_2=I_3-A_1\). These matrices are symmetric and sum to \(I_3\), so their quadratic forms sum to \(Z'Z\). But \(A_1\) is not a projection: its eigenvalues are \(2/3,1/3,0\), not just zero and one.

The first quadratic form is \(Q_1=(2/3)Z_1^2+(1/3)Z_2^2\). Its mean is \(\operatorname{tr}(A_1)=1\), which matches the mean of \(\chi^2_1\), but the variance does not match. From independence and \(\operatorname{Var}(Z_j^2)=2\),

\operatorname{Var}(Q_1) =2\left[(2/3)^2+(1/3)^2\right] =2(5/9) =10/9,

whereas \(\operatorname{Var}(\chi^2_1)=2\). Thus \(Q_1\) is not \(\chi^2_1\). The trace gives the mean, but cannot by itself establish a chi-square distribution or a projection matrix.

A common mistaken argument is: “The forms sum to \(Z'Z\), and their means give the stated degrees of freedom; therefore their matrices are projections.” The conclusion does not follow from the means. For a symmetric matrix, the mean of its Gaussian quadratic form is its trace, and many different eigenvalue collections can have the same trace. The chi-square marginal assumption supplies stronger information: its moment-generating function pins down the positive eigenvalues, while the sum identity supplies the matrix decomposition needed to prove orthogonality.

Translate the sum identity.
Equality of the quadratic forms almost surely gives \(\sum_i A_i=I_n\).

Use each chi-square law.
The Gaussian eigenvalue representation and its moment-generating function show that each \(A_i\) has only zero and one eigenvalues.

Use the projection sum.
Idempotence together with \(\sum_i A_i=I_n\) forces every cross-product \(A_iA_j\) to vanish.

Check Your Understanding

Try to justify each conclusion using the assumptions and arguments above.

Why does \(Z'BZ=0\) almost surely for a standard Gaussian \(Z\), with symmetric \(B\), imply \(B=0\)?
In the eigenvalue argument, why must a negative eigenvalue contradict a chi-square marginal distribution?
If \(Z'AZ\sim\chi^2_3\) and \(A\) is symmetric, what are the possible eigenvalues of \(A\), and what is its rank?
Why does \(\sum_iA_i=I_n\) imply that the degrees of freedom sum to \(n\) once the matrices are known to be projections?
What is wrong with concluding that a quadratic form is chi-square solely because its mean matches the proposed chi-square mean?