Coordinate Subspaces and Gaussian Sum-of-Squares Examples

Coordinate Sets as Projection Matrices

For a standard normal vector, a quadratic form built from a diagonal projection has a direct interpretation: it keeps selected coordinates and discards the rest. This makes coordinate subspaces useful for checking degrees of freedom, identifying independent sums of squares, and catching matrix mistakes before applying more general results.

Let \(Z=(Z_1,\ldots,Z_p)^\mathsf T\sim N_p(0,I_p)\), where \(I_p\) is the \(p\times p\) identity matrix. Its coordinates are independent \(N(0,1)\) random variables. For a set \(A\subseteq\{1,\ldots,p\}\), let \(\mathcal S_A\) be the subspace spanned by the coordinate vectors \(e_i\) with \(i\in A\).

Definition: The coordinate projection onto \(\mathcal S_A\) is the matrix \(P_A\) whose \(i\)th diagonal entry is 1 when \(i\in A\) and 0 otherwise. All off-diagonal entries are 0. Thus \(P_Az\) retains the coordinates indexed by \(A\) and sets every other coordinate to 0.

Because \(P_A\) is diagonal with entries 0 or 1, it is symmetric and idempotent: \(P_A^\mathsf T=P_A\) and \(P_A^2=P_A\). Its rank and trace are both \(|A|\), the number of indices in \(A\). The associated quadratic form is

Z^\mathsf T P_A Z=\sum_{i\in A}Z_i^2.

A chi-square random variable with \(r\) degrees of freedom, denoted \(\chi^2_r\), is the sum of the squares of \(r\) independent \(N(0,1)\) random variables. In particular, an empty coordinate set gives the constant zero, conventionally viewed as \(\chi^2_0\).

Distribution and Independence for a Partition

Theorem: Suppose \(A_1,\ldots,A_k\) are nonempty, pairwise disjoint sets whose union is \(\{1,\ldots,p\}\). Then \(Q_j=Z^\mathsf T P_{A_j}Z\sim\chi^2_{|A_j|}\), and \(Q_1,\ldots,Q_k\) are mutually independent. Moreover, \(\sum_{j=1}^kP_{A_j}=I_p\) and \(\sum_{j=1}^kQ_j=Z^\mathsf T Z\sim\chi^2_p\).

Proof. By the definition of \(P_{A_j}\), \(Q_j=\sum_{i\in A_j}Z_i^2\). The coordinates in each sum are independent standard normal variables, so this sum has the \(\chi^2_{|A_j|}\) distribution. Since the sets are disjoint, each \(Q_j\) is a function of a different block of coordinates. The blocks are independent, so their measurable functions \(Q_1,\ldots,Q_k\) are mutually independent. For each coordinate \(i\), exactly one set contains \(i\); hence the \(i\)th diagonal entry of \(\sum_jP_{A_j}\) is 1, proving that sum is \(I_p\). Applying the quadratic form to \(Z\) gives \(\sum_jQ_j=Z^\mathsf TZ\), the sum of the squares of all \(p\) independent standard normal coordinates. It therefore has the \(\chi^2_p\) distribution. \(\square\)

The independence here follows from disjoint coordinate blocks, not merely from the fact that each marginal sum has a chi-square distribution. The partition condition gives a transparent matrix check as well: the projection matrices add to the identity and their ranks add to \(p\).

Worked Example: A Three-Way Coordinate Partition

Let \(Z\sim N_6(0,I_6)\), and divide its coordinates into \(A_1=\{1,4\}\), \(A_2=\{2,5,6\}\), and \(A_3=\{3\}\). Construct the projection matrices and find the joint laws of the three sums of squares.

In the coordinate order \(1,\ldots,6\), the matrices are

P_{A_1}=\operatorname{diag}(1,0,0,1,0,0),\quad P_{A_2}=\operatorname{diag}(0,1,0,0,1,1),\quad P_{A_3}=\operatorname{diag}(0,0,1,0,0,0).

Their sum is \(\operatorname{diag}(1,1,1,1,1,1)=I_6\). Their ranks are 2, 3, and 1. The corresponding quadratic forms are \(Q_1=Z_1^2+Z_4^2\), \(Q_2=Z_2^2+Z_5^2+Z_6^2\), and \(Q_3=Z_3^2\). Therefore \(Q_1\sim\chi^2_2\), \(Q_2\sim\chi^2_3\), and \(Q_3\sim\chi^2_1\). The coordinate sets are disjoint, so these three random variables are mutually independent. As a check, their sum is \(Z^\mathsf TZ\sim\chi^2_6\), and their degrees of freedom add to \(2+3+1=6\).

Nested Coordinate Sets and Independent Increments

Coordinate sets need not form a full partition to yield a useful decomposition. When one set is contained in another, the difference between their sums of squares is exactly the sum over the added coordinates.

Proposition: If \(A\subseteq B\subseteq\{1,\ldots,p\}\), then \(P_B-P_A=P_{B\setminus A}\) and \[ Z^\mathsf TP_BZ-Z^\mathsf TP_AZ=Z^\mathsf TP_{B\setminus A}Z. \] The increment has distribution \(\chi^2_{|B\setminus A|}\) and is independent of \(Z^\mathsf TP_AZ\), which has distribution \(\chi^2_{|A|}\).

Proof. For any coordinate \(i\in A\), both \(P_A\) and \(P_B\) have diagonal entry 1, so their difference has entry 0. For \(i\in B\setminus A\), \(P_B\) has entry 1 and \(P_A\) has entry 0, so their difference has entry 1. Outside \(B\), both entries are 0. This proves \(P_B-P_A=P_{B\setminus A}\), and multiplying on the left and right by \(Z^\mathsf T\) and \(Z\) gives the quadratic-form identity. The sums indexed by \(A\) and \(B\setminus A\) use disjoint blocks of independent standard normal coordinates. Their quadratic forms are therefore independent and have chi-square degrees of freedom \(|A|\) and \(|B\setminus A|\), respectively. \(\square\)

Worked Example: An Increment in a Nested Pair

Let \(Z\sim N_7(0,I_7)\), \(A=\{1,2,4\}\), and \(B=\{1,2,3,4,5\}\). Find the distributions of the sum on \(A\), the increment from \(A\) to \(B\), and the sum on \(B\).

In the coordinate order \(1,\ldots,7\),

P_A=\operatorname{diag}(1,1,0,1,0,0,0),\qquad P_B=\operatorname{diag}(1,1,1,1,1,0,0).

Subtracting gives \(P_B-P_A=\operatorname{diag}(0,0,1,0,1,0,0)=P_{\{3,5\}}\). Thus \(Q_A=Z_1^2+Z_2^2+Z_4^2\sim\chi^2_3\), while \(Q_B-Q_A=Z_3^2+Z_5^2\sim\chi^2_2\). These two quantities are independent because they use disjoint coordinates. Their sum is \(Q_B=Z_1^2+Z_2^2+Z_3^2+Z_4^2+Z_5^2\sim\chi^2_5\), in agreement with the rank of \(P_B\). The total \(Q_B\) is not independent of its increment: it contains that increment as a summand.

Overlap Changes the Independence Conclusion

Two coordinate sums can each have a chi-square distribution without being independent. Disjointness is the key condition in the partition theorem. If two sets overlap, they reuse at least one random coordinate, so the sums share a random term.

Worked Example: Overlapping Sets Are Not Independent

Let \(Z\sim N_3(0,I_3)\), with \(A=\{1,2\}\) and \(B=\{2,3\}\). Both coordinate sums have \(\chi^2_2\) marginal distributions. Check whether they are independent.

The projection matrices are \(P_A=\operatorname{diag}(1,1,0)\) and \(P_B=\operatorname{diag}(0,1,1)\), so \(P_AP_B=\operatorname{diag}(0,1,0)\ne0\). The quadratic forms are \(Q_A=Z_1^2+Z_2^2\) and \(Q_B=Z_2^2+Z_3^2\). Their covariance is

\operatorname{Cov}(Q_A,Q_B) =\operatorname{Cov}(Z_1^2+Z_2^2,Z_2^2+Z_3^2) =\operatorname{Var}(Z_2^2)=2.

The cross-covariances involving different coordinates are zero by independence, and \(\operatorname{Var}(Z_2^2)=E[Z_2^4]-(E[Z_2^2])^2=3-1=2\). Since independent random variables with finite second moments must have covariance zero, \(Q_A\) and \(Q_B\) are not independent.

Common Mistakes and Practical Checks

Using the number of coordinates in the whole vector as the degrees of freedom. The degrees of freedom for \(Z^\mathsf TP_AZ\) are \(|A|=\operatorname{rank}(P_A)\), not \(p\) unless \(A\) contains every coordinate.
Assuming two chi-square marginals imply independence. The overlapping-set example has two \(\chi^2_2\) marginals but a nonzero covariance.
Confusing a nested total with its increment. In a nested decomposition, the increment is independent of the smaller sum. The larger total includes the increment and is generally not independent of it.
Adding projection matrices without checking their coordinate sets. The sum equals the identity only when the sets partition all coordinates. Overlap creates diagonal entries greater than 1, while omitted coordinates create zero entries.

These coordinate calculations give a concrete way to interpret more abstract projection arguments: the rank counts retained independent normal coordinates, and disjoint coordinate blocks explain independence directly. In applications, writing out the selected indices and checking the diagonal entries can reveal an incorrect degrees-of-freedom claim or an unjustified independence claim quickly.

Check Your Understanding

For each question, identify the coordinate sets and use them to justify your conclusion.

If \(Z\sim N_5(0,I_5)\) and \(A=\{2,3,5\}\), what is the matrix \(P_A\), and what is the distribution of \(Z^\mathsf TP_AZ\)?
Suppose \(A_1=\{1,3\}\), \(A_2=\{2,4,5\}\), and \(Z\sim N_5(0,I_5)\). Are the two projected sums independent, and what is the distribution of their sum?
If \(A=\{1,4\}\) and \(B=\{1,2,4,5\}\), what coordinate set indexes the increment \(Z^\mathsf TP_BZ-Z^\mathsf TP_AZ\)? State its distribution.
For \(A=\{1,2\}\) and \(B=\{2,3\}\), why is it not enough to know that both quadratic forms have chi-square marginal distributions to conclude independence?
What should the rank and trace of a coordinate projection \(P_A\) equal, and why?