Rank-Deficient Partitions and Redundant Components

Why Rank Checks Matter

A proposed sums-of-squares decomposition can look plausible because its components have familiar names, such as a treatment component and a residual component. But the labels do not establish that the components divide the data into distinct directions. If two components capture some of the same directions, their sums of squares count some variation more than once. If the components fail to span the whole data space, some variation is left out. Projection ranks make these problems visible as dimension-accounting errors.

For an \(n\)-dimensional observation vector, the rank of an orthogonal projection is the dimension of the subspace onto which it projects. In a valid orthogonal partition, each direction belongs to exactly one component. The component ranks therefore add to \(n\). This is a powerful diagnostic, but it must be used alongside checks of the projection matrices: matching rank totals alone do not prove that components are orthogonal or sum to the identity.

Definition: A matrix \(P\) is an orthogonal projection matrix if \(P'=P\) and \(P^2=P\). Its range is the subspace onto which it projects, and its rank \(\operatorname{rank}(P)\) is the dimension of that subspace. A collection \(P_1,\ldots,P_k\) is an orthogonal projection partition of \(\mathbb{R}^n\) if \(P_iP_j=0\) for \(i\ne j\) and \(\sum_{i=1}^k P_i=I_n\).

For any vector \(y\), the sum of squares associated with \(P_i\) is \(y'P_i y=\|P_i y\|^2\). In an orthogonal projection partition, these component sums of squares add to the total \(y'y\). Under \(Y\sim N_n(\mu,\sigma^2 I_n)\), Cochran’s theorem gives independent components \(Y'P_iY/\sigma^2\), each with a noncentral chi-square distribution having degrees of freedom \(\operatorname{rank}(P_i)\) and noncentrality parameter \(\mu'P_i\mu/\sigma^2\). A rank-zero component is identically zero; it supplies no positive degrees of freedom.

Ranks, Orthogonality, and Completeness

The rank count follows directly from the geometry of orthogonal subspaces. If the ranges of the projections are mutually orthogonal, their dimensions add to the dimension of their combined span. If the projections also sum to \(I_n\), that span is all of \(\mathbb{R}^n\). Thus a complete orthogonal partition has total rank \(n\).

Theorem: If \(P_1,\ldots,P_k\) are mutually orthogonal projection matrices, then \(\operatorname{rank}(\sum_i P_i)=\sum_i\operatorname{rank}(P_i)\). If, in addition, \(\sum_iP_i=I_n\), then \(\sum_i\operatorname{rank}(P_i)=n\).

Use the zero-product condition.
If \(P_iP_j=0\) for \(i\ne j\), then the ranges of \(P_i\) and \(P_j\) are orthogonal: for \(u=P_i x\) and \(v=P_j y\), \(u'v=x'P_iP_jy=0\).

Add dimensions of orthogonal subspaces.
The dimension of the span of mutually orthogonal subspaces is the sum of their dimensions. Since \(\operatorname{rank}(P_i)\) is the dimension of the range of \(P_i\), the rank of the sum is the sum of the ranks.

Check completeness.
If \(\sum_iP_i=I_n\), the combined range is \(\mathbb{R}^n\), of dimension \(n\). Therefore the ranks sum to \(n\). \(\square\)

There is also a useful converse diagnostic. For orthogonal projections, a sum equal to the identity already forces the projections to be mutually orthogonal. To see why, take any \(x\) in the range of \(P_i\). Then \(P_i x=x\), so \[ \|x\|^2=x'I_nx=\sum_j x'P_jx=\|x\|^2+\sum_{j\ne i}\|P_jx\|^2. \] Every term in the last sum is nonnegative, so each is zero. Hence every other projection annihilates the range of \(P_i\), and \(P_jP_i=0\) for \(j\ne i\).

Rank diagnostic: For proposed mutually orthogonal projection components in \(\mathbb{R}^n\), a rank sum greater than \(n\) is impossible. A rank sum less than \(n\) means that the components cannot cover \(\mathbb{R}^n\). A rank sum equal to \(n\) is necessary for a complete orthogonal partition, but does not by itself establish orthogonality or that the matrices sum to \(I_n\).

Worked Examples: Finding Redundancy and Checking Degrees of Freedom

Worked Example: A Duplicated Component Overcounts Variation

In \(\mathbb{R}^3\), let \(P_1\) and \(P_2\) both project onto the first coordinate axis, and let \(P_3\) project onto the plane spanned by the second and third coordinate axes. For \(y=(2,3,4)'\), check whether these components form a partition.

The matrices and ranks are \[ P_1=P_2= \begin{pmatrix}1&0&0\\0&0&0\\0&0&0\end{pmatrix}, \qquad P_3= \begin{pmatrix}0&0&0\\0&1&0\\0&0&1\end{pmatrix}, \] with ranks \(1,1,2\). Their ranks sum to \(4\), which exceeds \(n=3\). The two first-coordinate components are duplicates.

The component sums of squares are \(y'P_1y=4\), \(y'P_2y=4\), and \(y'P_3y=3^2+4^2=25\). Their sum is \(33\), whereas \(y'y=2^2+3^2+4^2=29\). Equivalently, \(P_1+P_2+P_3=\operatorname{diag}(2,1,1)\ne I_3\). The proposed components double-count the first-coordinate variation and cannot be a Cochran partition.

Worked Example: Redundant Contrast Rows Do Not Add Degrees of Freedom

Suppose three proposed contrast rows in \(\mathbb{R}^4\) are \(c_1=(1,-1,0,0)\), \(c_2=(0,0,1,-1)\), and \(c_3=2c_1\). Determine the rank of the contrast space and construct a complete orthogonal partition using the two distinct contrast directions.

The third row is a multiple of the first, so the row space has dimension \(2\), not \(3\). The projection onto the \(c_1\) direction is \[ P_1=\frac{c_1'c_1}{c_1c_1'}= \frac12 \begin{pmatrix}1&-1&0&0\\-1&1&0&0\\0&0&0&0\\0&0&0&0\end{pmatrix}. \] Similarly, the projection onto \(c_2\) is \[ P_2=\frac12 \begin{pmatrix}0&0&0&0\\0&0&0&0\\0&0&1&-1\\0&0&-1&1\end{pmatrix}. \] Each has rank \(1\), and their ranges are orthogonal. The remaining projection \(P_3=I_4-P_1-P_2\) has rank \(2\), projecting onto the span of \((1,1,0,0)'\) and \((0,0,1,1)'\). Thus the ranks are \(1+1+2=4\), and these three projections form a complete orthogonal partition.

For \(y=(3,1,5,1)'\), the first contrast sum of squares is \((c_1y)^2/(c_1c_1')=(2)^2/2=2\). The second is \((c_2y)^2/(c_2c_2')=(4)^2/2=8\). The remaining component is \(y'P_3y=36-2-8=26\), since \(y'y=9+1+25+1=36\). The components add to \(36\). Counting the three listed rows as three independent contrast degrees of freedom would be incorrect: the row rank, not the number of rows, determines the contrast-space dimension.

Worked Example: A Rank-Deficient Model Extension Has Zero Increment

Let the current model space be \(U=\operatorname{span}(e_1,e_2)\) in \(\mathbb{R}^4\), and suppose a proposed extension adds the column \(e_2\) again. Find the ranks of the current-model, increment, and residual projections, and calculate their sums of squares for \(y=(1,2,3,4)'\).

Because the added column is already in \(U\), the extended space \(V\) equals \(U\). Thus \(P_U=P_V=\operatorname{diag}(1,1,0,0)\), and the increment projection \(Q=P_V-P_U\) is the zero matrix. The residual projection is \(R=I_4-P_V=\operatorname{diag}(0,0,1,1)\). Their ranks are \(2,0,2\), which sum to \(4\).

The current-model sum of squares is \(y'P_Uy=1^2+2^2=5\). The increment is \(y'Qy=0\), and the residual sum of squares is \(y'Ry=3^2+4^2=25\). These add to \(30=y'y\). The proposed extension contributes no new direction and therefore has zero degrees of freedom. Treating the repeated column as a one-degree-of-freedom effect would assign variation and degrees of freedom to a component that is identically zero.

Nested Spaces and Rank Differences

Rank accounting is especially straightforward for nested model spaces. Let \(U\subseteq V\) be subspaces of \(\mathbb{R}^n\), with orthogonal projection matrices \(P_U\) and \(P_V\). Since projecting onto \(V\) leaves every vector in \(U\) unchanged, \(P_UP_V=P_VP_U=P_U\). Therefore the increment matrix \(Q=P_V-P_U\) is itself an orthogonal projection: \[ Q'=Q,\qquad Q^2=P_V^2-P_VP_U-P_UP_V+P_U^2=P_V-P_U=Q. \] Its range is the part of \(V\) orthogonal to \(U\), and its rank is \(\dim(V)-\dim(U)\). The residual projection \(I_n-P_V\) completes the partition because \[ P_U+(P_V-P_U)+(I_n-P_V)=I_n. \] The ranks telescope to \(\dim(U)+[\dim(V)-\dim(U)]+[n-\dim(V)]=n\). If the extension is redundant, then \(V=U\) in the relevant directions and the increment rank is zero.

Practical test: For each proposed component, identify its projected subspace and compute its rank. Then check whether component subspaces are mutually orthogonal, whether their projections sum to the identity (or to the intended total projection), and whether the ranks sum to the rank of that total projection. These checks distinguish duplicate directions, omitted directions, and genuine orthogonal increments.

Common Mistake: Counting Labels or Columns Instead of Directions

A frequent error is to assign one degree of freedom to every named effect, contrast row, or design column. Degrees of freedom count linearly independent directions, not labels. Repeated columns, dependent contrasts, and aliased effects can leave the model space unchanged even when the written design includes additional terms. The correction is to compute the rank of the relevant design or contrast space, then use projection matrices for the actual subspaces.

A second error is to regard a correct rank total as sufficient proof of a Cochran decomposition. For example, two rank-one projections in \(\mathbb{R}^2\) have ranks summing to \(2\), but if their ranges are not orthogonal, their projection matrices do not sum to \(I_2\). The rank count is a necessary dimension check; the matrix identities and orthogonality conditions must still be verified. When those conditions fail, the independent chi-square conclusion of Cochran’s theorem does not follow.

Check Your Understanding

Use ranks to diagnose each proposed decomposition before drawing conclusions about independence.

Three mutually orthogonal projections in \(\mathbb{R}^7\) have ranks \(2,2,2\). Can they form a complete partition? What dimension remains uncovered?
Two proposed projections in \(\mathbb{R}^5\) each have rank \(3\). What does the rank total tell you if they are claimed to be mutually orthogonal?
A contrast matrix has five rows but rank three. How many contrast directions, and hence how many projection degrees of freedom, does its row space contain?
For nested spaces \(U\subseteq V\), what does \(\operatorname{rank}(P_V-P_U)=0\) imply about the added model directions?
Why does a rank sum equal to \(n\) not, by itself, establish that a proposed collection of components satisfies Cochran’s theorem?