Projection Geometry of Nested Multivariate Models

Why Nested Model Geometry Matters

In a multivariate linear model, each response is observed on the same $n$ cases, so the response data can be arranged as an $n\times p$ matrix $Y$. A linear model restricts the fitted value vector for each response to a specified subspace of $\mathbb{R}^n$. When one model is nested in another, their relationship is therefore a relationship between subspaces: the larger model can fit everything the smaller model can, and possibly more.

This geometry gives a direct interpretation of the hypothesis and error sum-of-squares matrices. The hypothesis matrix records the part of the response explained by the enlargement from the restricted model to the full model. The error matrix records the part left orthogonal to the full model. Their degrees of freedom come from the dimensions of those two orthogonal spaces, not from counting columns in a design matrix without checking whether those columns are linearly independent.

Definition: Let $S_0\subseteq S_1\subseteq\mathbb{R}^n$ be the column spaces of a restricted and a full model, respectively. Let $P_0$ and $P_1$ be the orthogonal projection matrices onto these spaces. Write $r_0=\dim(S_0)$ and $r_1=\dim(S_1)$. The hypothesis and error sum-of-squares matrices for this nested comparison are $$ H=Y'(P_1-P_0)Y,\qquad E=Y'(I-P_1)Y. $$ Their corresponding projection degrees of freedom are $r_1-r_0$ and $n-r_1$.

The definition uses projection matrices, not a particular parameterization of the models. If a model is described by a design matrix $X$, its space is $\mathcal{C}(X)$, and its projection is onto that column space. A rank-deficient design matrix still defines a projection; its projection rank is $\operatorname{rank}(X)$, not necessarily its number of columns.

The Orthogonal Pieces of a Nested Comparison

The key geometric fact is that $P_1-P_0$ projects onto the part of the larger model space that is orthogonal to the smaller space. The residual projection $I-P_1$ projects onto the orthogonal complement of the full model space. These two pieces are orthogonal to one another.

Theorem: If $S_0\subseteq S_1$, then $Q=P_1-P_0$ is the orthogonal projection onto $S_1\cap S_0^\perp$. The matrix $R=I-P_1$ is the orthogonal projection onto $S_1^\perp$, and $QR=0$. In particular, $$ \operatorname{rank}(Q)=r_1-r_0,\qquad \operatorname{rank}(R)=n-r_1. $$

Proof. Since $S_0\subseteq S_1$, projecting first onto $S_1$ and then onto $S_0$ has the same effect as projecting directly onto $S_0$. Thus $P_1P_0=P_0$. Taking transposes gives $P_0P_1=P_0$, because both projection matrices are symmetric. Consequently $Q$ is symmetric, and $$ Q^2=(P_1-P_0)^2=P_1-P_1P_0-P_0P_1+P_0=P_1-P_0=Q. $$ A symmetric idempotent matrix is an orthogonal projection.

For any $x\in\mathbb{R}^n$, $Qx=P_1x-P_0x$ belongs to $S_1$. If $z\in S_0$, then $$ z'Qx=z'P_1x-z'P_0x=z'x-z'x=0, $$ so $Qx$ is also orthogonal to $S_0$. Conversely, if $x\in S_1\cap S_0^\perp$, then $P_1x=x$ and $P_0x=0$, hence $Qx=x$. This identifies the range of $Q$ as $S_1\cap S_0^\perp$. The larger space decomposes as the orthogonal direct sum $S_1=S_0\oplus(S_1\cap S_0^\perp)$, so the latter space has dimension $r_1-r_0$. The range of $R=I-P_1$ is $S_1^\perp$, of dimension $n-r_1$. Since the range of $Q$ lies in $S_1$ and the range of $R$ lies in $S_1^\perp$, $QR=0$. $\square$

The theorem yields both the matrix decomposition and its orthogonality. The fitted response under the full model is $P_1Y$. Its increment over the restricted fitted response is $(P_1-P_0)Y=QY$, while the full-model residual matrix is $(I-P_1)Y=RY$. Since $Q$ and $R$ project onto orthogonal spaces, their projected response matrices have zero cross-product.

Result: For nested spaces, the response variation not explained by the restricted model splits into hypothesis and error parts: $$ Y'(I-P_0)Y=H+E,\qquad ((P_1-P_0)Y)'((I-P_1)Y)=0. $$ Both $H$ and $E$ are positive semidefinite.

Derivation. The projectors satisfy $P_1-P_0+I-P_1=I-P_0$, so premultiplying by $Y'$ and postmultiplying by $Y$ gives $Y'(I-P_0)Y=H+E$. For the cross-product, $Q'=Q$ and $QR=0$, giving $$ (QY)'(RY)=Y'QRY=0. $$ Finally, for every $a\in\mathbb{R}^p$, $a'Ha=(Ya)'Q(Ya)\geq 0$, because an orthogonal projection has nonnegative quadratic forms. The same argument with $R$ proves $a'Ea\geq 0$. $\square$

Worked Examples

Worked Example: One Added Predictor in a Three-Case Model

Take $n=3$ and two responses, with $$ Y=\begin{pmatrix}2&1\\1&0\\0&0\end{pmatrix}. $$ Let the restricted model space be the span of $(1,1,1)'$, and let the full space also include $(1,0,-1)'$. These vectors are orthogonal, so $r_0=1$, $r_1=2$, and the increment and residual spaces each have dimension one.

The unit vector in the added direction is $q=(1,0,-1)'/\sqrt{2}$. Its response scores are $q'Y=(\sqrt{2},1/\sqrt{2})$, so $$ H=(q'Y)'(q'Y)= \begin{pmatrix}\sqrt{2}\\1/\sqrt{2}\end{pmatrix} \begin{pmatrix}\sqrt{2}&1/\sqrt{2}\end{pmatrix} =\begin{pmatrix}2&1\\1&1/2\end{pmatrix}. $$

A unit vector orthogonal to both model directions is $v=(1,-2,1)'/\sqrt{6}$. Its response scores are $v'Y=(0,1/\sqrt{6})$. Therefore $$ E=(v'Y)'(v'Y)= \begin{pmatrix}0\\1/\sqrt{6}\end{pmatrix} \begin{pmatrix}0&1/\sqrt{6}\end{pmatrix} =\begin{pmatrix}0&0\\0&1/6\end{pmatrix}. $$ The projection degrees of freedom are $r_1-r_0=1$ for $H$ and $n-r_1=1$ for $E$. The observed $H$ has rank one, while $E$ has rank one as well; the zero first diagonal entry of $E$ reflects that the first response has no residual component in this example.

Worked Example: Two Added Directions and Their Error Space

Use the following orthonormal vectors in $\mathbb{R}^4$: $$ q_0=\tfrac12(1,1,1,1)',\quad q_1=\tfrac12(1,-1,1,-1)',\quad q_2=\tfrac12(1,1,-1,-1)',\quad q_3=\tfrac12(1,-1,-1,1)'. $$ Let $S_0=\operatorname{span}(q_0)$ and $S_1=\operatorname{span}(q_0,q_1)$. Choose response columns $y_1=2q_0+q_1+2q_2$ and $y_2=-q_0+2q_1+q_3$. The scores in the added direction $q_1$ are $(1,2)$. Thus $$ H=\begin{pmatrix}1\\2\end{pmatrix}\begin{pmatrix}1&2\end{pmatrix} =\begin{pmatrix}1&2\\2&4\end{pmatrix}. $$

The full-model residual space is spanned by $q_2,q_3$. The scores for $q_2$ are $(2,0)$, and those for $q_3$ are $(0,1)$. Adding their outer products gives $$ E=\begin{pmatrix}2\\0\end{pmatrix}\begin{pmatrix}2&0\end{pmatrix} +\begin{pmatrix}0\\1\end{pmatrix}\begin{pmatrix}0&1\end{pmatrix} =\begin{pmatrix}4&0\\0&1\end{pmatrix}. $$ Here $r_0=1$, $r_1=2$, and $n=4$, so the hypothesis and error projection degrees of freedom are $1$ and $2$. The calculation also shows why the error matrix sums contributions from every residual direction, rather than just one.

Worked Example: Redundant Design Columns Do Not Add Degrees of Freedom

For $n=3$, let the restricted design have columns $x_1=(1,0,0)'$ and $2x_1$. Its column space is only $\operatorname{span}(x_1)$, so $r_0=1$, not 2. Let the full design add $x_2=(1,1,0)'$ and $2x_2$. Since $x_1,x_2$ are linearly independent, the full space is $\operatorname{span}(e_1,e_2)$, with $r_1=2$. The added direction is $e_2=(0,1,0)'$, and the residual direction is $e_3=(0,0,1)'$.

For $$ Y=\begin{pmatrix}2&1\\3&-1\\4&2\end{pmatrix}, $$ the added-direction scores are $e_2'Y=(3,-1)$, giving $$ H=\begin{pmatrix}3\\-1\end{pmatrix}\begin{pmatrix}3&-1\end{pmatrix} =\begin{pmatrix}9&-3\\-3&1\end{pmatrix}. $$ The residual scores are $e_3'Y=(4,2)$, giving $$ E=\begin{pmatrix}4\\2\end{pmatrix}\begin{pmatrix}4&2\end{pmatrix} =\begin{pmatrix}16&8\\8&4\end{pmatrix}. $$ The degrees of freedom are $r_1-r_0=1$ and $n-r_1=1$. Counting design columns instead would incorrectly suggest two restricted-model parameters and four full-model parameters; projection ranks correctly account for the duplicate columns.

Interpreting the Degrees of Freedom

The degrees of freedom $r_1-r_0$ and $n-r_1$ are ranks of projections in the observation space. They are determined by the model spaces, before looking at the numerical values of $Y$. By contrast, the matrix ranks of $H$ and $E$ can be smaller: each is also limited by the number of responses $p$, and a particular data matrix may have no variation in some projected directions.

This distinction is useful when setting up multivariate comparisons. It separates the model’s geometric contribution from what the observed responses happen to do within that geometry. No distributional assumption is needed for the projection identities or these rank calculations. Distributional assumptions are needed for inferential reference distributions, which are a separate step.

Common Mistake

A frequent error is to say that the hypothesis degrees of freedom equal the number of columns added to the design matrix. That is correct only when the added columns contribute that many new independent directions beyond the restricted model. If columns are redundant, or if they lie partly in the restricted space, their count overstates the dimension of the enlargement. The correct quantity is always $\operatorname{rank}(P_1-P_0)=r_1-r_0$. Similarly, the error degrees of freedom are $n-r_1$, not $n$ minus the number of columns as written.

Key takeaway: In a nested multivariate comparison, $P_1-P_0$ projects onto the newly available model directions and $I-P_1$ onto the full-model residual directions. Their ranks, $r_1-r_0$ and $n-r_1$, determine the hypothesis and error projection degrees of freedom.

Check Your Understanding

Use the nested-space geometry to answer the following questions.

If $r_0=3$, $r_1=7$, and $n=20$, what are the hypothesis and error projection degrees of freedom?
Why is $P_1-P_0$ an orthogonal projection when $S_0\subseteq S_1$? State the key product identities.
Can the observed matrix $H$ have rank smaller than $r_1-r_0$? Explain why.
A full design matrix has eight columns but rank six. Which quantity determines the full-model error degrees of freedom?
What does $QR=0$, for $Q=P_1-P_0$ and $R=I-P_1$, say about the hypothesis and residual response components?