Projections Onto Orthogonal Complementary Subspaces

Splitting a Vector into Two Orthogonal Components

A projection onto a subspace keeps the component of a vector that lies in that subspace. The projection onto the subspace’s orthogonal complement keeps the part perpendicular to it. Together, these components recover the original vector: nothing is lost, and no component is counted twice. This gives a useful way to understand both the geometry of orthogonal projections and the matrix identities they satisfy.

Work first in a finite-dimensional real inner product space \(V\), and let \(W\) be a subspace of \(V\). Its orthogonal complement is \(W^\perp=\{\mathbf{z}\in V:\langle\mathbf{z},\mathbf{w}\rangle=0\text{ for every }\mathbf{w}\in W\}\). In \(\mathbb{R}^n\), the inner product is \(\langle\mathbf{x},\mathbf{y}\rangle=\mathbf{x}^{\top}\mathbf{y}\). A vector in \(W\) and a vector in \(W^\perp\) are orthogonal to one another.

These projections are well-defined in finite-dimensional inner product spaces. For example, if \(\mathbf{e}_1,\dots,\mathbf{e}_k\) is an orthonormal basis of \(W\), then \(\operatorname{proj}_W\mathbf{x}=\sum_{i=1}^k\langle\mathbf{x},\mathbf{e}_i\rangle\mathbf{e}_i\). An orthonormal basis of \(W\) exists by applying Gram-Schmidt to a basis of \(W\). If \(W=\{\mathbf{0}\}\), the sum is empty and equals \(\mathbf{0}\). The defining property also makes the projection unique: two choices in \(W\) whose differences from \(\mathbf{x}\) lie in \(W^\perp\) would differ by a vector in both \(W\) and \(W^\perp\), and the only such vector is \(\mathbf{0}\).

The Decomposition Theorem

Proof. Choose an orthonormal basis \(\mathbf{e}_1,\dots,\mathbf{e}_k\) of \(W\), allowing \(k=0\) when \(W=\{\mathbf{0}\}\), and set \(\mathbf{w}=\sum_{i=1}^k\langle\mathbf{x},\mathbf{e}_i\rangle\mathbf{e}_i\) and \(\mathbf{z}=\mathbf{x}-\mathbf{w}\). The vector \(\mathbf{w}\) is in \(W\). For each basis vector \(\mathbf{e}_j\), orthonormality gives \(\langle\mathbf{z},\mathbf{e}_j\rangle=\langle\mathbf{x},\mathbf{e}_j\rangle-\sum_{i=1}^k\langle\mathbf{x},\mathbf{e}_i\rangle\langle\mathbf{e}_i,\mathbf{e}_j\rangle=0\). Every vector of \(W\) is a linear combination of the basis vectors, so \(\mathbf{z}\in W^\perp\). This constructs the required expression, including \(W=\{\mathbf{0}\}\), where \(\mathbf{w}=\mathbf{0}\). For uniqueness, suppose \(\mathbf{x}=\mathbf{w}_1+\mathbf{z}_1=\mathbf{w}_2+\mathbf{z}_2\) with \(\mathbf{w}_i\in W\) and \(\mathbf{z}_i\in W^\perp\). Then \(\mathbf{w}_1-\mathbf{w}_2=\mathbf{z}_2-\mathbf{z}_1\) belongs to both \(W\) and \(W^\perp\). A vector \(\mathbf{u}\) in both satisfies \(\langle\mathbf{u},\mathbf{u}\rangle=0\), so \(\mathbf{u}=\mathbf{0}\). Hence \(\mathbf{w}_1=\mathbf{w}_2\) and \(\mathbf{z}_1=\mathbf{z}_2\). The defining property of each projection now identifies the two components and proves their sum is \(\mathbf{x}\). \(\square\)

The theorem includes the extreme cases. If \(W=V\), then \(W^\perp=\{\mathbf{0}\}\), so the first projection is the identity and the second is zero. If \(W=\{\mathbf{0}\}\), the roles are reversed. For any other subspace, each vector is split uniquely between the two orthogonal parts.

Matrix Identities for Complementary Projections

In \(\mathbb{R}^n\), a linear map is represented by a matrix acting on column vectors. Write \(\mathbf{P}\) for the matrix of \(\operatorname{proj}_W\) and \(\mathbf{Q}\) for the matrix of \(\operatorname{proj}_{W^\perp}\). The decomposition theorem says that their sum acts as the identity on every vector. Orthogonality adds further identities: projecting twice onto the same subspace changes nothing, and projecting onto one subspace then the other gives zero.

Proof. For every \(\mathbf{x}\in\mathbb{R}^n\), the decomposition theorem gives \(\mathbf{P}\mathbf{x}+\mathbf{Q}\mathbf{x}=\mathbf{x}\), so \((\mathbf{P}+\mathbf{Q})\mathbf{x}=\mathbf{I}_n\mathbf{x}\) for every \(\mathbf{x}\), which implies \(\mathbf{P}+\mathbf{Q}=\mathbf{I}_n\). Since \(\mathbf{P}\mathbf{x}\in W\), projecting it onto \(W\) again leaves it unchanged; thus \(\mathbf{P}^2\mathbf{x}=\mathbf{P}\mathbf{x}\) for every \(\mathbf{x}\), and \(\mathbf{P}^2=\mathbf{P}\). The same argument gives \(\mathbf{Q}^2=\mathbf{Q}\). For symmetry, choose an orthonormal basis \(\mathbf{e}_1,\dots,\mathbf{e}_k\) of \(W\). The projection formula gives \(\mathbf{P}=\sum_{i=1}^k\mathbf{e}_i\mathbf{e}_i^{\top}\), a sum of symmetric matrices, so \(\mathbf{P}^{\top}=\mathbf{P}\). An orthonormal basis of \(W^\perp\) gives \(\mathbf{Q}^{\top}=\mathbf{Q}\) by the same formula. Finally, \(\mathbf{Q}\mathbf{x}\in W^\perp\), and the projection of that vector onto \(W\) is zero, so \(\mathbf{P}\mathbf{Q}\mathbf{x}=\mathbf{0}\). Also \(\mathbf{P}\mathbf{x}\in W\), whose projection onto \(W^\perp\) is zero, so \(\mathbf{Q}\mathbf{P}\mathbf{x}=\mathbf{0}\). These hold for every \(\mathbf{x}\), proving both product identities. \(\square\)

Conversely, a symmetric idempotent matrix determines an orthogonal projection. This gives a useful test when a projection matrix is supplied algebraically, without first being described geometrically.

Proof. For any \(\mathbf{x}\), \(\mathbf{P}\mathbf{x}\in\operatorname{Col}\mathbf{P}\), and \(\mathbf{x}-\mathbf{P}\mathbf{x}\) is orthogonal to every vector \(\mathbf{P}\mathbf{y}\in\operatorname{Col}\mathbf{P}\), because \(\langle\mathbf{x}-\mathbf{P}\mathbf{x},\mathbf{P}\mathbf{y}\rangle=((\mathbf{I}_n-\mathbf{P})\mathbf{x})^{\top}\mathbf{P}\mathbf{y}=\mathbf{x}^{\top}(\mathbf{P}-\mathbf{P}^2)\mathbf{y}=0\). Here symmetry gives \((\mathbf{I}_n-\mathbf{P})^{\top}=\mathbf{I}_n-\mathbf{P}\), and idempotence gives \(\mathbf{P}-\mathbf{P}^2=\mathbf{O}\). Thus \(\mathbf{P}\mathbf{x}\) has precisely the defining property of the orthogonal projection onto \(\operatorname{Col}\mathbf{P}\). The matrix \(\mathbf{I}_n-\mathbf{P}\) is also symmetric and idempotent, since \((\mathbf{I}_n-\mathbf{P})^2=\mathbf{I}_n-2\mathbf{P}+\mathbf{P}^2=\mathbf{I}_n-\mathbf{P}\). Its column space is \((\operatorname{Col}\mathbf{P})^\perp\): if \(\mathbf{z}=(\mathbf{I}_n-\mathbf{P})\mathbf{x}\), then \(\mathbf{P}\mathbf{z}=\mathbf{P}\mathbf{x}-\mathbf{P}^2\mathbf{x}=\mathbf{0}\), and for every \(\mathbf{y}\), \(\langle\mathbf{z},\mathbf{P}\mathbf{y}\rangle=0\) by the calculation above. Conversely, if \(\mathbf{z}\) is orthogonal to \(\operatorname{Col}\mathbf{P}\), then \(\mathbf{P}\mathbf{z}\in\operatorname{Col}\mathbf{P}\) and \(\langle\mathbf{z},\mathbf{P}\mathbf{z}\rangle=0\); symmetry and idempotence give \(\|\mathbf{P}\mathbf{z}\|^2=\mathbf{z}^{\top}\mathbf{P}^{\top}\mathbf{P}\mathbf{z}=\mathbf{z}^{\top}\mathbf{P}\mathbf{z}=0\), so \(\mathbf{P}\mathbf{z}=\mathbf{0}\) and \((\mathbf{I}_n-\mathbf{P})\mathbf{z}=\mathbf{z}\). This proves the asserted column space and projection. \(\square\)

Common Mistake: The Complement Is Not the Transpose

For an orthogonal projection, the transpose is the matrix itself: \(\mathbf{P}^{\top}=\mathbf{P}\). A tempting but incorrect conclusion is that the projection onto the orthogonal complement is therefore \(\mathbf{P}^{\top}\). In the last example, this would set \(\mathbf{Q}=\mathbf{P}^{\top}=\mathbf{P}\). Then \(\mathbf{P}+\mathbf{Q}=2\mathbf{P}=\frac25\begin{bmatrix}1&2\\2&4\end{bmatrix}\), which is not \(\mathbf{I}_2\). Applied to \((5,0)^{\top}\), this incorrect choice gives \(\mathbf{P}\mathbf{x}+\mathbf{Q}\mathbf{x}=(2,4)^{\top}\), not \((5,0)^{\top}\). The correction is \(\mathbf{Q}=\mathbf{I}_2-\mathbf{P}\). The transpose identity describes symmetry of each orthogonal projection matrix; subtraction from the identity constructs the complementary projection.

The identities are useful checks in computations. If two proposed matrices are complementary orthogonal projections, each should be symmetric and idempotent, their sum should be the identity, and their products in either order should be zero. A failed check points to a mistake in a formula or in the proposed subspaces. Conversely, the symmetric-idempotent criterion lets a matrix identify its projected subspace as its column space.