Orthogonal Projections and Projection Matrices

Geometric Meaning of Orthogonal Projection

A projection onto a subspace keeps the component of a vector that lies in the subspace and removes the component perpendicular to it. The defining feature of an orthogonal projection is that the discarded residual is perpendicular to every vector in the subspace. This geometric condition determines a unique answer, and it leads to a matrix with two important properties: symmetry and idempotence.

Work in \(\mathbb R^m\) with the usual inner product \(x^T y\). Let \(S\) be a linear subspace, and let \(S^\perp\) denote its orthogonal complement: the vectors whose inner product with every element of \(S\) is zero. The finite-dimensional orthogonal decomposition theorem gives \(\mathbb R^m=S\oplus S^\perp\). Thus every \(x\in\mathbb R^m\) has a unique representation as a sum of a vector in \(S\) and a vector in \(S^\perp\).

Definition: The orthogonal projection of \(x\) onto \(S\), written \(\operatorname{proj}_S(x)\), is the unique \(u\in S\) for which \(x-u\in S^\perp\). The function \(P_S:\mathbb R^m\to\mathbb R^m\) defined by \(P_Sx=\operatorname{proj}_S(x)\) is the orthogonal projection map onto \(S\). Its matrix in the standard coordinates is called the orthogonal projection matrix onto \(S\).

In the decomposition \(x=u+v\), with \(u\in S\) and \(v\in S^\perp\), the projection is \(u\), while the residual \(x-P_Sx\) is \(v\). The projection is therefore characterized not just by being in \(S\), but by leaving a residual perpendicular to \(S\).

Worked Example: Projecting onto a Line in \(\mathbb R^2\)

Let \(S=\operatorname{span}\{a\}\), where \(a=(1,2)^T\), and project \(x=(3,1)^T\) onto \(S\). A vector in \(S\) has the form \(ca\). For \(x-ca\) to be perpendicular to \(a\), its inner product with \(a\) must be zero:

a^T(x-ca)=0 \quad\Longrightarrow\quad c=\frac{a^Tx}{a^Ta} =\frac{3+2}{1+4}=1.

Therefore \(P_Sx=a=(1,2)^T\), and the residual is \(x-P_Sx=(2,-1)^T\). Indeed, \(a^T(2,-1)=2-2=0\). For a general vector \(z\), the same calculation gives \(P_Sz=a(a^Tz)/(a^Ta)\). Hence the matrix of the map is

P_S=\frac{aa^T}{a^Ta} =\frac{1}{5} \begin{pmatrix} 1&2\\ 2&4 \end{pmatrix}.

The output is always a multiple of \(a\), so the range is \(S\). A vector is sent to zero exactly when \(a^Tz=0\), so the null space is \(S^\perp\).

Why the Projection Map Is Linear

The matrix properties follow from the geometry of the map. First, the map is linear. To see this, write \(x=u+v\) and \(y=s+t\), where \(u,s\in S\) and \(v,t\in S^\perp\). For scalars \(\alpha,\beta\),

\alpha x+\beta y=(\alpha u+\beta s)+(\alpha v+\beta t).

Because \(S\) and \(S^\perp\) are subspaces, the first term belongs to \(S\) and the second to \(S^\perp\). By uniqueness of the orthogonal decomposition, the projection of \(\alpha x+\beta y\) is \(\alpha u+\beta s\). Thus \(P_S(\alpha x+\beta y)=\alpha P_Sx+\beta P_Sy\), proving linearity. Every linear map from \(\mathbb R^m\) to itself has a matrix representation; denote the matrix of \(P_S\) by \(P\).

Theorem: If \(P\) is the matrix of the orthogonal projection onto a subspace \(S\subseteq\mathbb R^m\), then \(P^T=P\), \(P^2=P\), \(\mathcal R(P)=S\), and \(\mathcal N(P)=S^\perp\).

Proof of the Matrix Properties

We establish each claim directly from the defining decomposition. For every \(x\), \(Px=P_Sx\) belongs to \(S\). Also, \(x-Px\) belongs to \(S^\perp\).

Idempotence. Since \(Px\in S\), projecting \(Px\) onto \(S\) leaves it unchanged. Therefore \(P(Px)=Px\) for every \(x\), which is exactly \(P^2=P\).

Range. Since every output \(Px\) belongs to \(S\), \(\mathcal R(P)\subseteq S\). Conversely, if \(s\in S\), its decomposition is \(s=s+0\), so \(Ps=s\). Thus every element of \(S\) is an output of \(P\), giving \(S\subseteq\mathcal R(P)\). Hence \(\mathcal R(P)=S\).

Null space. If \(Px=0\), then \(x-Px=x\in S^\perp\), so \(x\in S^\perp\). Conversely, if \(x\in S^\perp\), its decomposition is \(x=0+x\), and therefore \(Px=0\). It follows that \(\mathcal N(P)=S^\perp\).

Symmetry. Write \(x=Px+(x-Px)\) and \(y=Py+(y-Py)\). The projection components lie in \(S\); the residuals lie in \(S^\perp\). Consequently, the cross inner products between a projection component and a residual are zero. In particular,

$$ x^TPy=(Px)^T(Py)=(Px)^Ty. $$

The first equality follows because \(x-Px\) is perpendicular to \(Py\); the second follows because \(y-Py\) is perpendicular to \(Px\). Since these equalities hold for every \(x,y\), they imply \(P^T=P\): indeed, \(x^TPy=x^TP^Ty\) for all \(x,y\), so all entries of \(P-P^T\) are zero. This proves the theorem. \(\square\)

The range and null-space statements are complementary: the part retained by the projection belongs to \(S\), and the part removed lies in \(S^\perp\). Idempotence expresses the fact that once a vector has been projected into \(S\), projecting it again makes no change.

Worked Example: Projecting onto a Plane Defined by a Constraint

Let \(S=\{x\in\mathbb R^3:x_1+x_2+x_3=0\}\). The normal vector \(n=(1,1,1)^T\) is perpendicular to every vector in \(S\). For any \(x\), the residual from its projection onto this plane must be a multiple \(cn\). Requiring \(x-cn\in S\) gives

0=n^T(x-cn)=n^Tx-3c, \qquad c=\frac{n^Tx}{3}.

Thus the projection subtracts the component along \(n\):

P_Sx=x-\frac{n^Tx}{3}n, \qquad P_S=I-\frac{nn^T}{3} =\frac{1}{3} \begin{pmatrix} 2&-1&-1\\ -1&2&-1\\ -1&-1&2 \end{pmatrix}.

For \(x=(2,-1,3)^T\), \(n^Tx=4\), so

P_Sx= \begin{pmatrix}2\\-1\\3\end{pmatrix} -\frac{4}{3} \begin{pmatrix}1\\1\\1\end{pmatrix} = \begin{pmatrix}2/3\\-7/3\\5/3\end{pmatrix}.

The coordinates of the result sum to \(2/3-7/3+5/3=0\), as required for membership in \(S\). The residual is \((4/3,4/3,4/3)^T\), a multiple of the normal vector, so it is perpendicular to the plane.

Worked Example: A Projection with Two Independent Constraints

Consider the subspace \(S=\{(a,-a,b,0)^T:a,b\in\mathbb R\}\subseteq\mathbb R^4\). To project \(x=(4,2,-1,5)^T\), match its third coordinate and set its fourth coordinate to zero. The first two coordinates of a point in \(S\) must be opposites. The residual in those coordinates must be perpendicular to \((1,-1)^T\), so their projected value is determined by

\begin{pmatrix}1\\-1\end{pmatrix}^T \left[ \begin{pmatrix}4\\2\end{pmatrix} -c\begin{pmatrix}1\\-1\end{pmatrix} \right]=0 \quad\Longrightarrow\quad c=\frac{4-2}{2}=1.

Therefore \(P_Sx=(1,-1,-1,0)^T\), with residual \((3,3,0,5)^T\). For any \((a,-a,b,0)^T\in S\), the residual's inner product with that vector is \(3a-3a=0\), confirming orthogonality. The full projection matrix is

P_S= \begin{pmatrix} 1/2&-1/2&0&0\\ -1/2&1/2&0&0\\ 0&0&1&0\\ 0&0&0&0 \end{pmatrix}.

Its output has the form \((a,-a,b,0)^T\), so its range is \(S\). Its null space consists of vectors \((t,t,0,s)^T\), which are exactly the vectors perpendicular to every \((a,-a,b,0)^T\in S\). The matrix is symmetric, and applying it twice leaves each of its output coordinates unchanged, so \(P_S^2=P_S\).

A Converse and a Common Pitfall

The two matrix identities also characterize orthogonal projection matrices. This converse is useful when a matrix is given algebraically rather than through a subspace.

Theorem: If a real square matrix \(P\) satisfies \(P^T=P\) and \(P^2=P\), then \(P\) is the matrix of the orthogonal projection onto \(\mathcal R(P)\).

Proof. For any \(x\), \(Px\in\mathcal R(P)\). Also, \(P(x-Px)=Px-P^2x=0\), so \(x-Px\in\mathcal N(P)\). These give a decomposition of \(x\) into a range component and a null-space component. To show they are perpendicular, take \(u=Pa\in\mathcal R(P)\) and \(v\in\mathcal N(P)\). Symmetry gives

$$ u^Tv=(Pa)^Tv=a^TP^Tv=a^TPv=0. $$

Thus \(x-Px\) is perpendicular to \(\mathcal R(P)\). By the definition of orthogonal projection, \(P\) projects onto its range. \(\square\)

A common mistake is to infer that idempotence alone guarantees an orthogonal projection. It guarantees that applying the map twice has the same effect as applying it once, but does not ensure that the residual is perpendicular to the range. For example,

A=\begin{pmatrix}1&1\\0&0\end{pmatrix}, \qquad A^2=A, \qquad A^T\ne A.

Here the range is spanned by \((1,0)^T\), while \((-1,1)^T\) is in the null space; their inner product is \(-1\), not zero. The correction is to check both properties: an orthogonal projection matrix must be symmetric and idempotent.

Why These Properties Matter

Projection matrices provide a compact algebraic description of a geometric decomposition. The range tells which subspace is retained; the null space tells which directions are removed. Symmetry records the orthogonality of those directions, while idempotence records that the operation is already complete after one application. These facts are useful when checking a proposed projection matrix or reasoning about the parts of a vector it preserves and discards.

Check Your Understanding

Use the definitions and results above to answer the following questions.

For an orthogonal projection matrix \(P\) onto \(S\), what do \(\mathcal R(P)\) and \(\mathcal N(P)\) equal?
Explain geometrically why \(P^2=P\).
For \(S=\operatorname{span}\{(1,1)^T\}\), project \((4,0)^T\) onto \(S\), and verify that the residual is perpendicular to \(S\).
What two matrix conditions characterize an orthogonal projection in the real, finite-dimensional setting?
Why does the identity \(A^2=A\), by itself, not establish that \(A\) represents an orthogonal projection?