Projection Matrices from Orthonormal Bases

Why Orthonormal Columns Give a Simple Projection

An orthogonal projection replaces a vector by its closest point in a specified subspace. When the subspace is described by an orthonormal basis, finding that closest point does not require solving a general system of normal equations: each coordinate is obtained by taking an inner product with a basis vector. Collecting those coordinates into a matrix gives the compact formula $QQ^T$.

The order of the factors matters. If the subspace lies in $\mathbb{R}^m$ and has dimension $r$, its orthonormal basis vectors are the columns of an $m\times r$ matrix $Q$. The matrix $Q^TQ$ is $r\times r$, while $QQ^T$ is $m\times m$ and acts on vectors in the original space.

Definition: A matrix $Q\in\mathbb{R}^{m\times r}$ has orthonormal columns when $Q^TQ=I_r$. Equivalently, each column has length one and distinct columns are orthogonal. Its column space, denoted $\operatorname{col}(Q)$, is the subspace of all vectors $Qa$, where $a\in\mathbb{R}^r$.

Because $Q^TQ=I_r$, the columns of $Q$ are linearly independent, so they form an orthonormal basis for $\operatorname{col}(Q)$. For a vector $y\in\mathbb{R}^m$, the entries of $Q^Ty$ are precisely its inner products with the columns of $Q$. These are the coordinates needed to build the projection.

Deriving the Formula and Checking the Residual

Let $q_1,\ldots,q_r$ be the columns of $Q$. If $y$ is projected onto their span, the projected vector must be a linear combination $Qa$. The defining geometric condition for an orthogonal projection is that the residual $y-Qa$ be orthogonal to every vector in the subspace. It is enough to check orthogonality to each basis vector, or equivalently to require $Q^T(y-Qa)=0$.

Theorem: Suppose $Q\in\mathbb{R}^{m\times r}$ satisfies $Q^TQ=I_r$. For any $y\in\mathbb{R}^m$, its orthogonal projection onto $\operatorname{col}(Q)$ is $$ \widehat{y}=QQ^Ty. $$ The residual $e=y-QQ^Ty$ is orthogonal to $\operatorname{col}(Q)$, and $QQ^Ty$ is the unique closest vector in that subspace to $y$.

To derive the coefficient vector, impose the residual condition and use $Q^TQ=I_r$:

Q^T(y-Qa)=0 \quad\Longrightarrow\quad Q^Ty-Q^TQa=0 \quad\Longrightarrow\quad a=Q^Ty.

Substituting this coefficient vector gives $Qa=QQ^Ty$. The calculation also proves residual orthogonality directly:

Q^T\bigl(y-QQ^Ty\bigr) =Q^Ty-Q^TQ\,Q^Ty =Q^Ty-I_rQ^Ty =0.

Thus the residual has zero inner product with every column of $Q$, and hence with every linear combination of those columns. To verify that the projected point is the unique closest point, take any other vector $z=Qa$ in the column space and write

y-z =\bigl(y-QQ^Ty\bigr)+\bigl(QQ^Ty-Qa\bigr).

The first term is orthogonal to the column space. The second term belongs to the column space, since it equals $Q(Q^Ty-a)$. By the Pythagorean theorem,

\|y-z\|^2 =\|y-QQ^Ty\|^2+\|Q(Q^Ty-a)\|^2 \geq \|y-QQ^Ty\|^2.

Equality holds only if $Q(Q^Ty-a)=0$. Since $Q^TQ=I_r$, $Q$ has linearly independent columns, so this requires $a=Q^Ty$ and $z=QQ^Ty$. This proves both the projection formula and uniqueness of the closest point. $\square$

Worked Example: Projecting onto a Line

Let $q=(1,2)^T/\sqrt{5}$ and let $Q=q$. Its norm is one because $q^Tq=(1^2+2^2)/5=1$. Project $y=(4,1)^T$ onto the line spanned by $q$.

First calculate the coordinate along the unit vector:

Q^Ty=q^Ty=\frac{1}{\sqrt{5}}(4+2)=\frac{6}{\sqrt{5}}.

The projection is that coordinate multiplied by $q$:

QQ^Ty =\frac{6}{\sqrt{5}}\frac{1}{\sqrt{5}} \begin{pmatrix}1\\2\end{pmatrix} = \begin{pmatrix}6/5\\12/5\end{pmatrix}.

The residual is $e=y-QQ^Ty=(14/5,-7/5)^T$. Check orthogonality to the line:

q^Te =\frac{1}{\sqrt{5}}\left(\frac{14}{5}+2\left(-\frac{7}{5}\right)\right) =0.

The result lies on the line, and its residual is perpendicular to that line, as required. The check also catches arithmetic errors: a proposed point on this line is not the orthogonal projection unless its residual has zero inner product with $q$.

A Plane Example: Computing and Verifying Every Coordinate

When $r>1$, the same calculation works with several coordinates at once. The columns need to be mutually orthogonal as well as individually normalized; otherwise $Q^TQ$ is not the identity and the formula derived above does not apply as written.

Worked Example: Projecting onto a Plane in Three Dimensions

Consider the two vectors $q_1=(1,1,0)^T/\sqrt{2}$ and $q_2=(1,-1,2)^T/\sqrt{6}$. Their squared norms are $2/2=1$ and $6/6=1$, and their inner product is $(1-1+0)/\sqrt{12}=0$. They are therefore orthonormal. Set $Q=[q_1\ q_2]$ and project $y=(1,2,3)^T$ onto their span.

The two coordinates are

Q^Ty = \begin{pmatrix} q_1^Ty\\q_2^Ty \end{pmatrix} = \begin{pmatrix} 3/\sqrt{2}\\5/\sqrt{6} \end{pmatrix},

since $q_1^Ty=(1+2)/\sqrt{2}$ and $q_2^Ty=(1-2+6)/\sqrt{6}$. Multiplying these coordinates by their corresponding basis vectors gives

QQ^Ty =\frac{3}{\sqrt{2}}q_1+\frac{5}{\sqrt{6}}q_2 = \begin{pmatrix}3/2\\3/2\\0\end{pmatrix} + \begin{pmatrix}5/6\\-5/6\\5/3\end{pmatrix} = \begin{pmatrix}7/3\\2/3\\5/3\end{pmatrix}.

Subtract to obtain $e=y-QQ^Ty=(-4/3,4/3,4/3)^T$. Check both basis directions:

q_1^Te=\frac{-4/3+4/3}{\sqrt{2}}=0, \qquad q_2^Te=\frac{-4/3-4/3+8/3}{\sqrt{6}}=0.

Since every vector in the plane is a linear combination of $q_1$ and $q_2$, these two checks establish that the residual is orthogonal to the entire plane, not just to one selected direction.

The Matrix Form and the Choice of Basis

The matrix $P=QQ^T$ applies the projection to any input $y$ by multiplication, $Py=QQ^Ty$. Its dimensions are $m\times m$, even when the subspace has smaller dimension $r$. Two useful checks follow directly from orthonormality: $P^T=(QQ^T)^T=QQ^T=P$, and $P^2=Q(Q^TQ)Q^T=QQ^T=P$. These are the symmetry and idempotence properties of an orthogonal projection matrix.

A subspace has many orthonormal bases. The projection cannot depend on which one is chosen, because it is determined by the subspace itself. The next result verifies this fact algebraically.

Result: If $Q$ and $\widetilde Q$ are orthonormal basis matrices for the same $r$-dimensional subspace of $\mathbb{R}^m$, then $QQ^T=\widetilde Q\widetilde Q^T$.

To prove the result, since both matrices have orthonormal columns and span the same subspace, there is an $r\times r$ matrix $R$ such that $\widetilde Q=QR$. In fact, set $R=Q^T\widetilde Q$: each column of $\widetilde Q$ lies in $\operatorname{col}(Q)$, and the projection formula already established implies $QQ^T\widetilde Q=\widetilde Q$. Thus $QR=QQ^T\widetilde Q=\widetilde Q$. Now

R^TR =\widetilde Q^TQQ^T\widetilde Q =\widetilde Q^T\widetilde Q =I_r.

Because $R$ is square, $R^TR=I_r$ implies $RR^T=I_r$ as well. Therefore

\widetilde Q\widetilde Q^T =QRR^TQ^T =QI_rQ^T =QQ^T.

This proves that changing orthonormal coordinates within the same subspace changes the coordinate vector but not the projected vector or the projection matrix. $\square$

Worked Example: Rotating an Orthonormal Basis Without Changing the Projection

Let $Q=[e_1\ e_2]$ in $\mathbb{R}^3$, so its columns span the $x$-$y$ plane and $QQ^T=\operatorname{diag}(1,1,0)$. For an angle $\theta$, define

R= \begin{pmatrix} \cos\theta&-\sin\theta\\ \sin\theta&\cos\theta \end{pmatrix}.

The columns of $R$ are orthonormal, so $R^TR=RR^T=I_2$. The rotated basis matrix $\widetilde Q=QR$ has orthonormal columns and spans the same plane. Its projection matrix is

\widetilde Q\widetilde Q^T =QRR^TQ^T =QI_2Q^T =QQ^T = \begin{pmatrix} 1&0&0\\ 0&1&0\\ 0&0&0 \end{pmatrix}.

For example, this matrix sends $y=(2,-1,5)^T$ to $(2,-1,0)^T$, regardless of the angle used to rotate the basis within the plane. The third coordinate is removed because the direction perpendicular to the plane contributes only to the residual.

Common Pitfall and Practical Checks

The most common error is to reverse the factors and write $Q^TQ$ as the projection matrix. That product is $r\times r$, so it does not even act on $y\in\mathbb{R}^m$ when $r\ne m$. Under the orthonormal-column condition, $Q^TQ=I_r$; the projection in the original $m$-dimensional space is $QQ^T$.

A second mistake is to assume $QQ^T=I_m$. That holds when the columns of $Q$ form a full orthonormal basis of $\mathbb{R}^m$, but not generally when they span a proper subspace. For a proper subspace, projection leaves the component in the subspace unchanged and removes the perpendicular component. A reliable verification is to check that $QQ^Ty$ belongs to $\operatorname{col}(Q)$ and that $y-QQ^Ty$ is orthogonal to every column of $Q$.

Key takeaway: With orthonormal columns, $Q^Ty$ gives the coordinates of the projection in that basis, and multiplying by $Q$ returns the projected vector. Thus the projection matrix is $QQ^T$, and its residual is orthogonal to the column space.

Check Your Understanding

Use the dimensions and residual-orthogonality condition to justify each answer.

If $Q$ is $6\times 2$ with $Q^TQ=I_2$, what are the dimensions of $Q^TQ$ and $QQ^T$, and which one acts on vectors in $\mathbb{R}^6$?
For a vector $y$, explain why $Q^T(y-QQ^Ty)=0$ establishes that the residual is orthogonal to the whole column space.
If $Q$ is square and $Q^TQ=I$, what is $QQ^T$? Explain why this differs from the usual case of a proper subspace.
Suppose $\widetilde Q=QR$, where $R$ is an orthogonal square matrix. Show that the two basis matrices produce the same projection matrix.
A proposed projection has residual $e$ with $q_1^Te=0$, but $q_2^Te\ne0$, where $q_1,q_2$ are basis columns. What does this tell you about the proposed projection?