From Orthonormal Columns to a Matrix
A subspace may be described by a list of vectors that spans it, but computing a projection is especially direct when that list is orthonormal. The inner products with the columns give the coordinates of the projection, and a matrix product packages those calculations into one operation. This tutorial develops that construction and shows how to check both the projected vector and its residual.
Work over \(\mathbb{R}\). Let \(Q\in\mathbb{R}^{m\times n}\), with columns \(\mathbf{q}_1,\dots,\mathbf{q}_n\in\mathbb{R}^m\). The columns are orthonormal when \(\mathbf{q}_i^{\top}\mathbf{q}_j=1\) if \(i=j\) and \(0\) if \(i\ne j\). Equivalently, their pairwise dot products are recorded by \(Q^{\top}Q=\mathbf{I}_n\). Their span, also called the column space of \(Q\), is \(\operatorname{Col}Q=\operatorname{span}\{\mathbf{q}_1,\dots,\mathbf{q}_n\}\).
When the columns of \(Q\) are orthonormal, the coordinates of the projection onto \(\operatorname{Col}Q\) are \(Q^{\top}\mathbf{x}\): its \(j\)-th entry is \(\mathbf{q}_j^{\top}\mathbf{x}\). Multiplying these coordinates by \(Q\) converts them back into a vector in \(\mathbb{R}^m\). Thus the candidate projection is \(Q(Q^{\top}\mathbf{x})\), or \((QQ^{\top})\mathbf{x}\). The order matters: \(Q^{\top}Q\) is an \(n\times n\) matrix, whereas the projection acts on vectors in \(\mathbb{R}^m\), so its matrix must be \(m\times m\).
Proof. Set \(\mathbf{y}=QQ^{\top}\mathbf{x}\). Since \(\mathbf{y}\) is a linear combination of the columns of \(Q\), it belongs to \(\operatorname{Col}Q\). The residual is \(\mathbf{r}=\mathbf{x}-\mathbf{y}\). Using \(Q^{\top}Q=\mathbf{I}_n\), calculate $$ Q^{\top}\mathbf{r} =Q^{\top}\mathbf{x}-Q^{\top}QQ^{\top}\mathbf{x} =Q^{\top}\mathbf{x}-\mathbf{I}_nQ^{\top}\mathbf{x} =\mathbf{0}. $$ The entries of \(Q^{\top}\mathbf{r}\) are \(\mathbf{q}_j^{\top}\mathbf{r}\), so \(\mathbf{r}\) is orthogonal to every column. It is therefore orthogonal to every linear combination of those columns, which means \(\mathbf{r}\in(\operatorname{Col}Q)^\perp\). This proves that \(\mathbf{y}\) satisfies the definition of the projection. For uniqueness, suppose \(\mathbf{y}_1,\mathbf{y}_2\in\operatorname{Col}Q\) both satisfy the definition. Then \(\mathbf{y}_1-\mathbf{y}_2\in\operatorname{Col}Q\), and also \(\mathbf{y}_1-\mathbf{y}_2=(\mathbf{x}-\mathbf{y}_2)-(\mathbf{x}-\mathbf{y}_1)\in(\operatorname{Col}Q)^\perp\). A vector in a subspace and its orthogonal complement is orthogonal to itself, so \((\mathbf{y}_1-\mathbf{y}_2)^{\top}(\mathbf{y}_1-\mathbf{y}_2)=0\). Its squared norm is zero, which forces \(\mathbf{y}_1=\mathbf{y}_2\). Hence the formula gives the unique projection. \(\square\)
Properties of the Projection Matrix
Write \(\mathbf{P}=QQ^{\top}\). Its dimensions are \(m\times m\), regardless of the number \(n\) of orthonormal columns. The matrix gives the projection of every vector in \(\mathbb{R}^m\) onto the same subspace. It also has two useful algebraic properties: it is symmetric, and applying it twice has the same effect as applying it once.
Proof. Transposing the product gives \(\mathbf{P}^{\top}=(QQ^{\top})^{\top}=QQ^{\top}=\mathbf{P}\). For idempotence, use the orthonormal-column condition: $$ \mathbf{P}^2=(QQ^{\top})(QQ^{\top}) =Q(Q^{\top}Q)Q^{\top} =Q\mathbf{I}_nQ^{\top} =QQ^{\top} =\mathbf{P}. $$ Now take \(\mathbf{y}\in\operatorname{Col}Q\). There is a vector \(\mathbf{c}\in\mathbb{R}^n\) with \(\mathbf{y}=Q\mathbf{c}\), so \(\mathbf{P}\mathbf{y}=QQ^{\top}Q\mathbf{c}=Q\mathbf{I}_n\mathbf{c}=\mathbf{y}\). Finally, if \(\mathbf{z}\in(\operatorname{Col}Q)^\perp\), it is orthogonal to each column, giving \(Q^{\top}\mathbf{z}=\mathbf{0}\). Therefore \(\mathbf{P}\mathbf{z}=Q(Q^{\top}\mathbf{z})=\mathbf{0}\). \(\square\)
These properties offer useful checks, but they are consequences of the orthonormal-column construction. In particular, the calculation \(\mathbf{P}^2=\mathbf{P}\) confirms that a vector already projected into the subspace will not move under a second application of the matrix.
Computing the Matrix and Its Action
Worked Example: Projection onto a Two-Dimensional Subspace of R³
Let the columns of \(Q\) be \(\mathbf{q}_1=(1,1,0)^{\top}/\sqrt{2}\) and \(\mathbf{q}_2=(1,-1,2)^{\top}/\sqrt{6}\). Construct \(\mathbf{P}\), then project \(\mathbf{x}=(1,0,2)^{\top}\) and calculate the residual.
First check the columns. Their squared lengths are \((1^2+1^2+0^2)/2=1\) and \((1^2+(-1)^2+2^2)/6=1\). Their dot product is \((1-1+0)/\sqrt{12}=0\), so they are orthonormal. Thus $$ Q^{\top}Q=\mathbf{I}_2,\qquad \mathbf{P}=QQ^{\top} =\begin{bmatrix} \frac{2}{3}&\frac{1}{3}&\frac{1}{3}\\ \frac{1}{3}&\frac{2}{3}&-\frac{1}{3}\\ \frac{1}{3}&-\frac{1}{3}&\frac{2}{3} \end{bmatrix}. $$
For example, the \((1,1)\) entry is \(1/2+1/6=2/3\), and the \((1,2)\) entry is \(1/2-1/6=1/3\); the other entries follow by multiplying each column by its transpose. Apply the matrix: $$ \mathbf{y}=\mathbf{P}\mathbf{x} =\begin{bmatrix} \frac{2}{3}(1)+\frac{1}{3}(0)+\frac{1}{3}(2)\\ \frac{1}{3}(1)+\frac{2}{3}(0)-\frac{1}{3}(2)\\ \frac{1}{3}(1)-\frac{1}{3}(0)+\frac{2}{3}(2) \end{bmatrix} =\begin{bmatrix}\frac{4}{3}\\-\frac{1}{3}\\\frac{5}{3}\end{bmatrix}. $$
The residual is $$ \mathbf{r}=\mathbf{x}-\mathbf{y} =\begin{bmatrix}-\frac{1}{3}\\\frac{1}{3}\\\frac{1}{3}\end{bmatrix}. $$
Check: \(Q^{\top}\mathbf{r}=\bigl((-\frac13+\frac13)/\sqrt2,\;(-\frac13-\frac13+\frac23)/\sqrt6\bigr)^{\top}=\mathbf{0}\), so the residual is orthogonal to both columns. Also \(\mathbf{y}=Q(Q^{\top}\mathbf{x})\): the coordinates are \(Q^{\top}\mathbf{x}=(1/\sqrt2,5/\sqrt6)^{\top}\), and multiplying by \(Q\) gives \((4/3,-1/3,5/3)^{\top}\), as required.
The same method can be used without first writing down every entry of \(\mathbf{P}\). Compute the coordinates \(Q^{\top}\mathbf{x}\), then multiply by \(Q\). This is often efficient when \(Q\) has many rows but only a few columns.
Worked Example: Projection from Orthonormal Columns in R⁴
Take $$ Q=\begin{bmatrix} \frac12&\frac12\\ \frac12&\frac12\\ \frac12&-\frac12\\ \frac12&-\frac12 \end{bmatrix}, \qquad \mathbf{x}=\begin{bmatrix}1\\2\\0\\3\end{bmatrix}. $$ The columns have squared lengths \(4(1/4)=1\) and dot product \(1/4+1/4-1/4-1/4=0\), so \(Q^{\top}Q=\mathbf{I}_2\).
First calculate the coordinates: $$ Q^{\top}\mathbf{x} =\begin{bmatrix} (1+2+0+3)/2\\ (1+2-0-3)/2 \end{bmatrix} =\begin{bmatrix}3\\0\end{bmatrix}. $$ Thus $$ \mathbf{y}=Q\begin{bmatrix}3\\0\end{bmatrix} =\begin{bmatrix}\frac32\\\frac32\\\frac32\\\frac32\end{bmatrix}, \qquad \mathbf{r}=\mathbf{x}-\mathbf{y} =\begin{bmatrix}-\frac12\\\frac12\\-\frac32\\\frac32\end{bmatrix}. $$
Here is the same calculation through the projection matrix: $$ \mathbf{P}=QQ^{\top} =\begin{bmatrix} \frac12&\frac12&0&0\\ \frac12&\frac12&0&0\\ 0&0&\frac12&\frac12\\ 0&0&\frac12&\frac12 \end{bmatrix}. $$
Check: Multiplying \(\mathbf{P}\mathbf{x}\) gives \(((1+2)/2,(1+2)/2,(0+3)/2,(0+3)/2)^{\top}=(3/2,3/2,3/2,3/2)^{\top}\). The residual’s dot products with the first and second columns are \((-\frac12+\frac12-\frac32+\frac32)/2=0\) and \((-\frac12+\frac12+\frac32-\frac32)/2=0\). Thus the calculated residual is orthogonal to \(\operatorname{Col}Q\).
Residuals and Matrix Checks
For a particular vector, there are two convenient ways to verify a result. First, check the defining decomposition: \(\mathbf{x}=\mathbf{y}+\mathbf{r}\), with \(\mathbf{y}\in\operatorname{Col}Q\) and \(Q^{\top}\mathbf{r}=\mathbf{0}\). Second, check the matrix properties \(\mathbf{P}^{\top}=\mathbf{P}\) and \(\mathbf{P}^2=\mathbf{P}\). The first pair tests the actual projected vector and residual; the second tests the constructed matrix. For hand calculations, checking the residual is often the more direct way to catch an arithmetic error in a matrix-vector product.
The residual formula follows directly from subtracting the projection: $$ \mathbf{r}=\mathbf{x}-QQ^{\top}\mathbf{x} =(\mathbf{I}_m-QQ^{\top})\mathbf{x}. $$ The matrix \(\mathbf{I}_m-\mathbf{P}\) therefore returns the residual rather than the projection. Its output is orthogonal to \(\operatorname{Col}Q\), because \(Q^{\top}(\mathbf{I}_m-QQ^{\top})\mathbf{x}=\mathbf{0}\). This statement concerns the residual produced by this orthonormal-column construction.
Worked Example: Build and Check a Projection Matrix
Let $$ Q=\begin{bmatrix} \frac{1}{\sqrt2}&\frac{1}{\sqrt6}\\ 0&-\frac{2}{\sqrt6}\\ \frac{1}{\sqrt2}&-\frac{1}{\sqrt6} \end{bmatrix}, \qquad \mathbf{x}=\begin{bmatrix}3\\1\\0\end{bmatrix}. $$ The first column has squared length \(1/2+1/2=1\); the second has squared length \((1+4+1)/6=1\). Their dot product is \(1/\sqrt{12}+0-1/\sqrt{12}=0\), so the columns are orthonormal.
Forming the outer products of the columns gives $$ \mathbf{P}=QQ^{\top} =\begin{bmatrix} \frac23&-\frac13&\frac13\\ -\frac13&\frac23&\frac13\\ \frac13&\frac13&\frac23 \end{bmatrix}. $$
Multiplying by \(\mathbf{x}\) gives $$ \mathbf{y}=\mathbf{P}\mathbf{x} =\begin{bmatrix} 2-\frac13\\ -1+\frac23\\ 1+\frac13 \end{bmatrix} =\begin{bmatrix}\frac53\\-\frac13\\\frac43\end{bmatrix}, \qquad \mathbf{r}=\mathbf{x}-\mathbf{y} =\begin{bmatrix}\frac43\\\frac43\\-\frac43\end{bmatrix}. $$
Check: The first column of \(Q\) has dot product \((4/3-4/3)/\sqrt2=0\) with \(\mathbf{r}\); the second has dot product \((4/3-8/3+4/3)/\sqrt6=0\). Also \(Q^{\top}\mathbf{x}=(3/\sqrt2,1/\sqrt6)^{\top}\); multiplying this coordinate vector by \(Q\) gives \((5/3,-1/3,4/3)^{\top}\), confirming the projection calculated with \(\mathbf{P}\).
Common Mistake: Reversing the Matrix Product
A frequent error is to use \(Q^{\top}Q\) as the projection matrix in \(\mathbb{R}^m\). That product is \(n\times n\), while a matrix that acts on \(\mathbf{x}\in\mathbb{R}^m\) must have \(m\) rows and \(m\) columns. In the first example, \(Q^{\top}Q=\mathbf{I}_2\). Treating it as the projection matrix would both give the wrong dimensions for a transformation of \(\mathbb{R}^3\) and incorrectly suggest that every vector in \(\mathbb{R}^3\) is unchanged.
The correction is to preserve the order that first extracts coordinates and then rebuilds a vector: \(Q^{\top}\mathbf{x}\) lies in \(\mathbb{R}^n\), and \(Q(Q^{\top}\mathbf{x})\) lies in \(\mathbb{R}^m\). Consequently, the projection matrix is \(QQ^{\top}\), not \(Q^{\top}Q\). Nor should \(QQ^{\top}\) generally be replaced by \(\mathbf{I}_m\): in the first example, the projection matrix has entries such as \(P_{13}=1/3\) and is not \(\mathbf{I}_3\).
Another useful check is to verify orthonormality before using the formula. If the columns are not orthonormal, then \(Q^{\top}Q\ne\mathbf{I}_n\), and the proof of the formula above no longer applies. The construction here specifically uses orthonormal columns; other ways to form a projection matrix are a separate case.
Check Your Understanding
For each calculation, check the stated column condition and use the residual to verify the result.
- For \(Q=\begin{bmatrix}1/\sqrt2&0\\0&1\\1/\sqrt2&0\end{bmatrix}\), verify that its columns are orthonormal and find \(QQ^{\top}\).
- Using the \(Q\) in question 1, project \(\mathbf{x}=(2,3,0)^{\top}\). Find the residual and check its dot product with each column.
- Explain why \(Q^{\top}Q\) has size \(n\times n\) while \(QQ^{\top}\) has size \(m\times m\) when \(Q\in\mathbb{R}^{m\times n}\).
- Show directly from \(Q^{\top}Q=\mathbf{I}_n\) that \((QQ^{\top})^2=QQ^{\top}\).
- For the \(4\times2\) matrix in the second worked example, calculate \((\mathbf{I}_4-QQ^{\top})\mathbf{x}\) and compare it with the residual found there.
