From an Eigenvector to a Projector
An eigenvector describes a direction on which a matrix acts by scalar multiplication. A projector answers a different question: given a vector \(\mathbf{x}\), what part of it lies in a chosen subspace? When the subspace is an eigenspace of a real symmetric matrix, an orthonormal basis gives a direct formula for this component. The formula also makes the geometric meaning easy to check: the part left over is perpendicular to the eigenspace.
Throughout, let \(\mathbf{A}\in\mathbb{R}^{n\times n}\) be symmetric when a matrix is needed, and let \(\lambda\in\mathbb{R}\) be one of its eigenvalues. Its eigenspace is \(E_{\lambda}=\{\mathbf{v}\in\mathbb{R}^n:\mathbf{A}\mathbf{v}=\lambda\mathbf{v}\}\). It includes the zero vector. By the Spectral Theorem, a real symmetric matrix has real eigenvalues and each eigenspace has an orthonormal basis. We will use that earlier result to choose the vectors needed for the construction.
For a one-dimensional eigenspace, suppose \(\mathbf{q}\) is a unit eigenvector, so \(\mathbf{q}^{\top}\mathbf{q}=1\). The part of \(\mathbf{x}\) in the direction of \(\mathbf{q}\) is \((\mathbf{q}^{\top}\mathbf{x})\mathbf{q}\): the scalar \(\mathbf{q}^{\top}\mathbf{x}\) is its coefficient along that unit direction. In matrix notation, this vector is \(\mathbf{q}\mathbf{q}^{\top}\mathbf{x}\).
Worked Example: Projecting onto an Eigenline
Let \(\mathbf{A}=\begin{bmatrix}2&0\\0&5\end{bmatrix}\). The vector \(\mathbf{q}=\begin{bmatrix}1\\0\end{bmatrix}\) is a unit eigenvector for \(\lambda=2\), since \(\mathbf{A}\mathbf{q}=\begin{bmatrix}2\\0\end{bmatrix}=2\mathbf{q}\). Project \(\mathbf{x}=\begin{bmatrix}3\\-2\end{bmatrix}\) onto this eigenline.
First form the projector and apply it:
The projected vector belongs to the eigenline because \(\begin{bmatrix}3\\0\end{bmatrix}=3\mathbf{q}\). The residual is \(\mathbf{x}-\mathbf{P}\mathbf{x}=\begin{bmatrix}0\\-2\end{bmatrix}\), and its dot product with \(\mathbf{q}\) is \(0\cdot1+(-2)\cdot0=0\). Thus the projection lies in the eigenline and the residual is perpendicular to it, as required.
Projecting onto a Whole Eigenspace
An eigenspace can have dimension greater than one. In that case, put an orthonormal basis into the columns of a matrix \(\mathbf{Q}\). If the basis is \(\mathbf{q}_1,\dots,\mathbf{q}_k\), then \(\mathbf{Q}=\begin{bmatrix}\mathbf{q}_1&\cdots&\mathbf{q}_k\end{bmatrix}\) and orthonormality means \(\mathbf{Q}^{\top}\mathbf{Q}=\mathbf{I}_k\). The vector \(\mathbf{Q}^{\top}\mathbf{x}\) lists the coefficients of the projection along those basis directions. Multiplication by \(\mathbf{Q}\) rebuilds the projected vector in \(\mathbb{R}^n\).
Proof. Write \(\mathbf{p}=\mathbf{Q}\mathbf{Q}^{\top}\mathbf{x}\). Since \(\mathbf{p}\) is a linear combination of the columns of \(\mathbf{Q}\), it belongs to \(E_{\lambda}\). To check the residual, let \(\mathbf{q}_j\) be any column of \(\mathbf{Q}\). The \(j\)th entry of \(\mathbf{Q}^{\top}\mathbf{x}\) is \(\mathbf{q}_j^{\top}\mathbf{x}\), and orthonormality gives \(\mathbf{q}_j^{\top}\mathbf{Q}\) equal to the row vector whose \(j\)th entry is \(1\) and whose other entries are \(0\). Therefore \(\mathbf{q}_j^{\top}(\mathbf{x}-\mathbf{p})=\mathbf{q}_j^{\top}\mathbf{x}-\mathbf{q}_j^{\top}\mathbf{Q}\mathbf{Q}^{\top}\mathbf{x}=\mathbf{q}_j^{\top}\mathbf{x}-\mathbf{q}_j^{\top}\mathbf{x}=0\). Every vector in \(E_{\lambda}\) is a linear combination of these basis vectors, so the residual is orthogonal to every vector in \(E_{\lambda}\). To see uniqueness, suppose \(\mathbf{p}_1,\mathbf{p}_2\in E_{\lambda}\) both have residuals orthogonal to \(E_{\lambda}\). Their difference \(\mathbf{p}_1-\mathbf{p}_2=(\mathbf{x}-\mathbf{p}_2)-(\mathbf{x}-\mathbf{p}_1)\) is orthogonal to \(E_{\lambda}\), and it also belongs to \(E_{\lambda}\). Hence it is orthogonal to itself, so \(\|\mathbf{p}_1-\mathbf{p}_2\|^2=0\) and \(\mathbf{p}_1=\mathbf{p}_2\). Thus \(\mathbf{p}\) is the orthogonal projection. \(\square\)
Worked Example: Projecting onto a Two-Dimensional Eigenspace
Consider the symmetric matrix \(\mathbf{A}=\begin{bmatrix}3&1&0\\1&3&0\\0&0&2\end{bmatrix}\). Find the projection of \(\mathbf{x}=\begin{bmatrix}2\\1\\4\end{bmatrix}\) onto \(E_2\).
Solving \((\mathbf{A}-2\mathbf{I}_3)\mathbf{v}=\mathbf{0}\) gives \(v_1+v_2=0\), with \(v_3\) free. Thus \(E_2=\operatorname{span}\{\begin{bmatrix}1\\-1\\0\end{bmatrix},\begin{bmatrix}0\\0\\1\end{bmatrix}\}\). Normalize the first vector; the second already has length \(1\). An orthonormal basis matrix and its projector are
Applying the projector gives \(\mathbf{Q}\mathbf{Q}^{\top}\mathbf{x} =\begin{bmatrix}1/2&-1/2&0\\-1/2&1/2&0\\0&0&1\end{bmatrix}\begin{bmatrix}2\\1\\4\end{bmatrix} =\begin{bmatrix}1/2\\-1/2\\4\end{bmatrix}\). The result belongs to \(E_2\): it is \(\tfrac12\begin{bmatrix}1\\-1\\0\end{bmatrix}+4\begin{bmatrix}0\\0\\1\end{bmatrix}\). The residual is \(\mathbf{x}-\mathbf{Q}\mathbf{Q}^{\top}\mathbf{x}=\begin{bmatrix}3/2\\3/2\\0\end{bmatrix}\). Its dot products with the two orthonormal basis vectors are \((3/2-3/2)/\sqrt{2}=0\) and \(0\), respectively. The residual is therefore perpendicular to the entire eigenspace.
Why the Projection Is the Closest Vector
The defining conditions have a useful consequence: among all vectors in the eigenspace, the projection is the one closest to \(\mathbf{x}\). This conclusion follows directly from orthogonality rather than requiring a separate method for searching the eigenspace.
Proof. Since \(\mathbf{p},\mathbf{y}\in W\), their difference \(\mathbf{p}-\mathbf{y}\) belongs to \(W\). By the definition of \(\mathbf{p}\), the residual \(\mathbf{x}-\mathbf{p}\) is orthogonal to every vector in \(W\), in particular to \(\mathbf{p}-\mathbf{y}\). Now write \(\mathbf{x}-\mathbf{y}=(\mathbf{x}-\mathbf{p})+(\mathbf{p}-\mathbf{y})\). Expanding its squared norm using the dot product gives \(\|\mathbf{x}-\mathbf{y}\|^2=\|\mathbf{x}-\mathbf{p}\|^2+2(\mathbf{x}-\mathbf{p})^{\top}(\mathbf{p}-\mathbf{y})+\|\mathbf{p}-\mathbf{y}\|^2\). The middle term is zero by orthogonality, proving the identity. Since \(\|\mathbf{p}-\mathbf{y}\|^2\ge0\), the distance from \(\mathbf{x}\) to \(\mathbf{y}\) is at least the distance to \(\mathbf{p}\). Equality requires \(\|\mathbf{p}-\mathbf{y}\|^2=0\), which implies \(\mathbf{y}=\mathbf{p}\). So \(\mathbf{p}\) is the unique minimizer. \(\square\)
Worked Example: A Non-Coordinate Eigenline
Let \(\mathbf{A}=\begin{bmatrix}-\frac15&\frac85\\[2pt]\frac85&\frac{11}{5}\end{bmatrix}\), \(\mathbf{q}=\frac{1}{\sqrt{5}}\begin{bmatrix}1\\2\end{bmatrix}\), and \(\mathbf{x}=\begin{bmatrix}3\\1\end{bmatrix}\). First verify that \(\mathbf{q}\) is a unit eigenvector, then project \(\mathbf{x}\) onto its eigenline.
The length check is \(\mathbf{q}^{\top}\mathbf{q}=(1+4)/5=1\). Multiplication gives \(\mathbf{A}\mathbf{q}=\frac{1}{5\sqrt{5}}\begin{bmatrix}-1+16\\8+22\end{bmatrix} =\frac{1}{5\sqrt{5}}\begin{bmatrix}15\\30\end{bmatrix} =3\mathbf{q}\), so \(\mathbf{q}\) is an eigenvector for \(\lambda=3\). Now calculate the coefficient and projected vector: \(\mathbf{q}^{\top}\mathbf{x}=(3+2)/\sqrt{5}=\sqrt{5}\), and hence \(\mathbf{q}\mathbf{q}^{\top}\mathbf{x}=\sqrt{5}\mathbf{q}=\begin{bmatrix}1\\2\end{bmatrix}\). The residual is \(\mathbf{x}-\mathbf{q}\mathbf{q}^{\top}\mathbf{x}=\begin{bmatrix}2\\-1\end{bmatrix}\); its dot product with \(\mathbf{q}\) is \((2-2)/\sqrt{5}=0\). The projection is in the eigenline, the residual is perpendicular to it, and the proposition shows that this is the closest vector on that line to \(\mathbf{x}\).
Common Mistakes and Practical Checks
Using a nonunit vector in the unit-vector formula. The matrix \(\mathbf{v}\mathbf{v}^{\top}\) gives the stated projector only when \(\mathbf{v}\) has length \(1\). For example, take \(\mathbf{v}=\begin{bmatrix}1\\2\end{bmatrix}\) and \(\mathbf{x}=\begin{bmatrix}3\\1\end{bmatrix}\). The tempting calculation \(\mathbf{v}\mathbf{v}^{\top}\mathbf{x}=\begin{bmatrix}1&2\\2&4\end{bmatrix}\begin{bmatrix}3\\1\end{bmatrix}=\begin{bmatrix}5\\10\end{bmatrix}\) is wrong as a projection: its coefficient along \(\mathbf{v}\) has been scaled incorrectly. The vector \(\mathbf{v}\) has squared length \(5\), so its unit version is \(\mathbf{q}=\mathbf{v}/\sqrt{5}\), and the correct projector is \(\mathbf{q}\mathbf{q}^{\top}=\frac{1}{5}\mathbf{v}\mathbf{v}^{\top}\). It gives \(\frac15\begin{bmatrix}5\\10\end{bmatrix}=\begin{bmatrix}1\\2\end{bmatrix}\), which lies on the line and has residual \(\begin{bmatrix}2\\-1\end{bmatrix}\) perpendicular to \(\mathbf{v}\), since \(2-2=0\).
Using a basis that is not orthonormal in \(QQ^{\top}\). The formula requires orthonormal columns. If a basis of an eigenspace is not orthonormal, first replace it with an orthonormal basis; otherwise \(\mathbf{Q}^{\top}\mathbf{Q}\ne\mathbf{I}\), and the proof that the residual is perpendicular does not apply. For a one-dimensional eigenspace, this is the same normalization issue as above. For a higher-dimensional eigenspace, check each column has length \(1\) and that distinct columns have dot product \(0\).
For any calculation, three checks keep the geometry visible: the projected vector should be a linear combination of the eigenspace basis vectors; its residual should have dot product zero with each basis vector; and substituting the vector into \(\mathbf{A}\mathbf{v}=\lambda\mathbf{v}\) should confirm that the chosen basis vectors really belong to the stated eigenspace. The matrix \(\mathbf{Q}\mathbf{Q}^{\top}\) encodes this projection in standard coordinates, while the vector \(\mathbf{Q}^{\top}\mathbf{x}\) records the coordinates along the orthonormal basis directions.
Check Your Understanding
Use the defining conditions—membership in the eigenspace and an orthogonal residual—to check each answer.
- For \(\mathbf{q}=\frac{1}{\sqrt{2}}\begin{bmatrix}1\\-1\end{bmatrix}\) and \(\mathbf{x}=\begin{bmatrix}4\\2\end{bmatrix}\), calculate \(\mathbf{q}\mathbf{q}^{\top}\mathbf{x}\) and the residual.
- For \(\mathbf{Q}=\begin{bmatrix}1&0\\0&0\\0&1\end{bmatrix}\), calculate \(\mathbf{Q}\mathbf{Q}^{\top}\mathbf{x}\) when \(\mathbf{x}=\begin{bmatrix}2\\-3\\5\end{bmatrix}\). Which coordinate is removed?
- Explain why \(\mathbf{v}\mathbf{v}^{\top}\) is not the orthogonal projector onto \(\operatorname{span}\{\mathbf{v}\}\) when \(\mathbf{v}^{\top}\mathbf{v}\ne1\).
- State the two conditions that identify the orthogonal projection of \(\mathbf{x}\) onto a subspace \(W\), and explain why they determine at most one vector.
- Show using the distance identity why the orthogonal projection is the unique closest vector in its subspace.
