Three Different Ideas That Are Easy to Confuse
An orthogonal projection, a reflection, and an idempotent map are related, but they are not interchangeable. A projection keeps the component of a vector in a subspace and removes the perpendicular component. A reflection reverses the perpendicular component instead. Idempotence says that applying a map twice has the same effect as applying it once; by itself, it says nothing about whether the removed component is perpendicular.
Work over \(\mathbb{R}\) with the usual dot product \(\mathbf{x}\cdot\mathbf{y}=\mathbf{x}^{\top}\mathbf{y}\). For a subspace \(W\subseteq\mathbb{R}^n\), its orthogonal complement \(W^\perp\) consists of the vectors perpendicular to every vector in \(W\). An orthogonal projection onto \(W\) assigns to \(\mathbf{x}\) the component \(\mathbf{w}\in W\) in the decomposition \(\mathbf{x}=\mathbf{w}+\mathbf{z}\), where \(\mathbf{z}\in W^\perp\). The projection is the vector \(\mathbf{w}\), not the reflection of \(\mathbf{x}\) across \(W\).
Projection Is Not Reflection
For a projection onto \(W\), write \(\mathbf{x}=\mathbf{w}+\mathbf{z}\), with \(\mathbf{w}\in W\) and \(\mathbf{z}\in W^\perp\). Projection sends \(\mathbf{x}\) to \(\mathbf{w}\). Reflection across \(W\) sends it to \(\mathbf{w}-\mathbf{z}\): the part in \(W\) stays fixed while the perpendicular part changes sign. If \(\mathbf{P}\) is the projection matrix, the reflection matrix is therefore \(2\mathbf{P}-\mathbf{I}_n\), since \(2\mathbf{w}-(\mathbf{w}+\mathbf{z})=\mathbf{w}-\mathbf{z}\).
Worked Example: Projection Onto a Line Versus Reflection Across It
Let \(W=\operatorname{span}\{(1,1)^\top\}\). The vector \((1,1)^\top\) has squared length \(2\), so the projection matrix onto this line is
Apply the projection to \(\mathbf{x}=(2,0)^\top\):
The residual is \(\mathbf{x}-\mathbf{P}\mathbf{x}=(1,-1)^\top\), and it is perpendicular to the line’s direction vector because \((1,-1)\cdot(1,1)=1-1=0\). The projection therefore keeps the component on the line and removes the perpendicular one. The associated reflection matrix is
It sends \((2,0)^\top\) to \((0,2)^\top\), not to \((1,1)^\top\). Direct multiplication gives \(\mathbf{P}^2=\mathbf{P}\), whereas the reflection matrix squared is \(\mathbf{I}_2\). Check: projecting \((1,1)^\top\) leaves it unchanged, while reflecting \((2,0)^\top\) across the line swaps its coordinates, as the reflection matrix shows.
A reflection is thus generally not idempotent: applying it twice returns the original vector. For a projection, applying it twice does nothing further because the first output already lies in the target subspace. These different actions explain why substituting a reflection matrix where a projection is required gives the wrong vector.
Why \(AA^\top\) Requires Orthonormal Columns
Suppose the columns of \(\mathbf{A}\in\mathbb{R}^{n\times k}\) are orthonormal, meaning that each has length \(1\) and distinct columns are perpendicular. Then \(\mathbf{A}^{\top}\mathbf{A}=\mathbf{I}_k\), and \(\mathbf{A}\mathbf{A}^{\top}\) is the orthogonal projection onto the column space of \(\mathbf{A}\). If the columns merely span a subspace but are not orthonormal, that shortcut does not apply. When the columns are linearly independent, the correct formula includes their Gram matrix \(\mathbf{A}^{\top}\mathbf{A}\).
Proof. For any \(\mathbf{c}\in\mathbb{R}^k\), \(\mathbf{c}^{\top}\mathbf{A}^{\top}\mathbf{A}\mathbf{c}=(\mathbf{A}\mathbf{c})^{\top}(\mathbf{A}\mathbf{c})=\|\mathbf{A}\mathbf{c}\|^2\). If \(\mathbf{c}\ne\mathbf{0}\), column independence gives \(\mathbf{A}\mathbf{c}\ne\mathbf{0}\), so this squared length is positive. Thus \(\mathbf{A}^{\top}\mathbf{A}\) has no nonzero vector in its null space and, being square, is invertible. Set \(\mathbf{P}=\mathbf{A}(\mathbf{A}^{\top}\mathbf{A})^{-1}\mathbf{A}^{\top}\). The matrix \(\mathbf{A}^{\top}\mathbf{A}\) and its inverse are symmetric, so \(\mathbf{P}^{\top}=\mathbf{P}\). Also, direct multiplication gives \(\mathbf{P}^2=\mathbf{A}(\mathbf{A}^{\top}\mathbf{A})^{-1}(\mathbf{A}^{\top}\mathbf{A})(\mathbf{A}^{\top}\mathbf{A})^{-1}\mathbf{A}^{\top}=\mathbf{P}\). Every output \(\mathbf{P}\mathbf{x}\) lies in \(\operatorname{Col}\mathbf{A}\), and for any \(\mathbf{y}=\mathbf{A}\mathbf{c}\) in that column space, \(\mathbf{y}^{\top}(\mathbf{x}-\mathbf{P}\mathbf{x})=\mathbf{c}^{\top}\mathbf{A}^{\top}(\mathbf{x}-\mathbf{P}\mathbf{x})=\mathbf{0}\). The last equality follows from \(\mathbf{A}^{\top}\mathbf{P}=\mathbf{A}^{\top}\). Thus the output is in the column space and the residual is perpendicular to it, which is precisely the orthogonal projection. If the columns are orthonormal, \(\mathbf{A}^{\top}\mathbf{A}=\mathbf{I}_k\), giving the stated shorter formula. \(\square\)
Worked Example: Correcting the Formula for Nonorthogonal Columns
Let \(\mathbf{a}_1=(1,0,1)^\top\) and \(\mathbf{a}_2=(0,1,1)^\top\), and put them into the columns of \(\mathbf{A}\). They are independent, but they are not orthogonal because \(\mathbf{a}_1\cdot\mathbf{a}_2=1\). Compute the projection onto their span.
Substituting into the formula gives
For \(\mathbf{x}=(1,2,0)^\top\), the proposed projection is
The residual is \((1,1,-1)^\top\). Its dot products with the spanning columns are \(1+0-1=0\) and \(0+1-1=0\), confirming that it is perpendicular to their span. A common incorrect calculation is to use \(\mathbf{A}\mathbf{A}^{\top}\) anyway. Here that matrix is \(\begin{bmatrix}1&0&1\\0&1&1\\1&1&2\end{bmatrix}\), which sends \(\mathbf{x}\) to \((1,2,3)^\top\), not to the orthogonal projection. Indeed, applying it again gives \((4,5,9)^\top\), so it is not even idempotent. Check: the correct output \(\mathbf{a}_2\) belongs to the proposed subspace, and the residual has zero dot product with both independent spanning vectors.
Idempotence Alone Is Not Enough
Every orthogonal projection matrix is both idempotent and symmetric. The symmetry condition rules out idempotent maps that project onto a subspace along a direction that is not perpendicular to it. The following characterization makes the distinction exact.
Proof. First suppose \(\mathbf{P}\) represents the orthogonal projection onto \(W\). For any \(\mathbf{x}\), its projection \(\mathbf{P}\mathbf{x}\) lies in \(W\), and projecting that vector again leaves it unchanged. Hence \(\mathbf{P}^2\mathbf{x}=\mathbf{P}\mathbf{x}\) for every \(\mathbf{x}\), so \(\mathbf{P}^2=\mathbf{P}\). To show symmetry, take any \(\mathbf{x},\mathbf{y}\) and write their orthogonal decompositions as \(\mathbf{x}=\mathbf{w}+\mathbf{z}\) and \(\mathbf{y}=\mathbf{u}+\mathbf{q}\), with \(\mathbf{w},\mathbf{u}\in W\) and \(\mathbf{z},\mathbf{q}\in W^\perp\). Then \((\mathbf{P}\mathbf{x})^{\top}\mathbf{y}=\mathbf{w}^{\top}(\mathbf{u}+\mathbf{q})=\mathbf{w}^{\top}\mathbf{u}\), and \(\mathbf{x}^{\top}(\mathbf{P}\mathbf{y})=(\mathbf{w}+\mathbf{z})^{\top}\mathbf{u}=\mathbf{w}^{\top}\mathbf{u}\). Therefore \(\mathbf{x}^{\top}\mathbf{P}^{\top}\mathbf{y}=\mathbf{x}^{\top}\mathbf{P}\mathbf{y}\) for all \(\mathbf{x},\mathbf{y}\), which implies \(\mathbf{P}^{\top}=\mathbf{P}\).
Conversely, suppose \(\mathbf{P}^2=\mathbf{P}\) and \(\mathbf{P}^{\top}=\mathbf{P}\), and set \(W=\operatorname{im}\mathbf{P}\), the set of outputs of \(\mathbf{P}\). For every \(\mathbf{x}\), \(\mathbf{P}\mathbf{x}\in W\), and \(\mathbf{x}-\mathbf{P}\mathbf{x}\) is in the null space of \(\mathbf{P}\), since \(\mathbf{P}(\mathbf{x}-\mathbf{P}\mathbf{x})=\mathbf{P}\mathbf{x}-\mathbf{P}^2\mathbf{x}=\mathbf{0}\). Every \(\mathbf{u}\in W\) has the form \(\mathbf{u}=\mathbf{P}\mathbf{a}\). If \(\mathbf{z}\) is in the null space, then \(\mathbf{u}^{\top}\mathbf{z}=(\mathbf{P}\mathbf{a})^{\top}\mathbf{z}=\mathbf{a}^{\top}\mathbf{P}^{\top}\mathbf{z}=\mathbf{a}^{\top}\mathbf{P}\mathbf{z}=0\). Thus \(\mathbf{x}-\mathbf{P}\mathbf{x}\) is perpendicular to \(W\), and \(\mathbf{P}\mathbf{x}\) is its component in \(W\). This is the orthogonal projection onto \(W\). \(\square\)
Worked Example: An Idempotent Matrix That Is Not an Orthogonal Projection
Consider \(\mathbf{Q}=\begin{bmatrix}1&1\\0&0\end{bmatrix}\). Multiplication shows
A tempting but incorrect conclusion is that \(\mathbf{Q}\) must be an orthogonal projection because it is idempotent. But \(\mathbf{Q}^{\top}=\begin{bmatrix}1&0\\1&0\end{bmatrix}\ne\mathbf{Q}\). More concretely, \(\mathbf{Q}(0,1)^\top=(1,0)^\top\), so the residual is \((-1,1)^\top\). The output lies on the horizontal axis, but the residual is not perpendicular to it: \((-1,1)\cdot(1,0)=-1\ne0\). Thus this map is not an orthogonal projection. Check: the calculation of \(\mathbf{Q}^2\) verifies idempotence, while the unequal transpose and nonzero dot product each rule out an orthogonal projection.
A Reliable Test and a Common Pitfall
When given a square matrix and asked whether it represents an orthogonal projection, test both conditions: compute \(\mathbf{P}^2\) and compare it with \(\mathbf{P}\), then compare \(\mathbf{P}^{\top}\) with \(\mathbf{P}\). Passing only the first test is not enough. When constructing a projection from spanning vectors, check whether the columns are orthonormal before using \(\mathbf{A}\mathbf{A}^{\top}\). For independent but nonorthogonal columns, use the Gram-matrix formula proved above. If the given columns are dependent, the inverse in that formula does not exist; first choose an independent set spanning the same subspace, or use a method designed for dependent generators.
The central pitfall is treating a familiar formula or property as sufficient without checking its hypotheses. The formula \(\mathbf{A}\mathbf{A}^{\top}\) needs orthonormal columns, while idempotence needs symmetry as a companion condition to characterize orthogonal projections. A reflection has its own formula and reverses the perpendicular component rather than discarding it.
Check Your Understanding
For each question, state which condition or calculation distinguishes the relevant map.
- For an orthogonal projection matrix \(\mathbf{P}\), what does \(\mathbf{P}^2=\mathbf{P}\) express about applying the map twice?
- What matrix represents reflection across the image of an orthogonal projection \(\mathbf{P}\), and how does it act on the perpendicular component?
- When does \(\mathbf{A}\mathbf{A}^{\top}\) give the orthogonal projection onto \(\operatorname{Col}\mathbf{A}\)?
- Explain why \(\mathbf{A}^{\top}\mathbf{A}\) is invertible when the columns of \(\mathbf{A}\) are linearly independent.
- Give both matrix conditions that together characterize an orthogonal projection, and explain why idempotence alone is insufficient.
