Algebra of Spectral Projectors

From Eigenspaces to Projectors

For a real symmetric matrix, the spectral theorem gives an orthonormal basis of eigenvectors. Grouping the vectors by eigenvalue gives mutually orthogonal eigenspaces. An orthogonal projector selects the component of a vector lying in one specified subspace and discards its orthogonal component. The algebra of these projectors records how the eigenspaces fit together: projecting twice onto the same eigenspace changes nothing, projecting onto two distinct eigenspaces in succession gives zero, and projecting onto all the eigenspaces recovers the original vector.

Let \(\mathbf{A}\in\mathbb{R}^{n\times n}\) be symmetric, and let \(\lambda_1,\dots,\lambda_r\) be its distinct eigenvalues. For each \(i\), its eigenspace is \(E_{\lambda_i}=\{\mathbf{x}\in\mathbb{R}^n:\mathbf{A}\mathbf{x}=\lambda_i\mathbf{x}\}\). Each eigenspace is nonzero. By the spectral theorem, we can choose an orthonormal basis of each \(E_{\lambda_i}\), and the union of these bases is an orthonormal basis of \(\mathbb{R}^n\). Distinct eigenspaces are orthogonal.

In particular, the projector is defined for the whole eigenspace, not just for one eigenvector chosen from it. If an eigenvalue has a multidimensional eigenspace, its projector includes every direction in that space. The choice of orthonormal basis within an eigenspace does not change the projector; the identities below describe it using the full orthogonal decomposition into eigenspaces.

The Algebra of the Eigenspace Projectors

Proof. Fix \(i\), and write \(\mathbf{P}_i=\mathbf{U}_i\mathbf{U}_i^{\top}\). Transposing gives \(\mathbf{P}_i^{\top}=(\mathbf{U}_i\mathbf{U}_i^{\top})^{\top}=\mathbf{U}_i\mathbf{U}_i^{\top}=\mathbf{P}_i\). Using \(\mathbf{U}_i^{\top}\mathbf{U}_i=\mathbf{I}\) for the identity matrix of the column size of \(\mathbf{U}_i\), we also have \(\mathbf{P}_i^2=\mathbf{U}_i(\mathbf{U}_i^{\top}\mathbf{U}_i)\mathbf{U}_i^{\top}=\mathbf{U}_i\mathbf{U}_i^{\top}=\mathbf{P}_i\). If \(i\ne j\), the eigenspaces \(E_{\lambda_i}\) and \(E_{\lambda_j}\) are orthogonal. Therefore \(\mathbf{U}_i^{\top}\mathbf{U}_j=\mathbf{O}\), since every entry is an inner product between a column of \(\mathbf{U}_i\) and a column of \(\mathbf{U}_j\). It follows that \(\mathbf{P}_i\mathbf{P}_j=\mathbf{U}_i(\mathbf{U}_i^{\top}\mathbf{U}_j)\mathbf{U}_j^{\top}=\mathbf{O}\). Finally, concatenate the orthonormal bases of the eigenspaces into the square matrix \(\mathbf{U}=[\mathbf{U}_1\ \cdots\ \mathbf{U}_r]\). The spectral theorem says these columns form an orthonormal basis of \(\mathbb{R}^n\), so \(\mathbf{U}\mathbf{U}^{\top}=\mathbf{I}_n\). Multiplying out in blocks gives \(\mathbf{U}\mathbf{U}^{\top}=\sum_{i=1}^r\mathbf{U}_i\mathbf{U}_i^{\top}=\sum_{i=1}^r\mathbf{P}_i\). Thus the projectors sum to \(\mathbf{I}_n\), completing all four claims. \(\square\)

These identities have direct interpretations. Symmetry says that the projector is a symmetric matrix. Idempotence says that once a vector has been projected into an eigenspace, projecting again leaves it there. The zero product says that distinct eigenspaces have no nonzero component in common. The sum identity says that every vector is the sum of its components in the eigenspaces.

Spectral Reconstruction from the Projectors

Proof. Let \(\mathbf{x}\in\mathbb{R}^n\). The identity \(\sum_{i=1}^r\mathbf{P}_i=\mathbf{I}_n\) gives \(\mathbf{x}=\sum_{i=1}^r\mathbf{P}_i\mathbf{x}\). Each \(\mathbf{P}_i\mathbf{x}\) belongs to \(E_{\lambda_i}\), because it is a linear combination of the columns of \(\mathbf{U}_i\). By the definition of that eigenspace, \(\mathbf{A}(\mathbf{P}_i\mathbf{x})=\lambda_i\mathbf{P}_i\mathbf{x}\). Applying \(\mathbf{A}\) to the decomposition of \(\mathbf{x}\) therefore gives \(\mathbf{A}\mathbf{x}=\sum_{i=1}^r\lambda_i\mathbf{P}_i\mathbf{x}\). This holds for every \(\mathbf{x}\in\mathbb{R}^n\), so the two matrices \(\mathbf{A}\) and \(\sum_{i=1}^r\lambda_i\mathbf{P}_i\) act identically on every vector and are equal. \(\square\)

The formula is valid whether the eigenvalues are distinct or repeated, provided the sum is indexed by the distinct eigenvalues and \(\mathbf{P}_i\) projects onto the entire corresponding eigenspace. The rank of a projector is the dimension of the eigenspace onto which it projects; it need not be one.

A Three-Eigenspace Verification

Common Pitfall: Projecting onto Only Part of an Eigenspace

When an eigenvalue is repeated, using just one unit eigenvector gives a projector onto one line, not onto the full eigenspace. For the repeated-eigenvalue example, suppose we incorrectly use only \(\mathbf{e}_1\) to represent the \(2\)-eigenspace. This gives the wrong projector \(\widetilde{\mathbf{P}}_2=\mathbf{e}_1\mathbf{e}_1^{\top}=\begin{bmatrix}1&0&0\\0&0&0\\0&0&0\end{bmatrix}\). It is symmetric and idempotent, but it omits the direction \(\mathbf{u}\). In particular, \(\widetilde{\mathbf{P}}_2+\mathbf{P}_4=\operatorname{diag}(1,\tfrac12,\tfrac12)\ne\mathbf{I}_3\). The resulting reconstruction also fails:

The correction is to form the projector from an orthonormal basis of the entire eigenspace. Here that means adding \(\mathbf{e}_1\mathbf{e}_1^{\top}\) and \(\mathbf{u}\mathbf{u}^{\top}\). A quick diagnostic is to check both \(\sum_i\mathbf{P}_i=\mathbf{I}_n\) and \(\mathbf{A}=\sum_i\lambda_i\mathbf{P}_i\); either check exposes a missing eigendirection.