The Inductive Strategy
A real symmetric matrix has a special geometric feature: its action can be described using mutually perpendicular eigenvectors. The proof builds those directions one at a time. First, the Rayleigh quotient supplies a unit eigenvector. Symmetry then ensures that the vectors perpendicular to it form an invariant subspace. On that subspace, the matrix is again symmetric, so induction supplies the remaining eigenvectors.
For a real matrix \(\mathbf{A}\in\mathbb{R}^{n\times n}\), symmetry means \(\mathbf{A}^{\top}=\mathbf{A}\). A list \((\mathbf{q}_1,\dots,\mathbf{q}_n)\) is an orthonormal eigenbasis if it is a basis of \(\mathbb{R}^n\), each vector has norm \(1\), distinct vectors are perpendicular, and each \(\mathbf{q}_i\) is an eigenvector of \(\mathbf{A}\). The theorem below asserts existence; it does not say that every choice of eigenvectors is already orthonormal.
The proof uses the Rayleigh quotient \(R_{\mathbf{A}}(\mathbf{x})=\mathbf{x}^{\top}\mathbf{A}\mathbf{x}\) for unit vectors \(\mathbf{x}\). For a real symmetric matrix, it is continuous on the unit sphere, and that sphere is compact. Therefore the quotient attains a maximum. The next argument shows why a maximizing vector is an eigenvector.
Proof. Let \(\mathbf{w}\) be any vector perpendicular to \(\mathbf{q}\). The vector \(\mathbf{u}(t)=(\mathbf{q}+t\mathbf{w})/\sqrt{1+t^2\|\mathbf{w}\|^2}\) has norm \(1\), because \(\|\mathbf{q}+t\mathbf{w}\|^2=1+t^2\|\mathbf{w}\|^2\). The value of the Rayleigh quotient along this curve is \[ \frac{\mathbf{q}^{\top}\mathbf{A}\mathbf{q}+2t\mathbf{w}^{\top}\mathbf{A}\mathbf{q}+t^2\mathbf{w}^{\top}\mathbf{A}\mathbf{w}} {1+t^2\|\mathbf{w}\|^2}. \] Here symmetry gives \(\mathbf{q}^{\top}\mathbf{A}\mathbf{w}=\mathbf{w}^{\top}\mathbf{A}\mathbf{q}\). Since the value has a maximum at \(t=0\), its derivative there is zero; differentiating the displayed expression gives \(2\mathbf{w}^{\top}\mathbf{A}\mathbf{q}=0\). This holds for every \(\mathbf{w}\perp\mathbf{q}\). Set \(\lambda=\mathbf{q}^{\top}\mathbf{A}\mathbf{q}\) and \(\mathbf{r}=\mathbf{A}\mathbf{q}-\lambda\mathbf{q}\). Then \(\mathbf{r}\perp\mathbf{q}\), and the result just obtained, applied with \(\mathbf{w}=\mathbf{r}\), gives \(\mathbf{r}^{\top}\mathbf{A}\mathbf{q}=0\). But \(\mathbf{A}\mathbf{q}=\lambda\mathbf{q}+\mathbf{r}\), so \(\mathbf{r}^{\top}\mathbf{A}\mathbf{q}=\|\mathbf{r}\|^2\). Thus \(\mathbf{r}=\mathbf{0}\), and \(\mathbf{A}\mathbf{q}=\lambda\mathbf{q}\). \(\square\)
Why the Orthogonal Complement Works
For a unit eigenvector \(\mathbf{q}\), its orthogonal complement is the subspace \(W=\{\mathbf{w}\in\mathbb{R}^n:\mathbf{q}^{\top}\mathbf{w}=0\}\). It has dimension \(n-1\) and has an orthonormal basis. These standard finite-dimensional facts can be seen by extending \(\mathbf{q}\) to an orthonormal basis: the remaining \(n-1\) vectors span \(W\).
Proof. If \(\mathbf{w}\in W\), then \(\mathbf{q}^{\top}\mathbf{A}\mathbf{w}=(\mathbf{A}\mathbf{q})^{\top}\mathbf{w}=(\lambda\mathbf{q})^{\top}\mathbf{w}=0\). Thus \(\mathbf{A}\mathbf{w}\in W\), which proves invariance. For any \(\mathbf{u},\mathbf{w}\in W\), symmetry of \(\mathbf{A}\) gives \(\mathbf{u}^{\top}\mathbf{A}\mathbf{w}=(\mathbf{A}\mathbf{u})^{\top}\mathbf{w}\). Since both images remain in \(W\), this is exactly the symmetry condition for the restricted map with the inner product inherited from \(\mathbb{R}^n\). \(\square\)
To apply induction, choose an orthonormal basis \((\mathbf{w}_2,\dots,\mathbf{w}_n)\) of \(W\). The matrix \(\mathbf{B}\) representing the restriction in that basis has entries \(b_{ij}=\mathbf{w}_i^{\top}\mathbf{A}\mathbf{w}_j\), with indices \(i,j\) running from \(2\) to \(n\). Since \(\mathbf{A}\) is symmetric, \(b_{ij}=b_{ji}\), so \(\mathbf{B}\) is a real symmetric \((n-1)\times(n-1)\) matrix.
Proof by Induction on Dimension
Proof of the Real Spectral Theorem. We use induction on \(n\). If \(n=1\), the matrix is \(\begin{bmatrix}a\end{bmatrix}\), and the one-vector list \((\begin{bmatrix}1\end{bmatrix})\) is an orthonormal eigenbasis, with eigenvalue \(a\).
Now let \(n>1\), and assume the theorem holds for real symmetric matrices of size \((n-1)\times(n-1)\). The unit sphere in \(\mathbb{R}^n\) is compact, so the Rayleigh quotient of \(\mathbf{A}\) has a maximizing unit vector \(\mathbf{q}_1\). By the lemma, \(\mathbf{q}_1\) is an eigenvector. Let \(W=\{\mathbf{w}:\mathbf{q}_1^{\top}\mathbf{w}=0\}\). By the proposition, \(W\) is invariant and the restriction of \(\mathbf{A}\) to \(W\) is symmetric. Choose an orthonormal basis of \(W\); its restriction matrix \(\mathbf{B}\) is therefore real symmetric and has size \((n-1)\times(n-1)\).
The induction hypothesis gives an orthonormal eigenbasis of \(\mathbf{B}\). Interpreting its coordinate vectors in the chosen basis of \(W\) gives an orthonormal eigenbasis \((\mathbf{q}_2,\dots,\mathbf{q}_n)\) for the restriction of \(\mathbf{A}\) to \(W\). Because \(W\) is invariant, each of these is also an eigenvector of \(\mathbf{A}\) on \(\mathbb{R}^n\). Each is perpendicular to \(\mathbf{q}_1\), and the vectors \(\mathbf{q}_2,\dots,\mathbf{q}_n\) are mutually orthonormal. Hence \((\mathbf{q}_1,\mathbf{q}_2,\dots,\mathbf{q}_n)\) is an orthonormal list of \(n\) vectors in \(\mathbb{R}^n\), so it is an orthonormal basis and consists entirely of eigenvectors. This completes the induction. \(\square\)
Following the Induction in Examples
Worked Example: The One-Dimensional Base Case
Consider \(\mathbf{A}=\begin{bmatrix}5\end{bmatrix}\). The unit vector \(\mathbf{q}_1=\begin{bmatrix}1\end{bmatrix}\) satisfies \(\mathbf{A}\mathbf{q}_1=\begin{bmatrix}5\end{bmatrix}=5\mathbf{q}_1\). Its norm is \(1\), so the single vector is an orthonormal eigenbasis. The Rayleigh quotient on the unit sphere, which consists of \(1\) and \(-1\), equals \(5\) at both points:
Check: the proposed vector has norm \(1\), and multiplying by \(\mathbf{A}\) gives \(5\mathbf{q}_1\), as required.
Worked Example: A Two-Dimensional Symmetric Matrix
Let \(\mathbf{A}=\begin{bmatrix}2&1\\1&2\end{bmatrix}\). For a unit vector \(\mathbf{x}=\begin{bmatrix}x\\y\end{bmatrix}\), its Rayleigh quotient is \(2x^2+2xy+2y^2\). Since \(x^2+y^2=1\), this can be written \(3-(x-y)^2\), which is at most \(3\). Equality holds when \(x=y\), so choose the maximizing unit vector \(\mathbf{q}_1=\frac{1}{\sqrt{2}}\begin{bmatrix}1\\1\end{bmatrix}\).
The orthogonal complement is the line spanned by \(\mathbf{q}_2=\frac{1}{\sqrt{2}}\begin{bmatrix}1\\-1\end{bmatrix}\). Direct multiplication gives \(\mathbf{A}\mathbf{q}_2=\frac{1}{\sqrt{2}}\begin{bmatrix}1\\-1\end{bmatrix}=\mathbf{q}_2\). The vectors are unit length and \(\mathbf{q}_1^{\top}\mathbf{q}_2=(1-1)/2=0\).
Check: both eigenvector equations hold, and two orthonormal vectors in \(\mathbb{R}^2\) form a basis. Thus the eigenvalues are \(3\) and \(1\) for the displayed orthonormal eigenvectors.
Worked Example: The Restriction in Three Dimensions
Consider \(\mathbf{A}=\begin{bmatrix}2&1&0\\1&2&0\\0&0&4\end{bmatrix}\). For \(\mathbf{x}=\begin{bmatrix}x\\y\\z\end{bmatrix}\), calculate \(\mathbf{x}^{\top}\mathbf{A}\mathbf{x}=2x^2+2xy+2y^2+4z^2\). Therefore \(4\|\mathbf{x}\|^2-\mathbf{x}^{\top}\mathbf{A}\mathbf{x}=x^2+y^2+(x-y)^2\geq0\). On the unit sphere the Rayleigh quotient is at most \(4\), and equality occurs at \(\mathbf{q}_1=\begin{bmatrix}0\\0\\1\end{bmatrix}\). Multiplication confirms \(\mathbf{A}\mathbf{q}_1=4\mathbf{q}_1\).
The orthogonal complement \(W\) is the \(xy\)-plane. In the orthonormal basis \(\mathbf{e}_1,\mathbf{e}_2\), the restriction has matrix \(\begin{bmatrix}2&1\\1&2\end{bmatrix}\). From the preceding two-dimensional calculation, its orthonormal eigenvectors are \(\frac{1}{\sqrt{2}}\begin{bmatrix}1\\1\end{bmatrix}\) and \(\frac{1}{\sqrt{2}}\begin{bmatrix}1\\-1\end{bmatrix}\). Within \(\mathbb{R}^3\), these become \(\mathbf{q}_2=\frac{1}{\sqrt{2}}\begin{bmatrix}1\\1\\0\end{bmatrix}\) and \(\mathbf{q}_3=\frac{1}{\sqrt{2}}\begin{bmatrix}1\\-1\\0\end{bmatrix}\).
Check: \(\mathbf{q}_1,\mathbf{q}_2,\mathbf{q}_3\) each have norm \(1\). Their pairwise dot products are \(0\), since \(\mathbf{q}_1\) has only a third coordinate and \(\mathbf{q}_2^{\top}\mathbf{q}_3=(1-1)/2=0\). Each eigenvector equation is confirmed by the displayed matrix multiplication, so the three vectors give an orthonormal eigenbasis.
Common Pitfall: An Eigenbasis Need Not Be Orthonormal
It is incorrect to assume that any selection of eigenvectors is automatically orthogonal. For example, \(\mathbf{A}=\begin{bmatrix}1&0\\0&1\end{bmatrix}\) has eigenvalue \(1\), and both \(\mathbf{v}_1=\begin{bmatrix}1\\0\end{bmatrix}\) and \(\mathbf{v}_2=\begin{bmatrix}1\\1\end{bmatrix}\) are eigenvectors. A wrong calculation might call them an orthogonal eigenbasis; in fact, \(\mathbf{v}_1^{\top}\mathbf{v}_2=1\ne0\), and \(\|\mathbf{v}_2\|=\sqrt{2}\ne1\). They are not orthonormal.
The correction is to choose an orthonormal basis within the repeated eigenspace: \(\mathbf{e}_1=\begin{bmatrix}1\\0\end{bmatrix}\) and \(\mathbf{e}_2=\begin{bmatrix}0\\1\end{bmatrix}\). Both satisfy \(\mathbf{A}\mathbf{e}_i=\mathbf{e}_i\), both have norm \(1\), and \(\mathbf{e}_1^{\top}\mathbf{e}_2=0\). The induction proof makes this choice systematically: it obtains one unit eigenvector and then finds the rest inside its orthogonal complement.
The key point is that the induction is applied to the restriction, not to the original matrix again. Invariance ensures the restricted action stays inside the smaller space, and symmetry ensures that its matrix in an orthonormal basis remains symmetric. Those two facts are precisely what make the induction hypothesis applicable.
Check Your Understanding
Use the proof structure and verify each calculation by substitution.
- Why does a maximizing unit vector for the Rayleigh quotient have zero derivative along every curve \((\mathbf{q}+t\mathbf{w})/\sqrt{1+t^2\|\mathbf{w}\|^2}\) with \(\mathbf{w}\perp\mathbf{q}\)?
- In the invariant-complement proof, where is the assumption \(\mathbf{A}^{\top}=\mathbf{A}\) used?
- For \(\mathbf{A}=\begin{bmatrix}3&1\\1&3\end{bmatrix}\), find a maximizing unit vector and a unit eigenvector in its orthogonal complement.
- Explain why the restriction matrix in an orthonormal basis of \(W\) is symmetric.
- In the induction step, why do the eigenvectors of the restriction also satisfy eigenvector equations for the original matrix?
