Existence of Extremal Rayleigh Quotients

From a Bounded Range to an Attained Extreme

A continuous function on a compact set does more than have values that stay within bounds: it reaches its largest and smallest values. For a real symmetric matrix, the relevant function is the Rayleigh quotient, and the relevant compact set is the unit sphere. The key point of this tutorial is existence. We will prove that there are unit vectors where the quotient is greatest and least, without yet identifying those values in terms of eigenvalues.

Let \(n\) be a positive integer and let \(\mathbf{A}\in\mathbb{R}^{n\times n}\) be symmetric, meaning \(\mathbf{A}^{\top}=\mathbf{A}\). For every nonzero \(\mathbf{x}\in\mathbb{R}^n\), define the Rayleigh quotient by dividing the quadratic form by the squared length of \(\mathbf{x}\). The denominator is positive for every such vector.

The unit sphere is \(S^{n-1}=\{\mathbf{x}\in\mathbb{R}^n:\mathbf{x}^{\top}\mathbf{x}=1\}\). On this sphere, the denominator in the Rayleigh quotient is \(1\). Thus the quotient there is the function \(f_{\mathbf{A}}(\mathbf{x})=\mathbf{x}^{\top}\mathbf{A}\mathbf{x}\). In coordinates, this is a polynomial in the entries of \(\mathbf{x}\), so it is continuous.

Proof. The unit sphere is nonempty because it contains the standard basis vector \(\mathbf{e}_1\). It is bounded, since every \(\mathbf{x}\in S^{n-1}\) satisfies \(|x_i|\le1\) for every coordinate \(i\). It is also closed: if a sequence of unit vectors converges to \(\mathbf{x}\), continuity of the squared norm gives \(\mathbf{x}^{\top}\mathbf{x}=1\). By the Heine–Borel theorem, a closed and bounded subset of \(\mathbb{R}^n\) is compact; in particular, every sequence in the sphere has a subsequence converging to a point of the sphere.

Write \(a_{ij}\) for the entries of \(\mathbf{A}\). For every unit vector \(\mathbf{x}\), the coordinate formula and \(|x_i|\le1\) give

Therefore the values of \(f_{\mathbf{A}}\) on the sphere form a nonempty bounded subset of \(\mathbb{R}\). Let \(\alpha\) be their supremum. By the defining property of a supremum, for each \(k\in\mathbb{N}\) there is a unit vector \(\mathbf{x}_k\) such that \(\alpha-\tfrac1k<f_{\mathbf{A}}(\mathbf{x}_k)\le\alpha\). Compactness gives a convergent subsequence \(\mathbf{x}_{k_r}\to\mathbf{x}_{\max}\) with \(\mathbf{x}_{\max}\) on the sphere. Continuity then gives \(f_{\mathbf{A}}(\mathbf{x}_{k_r})\to f_{\mathbf{A}}(\mathbf{x}_{\max})\). The inequalities defining the sequence imply \(f_{\mathbf{A}}(\mathbf{x}_{k_r})\to\alpha\), so \(f_{\mathbf{A}}(\mathbf{x}_{\max})=\alpha\). Hence the supremum is a maximum.

For the minimum, let \(\beta\) be the infimum of the same bounded, nonempty set of values. Choose unit vectors \(\mathbf{y}_k\) with \(\beta\le f_{\mathbf{A}}(\mathbf{y}_k)<\beta+\tfrac1k\). A convergent subsequence has a limit \(\mathbf{x}_{\min}\) on the sphere, and continuity gives \(f_{\mathbf{A}}(\mathbf{x}_{\min})=\beta\). Thus the infimum is a minimum as well. Since \(R_{\mathbf{A}}=f_{\mathbf{A}}\) on the sphere, both conclusions hold for the Rayleigh quotient. \(\square\)

The proof has two separate jobs. The coordinate estimate ensures the quotient values cannot escape to infinity, so a supremum and an infimum exist as real numbers. Compactness supplies a convergent subsequence, and continuity ensures its limit has the limiting function value. Without that last step, a bound on the values alone would not show that either endpoint is attained.

Worked Applications on the Unit Sphere

Why Compactness Is the Right Hypothesis

The Rayleigh quotient is defined for every nonzero vector, but the existence proof is applied on the unit sphere. This restriction is useful for two reasons: it removes the denominator, and it gives a compact domain. Indeed, scaling a nonzero vector by a nonzero real number \(c\) does not change its quotient:

Also, every nonzero \(\mathbf{x}\) can be rescaled to the unit vector \(\mathbf{x}/\|\mathbf{x}\|\). Thus studying the quotient on the sphere does not discard any quotient values. The proof uses the finite number of matrix entries to bound the quadratic form and compactness to turn limiting values into actual values.

Symmetry is part of the stated setting for the spectral theory of real matrices. However, the existence argument itself uses only that the entries are real and the function \(\mathbf{x}^{\top}\mathbf{A}\mathbf{x}\) is continuous. The symmetry condition is therefore not needed for this particular existence claim. It becomes important when relating the extrema to the matrix’s eigenvalues and eigenvectors.

Common Pitfall: Confusing the Quadratic Form with the Quotient

Take \(\mathbf{A}=\begin{bmatrix}2&0\\0&5\end{bmatrix}\) and \(\mathbf{x}=\begin{bmatrix}1\\1\end{bmatrix}\). A common incorrect calculation says that the quotient is \(7\), because \(\mathbf{x}^{\top}\mathbf{A}\mathbf{x}=7\). But \(\mathbf{x}\) is not a unit vector: \(\mathbf{x}^{\top}\mathbf{x}=2\). The correct calculation is

Check: \(\mathbf{A}\mathbf{x}=\begin{bmatrix}2\\5\end{bmatrix}\), so \(\mathbf{x}^{\top}\mathbf{A}\mathbf{x}=1(2)+1(5)=7\), while the squared length is \(2\). The quotient is therefore \(\tfrac72\), not \(7\). On the unit sphere, and only because the denominator is then \(1\), the quadratic form itself equals the Rayleigh quotient.

Another logical distinction matters: a function being bounded does not by itself imply it reaches either bound. The proof establishes attainment by taking a sequence of values approaching the supremum or infimum, using compactness to obtain a convergent subsequence, and applying continuity at its limit. Omitting that sequence-and-limit step proves only boundedness, not existence of a maximum or minimum.