Computing a Rayleigh Quotient
For a real symmetric matrix, a Rayleigh quotient turns a nonzero vector into a scalar. Its value depends on both the matrix and the vector’s direction, not on the vector’s length. This makes it useful to compare how a matrix acts along different directions. The calculation itself is direct, but interpreting the result requires care: in two dimensions, equality with an eigenvalue has a particularly precise meaning, while in higher dimensions the same conclusion need not hold.
To calculate the quotient, first find \(\mathbf{A}\mathbf{x}\), then take the dot product \(\mathbf{x}^{\top}(\mathbf{A}\mathbf{x})\), and divide by \(\mathbf{x}^{\top}\mathbf{x}\). Keeping the denominator in the calculation is essential unless the vector has already been shown to be a unit vector.
Proof. Since \((c\mathbf{x})^{\top}=c\mathbf{x}^{\top}\), the numerator for \(c\mathbf{x}\) is \((c\mathbf{x})^{\top}\mathbf{A}(c\mathbf{x})=c^2\mathbf{x}^{\top}\mathbf{A}\mathbf{x}\), and its denominator is \((c\mathbf{x})^{\top}(c\mathbf{x})=c^2\mathbf{x}^{\top}\mathbf{x}\). Because \(c\ne0\), the denominator is nonzero, and the common factor \(c^2\) cancels. Thus \(R_{\mathbf{A}}(c\mathbf{x})=R_{\mathbf{A}}(\mathbf{x})\). \(\square\)
This proposition means that the quotient can be computed using any nonzero representative of a direction. It does not mean that the denominator can be omitted for an arbitrary vector: the numerator alone agrees with the quotient only when the vector is unit length.
Worked Example: A Quotient for a Non-Unit Vector
Compute the Rayleigh quotient of \(\mathbf{A}=\begin{bmatrix}3&1\\1&2\end{bmatrix}\) at \(\mathbf{x}=\begin{bmatrix}2\\1\end{bmatrix}\), then check it using a unit vector in the same direction.
First multiply the matrix by the vector:
The numerator is \(\mathbf{x}^{\top}\mathbf{A}\mathbf{x}=2(7)+1(4)=18\). The denominator is \(\mathbf{x}^{\top}\mathbf{x}=2^2+1^2=5\). Therefore
The vector \(\mathbf{u}=\frac{1}{\sqrt{5}}\begin{bmatrix}2\\1\end{bmatrix}\) is unit length. Using it gives \(\mathbf{A}\mathbf{u}=\frac{1}{\sqrt{5}}\begin{bmatrix}7\\4\end{bmatrix}\), so \(\mathbf{u}^{\top}\mathbf{A}\mathbf{u}=(2(7)+1(4))/5=18/5\).
Check: the original vector has squared length \(5\), and its quadratic-form numerator is \(18\); their ratio is \(18/5\). The unit-vector calculation gives the same value, as scale invariance requires.
Comparing Values on the Unit Circle
For a symmetric matrix in \(\mathbb{R}^{2\times2}\), write \(\mathbf{A}=\begin{bmatrix}a&b\\b&d\end{bmatrix}\). A unit vector on the circle can be written \(\mathbf{u}=\begin{bmatrix}\cos t\\\sin t\end{bmatrix}\), where \(t\) is a real angle. Direct multiplication gives
The mixed term has a factor of \(2\) because both off-diagonal entries contribute: one from the first row and one from the second. Using \(\cos^2t+\sin^2t=1\), the same value can also be written as
This formula lets us compare directions by substituting their coordinates or angles. Opposite points on the unit circle give the same quotient: replacing \(\mathbf{u}\) by \(-\mathbf{u}\) multiplies both appearances of the vector in \(\mathbf{u}^{\top}\mathbf{A}\mathbf{u}\) by \(-1\), leaving the value unchanged.
Worked Example: Comparing Directions on the Unit Circle
For \(\mathbf{A}=\begin{bmatrix}4&2\\2&1\end{bmatrix}\), compare the quotient in the direction \(\mathbf{u}=\frac{1}{\sqrt{5}}\begin{bmatrix}2\\1\end{bmatrix}\) with its value at \(\mathbf{v}=\frac{1}{\sqrt{5}}\begin{bmatrix}-1\\2\end{bmatrix}\).
Both vectors are unit vectors because \(2^2+1^2=5\) and \((-1)^2+2^2=5\). For the first one,
Thus \(R_{\mathbf{A}}(\mathbf{u})=\mathbf{u}^{\top}(5\mathbf{u})=5\), since \(\mathbf{u}^{\top}\mathbf{u}=1\). For the second direction,
It follows that \(R_{\mathbf{A}}(\mathbf{v})=\mathbf{v}^{\top}\mathbf{0}=0\). These directions are perpendicular because \((2)(-1)+(1)(2)=0\).
Check: \(\mathbf{u}\) has quotient \(5\) and \(\mathbf{v}\) has quotient \(0\), as the displayed products show. In fact, for any unit vector \(\mathbf{w}=\begin{bmatrix}x\\y\end{bmatrix}\) on the circle, \(R_{\mathbf{A}}(\mathbf{w})=4x^2+4xy+y^2=(2x+y)^2\), so the quotient is nonnegative; the two computed directions give \(5\) and \(0\).
When Does the Quotient Equal an Eigenvalue?
An eigenvalue \(\lambda\) of \(\mathbf{A}\) is a scalar for which there is a nonzero vector \(\mathbf{v}\) satisfying \(\mathbf{A}\mathbf{v}=\lambda\mathbf{v}\); such a \(\mathbf{v}\) is an eigenvector. If \(\mathbf{x}\) is an eigenvector, substitution into the quotient gives \(\mathbf{x}^{\top}\mathbf{A}\mathbf{x}=\mathbf{x}^{\top}(\lambda\mathbf{x})=\lambda\mathbf{x}^{\top}\mathbf{x}\), and therefore \(R_{\mathbf{A}}(\mathbf{x})=\lambda\). This implication holds whether or not the matrix is symmetric.
For a real symmetric \(2\times2\) matrix, the converse also holds: if the quotient equals one of its eigenvalues, the vector must be an eigenvector. The following result proves the claim in this dimension. It also explains why the dimension restriction matters.
Proof. Write \(\mathbf{A}=\begin{bmatrix}a&b\\b&d\end{bmatrix}\). Its characteristic equation is \((a-\lambda)(d-\lambda)-b^2=0\); the discriminant is \((a-d)^2+4b^2\), which is nonnegative. Thus its eigenvalues are real. For distinct eigenvalues, choose nonzero eigenvectors \(\mathbf{v}_1,\mathbf{v}_2\) and normalize them to unit vectors \(\mathbf{u}_1,\mathbf{u}_2\). Symmetry gives \(\lambda_2\mathbf{u}_1^{\top}\mathbf{u}_2 =\mathbf{u}_1^{\top}\mathbf{A}\mathbf{u}_2 =(\mathbf{A}\mathbf{u}_1)^{\top}\mathbf{u}_2 =\lambda_1\mathbf{u}_1^{\top}\mathbf{u}_2\). Since \(\lambda_1\ne\lambda_2\), the dot product is zero. The two nonzero orthogonal vectors form a basis of \(\mathbb{R}^2\). Write \(\mathbf{x}=c_1\mathbf{u}_1+c_2\mathbf{u}_2\); orthonormality gives \(\mathbf{x}^{\top}\mathbf{x}=c_1^2+c_2^2\) and \(\mathbf{x}^{\top}\mathbf{A}\mathbf{x}=\lambda_1c_1^2+\lambda_2c_2^2\). Consequently, $$ R_{\mathbf{A}}(\mathbf{x}) = \frac{\lambda_1c_1^2+\lambda_2c_2^2}{c_1^2+c_2^2}. $$ The denominator is positive. Equality with \(\lambda_1\) is equivalent to \((\lambda_2-\lambda_1)c_2^2=0\), hence to \(c_2=0\), which means \(\mathbf{x}\) is an eigenvector for \(\lambda_1\). Equality with \(\lambda_2\) is equivalent to \((\lambda_2-\lambda_1)c_1^2=0\), hence to \(c_1=0\), the eigenvector condition for \(\lambda_2\). If the eigenvalue is repeated, the discriminant is zero, so \((a-d)^2+4b^2=0\). Both summands are nonnegative, forcing \(a=d\) and \(b=0\). Hence \(\mathbf{A}=a\mathbf{I}_2\), and direct substitution gives \(R_{\mathbf{A}}(\mathbf{x})=a=\lambda\) for every nonzero \(\mathbf{x}\). \(\square\)
Worked Example: Matching Both Eigenvalues
For \(\mathbf{A}=\begin{bmatrix}2&1\\1&2\end{bmatrix}\), find the eigenvalues and determine which unit directions give a quotient equal to each one.
The characteristic equation is \((2-\lambda)^2-1=0\), so \(\lambda=1\) or \(\lambda=3\). For \(\lambda=3\), \(\mathbf{u}_3=\frac{1}{\sqrt{2}}\begin{bmatrix}1\\1\end{bmatrix}\) is an eigenvector, since \(\mathbf{A}\mathbf{u}_3=\frac{1}{\sqrt{2}}\begin{bmatrix}3\\3\end{bmatrix}=3\mathbf{u}_3\). For \(\lambda=1\), \(\mathbf{u}_1=\frac{1}{\sqrt{2}}\begin{bmatrix}1\\-1\end{bmatrix}\) is an eigenvector, since \(\mathbf{A}\mathbf{u}_1=\frac{1}{\sqrt{2}}\begin{bmatrix}1\\-1\end{bmatrix}=\mathbf{u}_1\). Both vectors are unit length.
For a direct comparison, if \(\mathbf{w}=\begin{bmatrix}x\\y\end{bmatrix}\) is any unit vector, then \(R_{\mathbf{A}}(\mathbf{w})=2x^2+2xy+2y^2\). At \(\mathbf{u}_3\), the numerator is \(2(1/2)+2(1/2)+2(1/2)=3\); at \(\mathbf{u}_1\), it is \(2(1/2)-2(1/2)+2(1/2)=1\). The theorem shows these are the only directions, up to sign, where the quotient equals the respective eigenvalue.
Check: the two matrix-vector products above verify both eigenvector equations. Their unit lengths make their quotients equal to the corresponding quadratic-form values, \(3\) and \(1\).
A Higher-Dimensional Caution
The two-dimensional equality criterion should not be extended to all symmetric matrices. In more than two dimensions, a quotient can equal an eigenvalue even when the vector is not an eigenvector. The reason is that a value between other eigenvalues can also be a weighted average. For example, consider a diagonal symmetric matrix with three distinct diagonal entries.
Worked Example: Equality Without an Eigenvector
Let \(\mathbf{A}=\begin{bmatrix}0&0&0\\0&1&0\\0&0&2\end{bmatrix}\) and \(\mathbf{x}=\begin{bmatrix}1\\0\\1\end{bmatrix}\). Compute the quotient and check whether \(\mathbf{x}\) is an eigenvector.
Multiplication gives \(\mathbf{A}\mathbf{x}=\begin{bmatrix}0\\0\\2\end{bmatrix}\), so \(\mathbf{x}^{\top}\mathbf{A}\mathbf{x}=1(0)+0(0)+1(2)=2\). Also, \(\mathbf{x}^{\top}\mathbf{x}=1^2+0^2+1^2=2\). Thus
The value \(1\) is an eigenvalue, since \(\mathbf{A}\begin{bmatrix}0\\1\\0\end{bmatrix}=\begin{bmatrix}0\\1\\0\end{bmatrix}\). But \(\mathbf{A}\mathbf{x}=\begin{bmatrix}0\\0\\2\end{bmatrix}\) is not a scalar multiple of \(\mathbf{x}=\begin{bmatrix}1\\0\\1\end{bmatrix}\), so \(\mathbf{x}\) is not an eigenvector.
Check: the numerator and denominator are both \(2\), confirming the quotient \(1\); the unequal coordinate patterns in \(\mathbf{A}\mathbf{x}\) and \(\mathbf{x}\) confirm that the vector is not an eigenvector.
Common Mistake: Omitting the Denominator
A frequent error is to treat \(\mathbf{x}^{\top}\mathbf{A}\mathbf{x}\) as the quotient even when \(\mathbf{x}\) is not unit length. For the matrix and vector in the first worked example, the incorrect calculation would report \(18\) as the quotient because \(18=\mathbf{x}^{\top}\mathbf{A}\mathbf{x}\). But \(\mathbf{x}^{\top}\mathbf{x}=5\), so the correct value is \(18/5\). The correction is to divide by the squared length, or first normalize the vector and then use its quadratic form.
When comparing directions on the unit circle, check the unit-length condition before dropping the denominator. When testing whether a computed value is an eigenvalue, distinguish the statement “the quotient equals an eigenvalue” from “the vector is an eigenvector”: they are equivalent for a symmetric \(2\times2\) matrix with distinct eigenvalues, but the three-dimensional example shows they are not equivalent in general.
Check Your Understanding
For each calculation, keep track of whether the vector is unit length and verify any claimed eigenvector equation by multiplication.
- Compute \(R_{\mathbf{A}}(\mathbf{x})\) for \(\mathbf{A}=\begin{bmatrix}1&2\\2&4\end{bmatrix}\) and \(\mathbf{x}=\begin{bmatrix}1\\1\end{bmatrix}\).
- For \(\mathbf{A}=\begin{bmatrix}3&0\\0&1\end{bmatrix}\), compare the quotient at \(\frac{1}{\sqrt{2}}\begin{bmatrix}1\\1\end{bmatrix}\) and \(\frac{1}{\sqrt{2}}\begin{bmatrix}1\\-1\end{bmatrix}\).
- Show directly that scaling a nonzero vector by \(c\ne0\) leaves its Rayleigh quotient unchanged.
- For the matrix in the previous question, identify unit directions where the quotient equals \(3\) and where it equals \(1\).
- Give a reason the conclusion “the quotient equals an eigenvalue, so the vector is an eigenvector” cannot be used for every symmetric matrix.
