Rayleigh Quotients and Unit-Sphere Geometry

The Quotient Depends on Direction, Not Length

A matrix sends a vector to another vector, but a quadratic expression such as \(\mathbf{x}^{\top}\mathbf{A}\mathbf{x}\) assigns a single number to the input. That number changes when \(\mathbf{x}\) is lengthened. Dividing by the squared length removes this change and gives a quantity associated with the direction of \(\mathbf{x}\). For a real symmetric matrix, this quantity is called the Rayleigh quotient.

Throughout, \(\mathbf{A}\in\mathbb{R}^{n\times n}\) is symmetric, so \(\mathbf{A}^{\top}=\mathbf{A}\), and \(\mathbf{x}\in\mathbb{R}^n\) is nonzero. The Euclidean norm satisfies \(\|\mathbf{x}\|^2=\mathbf{x}^{\top}\mathbf{x}>0\) for every nonzero \(\mathbf{x}\). This positivity ensures that the quotient below is defined.

The numerator is a quadratic form: it uses the vector twice, once on each side of the matrix. If \(\mathbf{A}\) is symmetric, then \(\mathbf{x}^{\top}\mathbf{A}\mathbf{x}\) is real, and the quotient is a real number. Symmetry also gives the quadratic form its standard connection with eigenvectors and orthogonal directions. The denominator is the squared length, not the length itself; that matching degree is what makes the quotient unchanged by rescaling.

Scale Invariance and the Unit Sphere

Proof. Since \(c\ne0\) and \(\mathbf{x}\ne\mathbf{0}\), the vector \(c\mathbf{x}\) is nonzero, so its quotient is defined. Using the rule for transposes and scalar multiplication, the numerator becomes \((c\mathbf{x})^{\top}\mathbf{A}(c\mathbf{x})=c^2\mathbf{x}^{\top}\mathbf{A}\mathbf{x}\), and the denominator becomes \((c\mathbf{x})^{\top}(c\mathbf{x})=c^2\mathbf{x}^{\top}\mathbf{x}\). Because \(c^2\ne0\), it cancels from the ratio. Therefore \(R_{\mathbf{A}}(c\mathbf{x})=\dfrac{\mathbf{x}^{\top}\mathbf{A}\mathbf{x}}{\mathbf{x}^{\top}\mathbf{x}}=R_{\mathbf{A}}(\mathbf{x})\). \(\square\)

For each nonzero \(\mathbf{x}\), the vector \(\mathbf{u}=\mathbf{x}/\|\mathbf{x}\|\) is a unit vector pointing in the same direction, since \(\|\mathbf{u}\|=\|\mathbf{x}\|/\|\mathbf{x}\|=1\). Scale invariance says the quotient at \(\mathbf{x}\) equals its value at \(\mathbf{u}\). On the sphere, \(\mathbf{u}^{\top}\mathbf{u}=1\), so the denominator disappears:

This gives a precise geometric interpretation: the quotient at a nonzero vector is the value of the quadratic form at the unit vector in that direction. Every nonzero vector can be replaced by a unit vector without changing the quotient. In \(\mathbb{R}^2\), the unit sphere is the unit circle; in \(\mathbb{R}^3\), it is the unit sphere in ordinary space. For any dimension, it is the set of all unit-length vectors.

In real space, a direction through the origin has two unit representatives, \(\mathbf{u}\) and \(-\mathbf{u}\). They give the same quotient because \((-\mathbf{u})^{\top}\mathbf{A}(-\mathbf{u})=\mathbf{u}^{\top}\mathbf{A}\mathbf{u}\). Thus the sphere displays all possible directions, while antipodal points represent the same direction for this calculation. The quotient is not a function of vector length: it is a function of direction.

Reading the Quotient on a Circle

A unit vector in \(\mathbb{R}^2\) can be written as \(\mathbf{u}=\begin{bmatrix}\cos\theta\\\sin\theta\end{bmatrix}\), since \(\cos^2\theta+\sin^2\theta=1\). Substituting such a vector into the quadratic form shows how the quotient varies as the point moves around the unit circle. This is a useful way to see that the quotient is tied to direction; it does not require a general conclusion about where the quotient is largest or smallest.

Coordinates Along Orthogonal Eigenvectors

The unit-sphere viewpoint becomes especially transparent when a vector is expressed using orthonormal eigenvectors. An eigenvector \(\mathbf{q}\ne\mathbf{0}\) of \(\mathbf{A}\) with eigenvalue \(\lambda\) satisfies \(\mathbf{A}\mathbf{q}=\lambda\mathbf{q}\). A list of vectors is orthonormal if every vector has norm \(1\) and distinct vectors in the list have dot product \(0\). The next result concerns a vector that is already expressed as a combination of such eigenvectors; it does not require us to assume that every vector has such an expression.

Proof. From \(\mathbf{A}\mathbf{q}_i=\lambda_i\mathbf{q}_i\), linearity gives \(\mathbf{A}\mathbf{x}=\sum_{i=1}^k c_i\lambda_i\mathbf{q}_i\). Therefore \(\mathbf{x}^{\top}\mathbf{A}\mathbf{x}=\sum_{i=1}^k\sum_{j=1}^k c_i c_j\lambda_j\mathbf{q}_i^{\top}\mathbf{q}_j\). Orthonormality means \(\mathbf{q}_i^{\top}\mathbf{q}_j=0\) for \(i\ne j\) and \(\mathbf{q}_i^{\top}\mathbf{q}_i=1\), so only the terms with \(i=j\) remain. The numerator is consequently \(\sum_{i=1}^k\lambda_i c_i^2\). Applying the same orthonormality calculation to \(\mathbf{x}^{\top}\mathbf{x}\) gives \(\sum_{i=1}^k c_i^2\). Since \(\mathbf{x}\ne\mathbf{0}\), this denominator is positive, and division gives the claimed expression. \(\square\)

The coefficients appear squared, so their signs do not affect their contribution. The denominator is the sum of those squared coefficients, and each eigenvalue is weighted by its corresponding squared coefficient. In particular, if \(\mathbf{x}\) is itself an eigenvector with eigenvalue \(\lambda\), its quotient is \(\lambda\): for any nonzero eigenvector, \(\mathbf{x}^{\top}\mathbf{A}\mathbf{x}=\mathbf{x}^{\top}(\lambda\mathbf{x})=\lambda\mathbf{x}^{\top}\mathbf{x}\).

A Common Scaling Error

A frequent mistake is to scale the numerator and denominator by different powers of the scale factor. For the first worked example, the original numerator is \(18\) and the denominator is \(5\). If \(\mathbf{x}\) is replaced by \(2\mathbf{x}\), it is wrong to double the numerator and square the denominator, giving \(36/20=9/5\). Both expressions are quadratic in \(\mathbf{x}\), so both must be multiplied by \(2^2=4\).

The corrected quotient is \(72/20=18/5\), exactly the original value. The reliable check is to count the factors: scaling the vector by \(c\) puts one factor \(c\) on each side of the numerator and also gives \(c^2\) in the denominator.

It is also important not to drop the denominator for a vector that is not known to have unit length. The expression \(\mathbf{x}^{\top}\mathbf{A}\mathbf{x}\) equals the Rayleigh quotient only when \(\mathbf{x}^{\top}\mathbf{x}=1\). For an arbitrary nonzero vector, divide by its squared norm, or first normalize it to a unit vector.