Stationary Points on the Unit Sphere
For a real matrix \(\mathbf{A}\), the Rayleigh quotient at a nonzero vector \(\mathbf{x}\) is \(R_{\mathbf{A}}(\mathbf{x})=\frac{\mathbf{x}^{\top}\mathbf{A}\mathbf{x}}{\mathbf{x}^{\top}\mathbf{x}}\). It is unchanged when \(\mathbf{x}\) is multiplied by a nonzero scalar. Consequently, studying the quotient on nonzero vectors is equivalent to studying the quadratic form \(f(\mathbf{x})=\mathbf{x}^{\top}\mathbf{A}\mathbf{x}\) on the unit sphere \(\mathbf{x}^{\top}\mathbf{x}=1\): on that sphere, \(f(\mathbf{x})=R_{\mathbf{A}}(\mathbf{x})\). This tutorial identifies the stationary points of that constrained problem when \(\mathbf{A}\) is real and symmetric.
A point on the sphere is stationary if the first derivative of the function along every differentiable curve on the sphere passing through that point is zero. In particular, it is not enough to differentiate in just one direction. The constraint matters: allowable directions are tangent to the sphere, rather than arbitrary directions in \(\mathbb{R}^n\).
The symmetry assumption makes the gradient of the quadratic form particularly simple. For a direction \(\mathbf{h}\), expand the quadratic form at \(\mathbf{x}+t\mathbf{h}\):
Because \(\mathbf{A}^{\top}=\mathbf{A}\), the two coefficients of \(t\) are equal. Thus the directional derivative is \(2\mathbf{h}^{\top}\mathbf{A}\mathbf{x}\), and \(\nabla f(\mathbf{x})=2\mathbf{A}\mathbf{x}\). For the constraint function \(g(\mathbf{x})=\mathbf{x}^{\top}\mathbf{x}\), the same expansion gives \(\nabla g(\mathbf{x})=2\mathbf{x}\).
The Lagrange Multiplier Condition
For a constrained stationary point, the Lagrange multiplier equation sets the gradient of the function equal to a scalar multiple of the gradient of the constraint. Here it takes the form \(\nabla f(\mathbf{x})=\mu\nabla g(\mathbf{x})\). The constraint gradient does not vanish on the unit sphere, since \(\nabla g(\mathbf{x})=2\mathbf{x}\) and \(\mathbf{x}\ne\mathbf{0}\). We can justify the condition directly for this sphere, rather than assume a general multiplier theorem.
Proof. Suppose first that \(\mathbf{x}\) is a stationary point on the unit sphere. If \(\mathbf{y}^{\top}\mathbf{x}=0\), then \(\mathbf{y}(t)=\frac{\mathbf{x}+t\mathbf{y}}{\|\mathbf{x}+t\mathbf{y}\|}\) is a curve on the sphere for \(t\) near zero: its denominator is nonzero near zero and its norm is one. Since \(\mathbf{x}^{\top}\mathbf{y}=0\), differentiating the denominator at \(t=0\) gives zero, so \(\mathbf{y}'(0)=\mathbf{y}\). Stationarity and the derivative formula therefore give \(0=2\mathbf{y}^{\top}\mathbf{A}\mathbf{x}\) for every \(\mathbf{y}\) perpendicular to \(\mathbf{x}\).
Set \(\mathbf{z}=\mathbf{A}\mathbf{x}-(\mathbf{x}^{\top}\mathbf{A}\mathbf{x})\mathbf{x}\). Because \(\mathbf{x}^{\top}\mathbf{x}=1\), we have \(\mathbf{x}^{\top}\mathbf{z}=0\). We may use \(\mathbf{y}=\mathbf{z}\) in the condition just obtained. Also, \(\mathbf{z}^{\top}\mathbf{A}\mathbf{x}=\mathbf{z}^{\top}\mathbf{z}\), since \(\mathbf{A}\mathbf{x}=\mathbf{z}+(\mathbf{x}^{\top}\mathbf{A}\mathbf{x})\mathbf{x}\) and \(\mathbf{z}^{\top}\mathbf{x}=0\). Hence \(0=2\mathbf{z}^{\top}\mathbf{z}\), which forces \(\mathbf{z}=\mathbf{0}\). It follows that \(\mathbf{A}\mathbf{x}=\mu\mathbf{x}\), where \(\mu=\mathbf{x}^{\top}\mathbf{A}\mathbf{x}\). Thus \(\mathbf{x}\) is a unit eigenvector.
Conversely, suppose \(\mathbf{x}^{\top}\mathbf{x}=1\) and \(\mathbf{A}\mathbf{x}=\mu\mathbf{x}\). For any differentiable curve \(\mathbf{y}(t)\) on the sphere with \(\mathbf{y}(0)=\mathbf{x}\), differentiating \(\mathbf{y}(t)^{\top}\mathbf{y}(t)=1\) at zero gives \(2\mathbf{x}^{\top}\mathbf{y}'(0)=0\). The derivative of the quadratic form along the curve is therefore \(2\mathbf{y}'(0)^{\top}\mathbf{A}\mathbf{x}=2\mu\mathbf{y}'(0)^{\top}\mathbf{x}=0\). So \(\mathbf{x}\) is stationary. Finally, \(\nabla f=2\mathbf{A}\mathbf{x}\) and \(\nabla g=2\mathbf{x}\), so \(\nabla f=\mu\nabla g\) is exactly \(\mathbf{A}\mathbf{x}=\mu\mathbf{x}\). This proves both directions and the multiplier equation. \(\square\)
Proof. The theorem gives \(\mathbf{A}\mathbf{x}=\mu\mathbf{x}\) and \(\mathbf{x}^{\top}\mathbf{x}=1\). Multiplying the eigenvector equation on the left by \(\mathbf{x}^{\top}\) gives \(\mathbf{x}^{\top}\mathbf{A}\mathbf{x}=\mu\mathbf{x}^{\top}\mathbf{x}=\mu\). On the unit sphere, the numerator is the Rayleigh quotient, so \(R_{\mathbf{A}}(\mathbf{x})=\mu\). \(\square\)
Worked Stationary-Point Equations
Worked Example: Two Distinct Eigenvalues
Find all stationary points of the quadratic form for \(\mathbf{A}=\begin{bmatrix}2&1\\1&2\end{bmatrix}\) on the unit circle.
Write \(\mathbf{x}=(x_1,x_2)^{\top}\). The multiplier equation and the constraint are
Add the first two equations to obtain \(3(x_1+x_2)=\mu(x_1+x_2)\), so \((3-\mu)(x_1+x_2)=0\). Subtract the second equation from the first to obtain \((x_1-x_2)=\mu(x_1-x_2)\), so \((1-\mu)(x_1-x_2)=0\). Since the unit constraint rules out \(x_1=x_2=0\), at least one of \(x_1+x_2\) and \(x_1-x_2\) is nonzero. Therefore \(\mu=3\) or \(\mu=1\).
If \(\mu=3\), the equations require \(x_1=x_2\). Substitution into the constraint gives \(2x_1^2=1\), so the stationary points are \(\mathbf{x}=\pm(1,1)^{\top}/\sqrt{2}\). If \(\mu=1\), they require \(x_1=-x_2\), giving \(\mathbf{x}=\pm(1,-1)^{\top}/\sqrt{2}\).
Check: For \(\mathbf{x}=(1,1)^{\top}/\sqrt{2}\), \(\mathbf{A}\mathbf{x}=(3,3)^{\top}/\sqrt{2}=3\mathbf{x}\), and \(\mathbf{x}^{\top}\mathbf{A}\mathbf{x}=3\). For \(\mathbf{x}=(1,-1)^{\top}/\sqrt{2}\), \(\mathbf{A}\mathbf{x}=(1,-1)^{\top}/\sqrt{2}=\mathbf{x}\), and \(\mathbf{x}^{\top}\mathbf{A}\mathbf{x}=1\). Negating either vector preserves its eigenvector equation and unit norm, so all four listed points check.
Worked Example: A Repeated Eigenvalue
Find all stationary points for \(\mathbf{A}=\begin{bmatrix}5&0&0\\0&5&0\\0&0&1\end{bmatrix}\). This example shows why the answer need not be a finite list of points.
The multiplier equations are \((5-\mu)x_1=0\), \((5-\mu)x_2=0\), and \((1-\mu)x_3=0\), together with \(x_1^2+x_2^2+x_3^2=1\). If \(\mu=5\), the third equation forces \(x_3=0\), while the constraint becomes \(x_1^2+x_2^2=1\). Every unit vector in that circle is stationary. If \(\mu=1\), the first two equations force \(x_1=x_2=0\), and the constraint gives \(x_3=1\) or \(x_3=-1\). No other multiplier is possible: if \(\mu\) is neither 5 nor 1, all three coordinates must be zero, contradicting the unit constraint.
Check: In the first case, \(\mathbf{x}=(x_1,x_2,0)^{\top}\) with \(x_1^2+x_2^2=1\) satisfies \(\mathbf{A}\mathbf{x}=(5x_1,5x_2,0)^{\top}=5\mathbf{x}\), and its quadratic form is \(5x_1^2+5x_2^2=5\). In the second case, \(\mathbf{x}=(0,0,\pm1)^{\top}\) satisfies \(\mathbf{A}\mathbf{x}=\mathbf{x}\), and its quadratic form is \(1\). Both families satisfy the multiplier equation and constraint.
Worked Example: A Symmetric Matrix with Off-Diagonal Entries
Determine all stationary points for \(\mathbf{A}=\begin{bmatrix}2&0&0\\0&1&1\\0&1&1\end{bmatrix}\).
For \(\mathbf{x}=(x_1,x_2,x_3)^{\top}\), the equations are \((2-\mu)x_1=0\), \((1-\mu)x_2+x_3=0\), \(x_2+(1-\mu)x_3=0\), and \(x_1^2+x_2^2+x_3^2=1\). Subtracting the third multiplier equation from the second gives \(\mu(x_3-x_2)=0\).
If \(\mu=0\), the first equation gives \(x_1=0\), and the second gives \(x_3=-x_2\). The constraint then gives \(2x_2^2=1\), so \(\mathbf{x}=(0,1,-1)^{\top}/\sqrt{2}\) or its negative. If \(\mu\ne0\), the subtraction gives \(x_3=x_2\). Substitution into the second equation gives \((2-\mu)x_2=0\). If \(x_2\ne0\), this forces \(\mu=2\). If \(x_2=0\), then \(x_3=0\); the unit constraint makes \(x_1\ne0\), and the first equation again forces \(\mu=2\). Thus the remaining stationary points have \(\mu=2\), \(x_3=x_2\), and \(x_1^2+2x_2^2=1\).
Check: When \(\mu=0\), multiplication gives \(\mathbf{A}(0,1,-1)^{\top}=(0,0,0)^{\top}\), and the normalized vectors have quadratic form \(0\). When \(\mu=2\) and \(x_3=x_2\), multiplication gives \(\mathbf{A}\mathbf{x}=(2x_1,2x_2,2x_2)^{\top}=2\mathbf{x}\). Their quadratic form is \(2x_1^2+4x_2^2=2(x_1^2+2x_2^2)=2\). The stated conditions already impose unit norm, so both families satisfy the constraint and the eigenvector equation.
What the Stationarity Test Does—and Does Not—Say
The theorem is a first-order characterization: it converts a constrained derivative question into an eigenvector equation. It does not say that every stationary point is a maximum or minimum. In the examples, the stationary values include distinct eigenvalues, and a repeated eigenvalue gives an entire family of stationary points. The theorem alone does not compare the values at different stationary points or establish that an extremal value exists.
For the symmetric matrix in the theorem, a practical method is to form the equations \(\mathbf{A}\mathbf{x}=\mu\mathbf{x}\) together with \(\mathbf{x}^{\top}\mathbf{x}=1\), then solve for the allowable multipliers and unit vectors. The constraint must remain part of the calculation: an eigenvector direction gives stationary points on the sphere only after it is normalized to unit length.
Common pitfall: using the symmetric-matrix gradient formula without symmetry. The identity \(\nabla(\mathbf{x}^{\top}\mathbf{A}\mathbf{x})=2\mathbf{A}\mathbf{x}\) requires \(\mathbf{A}^{\top}=\mathbf{A}\). For example, take the nonsymmetric matrix \(\mathbf{B}=\begin{bmatrix}0&1\\0&0\end{bmatrix}\) and \(\mathbf{x}=(1,0)^{\top}\). A mistaken use of the formula gives \(2\mathbf{B}\mathbf{x}=\mathbf{0}\), which might suggest that the quadratic form is stationary there. But \(\mathbf{x}^{\top}\mathbf{B}\mathbf{x}=x_1x_2\), whose gradient at \((1,0)^{\top}\) is \((0,1)^{\top}\), not zero. More concretely, the unit-circle curve \(\mathbf{y}(t)=(\cos t,\sin t)^{\top}\) gives \(\mathbf{y}(t)^{\top}\mathbf{B}\mathbf{y}(t)=\cos t\sin t\), whose derivative at \(t=0\) is \(1\). The point is not stationary. In general the gradient is \((\mathbf{B}+\mathbf{B}^{\top})\mathbf{x}\); it reduces to \(2\mathbf{A}\mathbf{x}\) precisely in the symmetric setting used above.
A second check is normalization. For \(\mathbf{A}=\begin{bmatrix}2&1\\1&2\end{bmatrix}\), the vector \((1,1)^{\top}\) is an eigenvector with eigenvalue \(3\), but it is not on the unit circle. Its quadratic form is \((1,1)\begin{bmatrix}3\\3\end{bmatrix}=6\), not \(3\). The unit eigenvector \((1,1)^{\top}/\sqrt{2}\) has quadratic form \(3\), as the constraint requires. The direction identifies the eigenvalue; normalization makes it a point in the constrained domain.
Check Your Understanding
For each question, use the constraint as well as the multiplier equation, and check candidate vectors by multiplication.
- For \(\mathbf{A}=\begin{bmatrix}4&0\\0&-2\end{bmatrix}\), find all stationary points on the unit circle and their stationary values.
- Explain why every unit vector in the plane \(x_3=0\) is stationary for \(\mathbf{A}=\begin{bmatrix}3&0&0\\0&3&0\\0&0&1\end{bmatrix}\).
- For a symmetric matrix, derive the gradient \(2\mathbf{A}\mathbf{x}\) by expanding the quadratic form along \(\mathbf{x}+t\mathbf{h}\).
- At a stationary unit vector satisfying \(\mathbf{A}\mathbf{x}=\mu\mathbf{x}\), show directly that \(R_{\mathbf{A}}(\mathbf{x})=\mu\).
- Why does the calculation with \(\mathbf{B}=\begin{bmatrix}0&1\\0&0\end{bmatrix}\) not contradict the theorem for real symmetric matrices?
