Extrema On The Unit Sphere

The Unit-Sphere Extremum Problem

Given a real matrix \(\mathbf{A}\), consider the quadratic form \(q(\mathbf{x})=\mathbf{x}^{\top}\mathbf{A}\mathbf{x}\) while \(\mathbf{x}\) is restricted to have unit length. The restriction matters: without it, scaling \(\mathbf{x}\) by a real number \(c\) scales \(q(\mathbf{x})\) by \(c^2\), so a nonzero value can be made arbitrarily large in magnitude. On the unit sphere, the values are bounded when \(\mathbf{A}\) is symmetric, and the exact bounds are determined by its eigenvalues.

The symmetry condition is essential to the eigenvalue statement. For a real symmetric matrix, the Spectral Theorem says there is an orthogonal matrix \(\mathbf{Q}\) and a real diagonal matrix \(\boldsymbol{\Lambda}\) such that \(\mathbf{A}=\mathbf{Q}\boldsymbol{\Lambda}\mathbf{Q}^{\top}\). Here orthogonal means \(\mathbf{Q}^{\top}\mathbf{Q}=\mathbf{I}_n\). We will use this standard result from the earlier material on the Spectral Theorem; its consequence for the optimization problem can then be proved directly.

The Extremal Values

Proof. By the Spectral Theorem, write \(\mathbf{A}=\mathbf{Q}\boldsymbol{\Lambda}\mathbf{Q}^{\top}\), where \(\mathbf{Q}^{\top}\mathbf{Q}=\mathbf{I}_n\) and \(\boldsymbol{\Lambda}\) has diagonal entries \(\lambda_1,\dots,\lambda_n\). Set \(\mathbf{y}=\mathbf{Q}^{\top}\mathbf{x}\). Orthogonality gives \(\mathbf{y}^{\top}\mathbf{y}=\mathbf{x}^{\top}\mathbf{Q}\mathbf{Q}^{\top}\mathbf{x}=\mathbf{x}^{\top}\mathbf{x}=1\). Writing the entries of \(\mathbf{y}\) as \(y_1,\dots,y_n\), we obtain \(\mathbf{x}^{\top}\mathbf{A}\mathbf{x}=\mathbf{y}^{\top}\boldsymbol{\Lambda}\mathbf{y}=\sum_{i=1}^n\lambda_i y_i^2\), with \(\sum_{i=1}^n y_i^2=1\). Since \(\lambda_1\leq\lambda_i\leq\lambda_n\) and \(y_i^2\geq0\), summing these inequalities after multiplying by \(y_i^2\) gives \(\lambda_1\sum_i y_i^2\leq\sum_i\lambda_i y_i^2\leq\lambda_n\sum_i y_i^2\). The sum of the squares is \(1\), proving the bounds. Taking \(\mathbf{y}=\mathbf{e}_1\) gives the value \(\lambda_1\), and taking \(\mathbf{y}=\mathbf{e}_n\) gives \(\lambda_n\). These choices correspond to unit vectors \(\mathbf{x}=\mathbf{Q}\mathbf{e}_1\) and \(\mathbf{x}=\mathbf{Q}\mathbf{e}_n\), so both bounds are attained. \(\square\)

The proof expresses the value as a weighted average of the eigenvalues: the weights \(y_i^2\) are nonnegative and add to \(1\). This explains why no unit vector can produce a value outside the eigenvalue interval, while an eigenvector for either endpoint produces that endpoint exactly.

Which Vectors Attain the Bounds?

The values alone do not describe all the extremizers. If an extreme eigenvalue is repeated, its eigenspace contains more than one direction, and every unit vector in that eigenspace attains the same extreme. The next result gives the exact condition, including the case where all eigenvalues are equal.

Proof. Use the orthogonal diagonalization and the coordinates \(\mathbf{y}=\mathbf{Q}^{\top}\mathbf{x}\) from the preceding proof. For the minimum, subtract the smallest eigenvalue from the quadratic form and use \(\sum_i y_i^2=1\): \(\mathbf{x}^{\top}\mathbf{A}\mathbf{x}-\lambda_1=\sum_{i=1}^n(\lambda_i-\lambda_1)y_i^2\). Each term on the right is nonnegative. The sum equals zero if and only if every term with \(\lambda_i-\lambda_1>0\) has \(y_i=0\). This says that \(\mathbf{y}\) has nonzero entries only in positions whose eigenvalue equals \(\lambda_1\). In the diagonal coordinates, that is exactly \(\boldsymbol{\Lambda}\mathbf{y}=\lambda_1\mathbf{y}\). Multiplying by \(\mathbf{Q}\) and using \(\mathbf{x}=\mathbf{Q}\mathbf{y}\) gives \(\mathbf{A}\mathbf{x}=\lambda_1\mathbf{x}\). Conversely, if \(\mathbf{A}\mathbf{x}=\lambda_1\mathbf{x}\) and \(\mathbf{x}^{\top}\mathbf{x}=1\), then \(\mathbf{x}^{\top}\mathbf{A}\mathbf{x}=\lambda_1\mathbf{x}^{\top}\mathbf{x}=\lambda_1\), so \(\mathbf{x}\) attains the minimum. For the maximum, use \(\lambda_n-\mathbf{x}^{\top}\mathbf{A}\mathbf{x}=\sum_{i=1}^n(\lambda_n-\lambda_i)y_i^2\). This sum is zero exactly when \(y_i=0\) wherever \(\lambda_i<\lambda_n\), which is equivalent to \(\boldsymbol{\Lambda}\mathbf{y}=\lambda_n\mathbf{y}\), and hence to \(\mathbf{A}\mathbf{x}=\lambda_n\mathbf{x}\). Conversely, a unit vector satisfying this eigenvector equation has quadratic-form value \(\lambda_n\). These arguments also apply when \(\lambda_1=\lambda_n\): all coefficients in both sums are zero, and every vector is in the eigenspace for that common eigenvalue. \(\square\)

Why Symmetry Cannot Be Dropped

The eigenvalue conclusion does not hold for an arbitrary real matrix. In fact, the quadratic form of any real square matrix depends only on its symmetric part. This fact explains why the optimization problem remains meaningful for a nonsymmetric matrix, but its extrema are governed by the symmetric part rather than, in general, by the original matrix's eigenvalues.

Proof. Let \(\mathbf{K}=\tfrac12(\mathbf{A}-\mathbf{A}^{\top})\), so \(\mathbf{A}=\mathbf{S}+\mathbf{K}\) and \(\mathbf{K}^{\top}=-\mathbf{K}\). The quantity \(r=\mathbf{x}^{\top}\mathbf{K}\mathbf{x}\) is a real scalar, so it equals its transpose. Therefore \(r=r^{\top}=\mathbf{x}^{\top}\mathbf{K}^{\top}\mathbf{x}=-\mathbf{x}^{\top}\mathbf{K}\mathbf{x}=-r\). Thus \(r=0\). It follows that \(\mathbf{x}^{\top}\mathbf{A}\mathbf{x}=\mathbf{x}^{\top}\mathbf{S}\mathbf{x}+\mathbf{x}^{\top}\mathbf{K}\mathbf{x}=\mathbf{x}^{\top}\mathbf{S}\mathbf{x}\), as required. \(\square\)

Common Pitfall: Using Eigenvalues Without Checking Symmetry

For example, consider the nonsymmetric real matrix \(\mathbf{A}=\begin{bmatrix}0&-1\\1&0\end{bmatrix}\). A mistaken application of the symmetric-matrix theorem might try to assign the extrema to its eigenvalues. But direct multiplication, for \(\mathbf{x}=(x_1,x_2)^{\top}\), gives \(\mathbf{x}^{\top}\mathbf{A}\mathbf{x}=-x_1x_2+x_2x_1=0\). So on the unit sphere the minimum and maximum are both \(0\), attained by every unit vector. The correction is to check the hypothesis before using the theorem: this matrix is not symmetric. Its symmetric part is \(\mathbf{O}\), which agrees with the proposition and gives the zero quadratic form. For a symmetric matrix, by contrast, orthogonal diagonalization supplies real eigenvalues and the proved extrema result applies.

The unit-sphere problem is useful because it converts directional variation of a quadratic form into a precise comparison with eigenvalues. The minimum and maximum tell us the smallest and largest possible values over all unit directions; the equality condition identifies exactly which directions attain them. When an endpoint eigenvalue is repeated, its full eigenspace must be considered. In applications, this is why reporting only one maximizing or minimizing vector can omit valid solutions.