Eigenvalue Clusters and Eigenspace Limits

Why the Eigenspace, Not Its Basis, Is the Target

When a population covariance matrix has a repeated eigenvalue, any orthonormal basis of its associated eigenspace is a valid set of principal directions. The data may select one particular basis for the sample covariance, but that choice need not approach any uniquely defined population basis. What is identified is the subspace itself.

This distinction changes the asymptotic target. For an isolated repeated eigenvalue, the sample eigenspace can be estimated consistently, and its projector has a first-order limit. That limit describes how the subspace tilts toward directions outside the population eigenspace. Perturbations wholly within the repeated-eigenvalue space can split the sample eigenvalues and rotate the sample axes there, but they do not change the subspace to first order.

Write the distinct eigenvalues of \(\Sigma\) other than \(\lambda\) as \(\mu\), and let \(P_\mu\) be the orthogonal projector onto the eigenspace for \(\mu\). Assume the repeated eigenvalue is separated from the rest of the spectrum:

This gap lets us define the sample cluster: for a covariance estimate \(S_n\) sufficiently close to \(\Sigma\), select the \(r\) eigenvalues near \(\lambda\), and let \(\widehat P_n\) project onto their combined eigenspace. The sample eigenvalues inside this cluster need not remain equal.

Three Worked Views of a Cluster

The First-Order Limit for the Projector

Let \(S_n\) be a symmetric covariance estimator and assume \(\sqrt n(S_n-\Sigma)\xrightarrow{d}G\), where \(G\) is a symmetric random matrix. This is the covariance-matrix central limit condition; for example, it holds for the usual sample covariance under independent identically distributed observations with finite fourth moments (with the corresponding covariance limit). The result below uses this condition and the positive gap \(\delta\).

Here is a derivation that also explains why the gap is needed. Put \(E=S_n-\Sigma\). A small contour in the complex plane enclosing \(\lambda\), and no other eigenvalue of \(\Sigma\), defines the projector through the spectral theorem:

For sufficiently small \(\lVert E\rVert\), the same contour encloses exactly the cluster eigenvalues of \(S_n\). Write \(R(z)=(zI-\Sigma)^{-1}\). The resolvent identity, applied twice, gives

On the fixed contour, the norm of \(R(z)\) is bounded because the contour does not meet the spectrum. For small enough \(\lVert E\rVert\), the perturbed resolvent is bounded there too. Hence the last term has norm at most \(C\lVert E\rVert^2\), uniformly on the contour, for a fixed constant \(C\). Integrating gives

To evaluate the linear term, use the spectral decomposition \(R(z)=\sum_\nu P_\nu/(z-\nu)\), where the sum is over distinct eigenvalues and \(P_\lambda=P\). The coefficient of \(P_\nu E P_\kappa\) in the integrand is \(1/((z-\nu)(z-\kappa))\). Its contour integral is zero if neither eigenvalue is \(\lambda\), and also zero for \(\nu=\kappa=\lambda\), since the integral of \((z-\lambda)^{-2}\) is zero. If exactly one index is \(\lambda\), the integral divided by \(2\pi i\) is \(1/(\lambda-\mu)\) for the ordered pair \((\nu,\kappa)=(\mu,\lambda)\) or \((\lambda,\mu)\). Therefore

The covariance central limit condition implies \(E=O_p(n^{-1/2})\). Thus \(\sqrt n\lVert E\rVert^2\xrightarrow{p}0\). Multiplying the expansion by \(\sqrt n\), then applying the continuous linear map in the displayed sum and Slutsky’s theorem, proves the distributional limit. It also gives \(\widehat P_n\xrightarrow{p}P\). \(\square\)

What the Limit Says About Random Axes

For intuition about the sample directions, choose an orthonormal basis \(U\) for the population cluster, so \(P=UU^\mathsf T\). Within that basis, the leading perturbation of the repeated eigenvalue is the \(r\times r\) symmetric matrix \(U^\mathsf T G U\). If this matrix has distinct eigenvalues, its eigenvectors determine the leading orientation of the sample axes within the cluster. In repeated-eigenvalue models, this matrix is generally random. Thus the sample axes may have random limiting orientations even while \(\widehat P_n\) converges to \(P\).

For example, suppose the observations are independent \(N(0,\Sigma)\) with \(\Sigma=\operatorname{diag}(6,6,2)\), and \(S_n\) is the sample covariance. The covariance limit \(G\) satisfies \(\operatorname{Var}(G_{13})=6\cdot2=12\), \(\operatorname{Var}(G_{23})=12\), and \(\operatorname{Cov}(G_{13},G_{23})=0\). These follow from the Gaussian covariance formula \(\operatorname{Cov}(G_{ij},G_{k\ell})=\Sigma_{ik}\Sigma_{j\ell}+\Sigma_{i\ell}\Sigma_{jk}\). Consequently the two random off-diagonal entries in the preceding projector limit have variance \(12/16=3/4\) and covariance zero. This gives a concrete measure of uncertainty in the plane’s tilt, without assigning a preferred population axis within that plane.

The projector is a basis-free summary: for any orthogonal change of basis \(Q\) within the cluster, replacing \(U\) by \(UQ\) leaves \(UU^\mathsf T\) unchanged. This is why asymptotic statements about a repeated-eigenvalue cluster are naturally made for \(\widehat P_n\), rather than for a selected list of sample eigenvectors.

Common Mistake

• Wrong reasoning: “The sample eigenvectors are consistent, so each must converge to a fixed population eigenvector.” A repeated eigenvalue does not define such fixed individual eigenvectors. Rotations within its eigenspace are equally valid population choices, and small sample perturbations can select different axes.
• Correction: Identify the population cluster and estimate its projector. Under a gap separating the cluster from the rest of the spectrum, the projector is consistent and has the first-order limit derived above. Do not interpret convergence of the subspace as convergence of a particular basis inside it.