Eigenvector Fluctuations for Supercritical Spikes

From Eigenvalue Separation to Directional Uncertainty

A separated sample eigenvalue identifies a signal in the spectrum, but it does not imply that the corresponding sample eigenvector is nearly parallel to the population direction. In the high-dimensional regime, the angle between these directions can remain bounded away from zero even when the signal is above the BBP threshold. A useful description therefore separates the estimated direction into its component along the population direction and its component in the perpendicular, or tangent, space.

We study the rank-one Gaussian spiked covariance model. Let $X_1,\ldots,X_n$ be independent $p$-dimensional observations with distribution $N_p(0,\Sigma)$, where $\Sigma=I_p+\beta vv^T$, $v$ is a unit vector, and $\beta>0$. Thus the population eigenvalue in direction $v$ is $1+\beta$, and all other population eigenvalues are 1. Suppose $p/n\to\gamma\in(0,\infty)$ and $\beta>\sqrt{\gamma}$. This is the supercritical regime. Let $\hat v$ be a unit eigenvector for the largest eigenvalue of the sample covariance matrix, with its sign chosen so that $v^T\hat v\geq 0$.

Definition: The tangent space at the population direction $v$ is the subspace $v^\perp=\{x\in\mathbb R^p:v^Tx=0\}$. The tangent component of $\hat v$ is $P_{v^\perp}\hat v=(I_p-vv^T)\hat v$. Its length measures how much of the estimated direction lies outside the population direction.

The eigenvector-alignment limit for this model gives $ (v^T\hat v)^2\xrightarrow{p}q$, where $q=(1-\gamma/\beta^2)/(1+\gamma/\beta)$. Because the sign is aligned, $v^T\hat v\xrightarrow{p}\sqrt q$. The limit $q$ is strictly between zero and one in the supercritical regime: the sample eigenvector has nonzero asymptotic alignment, but its tangent component does not vanish.

Key formula: In the rank-one Gaussian model with $p/n\to\gamma$ and $\beta>\sqrt{\gamma}$, the limiting squared alignment and tangent length are $$ q=\frac{1-\gamma/\beta^2}{1+\gamma/\beta}, \qquad \|P_{v^\perp}\hat v\|\xrightarrow{p}\sqrt{1-q}. $$

Worked Examples of Alignment and Tangent Fluctuations

Worked Example: Computing the Limiting Angle

Take $\gamma=1/4$ and $\beta=2$. Since $\sqrt{\gamma}=1/2$, the spike is supercritical. The limiting squared alignment is $$ q=\frac{1-(1/4)/4}{1+(1/4)/2} =\frac{15/16}{9/8} =\frac{15}{16}\cdot\frac{8}{9} =\frac{5}{6}. $$ Thus the limiting absolute inner product is $\sqrt{5/6}$, and the limiting tangent length is $\sqrt{1/6}$. The angle between the sign-aligned vectors converges to $\arccos(\sqrt{5/6})$, about $24.1$ degrees. A supercritical spike produces a well-aligned estimate here, but not a direction whose angle converges to zero.

Worked Example: A Two-Dimensional Tangent Projection

Suppose $p/n\to1/4$, $\beta=2$, and $b_1,b_2$ are fixed orthonormal vectors in $v^\perp$. For this example $q=5/6$, so the limiting tangent variance in each fixed unit direction is $1-q=1/6$. The derived limit below implies $$ \sqrt p \begin{pmatrix} b_1^T\hat v\\ b_2^T\hat v \end{pmatrix} \xrightarrow{d} N_2\left(0,\frac{1}{6}I_2\right). $$ For $p=1000$, the corresponding large-$p$ standard deviation of either unscaled projection is approximately $\sqrt{(1/6)/1000}=0.0129$. These are fluctuations in fixed tangent directions; they do not say that the entire tangent component has length $0.0129$. Its length instead approaches $\sqrt{1/6}$.

Why Tangent Projections Have a Gaussian Limit

The Gaussian spiked model has rotational symmetry around $v$: rotating the observations by any orthogonal transformation that fixes $v$ leaves their distribution unchanged. Consequently, once the length of the tangent component is fixed, its orientation is uniform over the unit sphere in $v^\perp$. This symmetry gives a direct derivation of the finite-dimensional fluctuation law.

Theorem: Under the Gaussian rank-one model and supercritical assumptions above, let $b_1,\ldots,b_k$ be any fixed orthonormal vectors in $v^\perp$, where $k$ does not grow with $p$. Then $$ \sqrt p\,(b_1^T\hat v,\ldots,b_k^T\hat v)^T \xrightarrow{d}N_k(0,(1-q)I_k). $$

Proof. Write $T=(I_p-vv^T)\hat v$ and $R=\|T\|$. With probability one, $R>0$ for the nondegenerate model. Rotational invariance about $v$ implies that, conditional on $R$, $T/R$ is uniform on the unit sphere of the $(p-1)$-dimensional space $v^\perp$. This is the unique rotation-invariant probability distribution on that sphere.

Let $G_1,\ldots,G_{p-1}$ be independent $N(0,1)$ random variables. A uniform vector on this sphere has the same distribution as a standard Gaussian vector divided by its length. In coordinates in an orthonormal basis of $v^\perp$, the first $k$ coordinates of $T$ therefore have the distribution $$ R\,\frac{(G_1,\ldots,G_k)^T}{\sqrt{\sum_{j=1}^{p-1}G_j^2}}. $$ The weak law of large numbers gives $(p-1)^{-1}\sum_{j=1}^{p-1}G_j^2\xrightarrow{p}1$, while the first $k$ Gaussian variables have exactly the $N_k(0,I_k)$ distribution. Also $R^2=1-(v^T\hat v)^2\xrightarrow{p}1-q$, so $R\xrightarrow{p}\sqrt{1-q}$. It follows that the displayed vector, multiplied by $\sqrt p$, converges in distribution to $\sqrt{1-q}(G_1,\ldots,G_k)^T$. This is $N_k(0,(1-q)I_k)$, as claimed. $\square$

The theorem describes any fixed number of tangent coordinates. It is not a claim that the whole $(p-1)$-dimensional tangent vector converges to a fixed-dimensional Gaussian vector. In fact, its length tends to a positive constant. The Gaussian limit arises because any fixed set of coordinates captures a vanishing fraction of the energy of a high-dimensional, uniformly oriented vector.

Turning Alignment into an Uncertainty Cone

The same alignment limit has a direct geometric interpretation. For unit vectors, define the angle between unoriented directions by $\angle(u,w)=\arccos(|u^Tw|)$, which does not depend on the arbitrary sign of an eigenvector. Then $\angle(\hat v,v)\xrightarrow{p}\theta$, where $\theta=\arccos(\sqrt q)$. In particular, the angle does not vanish. A confidence statement for the full population direction should reflect this nonvanishing uncertainty rather than imply high-dimensional consistency that the model does not provide.

Asymptotic confidence cone: For a unit-vector estimate $\hat v$ and radius $r\in[0,\pi/2]$, define the cone of unoriented directions $$ \mathcal C(\hat v,r)=\{u\in\mathbb R^p:\|u\|=1,\ \arccos(|u^T\hat v|)\leq r\}. $$ If $q$ is known and $r=\arccos(\sqrt q)+\varepsilon$, for fixed $\varepsilon>0$ small enough that $r<\pi/2$, then $P(v\in\mathcal C(\hat v,r))\to1$. The cone is conservative: its limiting coverage is one, not a calibrated finite-sample confidence level.

This construction follows from angle convergence: the random angle approaches $\arccos(\sqrt q)$, while the cone radius exceeds that limit by a fixed positive amount. If $q$ is unknown, it can be replaced by a consistent estimate. For example, write $\hat\lambda_1$ for the largest sample eigenvalue. The outlier limit is $\lambda_{\mathrm{out}}=(1+\beta)(1+\gamma/\beta)$. When $\gamma$ is known and the model assumptions hold, setting $y=\hat\lambda_1-1-\gamma$ and $\hat\beta=(y+\sqrt{y^2-4\gamma})/2$ gives a consistent estimate on the supercritical branch, with the formula defined whenever $y^2>4\gamma$. Then use $\hat q=(1-\gamma/\hat\beta^2)/(1+\gamma/\hat\beta)$ and radius $\arccos(\sqrt{\hat q})+\varepsilon$, restricting to cases where the estimates lie in their valid ranges. Consistency and the fixed slack again give asymptotic coverage one. The slack is important: this argument does not establish accurate coverage for a cone whose radius is merely the plug-in angle.

Worked Example: Building a Conservative Cone

Let $\gamma=1/2$ and $\beta=3/2$. The spike is supercritical because $3/2>\sqrt{1/2}$, and $$ q=\frac{1-(1/2)/(9/4)}{1+(1/2)/(3/2)} =\frac{1-2/9}{1+1/3} =\frac{7/9}{4/3} =\frac{7}{12}. $$ The limiting angle is $\arccos(\sqrt{7/12})$, about $40.2$ degrees. Choosing an additional margin of $3$ degrees gives a cone radius of about $43.2$ degrees; by angle convergence, its coverage tends to one. As a check on the spike inversion, the outlier eigenvalue limit is $(1+3/2)(1+(1/2)/(3/2))=(5/2)(4/3)=10/3$. Thus $y=10/3-1-1/2=11/6$, and $$ \frac{y+\sqrt{y^2-4\gamma}}{2} =\frac{11/6+\sqrt{121/36-2}}{2} =\frac{11/6+7/6}{2} =\frac32, $$ recovering the stated spike strength.

What the Fluctuations Do—and Do Not—Support

The tangent limit is useful for describing directional uncertainty in specified contrasts. If $b\in v^\perp$ is a fixed unit vector, then $\sqrt p\,b^T\hat v\xrightarrow{d}N(0,1-q)$. This gives a large-dimensional approximation to the dispersion of the estimated direction along $b$. For several orthonormal contrasts, the limiting coordinates are independent. These statements are theoretical descriptions relative to the population tangent space; they do not by themselves provide a practical interval for a contrast whose definition requires the unknown $v$.

For the full direction, the nonzero limiting angle is the central issue. The estimate retains a positive projection on the signal, but the tangent component has a positive limiting norm. Thus an uncertainty region for the population direction generally cannot shrink to an arbitrarily narrow cone in this regime. The conservative cone makes this limitation explicit. More precise, nominal finite-sample coverage requires additional distributional analysis and estimation of the model parameters; it does not follow from the fixed-coordinate Gaussian limit alone.

Common mistake: It is tempting to reason that a separated eigenvalue makes $\hat v$ consistent, or that the Gaussian limit of each fixed tangent coordinate means the total tangent error vanishes. Neither conclusion follows. Separation gives an outlier eigenvalue, while the squared alignment converges to $q<1$; at the same time, many small tangent coordinates combine to produce a tangent norm approaching $\sqrt{1-q}>0$. Eigenvalue fluctuation results and eigenvector fluctuation results concern different quantities.

Key takeaway: For a supercritical rank-one spike, fixed tangent projections of the sample eigenvector have a Gaussian limit after multiplication by $\sqrt p$, but the full direction retains a nonzero limiting angle from the population direction. Use the former for finite-dimensional tangent contrasts and the latter to calibrate nonshrinking uncertainty cones.

Check Your Understanding

Use the model assumptions and results stated above to answer the following questions.

For $\gamma=1$ and $\beta=2$, calculate $q$ and the limiting tangent length.
Why does a Gaussian limit for a fixed number of tangent coordinates not imply that the full tangent component vanishes?
State the limiting distribution of $\sqrt p\,b^T\hat v$ when $b$ is a fixed unit vector perpendicular to $v$.
Why is a fixed positive margin added to the limiting angle when constructing the conservative cone?
What is the difference between the limiting coverage of the cone described here and a nominal finite-sample confidence level?