Eigenvector Alignment Across the BBP Threshold

Why Eigenvalue Separation Is Not the Whole Story

A sample principal component can have an eigenvalue that separates from the noise spectrum and still fail to point exactly along the population direction that generated the spike. The relevant measure of alignment is the squared inner product between the population spike direction and the sample eigenvector. It is invariant to the arbitrary sign of an eigenvector, and it quantifies how much of the sample direction lies along the signal.

We use the rank-one spiked covariance model and the BBP eigenvalue phase transition. The central result is that, above the threshold, the overlap has a positive limit determined by the spike strength and the dimension ratio; at and below the threshold, that limit is zero. The derivation uses an exact finite-sample identity and the limiting sample eigenvalue, rather than deriving the eigenvalue equation.

Model and Overlap

Definition: Let $Z$ be a $p\times n$ matrix with independent $N(0,1)$ entries, let $v\in\mathbb{R}^p$ be a deterministic unit vector, and let $\beta>0$. The population covariance and sample covariance are $$ \Sigma=I_p+\beta vv^T,\qquad S=\frac{1}{n}\Sigma^{1/2}ZZ^T\Sigma^{1/2}. $$ The population eigenvalue in direction $v$ is $1+\beta$; all directions orthogonal to $v$ have population eigenvalue $1$. Write $\hat u$ for a unit eigenvector of $S$ corresponding to its largest eigenvalue. The squared overlap is $|v^T\hat u|^2$.

Assume $p,n\to\infty$ with $p/n\to\gamma\in(0,\infty)$. For this normalization, the BBP threshold for the excess spike strength is $\beta=\sqrt{\gamma}$. The eigenvalue phase transition gives the following limit for the largest sample eigenvalue:

\lambda_{\max}(S)\xrightarrow{a.s.} \begin{cases} (1+\sqrt{\gamma})^2, & 0<\beta\leq\sqrt{\gamma},\\[4pt] (1+\beta)\left(1+\dfrac{\gamma}{\beta}\right), & \beta>\sqrt{\gamma}. \end{cases}

The first limit is the upper edge of the Marchenko–Pastur bulk. Above the threshold, the largest eigenvalue has an outlier limit. The distinction between those eigenvalue limits will also control the eigenvector overlap.

Worked Example: A Supercritical Spike

Take $\gamma=1/4$ and $\beta=1$. The threshold is $\sqrt{\gamma}=1/2$, so the spike is supercritical. Its limiting largest sample eigenvalue is

(1+\beta)\left(1+\frac{\gamma}{\beta}\right) =2\left(1+\frac14\right)=\frac52.

The bulk edge is $(1+\sqrt{\gamma})^2=(1+1/2)^2=9/4$, so the limiting outlier $5/2$ is strictly above the edge. The overlap formula derived below gives

\frac{1-\gamma/\beta^2}{1+\gamma/\beta} =\frac{1-1/4}{1+1/4} =\frac{3/4}{5/4} =\frac35.

Thus the leading sample eigenvector retains a limiting squared projection of $0.6$ onto the population direction. It is aligned with the spike, but it is not identical to it.

An Exact Identity for the Overlap

The useful finite-sample representation is obtained by looking at the $n\times n$ matrix $C=S$'s dual Gram matrix. Put

B=\frac{1}{n}Z^TZ,\qquad z=\frac{1}{\sqrt n}Z^Tv,\qquad C(\beta)=\frac{1}{n}Z^T\Sigma Z=B+\beta zz^T.

The nonzero eigenvalues of $C(\beta)$ and $S$ agree. Let $\lambda_1(\beta)$ be the largest eigenvalue of $C(\beta)$, and let $q$ be a corresponding unit eigenvector. With probability one the largest eigenvalue is simple in this Gaussian model. The next identity connects its sensitivity to $\beta$ with the desired overlap.

Result: At any value of $\beta$ where the largest eigenvalue is simple, $$ \lambda_1'(\beta)=|z^Tq|^2,\qquad |v^T\hat u|^2=\frac{1+\beta}{\lambda_1(\beta)}\lambda_1'(\beta). $$

To prove the first equality, differentiate $C(\beta)q=\lambda_1(\beta)q$, using a locally differentiable unit eigenvector $q$. Since $q^Tq=1$, differentiation gives $q^Tq'=0$. Multiplying the differentiated eigenvalue equation on the left by $q^T$ therefore yields

q^TC'(\beta)q+q^TC(\beta)q' =\lambda_1'(\beta)+\lambda_1(\beta)q^Tq', \qquad \lambda_1'(\beta)=q^Tzz^Tq=|z^Tq|^2.

For the second equality, set $X=\Sigma^{1/2}Z$. The corresponding unit eigenvector of $S=XX^T/n$ is $\hat u=Xq/\sqrt{n\lambda_1(\beta)}$. Because $\Sigma^{1/2}v=\sqrt{1+\beta}\,v$,

v^T\hat u =\frac{\sqrt{1+\beta}\,v^TZq}{\sqrt{n\lambda_1(\beta)}} =\frac{\sqrt{1+\beta}\,z^Tq}{\sqrt{\lambda_1(\beta)}}.

Squaring and using $\lambda_1'(\beta)=|z^Tq|^2$ proves the identity. $\square$

Worked Example: Reading the Identity at Finite Sample Size

Suppose a particular realization has $\beta=1$, largest eigenvalue $\lambda_1(1)=2.5$, and eigenvalue derivative $\lambda_1'(1)=0.75$. The identity gives

|v^T\hat u|^2 =\frac{1+1}{2.5}(0.75) =\frac{2}{2.5}(0.75) =0.6.

This calculation is an exact relation for that realization, not an approximation that requires a large sample. In an asymptotic calculation, the task is to find the limit of the eigenvalue derivative.

Why the Eigenvalue Derivative Converges

Convergence of eigenvalues alone does not automatically imply convergence of derivatives. Here it does because $\lambda_1(\beta)$, as a function of $\beta$, is convex: for each unit vector $x$, $x^TC(\beta)x$ is affine in $\beta$, and the largest eigenvalue is the supremum of these affine functions.

Lemma: Suppose differentiable convex functions $f_n$ converge pointwise to a function $f$ on an open interval. If $f$ is differentiable at $\beta$, then $f_n'(\beta)\to f'(\beta)$, provided the pointwise convergence holds almost surely at every point in a countable sequence of neighborhoods used below.

Proof. For any sufficiently small $h>0$, convexity gives

\frac{f_n(\beta)-f_n(\beta-h)}{h} \leq f_n'(\beta) \leq \frac{f_n(\beta+h)-f_n(\beta)}{h}.

Take limits in the two secant slopes for a fixed $h$ at which pointwise convergence holds. The resulting lower and upper bounds are the corresponding secant slopes of $f$. Now take a sequence of such positive $h$'s decreasing to zero. Differentiability of $f$ at $\beta$ makes both secant slopes converge to $f'(\beta)$. The inequalities then force $f_n'(\beta)\to f'(\beta)$. For random functions, the countable sequence can be chosen in advance, and the almost-sure convergence events at those points can be intersected. $\square$

Apply the lemma to $f_n(\beta)=\lambda_1(\beta)$. The BBP eigenvalue limit is constant in $\beta$ on the subcritical interval, so its derivative there is zero. Above the threshold, differentiate the outlier limit:

\frac{d}{d\beta}\left[(1+\beta)\left(1+\frac{\gamma}{\beta}\right)\right] =\frac{d}{d\beta}\left(1+\beta+\gamma+\frac{\gamma}{\beta}\right) =1-\frac{\gamma}{\beta^2}.

At the threshold, the two pieces of the limiting eigenvalue function meet with derivative zero: the derivative of the outlier expression approaches $1-\gamma/(\sqrt{\gamma})^2=0$. Thus the derivative argument also gives a vanishing overlap at criticality.

The Limiting Alignment

Theorem: Under the rank-one Gaussian spiked covariance model above, with $p/n\to\gamma\in(0,\infty)$ and fixed $\beta>0$, the leading sample eigenvector satisfies $$ |v^T\hat u|^2\xrightarrow{a.s.} \begin{cases} 0, & 0<\beta\leq\sqrt{\gamma},\\[6pt] \dfrac{1-\gamma/\beta^2}{1+\gamma/\beta}, & \beta>\sqrt{\gamma}. \end{cases} $$

Proof. The exact identity gives $|v^T\hat u|^2=(1+\beta)\lambda_1'(\beta)/\lambda_1(\beta)$. Below the threshold, the limiting eigenvalue is constant, so the convex derivative lemma gives $\lambda_1'(\beta)\to0$. The eigenvalue limit is positive, and hence the overlap converges to zero. At the threshold, the limiting eigenvalue function is differentiable with derivative zero, giving the same conclusion.

Above the threshold, the derivative converges to $1-\gamma/\beta^2$, and the eigenvalue converges to $(1+\beta)(1+\gamma/\beta)$. Substitution yields

|v^T\hat u|^2 \longrightarrow \frac{(1+\beta)(1-\gamma/\beta^2)} {(1+\beta)(1+\gamma/\beta)} = \frac{1-\gamma/\beta^2}{1+\gamma/\beta}.

This expression is positive exactly when $\beta>\sqrt{\gamma}$, and it tends to zero as $\beta$ decreases to the threshold. $\square$

Worked Example: Subcritical and Critical Alignment

Let $\gamma=1/4$, so the threshold is $\beta=1/2$. First take $\beta=0.4$. Since $0.4<0.5$, the limiting largest eigenvalue is the bulk edge:

(1+\sqrt{\gamma})^2=(1+0.5)^2=2.25.

The limiting overlap is zero. This conclusion is not that the sample eigenvector is exactly orthogonal to $v$ at every finite $n$; it is that its squared projection converges to zero as the dimensions grow.

At the critical value $\beta=0.5$, the eigenvalue limit is still $2.25$. The supercritical overlap formula approaches

\frac{1-(1/4)/(1/2)^2}{1+(1/4)/(1/2)} =\frac{1-1}{1+1/2}=0.

Thus an eigenvalue at the transition does not come with persistent eigenvector alignment.

Worked Example: A Stronger Spike at a Larger Aspect Ratio

Take $\gamma=1$ and $\beta=2$. The threshold is $1$, so the spike is supercritical. The limiting outlier eigenvalue is

(1+2)\left(1+\frac{1}{2}\right)=\frac{9}{2},

while the bulk edge is $(1+1)^2=4$. The limiting squared overlap is

\frac{1-1/2^2}{1+1/2} =\frac{3/4}{3/2} =\frac12.

Even though the sample eigenvalue is separated from the bulk, half of the squared unit-vector length is not aligned with the population direction. The remaining component lies in directions orthogonal to $v$.

Interpretation and a Common Pitfall

The overlap formula makes the role of both signal strength and dimension ratio explicit. For fixed $\gamma$, the overlap is zero at and below the threshold, then increases as the spike strengthens. Increasing $\gamma=p/n$ raises the threshold: when the number of coordinates grows quickly relative to the sample size, a stronger population spike is needed for the leading sample eigenvector to retain nonzero asymptotic alignment.

Common mistake: “The largest sample eigenvalue is an outlier, so its eigenvector must point almost exactly in the population spike direction.” The first part identifies a supercritical spike, but the conclusion does not follow. For example, with $\gamma=1$ and $\beta=2$, the eigenvalue is an outlier while the limiting squared overlap is only $1/2$. The correct statement is that a fixed supercritical spike produces a positive limiting overlap given by the formula; perfect alignment requires a much stronger signal regime, not merely eigenvalue separation.

Another mistake is to use the population eigenvalue $1+\beta$ as the threshold parameter. In this convention the threshold is on the excess $\beta$: the population eigenvalue must exceed $1+\sqrt{\gamma}$. Keeping those two quantities distinct prevents a common off-by-one error.

Key takeaway: The eigenvalue phase transition has an eigenvector counterpart. Above the BBP threshold, the limiting squared overlap is $(1-\gamma/\beta^2)/(1+\gamma/\beta)$; at and below the threshold, the leading sample eigenvector's squared overlap with the population spike direction vanishes.

Check Your Understanding

Use the model and conventions defined above.

For $\gamma=1/9$ and $\beta=1/2$, determine whether the spike is supercritical and calculate the limiting squared overlap.
Explain why the dual matrix $C(\beta)$ has the same nonzero eigenvalues as the sample covariance $S$.
In the exact overlap identity, what quantity does the derivative $\lambda_1'(\beta)$ represent?
Why does the constant subcritical eigenvalue limit imply a vanishing overlap, rather than merely a sample eigenvalue with no outlier?
For $\gamma=1$, what is the limiting squared overlap when $\beta=3$?