Worked Analysis of a Spiked Covariance Spectrum

Why a Spiked Spectrum Needs a High-Dimensional Benchmark

In principal component analysis, a large sample eigenvalue can suggest a dominant population direction. But when the dimension is comparable to the sample size, even data with no population signal produce large sample eigenvalues. The correct question is therefore not simply whether the largest sample eigenvalue exceeds one: it is whether it separates from the noise spectrum expected at that dimension-to-sample-size ratio.

A rank-one Gaussian spike gives a precise setting in which to make this comparison. We will calculate the noise bulk edge, identify the signal-strength threshold for a separated eigenvalue, and quantify how well the corresponding sample eigenvector recovers the population direction. All limits below are as \(p,n\to\infty\) with \(p/n\to\gamma\), where \(0<\gamma<\infty\).

The Noise Bulk and the Outlier Threshold

When \(\beta=0\), the sample covariance is a Gaussian Wishart matrix with identity population covariance. The Marchenko–Pastur law describes its limiting empirical eigenvalue distribution. Its upper support edge is

This edge is the high-dimensional noise benchmark. The sample largest eigenvalue approaches \(b_+\) in the identity model, rather than approaching the population eigenvalue \(1\). In the spiked model, the population spike must be sufficiently strong to pull a sample eigenvalue strictly above this edge.

The transition says that a population eigenvalue greater than one is not automatically detectable as a separated sample eigenvalue. The comparison is between the excess spike \(\beta=\ell-1\) and \(\sqrt{\gamma}\), not between \(\ell\) and the sample bulk edge.

Deriving the Supercritical Eigenvalue

The outlier formula follows from a one-dimensional Schur complement. By rotating coordinates, take \(v\) to be the first coordinate vector. Write the data matrix, whose columns are the observations, as \(X=\begin{pmatrix}\sqrt{\ell}\,z^\top\\Y\end{pmatrix}\), where \(z\in\mathbb{R}^n\) has independent standard normal entries and \(Y\) is a \((p-1)\times n\) matrix of independent standard normal entries, independent of \(z\). Partition \(S\) as

For a candidate eigenvalue \(\lambda\) above the spectrum of \(C\), the eigenvalue equation reduces to \(\lambda=a+b^\top(\lambda I-C)^{-1}b\). The law of large numbers gives \(a\xrightarrow{p}\ell\). Conditional on \(Y\), the Gaussian quadratic form in the second term concentrates around its trace. Since \(C=YY^\top/n\), that trace gives

where \(F_\gamma\) is the Marchenko–Pastur limiting law for the eigenvalues of \(C\), normalized to have total mass one. Thus the limiting outlier equation is \(\lambda=\ell\{1+q(\lambda)\}\), where \(q(\lambda)=\gamma\int x/(\lambda-x)\,dF_\gamma(x)\).

For clarity, the transform identity needed to solve this equation can be written in terms of \(m(\lambda)=\int(\lambda-x)^{-1}\,dF_\gamma(x)\). For \(\lambda>b_+\), the Marchenko–Pastur transform satisfies \(\lambda=1/m+1/(1-\gamma m)\). Set \(s=\lambda m-1\). Substitution gives

The outlier equation requires \(\lambda=\ell(1+\gamma s)\). Equating this with the transform expression yields \(s=1/\beta\); substituting gives \(\lambda=(1+\beta)(1+\gamma/\beta)\). This solution lies on the physical branch above the bulk edge precisely when \(1/\beta<1/\sqrt{\gamma}\), or \(\beta>\sqrt{\gamma}\). At the threshold it reaches the edge; below it, there is no separated solution.

Deriving the Eigenvector Overlap

The same block representation explains why an outlier eigenvector can recover the signal direction. Write an eigenvector for an eigenvalue \(\lambda\) as \((u,w^\top)^\top\), where \(u\) is its coordinate along \(v\). The lower block of the eigenvalue equation gives \(w=(\lambda I-C)^{-1}bu\). Normalizing the eigenvector therefore yields

The quadratic form in the denominator converges, by the same conditional Gaussian concentration, to \(\ell\gamma\int x/(\lambda-x)^2\,dF_\gamma(x)=-\ell q'(\lambda)\). To evaluate it, use \(q=\gamma s\) and \(\lambda=(1+s)(\gamma+1/s)\). Differentiating gives \(d\lambda/ds=\gamma-1/s^2\), and hence \(q'(\lambda)=\gamma/(\gamma-1/s^2)\). At the outlier, \(s=1/\beta\), so \(q'(\lambda)=-\gamma/(\beta^2-\gamma)\). It follows that

Worked Analyses

Common Mistake: Comparing the Population Spike to the Noise Edge

A tempting but incorrect argument is: “The population eigenvalue is \(\ell=1+\beta\), so compare it directly with the sample noise edge \(b_+=(1+\sqrt{\gamma})^2\). If \(\ell<b_+\), the spike cannot be detected.” This compares a population eigenvalue with a sample eigenvalue, which are on different scales in the transition calculation. For instance, in the first worked example, \(\ell=2<2.25=b_+\), yet the sample outlier converges to \(2.5\), above the edge.

The correction is to compare the population excess \(\beta=\ell-1\) with \(\sqrt{\gamma}\). If \(\beta>\sqrt{\gamma}\), calculate the sample outlier using \((1+\beta)(1+\gamma/\beta)\), then compare that result with the bulk edge. If \(\beta\leq\sqrt{\gamma}\), the largest sample eigenvalue converges to the edge and the limiting overlap is zero.

What the Calculation Says About Recovery

The threshold depends on both the signal strength and the aspect ratio. Holding \(\beta\) fixed while increasing \(\gamma\) raises the threshold; a spike that separates when the sample size is large relative to dimension can merge into the bulk when dimension grows relative to sample size. Above threshold, the outlier location gives a spectral indication of the spike, while the overlap formula measures how much of its direction is recovered.

These are asymptotic limits, not exact finite-sample identities. In finite data, eigenvalues fluctuate around their limits, especially near the critical boundary. A value just above the theoretical edge is not by itself conclusive evidence of a supercritical population spike. The model, the dimension-to-sample-size ratio, and the eigenvector alignment all matter when interpreting a spectrum.