Resolvent Derivation of the Outlier Equation

Why an Outlier Equation Is Useful

In a spiked covariance model, a small number of population directions have larger variance than the background. The sample eigenvalues do not simply reproduce those population variances: the noise changes their locations, and a weak spike may not produce a sample eigenvalue separated from the bulk. The outlier equation describes the location of a separated sample eigenvalue by combining two ingredients: a finite-dimensional determinant identity and the limiting resolvent of the noise covariance.

We derive the equation for a fixed number of population spikes as the dimension grows. The derivation makes clear what the equation does—and does not—say: a root represents an outlier only when it lies above the limiting noise spectrum.

Model and Resolvent

Let \(p,n\to\infty\) with \(p/n\to\gamma\), where \(0<\gamma<\infty\). Let \(Z\) be a \(p\times n\) matrix with independent \(N(0,1)\) entries, and define the white-noise sample covariance matrix \(W=ZZ^T/n\). Let \(U\) be a deterministic \(p\times r\) matrix with orthonormal columns, where \(r\) is fixed, and let \(\Theta=\operatorname{diag}(\theta_1,\ldots,\theta_r)\), with each \(\theta_j>0\). The population covariance is \(\Sigma=I_p+U\Theta U^T\), so its spiked eigenvalues are \(1+\theta_j\). The observed sample covariance is \(S=\Sigma^{1/2}W\Sigma^{1/2}\).

For \(\lambda\) above the largest eigenvalue of \(W\), the noise resolvent is \((\lambda I_p-W)^{-1}\). The particular scalar quantity that enters the outlier equation is the weighted resolvent \(g_p(\lambda)=p^{-1}\operatorname{tr}\{W(\lambda I_p-W)^{-1}\}\). It weights each noise eigenvalue \(t\) by \(t/(\lambda-t)\), rather than using the unweighted resolvent alone.

Exact Finite-Sample Determinant Reduction

The covariance form \(S\) is convenient for describing the model, but its eigenvalues are more directly analyzed after moving the population perturbation onto the noise matrix. Set \(T=W^{1/2}\Sigma W^{1/2}\). The matrices \(S\) and \(T\) have the same eigenvalues: they are respectively \(AA^T\) and \(A^TA\) for \(A=\Sigma^{1/2}W^{1/2}\), a square matrix. Since \(\Sigma=I_p+U\Theta U^T\), \(T=W+W^{1/2}U\Theta U^TW^{1/2}\), a rank-at-most-\(r\) perturbation of \(W\).

Proof. Write \(H=W^{1/2}U\). Then \(T=W+H\Theta H^T\). When \(\lambda I_p-W\) is invertible, factor \(\lambda I_p-W\) out of \(\lambda I_p-T\) and use \(\det(I+AB)=\det(I+BA)\):

Substituting \(H=W^{1/2}U\) gives the stated identity. Above the spectrum of \(W\), the first determinant is nonzero, so an eigenvalue of \(T\) there must make the \(r\times r\) determinant vanish. \(\square\)

This is an exact identity at finite \(p,n\), not an asymptotic approximation. The reduction is useful because the determinant whose zeros matter has fixed dimension \(r\), even as the sample covariance dimension grows.

From the Noise Resolvent to the Secular Equation

For a fixed \(\lambda>b_\gamma\), the matrix inside the reduced determinant is a quadratic form in the columns of \(U\). In the Gaussian white-noise model, the eigenvectors of \(W\) are rotationally invariant. Their quadratic forms in any fixed finite collection of orthonormal directions concentrate around the normalized trace. Thus, for fixed \(r\),

The Marchenko–Pastur law supplies the limit of that normalized trace. If \(m_\gamma(\lambda)=\int(\lambda-t)^{-1}\,dF_\gamma(t)\), then \(g(\lambda)=\lambda m_\gamma(\lambda)-1\). For \(\lambda>b_\gamma\), the Marchenko–Pastur transform gives

The square root is taken positive for \(\lambda>b_\gamma\). Substitution yields \(g(\lambda)=\{\lambda-\gamma-1-\sqrt{(\lambda-a_\gamma)(\lambda-b_\gamma)}\}/(2\gamma)\). Equivalently, \(g\) is the positive solution on the upper spectral branch of

For clarity, this relation follows by substituting \(m_\gamma=(1+g)/\lambda\) into the Marchenko–Pastur transform equation \(\gamma\lambda m_\gamma^2-(\lambda+\gamma-1)m_\gamma+1=0\). After multiplication by \(\lambda\) and simplification, the equation becomes \(1+(1+\gamma-\lambda)g+\gamma g^2=0\), which rearranges to the displayed relation.

Derivation and location check. Replacing the quadratic-form matrix in the exact determinant by its limit gives \(\det(I_r-\Theta g(\lambda)I_r)=\prod_{j=1}^r(1-\theta_jg(\lambda))\). Thus a separated eigenvalue associated with a spike \(\theta\) must solve \(g(\lambda)=1/\theta\). Inserting this into the transform relation gives \(\lambda=1+\gamma+\theta+\gamma/\theta=(1+\theta)(1+\gamma/\theta)\).

It remains to check whether this algebraic solution is outside the bulk. On \(0<g<1/\sqrt{\gamma}\), the function \(\lambda(g)=1+\gamma+1/g+\gamma g\) is strictly decreasing, because \(\lambda'(g)=-1/g^2+\gamma<0\). It maps that interval onto \((b_\gamma,\infty)\): as \(g\) approaches \(0\), \(\lambda(g)\) diverges, and as \(g\) increases to \(1/\sqrt{\gamma}\), \(\lambda(g)\) decreases to \(b_\gamma\). Therefore \(g=1/\theta\) lies on this branch exactly when \(\theta>\sqrt{\gamma}\). At equality the solution is the edge, not a separated outlier.

For a rank-one spike, the root also describes the limiting sample eigenvalue, not just a necessary algebraic condition. For any fixed \(\lambda>b_\gamma\), the finite-sample quantity \(f_p(\lambda)=u^TW^{1/2}(\lambda I_p-W)^{-1}W^{1/2}u\) converges in probability to \(g(\lambda)\). When \(\theta>\sqrt{\gamma}\), choose \(\lambda_-<\lambda_{\mathrm{out}}<\lambda_+\), both above the edge. Since \(g\) is strictly decreasing, \(\theta g(\lambda_-)>1\) and \(\theta g(\lambda_+)<1\). With probability tending to one, the same inequalities hold with \(f_p\) in place of \(g\). Continuity then gives a root of \(1-\theta f_p(\lambda)=0\) between \(\lambda_-\) and \(\lambda_+\). The determinant identity makes this root an eigenvalue of \(T\); shrinking the bracket around \(\lambda_{\mathrm{out}}\) proves convergence in probability to that location. \(\square\)

Worked Examples

Common Pitfall and Interpretation

A common mistake is to solve \(1-\theta g(\lambda)=0\) algebraically and automatically call the resulting value an outlier. The equation was derived for \(\lambda\) outside the noise spectrum, where the resolvent is well behaved and the limiting transform is on its upper branch. A root that does not satisfy \(\lambda>b_\gamma\) is not a separated eigenvalue prediction. The correction is to check the branch and edge condition after solving: \(\theta>\sqrt{\gamma}\) is exactly the condition for a root strictly above the edge.

The reduction also explains why the noise resolvent matters. The population spike enters through \(\theta\), while the high-dimensional sampling environment enters through \(g(\lambda)\). Their balance determines whether a sample eigenvalue separates from the bulk and, if so, its limiting location. The determinant identity is exact; the scalar equation arises only after taking the high-dimensional limit.