Marchenko–Pastur Limits for Sample Covariance Spectra

Why the Sample Spectrum Spreads

When the population covariance is the identity matrix, every population eigenvalue equals one. It is tempting to expect the eigenvalues of the sample covariance matrix to lie close to one as well. That expectation can fail when the number of variables is a substantial fraction of the sample size: random sampling alone can spread the sample eigenvalues across a wide interval, and if there are more variables than observations, it also creates exact zero eigenvalues.

The Marchenko–Pastur limit describes this phenomenon. Its key parameter is the ratio of dimension to sample size. In high-dimensional problems, that ratio can remain non-negligible as both dimension and sample size grow, so the limiting spectrum is not concentrated at the population eigenvalue.

Setup and the Limiting Law

Definition: Let \(X_1,\ldots,X_n\) be independent random vectors in \(\mathbb{R}^p\), each with mean zero and covariance \(I_p\). Form the uncentered sample covariance matrix \(S=\frac{1}{n}\sum_{j=1}^n X_jX_j^\mathsf{T}\). Write its eigenvalues, including multiplicities, as \(\lambda_1,\ldots,\lambda_p\). The empirical spectral distribution (ESD) of \(S\) is the probability measure \(F_p^S=\frac{1}{p}\sum_{i=1}^p\delta_{\lambda_i}\), where \(\delta_x\) is a unit point mass at \(x\).

The ESD assigns equal weight \(1/p\) to each eigenvalue. It therefore records the overall distribution of sample eigenvalues, rather than tracking any one eigenvalue. To state its large-dimensional limit, suppose the coordinates of the \(X_j\)'s are independent and identically distributed with mean zero, variance one, and finite fourth moment. Consider a sequence of problems in which \(p,n\) both tend to infinity and \(p/n\) tends to a fixed number \(\gamma\) with \(0<\gamma<\infty\).

Theorem (Marchenko–Pastur limit): Under the conditions just stated, \(F_p^S\) converges almost surely in distribution to the Marchenko–Pastur law with aspect ratio \(\gamma\). Define \(a_\gamma=(1-\sqrt{\gamma})^2\) and \(b_\gamma=(1+\sqrt{\gamma})^2\). The limiting law has density \(f_\gamma(x)=\frac{\sqrt{(b_\gamma-x)(x-a_\gamma)}}{2\pi\gamma x}\) for \(a_\gamma\leq x\leq b_\gamma\), and density zero outside this interval. It also has an atom at zero of size \((1-1/\gamma)_+=\max(1-1/\gamma,0)\).

The continuous part has total mass \(\min(1,1/\gamma)\), so the atom and density together make a probability distribution. When \(\gamma<1\), there is no limiting mass at zero. When \(\gamma>1\), the atom has mass \(1-1/\gamma\); the density then accounts for the remaining mass \(1/\gamma\). At \(\gamma=1\), the support begins at zero, but there is no atom there. The density is unbounded near zero in that case, yet its singularity is integrable.

The support endpoints follow directly from the density: the square root is positive precisely between \(a_\gamma\) and \(b_\gamma\), except at the endpoints themselves. Thus the limiting continuous spectrum occupies the interval from \((1-\sqrt{\gamma})^2\) to \((1+\sqrt{\gamma})^2\). For \(\gamma=1\), the density behaves like a constant times \(x^{-1/2}\) near zero, which is integrable; a diverging density at a point does not imply a point mass there.

Rank and the Zero-Eigenvalue Mass

The atom at zero has a simple finite-dimensional explanation. Write \(Z\) for the \(p\times n\) matrix whose columns are \(X_1,\ldots,X_n\), so that \(S=ZZ^\mathsf{T}/n\). The next fact gives the exact zero count whenever \(Z\) has its largest possible rank.

Proposition: If \(\operatorname{rank}(Z)=\min(p,n)\), then the fraction of eigenvalues of \(S\) equal to zero is \((1-n/p)_+\). If \(p/n\to\gamma\), this fraction tends to \((1-1/\gamma)_+\).

Proof. For any vector \(v\in\mathbb{R}^p\), \(v^\mathsf{T}Sv=\|Z^\mathsf{T}v\|^2/n\), so \(Sv=0\) if and only if \(Z^\mathsf{T}v=0\). Thus \(\operatorname{nullity}(S)=p-\operatorname{rank}(Z)\). Under the stated rank condition, this is \(p-\min(p,n)=(p-n)_+\). Dividing by \(p\) gives \((1-n/p)_+\), which tends to \((1-1/\gamma)_+\) when \(p/n\to\gamma\). \(\square\)

For example, a matrix with independent standard normal entries has rank \(\min(p,n)\) with probability one. This follows because at least one maximal square minor is a nonzero polynomial in the entries, and the zero set of a nonzero polynomial has probability zero under a joint density. For such data, the finite-sample zero fraction already equals the limiting atom formula with the finite ratio \(p/n\). The Marchenko–Pastur theorem describes the distribution of the remaining eigenvalues as well.

Worked Example: More Variables Than Observations

Suppose \(p=600\) and \(n=300\), with independent standard normal coordinates. The aspect ratio is \(\gamma=600/300=2\). Since \(p>n\), the rank is 300 with probability one, so \(600-300=300\) eigenvalues are exactly zero. Their fraction is \(300/600=1/2\).

The limiting atom is \((1-1/2)_+=1/2\), matching the rank calculation. The continuous support has endpoints \(a_2=(1-\sqrt{2})^2\approx0.1716\) and \(b_2=(1+\sqrt{2})^2\approx5.8284\). Thus half of the eigenvalues are zero, while the nonzero eigenvalues have an asymptotic distribution across this positive interval. The population covariance is still \(I_p\): the zeros are caused by insufficient sample rank, not by zero population variances.

What the Support Says About Eigenvalue Estimation

When the population covariance is \(I_p\), its entire spectrum is the single value one. The Marchenko–Pastur support shows why a sample eigenvalue is not automatically a reliable estimate of its corresponding population eigenvalue when \(p/n\) stays away from zero. For \(\gamma<1\), the limiting sample spectrum extends below and above one. For \(\gamma>1\), it includes a positive fraction of exact zeros and its nonzero portion still spreads. Only in a regime where \(p/n\) tends to zero do both support endpoints tend to one.

The following finite-sample identity is useful context. It concerns the average of all sample eigenvalues, not their individual locations.

Proposition: Under the setup above, \(E[\operatorname{tr}(S)/p]=1\).

Proof. The trace of a matrix is the sum of its diagonal entries and, for a symmetric matrix, also the sum of its eigenvalues. Therefore \(\operatorname{tr}(S)=\frac{1}{n}\sum_{j=1}^n\sum_{k=1}^p X_{kj}^2\), where \(X_{kj}\) is coordinate \(k\) of observation \(j\). Each coordinate has variance one and mean zero, so \(E[X_{kj}^2]=1\). Taking expectations gives \(E[\operatorname{tr}(S)]=np/n=p\), and division by \(p\) proves the claim. \(\square\)

An average of one does not mean that each eigenvalue is near one. In particular, an average alone does not describe the spread or the proportion of eigenvalues at zero. The limiting law supplies that distributional information.

Worked Example: A Ratio Below One

Let \(p/n\to0.25\), so \(\gamma=0.25\) and \(\sqrt{\gamma}=0.5\). The support endpoints are \(a_{0.25}=(1-0.5)^2=0.25\) and \(b_{0.25}=(1+0.5)^2=2.25\).

There is no atom at zero because \((1-1/0.25)_+=0\). Although the population eigenvalues are all one, the limiting sample spectrum extends from 0.25 to 2.25. This is not a statement that every finite-sample eigenvalue must lie inside those endpoints; the theorem concerns convergence of the empirical distribution. It does show that, at a fixed nonzero aspect ratio, random sampling produces substantial bulk spread even under identity covariance.

Worked Example: Equal Dimension and Sample Size

Suppose \(p/n\to1\). Then \(\gamma=1\), \(a_1=(1-1)^2=0\), and \(b_1=(1+1)^2=4\). There is no atom at zero because \((1-1)_+=0\). The density on \(0<x<4\) is \(f_1(x)=\frac{\sqrt{(4-x)x}}{2\pi x}\).

As \(x\) decreases toward zero, \(\sqrt{(4-x)x}/(2\pi x)\) behaves like \(1/(\pi\sqrt{x})\). Its integral over a small interval \((0,\varepsilon)\) is finite, since \(\int_0^\varepsilon x^{-1/2}\,dx=2\sqrt{\varepsilon}\). Thus the limiting law can have many eigenvalues near zero without assigning positive probability to the single point zero. This distinction separates a continuous pile-up near zero from exact rank deficiency.

Reading the Limit Carefully

The Marchenko–Pastur law is a baseline for understanding sample eigenvalues under identity covariance. In a real analysis, a spread sample spectrum may reflect sampling noise even when population eigenvalues are all equal. Consequently, interpreting the largest or smallest sample eigenvalues as direct measurements of population eigenvalues can be misleading. Methods that adjust or regularize eigenvalues address this gap; the limit explains why such adjustment may be needed, without specifying an estimator.

The ratio \(\gamma\) also determines whether zero eigenvalues are structural. With \(p>n\), the sample covariance has rank at most \(n\), so at least \(p-n\) eigenvalues vanish. This is different from a small but positive eigenvalue, which may arise from the continuous part of the spectrum. In finite samples, the exact zero count depends on the rank of the data matrix; for continuous models such as Gaussian data, it reaches the maximum possible rank almost surely.

Common Mistake

Wrong reasoning: “The population covariance is the identity, so all sample covariance eigenvalues should be close to one; any large or small eigenvalue must indicate a real population feature.”

Correction: That conclusion ignores the aspect ratio. When \(p/n\) approaches a positive constant, the Marchenko–Pastur limit predicts a non-degenerate interval of sample eigenvalues even under identity covariance, and when \(p/n>1\) it predicts a zero-eigenvalue fraction. The sample spectrum must be interpreted relative to this high-dimensional sampling baseline.

Key takeaway: Under identity covariance, the limiting sample spectrum is governed by \(\gamma=p/n\): its continuous support is \([(1-\sqrt{\gamma})^2,(1+\sqrt{\gamma})^2]\), and its zero atom has mass \((1-1/\gamma)_+\). The population spectrum and the sample spectrum need not look alike.

Check Your Understanding

Use the Marchenko–Pastur law and rank reasoning to answer the following.

For \(\gamma=4\), calculate both support endpoints and the limiting zero-eigenvalue mass.
Why can the limiting density be unbounded near zero when \(\gamma=1\) without having an atom at zero?
If \(p=80\) and \(n=50\), how many zero eigenvalues occur when the data matrix has full possible rank, and what fraction is that?
Explain why an expected average sample eigenvalue of one does not imply that the eigenvalues are individually close to one.
How do the support endpoints behave as \(\gamma\) tends to zero, and what does that suggest about the sample spectrum in that regime?