Cross-Sectional Dependence in Factor-Model Noise

Why Cross-Sectional Dependence Changes the PCA Picture

In a high-dimensional factor model, each observation is a vector of measurements. The factor component creates shared variation across coordinates, while the idiosyncratic component captures what is left over. “Idiosyncratic” does not mean independent across coordinates: measurement errors, local shocks, or common unmodelled effects can make different coordinates move together. Such dependence changes the noise covariance spectrum and can create large eigenvalues that look like factors.

The key comparison is between the scale of the factor eigenvalues and the scale of the largest noise eigenvalues. If a fixed number of factors have eigenvalues proportional to the number of coordinates, while the noise covariance has bounded operator norm, the factor directions remain spectrally separated. If the noise itself has eigenvalues of the same order, ordinary PCA may instead select a noise direction. The relevant condition is therefore not that noise coordinates be uncorrelated, but that their dependence does not create competing eigenvalues.

Factor Model and Noise Spectrum

Definition: A (static) factor model for a $p$-dimensional observation $X_t$ is $$ X_t=\Lambda F_t+U_t, $$ where $F_t\in\mathbb{R}^r$ is a vector of $r$ factors, $\Lambda\in\mathbb{R}^{p\times r}$ is the loading matrix, and $U_t\in\mathbb{R}^p$ is the idiosyncratic component. Assume $E[F_t]=E[U_t]=0$ and $\operatorname{Cov}(F_t,U_t)=0$. Write $\Sigma_F=\operatorname{Cov}(F_t)$ and $\Sigma_U=\operatorname{Cov}(U_t)$. Then $$ \Sigma_X=\operatorname{Cov}(X_t)=A+\Sigma_U,\qquad A=\Lambda\Sigma_F\Lambda^\top. $$ Cross-sectional dependence is allowed: $\Sigma_U$ need not be diagonal.

The signal matrix $A$ is positive semidefinite and has rank at most $r$. When $\Sigma_F$ is positive definite and $\Lambda$ has full column rank, its range is the factor loading space $\mathcal{M}(\Lambda)$. In a pervasive-factor regime, the nonzero eigenvalues of $A$ grow proportionally to $p$. For precision, suppose $r$ is fixed and there are constants $c,C>0$, not depending on $p$, such that $$ cp\leq \lambda_r(A)\leq\lambda_1(A)\leq Cp. $$ Here $\lambda_j(M)$ denotes the $j$th largest eigenvalue of a symmetric matrix $M$.

With independent equal-variance noise, familiar high-dimensional bulk approximations may apply under additional distributional assumptions. With cross-sectional dependence, there is no general reason for the noise eigenvalues to follow that same bulk law. A structured covariance can produce a different bulk, isolated noise eigenvalues, or both. The matrix $\Sigma_U$, rather than the label “noise,” determines the population noise spectrum.

Worked Example: Correlated Errors in Small Blocks

Let $p=1200$, and partition the coordinates into blocks of size $m=4$. In each block let the noise covariance be $$ B=I_4+0.2(\mathbf{1}\mathbf{1}^\top-I_4), $$ with zero covariance between different blocks. Each diagonal entry is $1$, and each within-block off-diagonal entry is $0.2$; thus the errors are correlated. The eigenvalue in the all-ones direction of a block is $1+3(0.2)=1.6$. Each of the other three block eigenvalues is $1-0.2=0.8$. Consequently, $\|\Sigma_U\|_{\mathrm{op}}=1.6$, regardless of the number of blocks.

Now take one factor with covariance $1$ and loading vector $\lambda=\sqrt{6p}\,v$, where $v$ is a unit vector. The signal is $A=6p\,vv^\top$, so its only nonzero eigenvalue is $6p=7200$. The factor eigenvalue grows with $p$, whereas every noise eigenvalue is at most $1.6$. The coordinates of the noise are not independent, but the noise cannot create an eigenvalue on the factor’s scale.

Eigenvalue Separation with Bounded Noise

The following result makes the scale comparison explicit. It uses the variational characterization of eigenvalues: for a symmetric matrix, the $j$th eigenvalue can be characterized by optimizing its Rayleigh quotient over subspaces of the appropriate dimension.

Theorem: Suppose $A$ is positive semidefinite with rank $r$, $\lambda_r(A)\geq cp$, and $\Sigma_U$ is positive semidefinite with $\|\Sigma_U\|_{\mathrm{op}}\leq C$. For $\Sigma_X=A+\Sigma_U$, $$ \lambda_r(\Sigma_X)\geq cp,\qquad \lambda_{r+1}(\Sigma_X)\leq C. $$ Thus the gap between the $r$th and $(r+1)$st population eigenvalues is at least $cp-C$.

Proof. For any unit vector $x$, $x^\top\Sigma_Ux\geq0$, so adding $\Sigma_U$ cannot decrease any eigenvalue; in particular, $\lambda_r(\Sigma_X)\geq\lambda_r(A)\geq cp$. For the upper bound, use the min–max characterization of $\lambda_{r+1}$: choose the subspace $\mathcal{M}(A)^\perp$, which has codimension $r$. For every unit $x$ in that subspace, $Ax=0$, and hence $$ x^\top\Sigma_Xx=x^\top\Sigma_Ux\leq\|\Sigma_U\|_{\mathrm{op}}\leq C. $$ The characterization gives $\lambda_{r+1}(\Sigma_X)\leq C$. Subtracting the bounds proves the stated gap. $\square$

This theorem does not say that the leading population eigenvectors equal the loading directions exactly. If $\Sigma_U$ has off-diagonal components connecting the signal space and its orthogonal complement, it can rotate those eigenvectors. But when the factor eigenvalues are of order $p$ and the noise operator norm is bounded, that rotation is small: a standard Davis–Kahan eigenspace perturbation bound gives an angle of order $C/(cp-C)$, which tends to zero.

When Sample PCA Estimates the Factor Space

Let $X_1,\ldots,X_n$ be independent, identically distributed, mean-zero observations from the factor model, and let $S=n^{-1}\sum_{t=1}^n X_tX_t^\top$ be their sample covariance. (The same conclusions apply to a centered sample covariance when estimation of the mean contributes a negligible perturbation.) Let $\widehat{\mathcal{V}}_r$ be the span of the top $r$ eigenvectors of $S$. The population factor space is $\mathcal{M}(\Lambda)=\mathcal{M}(A)$ under the rank assumptions above.

Result: Suppose $r$ is fixed, $\lambda_r(A)\geq cp$, and $\|\Sigma_U\|_{\mathrm{op}}\leq C$. If $$ \|S-\Sigma_X\|_{\mathrm{op}}=o_p(p), $$ then the largest principal angle between $\widehat{\mathcal{V}}_r$ and $\mathcal{M}(\Lambda)$ converges to zero in probability. In particular, ordinary PCA consistently estimates the factor space even though the idiosyncratic errors may be cross-sectionally dependent.

Justification. The previous eigenvalue bounds give a population gap of at least $cp-C$, which is of order $p$. The Davis–Kahan perturbation bound applied to $S$ and $\Sigma_X$ therefore bounds the distance between their leading $r$-dimensional eigenspaces by a constant times $\|S-\Sigma_X\|_{\mathrm{op}}/p$. This tends to zero in probability by assumption. Applying the same bound to $\Sigma_X=A+\Sigma_U$ and $A$, whose signal gap is at least $cp-C$, bounds the distance between the leading population eigenspace and $\mathcal{M}(A)$ by a constant times $C/p$. This also tends to zero. The triangle inequality for the corresponding subspace distances proves the claim.

The sample perturbation condition is explicit and important: population separation alone does not ensure that an estimated eigenspace is accurate if the sample covariance is too noisy. One familiar sufficient regime is fixed $r$, $p/n=O(1)$, independent sub-Gaussian factors and idiosyncratic vectors, factors independent of the idiosyncratic vectors, $\|\Lambda\|_{\mathrm{op}}=O(\sqrt p)$, and uniformly bounded $\|\Sigma_U\|_{\mathrm{op}}$ and factor covariance eigenvalues. Under these conditions, standard covariance concentration applied to the noise term, the fixed-dimensional factor covariance, and the factor–noise cross terms yields $\|S-\Sigma_X\|_{\mathrm{op}}=o_p(p)$. More restrictive dependence across observations can also be allowed, but requires conditions that control the same sample perturbation.

Worked Example: A Large Noise Spike That Does Not Win

Let $v,w\in\mathbb{R}^p$ be orthogonal unit vectors. Suppose the signal and noise covariances are $$ A=5p\,vv^\top,\qquad \Sigma_U=2p\,ww^\top+I_p. $$ The signal eigenvalue in direction $v$ is $5p$. The noise eigenvalue in direction $w$ is $2p+1$, while each direction orthogonal to both $v$ and $w$ has noise eigenvalue $1$. Since $v$ and $w$ are orthogonal, $\Sigma_X=A+\Sigma_U$ has eigenvalues $5p$ in direction $v$, $2p+1$ in direction $w$, and $1$ in the remaining directions.

For $p>1/3$, $5p>2p+1$, so the leading population eigenvector is the factor direction $v$. The noise operator norm nevertheless grows like $p$, so the bounded-noise theorem does not apply. This example shows that bounded noise is a convenient sufficient condition, not a necessary one: a large noise spike can coexist with consistent factor recovery if the factor eigenvalue remains larger and the competing directions are separated. Sample PCA additionally requires sample error small relative to the gap $5p-(2p+1)=3p-1$.

Worked Example: When a Noise Spike Masquerades as a Factor

Keep $v,w$ orthogonal, but set $$ A=3p\,vv^\top,\qquad \Sigma_U=4p\,ww^\top+I_p. $$ The population covariance has eigenvalue $3p$ in the true factor direction and $4p+1$ in the noise direction. Since $4p+1>3p$ for every positive $p$, the leading eigenvector is $w$, not $v$. PCA that selects one component therefore selects a noise direction.

The issue is not merely that the noise coordinates are correlated. It is that their dependence creates an eigenvalue larger than the factor eigenvalue. If the aim is to recover the factor space, a large sample size cannot correct this population ordering: as the sample covariance approaches $\Sigma_X$, its leading eigenvector approaches $w$. A spectral factor-counting rule can also mistake such a noise spike for an additional factor.

Growing Noise and the Relevant Separation

The bounded-noise condition can be relaxed. What matters for the perturbation argument is that the factor eigenvalues dominate the noise scale and that the sample covariance estimates the population covariance accurately enough relative to the resulting gap. For instance, if $\|\Sigma_U\|_{\mathrm{op}}=O(p^\alpha)$ for some $0\leq\alpha<1$, then the noise is $o(p)$. The factor eigenvalues remain of order $p$, and the upper noise eigenvalues cannot compete asymptotically. In the signal-plus-noise setting above, the eigenvalue gap is at least $cp-O(p^\alpha)$, still of order $p$. The population leading eigenspace consequently approaches the loading space, and sample consistency follows if $\|S-\Sigma_X\|_{\mathrm{op}}=o_p(p)$.

Worked Example: Sublinear Growth in the Noise Spectrum

Let $p=1000$, $A=4p\,vv^\top$, and let the noise covariance have largest eigenvalue $p^{1/2}$, with all its other eigenvalues between $0$ and $p^{1/2}$. The factor eigenvalue is $4000$, whereas the largest noise eigenvalue is $\sqrt{1000}\approx31.62$. The factor-to-noise scale ratio is $4000/31.62\approx126.5$.

As $p$ increases, the ratio $4p/p^{1/2}=4p^{1/2}$ diverges. Thus the noise is not bounded in operator norm, but it is still small relative to the pervasive factor signal. The factor eigenvalue remains separated from the noise spectrum by a gap of order $p$. If the sample covariance perturbation is $o_p(p)$, ordinary PCA consistently estimates the factor space.

Common Pitfall and Practical Interpretation

A common mistake is to reason, “The errors are idiosyncratic, so PCA will ignore them.” The term “idiosyncratic” does not imply cross-sectional independence, bounded eigenvalues, or a harmless spectrum. For example, if every noise coordinate contains the same unmodelled component, the noise covariance may have a large rank-one eigenvalue. That component can appear as a principal component even when it is not part of the intended factor model.

The correction is to inspect the scale and structure of $\Sigma_U$, not just its individual variances or pairwise correlations. Bounded operator norm is a strong, transparent guarantee; more generally, one must compare the largest noise eigenvalues with the factor eigenvalues and verify a suitable eigengap. Then, for sample PCA, one must also check that sampling error is small relative to that gap. Correlated errors with bounded or sublinear spectral growth need not obstruct factor recovery, while a competing noise spike can defeat ordinary PCA.

Key takeaway: Cross-sectional dependence can reshape the noise spectrum and produce apparent factors. Ordinary PCA remains consistent when factor eigenvalues dominate the noise eigenvalues and sample covariance error is small relative to the separating gap; independence of the idiosyncratic coordinates is not required.

Check Your Understanding

Answer each question using the distinction between factor scale, noise scale, and sample error.

Why does a bounded operator norm for $\Sigma_U$ prevent the noise eigenvalues from matching pervasive factor eigenvalues of order $p$?
Can a noise covariance have an eigenvalue that grows with $p$ while ordinary PCA still consistently estimates a factor space? State a sufficient scale comparison.
In the competing-spike example, which eigenvector does one-component PCA select, and why?
Why does a population eigengap not by itself guarantee sample eigenspace consistency?
What assumption about the idiosyncratic covariance is sufficient in the theorem, and does it require independent noise coordinates?