PCA Estimation Rates in Large Panels

Why Large-Panel PCA Has Several Error Rates

In a large panel, \(N\) variables are observed over \(T\) time periods. Principal components estimate a low-rank common component from this \(N\)-by-\(T\) data matrix. But “how accurately does PCA estimate the factors?” is not the same question as “how accurately does it reconstruct the common component?” Factor coordinates depend on a choice of scale and rotation; the reconstructed matrix does not. A rate statement must say which object is being measured and how factor estimates are aligned.

This tutorial works with a fixed number of strong factors, while both panel dimensions grow. The main rates are average squared errors. Their square roots are the corresponding root-mean-square errors. The results are bounds under explicit assumptions, not a claim that every term is a sharp lower bound in every model.

Model, Normalization, and PCA Estimator

The normalization fixes a scale, but not a rotation: for any orthogonal \(r\)-by-\(r\) matrix \(Q\), \(FQ\) and \(\Lambda Q\) give the same common component because \((FQ)(\Lambda Q)'=F\Lambda'\). Thus, estimated factors and loadings must be compared with their targets after a common orthogonal alignment.

For a concrete rate calculation, assume the entries of \(E\) are independent, mean zero, and sub-Gaussian with uniformly bounded variance and sub-Gaussian norm, and that \(E\) is independent of the signal. These assumptions imply \(\|E\|_{\mathrm{op}}=O_p(\sqrt N+\sqrt T)\), where \(\|\cdot\|_{\mathrm{op}}\) is the largest singular value. Let \(\widehat S\) be a best rank-\(r\) approximation to \(X\) in squared Frobenius loss, equivalently its rank-\(r\) truncated singular-value decomposition. Write \(\widehat S=\widehat F\widehat\Lambda'\), with \(\widehat F'\widehat F/T=I_r\).

Common-Component Rate

Proof. Set \(D=\widehat S-S\). Since \(\widehat S\) minimizes \(\|X-A\|_F^2\) over rank-\(r\) matrices \(A\), and \(S\) has rank at most \(r\),

Substituting \(X=S+E\) and expanding both squared norms gives \(\|E-D\|_F^2\leq\|E\|_F^2\), hence

Here \(\langle A,B\rangle=\operatorname{tr}(A'B)\), the middle inequality follows from the singular-value inequality \(|\langle E,D\rangle|\leq\|E\|_{\mathrm{op}}\|D\|_*\), and \(\|D\|_*\leq\sqrt{\operatorname{rank}(D)}\|D\|_F\). The last bound uses \(\operatorname{rank}(D)\leq 2r\). If \(\|D\|_F=0\), the claimed inequality holds directly; otherwise divide by \(\|D\|_F\) and square to obtain \(\|D\|_F^2\leq 8r\|E\|_{\mathrm{op}}^2\). Since \(r\) is fixed and \(\|E\|_{\mathrm{op}}^2=O_p(N+T)\), dividing by \(NT\) proves the stated rate. \(\square\)

The two terms have a direct interpretation: \((N+T)/(NT)=1/T+1/N\). Increasing the number of variables or the number of periods reduces average reconstruction error. This Frobenius calculation targets \(S\) as a whole; it does not yet say how its factorization is estimated.

Aligned Factor and Loading Rates

To compare coordinates, use a common orthogonal rotation. Define normalized matrices \(A=F/\sqrt T\), \(B=\Lambda/\sqrt N\), \(\widehat A=\widehat F/\sqrt T\), and \(\widehat B=\widehat\Lambda/\sqrt N\). Then \(A'A=\widehat A'\widehat A=I_r\), and \(S/\sqrt{NT}=AB'\). Pervasiveness says the singular values of \(B\) are bounded above and away from zero.

Justification. The normalized signal \(AB'\) has its \(r\) nonzero singular values bounded away from zero by the lower bound on the singular values of \(B\). The normalized fitted matrix \(\widehat A\widehat B'=\widehat S/\sqrt{NT}\) differs from it in Frobenius norm by \[ \frac{\|\widehat S-S\|_F}{\sqrt{NT}} =O_p\left(\sqrt{\frac{1}{N}+\frac{1}{T}}\right), \] by the common-component theorem.

The column space of \(\widehat A\) is the leading left singular subspace of the fitted matrix. To see why its distance from the column space of \(A\) is controlled by the matrix error, let \(P_A=AA'\). The part of the fitted matrix outside the true space satisfies \[ \|(I-P_A)\widehat A\widehat B'\|_F =\|(I-P_A)(\widehat A\widehat B'-AB')\|_F \leq\|\widehat A\widehat B'-AB'\|_F. \] The nonzero singular values of \(\widehat B\) are bounded away from zero with probability tending to one, because the fitted matrix is close to a matrix whose \(r\) nonzero singular values are bounded away from zero. Consequently, the squared distance between the two \(r\)-dimensional spaces, and hence the squared Frobenius distance between orthogonally aligned bases, is bounded by a constant times \(\|\widehat A\widehat B'-AB'\|_F^2\). This yields an orthogonal \(Q\) with \(\|\widehat A-AQ\|_F^2=O_p(1/N+1/T)\).

For the loadings, the normalized PCA factorization gives \(\widehat B=(\widehat A\widehat B')'\widehat A\), while \(B=(AB')'A\). After the same alignment, subtract these expressions and use the triangle inequality: \[ \|\widehat B-BQ\|_F \leq \|\widehat A\widehat B'-AB'\|_F +\|AB'\|_{\mathrm{op}}\|\widehat A-AQ\|_F. \] The operator norm \(\|AB'\|_{\mathrm{op}}\) is bounded by pervasiveness. Squaring the resulting order bound gives \(\|\widehat B-BQ\|_F^2=O_p(1/N+1/T)\). By the definitions of \(A,B,\widehat A,\widehat B\), these are exactly the two average squared errors in the theorem. \(\square\)

This argument uses the stability of a well-conditioned factorization: a small matrix error implies small errors in its factors once the arbitrary rotation is resolved. The common-component theorem supplies the matrix error; strong factors ensure that the factorization is not nearly singular.

Why the Distinctions Matter

The common component \(S\) is often the target in forecasting, denoising, or reconstructing a panel. Its error is measured per entry, so it remains meaningful without choosing coordinates for the factors. Factor-score and loading errors instead assess a particular factor representation, and must be evaluated after alignment. The oracle benchmarks explain the natural direction of information: each factor score pools information across \(N\) variables, while each loading pools information across \(T\) periods.

A useful way to organize the three targets is:

A small common-component error does not identify uniquely named or oriented factors: rotating the factors and loadings together leaves every fitted entry unchanged. Conversely, a factor-score error without alignment can look large merely because PCA selected a different orthogonal coordinate system.

Common Mistake

Wrong reasoning: “The component estimate is accurate, so the estimated factors must be close to the true factors without adjustment.” This ignores rotational indeterminacy. For example, if \(r=2\) and \(Q\) rotates the two coordinates by \(90\) degrees, \(FQ(\Lambda Q)'=F\Lambda'\), even though the columns of \(FQ\) need not resemble the corresponding columns of \(F\).

Correction: State the target and its normalization. For factor and loading comparisons, find a common orthogonal alignment \(Q\), then report the corresponding average errors. For reconstruction, compare \(\widehat S\) directly with \(S\); no factor-coordinate alignment is needed.