Delta-Method Inference for Explained Variance

Why Explained-Variance Shares Need Their Own Inference

A principal component’s eigenvalue measures variance in its direction, while its explained-variance share compares that eigenvalue with the variance in all directions. That denominator matters: uncertainty in a share depends not only on the leading eigenvalue, but also on the fluctuations of the other eigenvalues and their dependence. The delta method turns the joint limit for sample eigenvalues into an asymptotic distribution for this ratio.

The target here is inference for a fixed number of leading components when the population covariance matrix has distinct eigenvalues. We take the joint eigenvalue limit as input, rather than re-deriving the covariance matrix or the eigenvalue perturbation result.

Population and Sample Variance Shares

Let \(X_1,\ldots,X_n\) be independent and identically distributed \(p\)-dimensional observations with mean \(\mu\) and positive-definite covariance matrix \(\Sigma\). Write the ordered population eigenvalues as \(\lambda_1>\lambda_2>\cdots>\lambda_p>0\), and the ordered eigenvalues of the sample covariance matrix \(S\) as \(\hat\lambda_1,\ldots,\hat\lambda_p\). The total population variance is \(T=\sum_{j=1}^p\lambda_j=\operatorname{tr}(\Sigma)\), and the total sample variance is \(\hat T=\sum_{j=1}^p\hat\lambda_j=\operatorname{tr}(S)\).

A share is a ratio of random quantities when estimated from data. Treating the denominator as fixed generally misses some uncertainty; treating the eigenvalues as independent can miss it as well. Both issues are handled by differentiating the ratio with respect to the whole eigenvalue vector.

The Joint Eigenvalue Limit and the Ratio Gradient

Assume the population eigenvalues are distinct and the observations have finite fourth moments. For the usual sample covariance matrix, the joint eigenvalue limit has the form \(\sqrt n(\hat{\boldsymbol\lambda}-\boldsymbol\lambda)\xrightarrow{d}N_p(0,V)\), where \(\boldsymbol\lambda=(\lambda_1,\ldots,\lambda_p)^\mathsf{T}\), \(\hat{\boldsymbol\lambda}=(\hat\lambda_1,\ldots,\hat\lambda_p)^\mathsf{T}\), and \(V\) is the asymptotic covariance matrix. The joint limit may be obtained from the eigenvalue perturbation and covariance matrix CLT results. For example, if \(u_i\) is a unit eigenvector of \(\Sigma\) for \(\lambda_i\), then under these conditions \(V_{ij}=\operatorname{Cov}([(u_i^\mathsf{T}(X_1-\mu))^2],[(u_j^\mathsf{T}(X_1-\mu))^2])\). The choice of sign for \(u_i\) does not affect its squared projection.

For a fixed \(k\), regard the cumulative share as the function \(h_k(x_1,\ldots,x_p)=(\sum_{i=1}^k x_i)/(\sum_{j=1}^p x_j)\), defined whenever the denominator is positive. Differentiating with the quotient rule gives \(\partial h_k/\partial x_j=(\mathbf{1}\{j\le k\}\sum_{\ell=1}^p x_\ell-\sum_{i=1}^k x_i)/(\sum_{\ell=1}^p x_\ell)^2\). At the population eigenvalues this simplifies to \(g_{k,j}=(\mathbf{1}\{j\le k\}-R_k)/T\). Thus every eigenvalue contributes: eigenvalues in the leading group have one common derivative, while those outside it have another.

For the individual share, direct differentiation gives gradient entries \((\mathbf{1}\{j=i\}-r_i)/T\). This differs from the cumulative-share gradient because only eigenvalue \(i\) appears in the numerator. Both formulas reflect the same denominator effect.

Derivation of the Asymptotic Distribution

The theorem is a direct application of the multivariate delta method. The proof also shows why the full covariance matrix \(V\), not just its diagonal, enters the answer.

For several cumulative shares at once, let \(\hat{\boldsymbol R}=(\hat R_1,\ldots,\hat R_K)^\mathsf{T}\) for fixed \(K\le p\). Define the \(K\)-by-\(p\) matrix \(G\) with row \(k\) equal to \(g_k^\mathsf{T}\). Applying the same Taylor expansion to the vector of ratios yields \(\sqrt n(\hat{\boldsymbol R}-\boldsymbol R)\xrightarrow{d}N_K(0,GVG^\mathsf{T})\). This follows because the vector function is differentiable with Jacobian \(G\), so its first-order term is \(G\sqrt n(\hat{\boldsymbol\lambda}-\boldsymbol\lambda)\); the vector Taylor remainder is \(o_p(1)\) after scaling, and the continuous linear transformation has covariance \(GVG^\mathsf{T}\).

Under multivariate normal sampling, the asymptotic covariance simplifies to \(V_{ii}=2\lambda_i^2\) and \(V_{ij}=0\) for \(i\ne j\). This follows because the population eigenvector coordinates are independent normal variables, so their squares have variances \(2\lambda_i^2\) and zero cross-covariances. Outside the Gaussian model, the off-diagonal entries need not vanish.

Worked Calculations

Interpretation and Common Pitfalls

A cumulative share is constrained to lie between zero and one, but its first-order normal approximation is not. For large samples and shares well inside those boundaries, a normal interval can be a useful summary. Near a boundary, the approximation may produce endpoints outside the allowable range or represent sampling uncertainty poorly. The formula is also a fixed-dimension, fixed-\(k\) result: it is not a guarantee for a growing number of variables or components.

• Holding the denominator fixed. The total variance is estimated from the same data. The gradient entries for eigenvalues outside the leading group are \(-R_k/T\), not zero.
• Ignoring covariance. In general, the variance is \(g_k^\mathsf{T}Vg_k\), including all cross-covariance terms. Diagonal \(V\) is a Gaussian-model simplification, not a universal rule.
• Using eigenvalue intervals as share intervals. A share is a ratio involving the full spectrum. Its uncertainty must be derived for the ratio itself.
• Ignoring the regularity conditions. The stated joint normal limit assumes distinct population eigenvalues and the appropriate sampling conditions. If these fail, this particular differentiable-eigenvalue argument need not apply.