Confidence Intervals for Population Eigenvalues

Why Eigenvalue Uncertainty Needs Its Own Interval

A sample eigenvalue is an estimate of a population variance along a principal direction, but it also varies from sample to sample. A confidence interval quantifies that sampling uncertainty. For a simple population eigenvalue, first-order perturbation theory reduces its leading fluctuation to a scalar covariance fluctuation, so asymptotic normal methods can give a pointwise interval.

The interval depends on more than the estimated eigenvalue. Its standard error reflects the fourth moment of observations projected onto the corresponding population direction. The familiar Gaussian formula is valid when that fourth-moment structure is justified; it should not automatically be used for every population distribution.

Setup and the Asymptotic Standard Error

Let $X_1,\ldots,X_n$ be independent and identically distributed $p$-dimensional random vectors with mean $\mu$, covariance matrix $\Sigma$, and finite fourth moments. Take $p$ to be fixed as $n$ increases. Define the sample covariance matrix using divisor $n$:

\bar X=\frac{1}{n}\sum_{i=1}^n X_i,\qquad S_n=\frac{1}{n}\sum_{i=1}^n(X_i-\bar X)(X_i-\bar X)^{\mathsf T}.

Write the ordered population eigenvalues as $\lambda_1\geq\cdots\geq\lambda_p$, with corresponding unit eigenvectors $u_1,\ldots,u_p$. The sample eigenvalue matching $\lambda_j$ is denoted by $\hat\lambda_j$. For this tutorial, assume that $\lambda_j$ is positive and simple: it has multiplicity one and is separated from the other population eigenvalues. Let $Z_j=u_j^{\mathsf T}(X-\mu)$ be the population score in direction $u_j$.

Definition: The asymptotic variance factor for a simple population eigenvalue $\lambda_j$ is $$ \tau_j^2=\operatorname{Var}(Z_j^2) =E[Z_j^4]-\lambda_j^2, $$ because $E[Z_j^2]=\lambda_j$. When $\tau_j^2>0$, the asymptotic standard error of $\hat\lambda_j$ is $\tau_j/\sqrt n$.

This formula is distribution-sensitive: the covariance matrix determines $E[Z_j^2]$, but not generally $E[Z_j^4]$. A sample estimate of the standard error can use the projected sample observations. Choose a unit sample eigenvector $\hat u_j$ for $\hat\lambda_j$ and define $\hat z_{ij}=\hat u_j^{\mathsf T}(X_i-\bar X)$. The sign chosen for $\hat u_j$ does not affect the squared scores. Let $s_{q,j}^2$ be the usual sample variance of the $n$ values $\hat z_{ij}^2$, using divisor $n-1$. Then the plug-in standard error is $s_{q,j}/\sqrt n$.

Why the Fourth Moment Appears

The first-order change in a simple eigenvalue is the covariance perturbation in its population eigenvector direction. Applying that fact to $S_n-\Sigma$ gives the leading term $u_j^{\mathsf T}(S_n-\Sigma)u_j$. The following derivation also explains the role of centering.

Result: Under the setup above, if $\lambda_j$ is simple and $\tau_j^2>0$, then $$ \sqrt n(\hat\lambda_j-\lambda_j)\xrightarrow{d}N(0,\tau_j^2). $$

Derivation. For a simple eigenvalue, first-order eigenvalue perturbation gives $\hat\lambda_j-\lambda_j=u_j^{\mathsf T}(S_n-\Sigma)u_j+O_p(\lVert S_n-\Sigma\rVert^2)$, where $\lVert\cdot\rVert$ is the operator norm. The finite fourth moments and fixed $p$ imply $S_n-\Sigma=O_p(n^{-1/2})$, so the remainder is $O_p(n^{-1})$. The projected sample covariance is

u_j^{\mathsf T}S_nu_j =\frac{1}{n}\sum_{i=1}^n \left\{u_j^{\mathsf T}(X_i-\mu)-u_j^{\mathsf T}(\bar X-\mu)\right\}^2 =\frac{1}{n}\sum_{i=1}^n Z_{ij}^2- \left(\frac{1}{n}\sum_{i=1}^n Z_{ij}\right)^2,

where $Z_{ij}=u_j^{\mathsf T}(X_i-\mu)$. The second term is $O_p(n^{-1})$, since the sample mean of the $Z_{ij}$ is $O_p(n^{-1/2})$. Consequently,

\sqrt n(\hat\lambda_j-\lambda_j) =\frac{1}{\sqrt n}\sum_{i=1}^n(Z_{ij}^2-\lambda_j)+o_p(1).

The summands are independent and identically distributed, have mean zero, and have finite variance $\tau_j^2$. The central limit theorem therefore gives the stated limit. The perturbation remainder and the centering term vanish after multiplication by $\sqrt n$, completing the derivation. $\square$

The plug-in standard error is consistent under these conditions: the sample eigenvector converges to $u_j$ up to sign when the eigenvalue is simple, and sample moments of the squared projected scores converge to their population counterparts under the finite fourth-moment assumption. Thus $s_{q,j}^2\xrightarrow{p}\tau_j^2$.

Pointwise Wald Confidence Intervals

A pointwise interval targets one specified eigenvalue $j$ and has asymptotic coverage for that parameter considered by itself. Let $z_{1-\alpha/2}$ denote the $1-\alpha/2$ quantile of the standard normal distribution. Provided $\tau_j^2>0$, a two-sided asymptotic $100(1-\alpha)\%$ Wald interval is

\left[ \hat\lambda_j-z_{1-\alpha/2}\frac{s_{q,j}}{\sqrt n},\quad \hat\lambda_j+z_{1-\alpha/2}\frac{s_{q,j}}{\sqrt n} \right].

Coverage justification. The asymptotic normal result and consistency of $s_{q,j}$ imply, by Slutsky’s theorem, $\sqrt n(\hat\lambda_j-\lambda_j)/s_{q,j}\xrightarrow{d}N(0,1)$. Therefore the probability that this statistic lies between $-z_{1-\alpha/2}$ and $z_{1-\alpha/2}$ tends to $1-\alpha$. Rearranging that event gives exactly the interval above, so its coverage tends to $1-\alpha$. $\square$

This is an asymptotic coverage statement, not a claim of exact coverage at finite $n$. It also concerns one eigenvalue at a time. Constructing a region with a specified coverage for several eigenvalues jointly is a different task.

When Gaussian Calibration Is Justified

If $X$ is multivariate normal, its projection $Z_j$ is normal with variance $\lambda_j$. The fourth moment of a centered normal variable is $3\lambda_j^2$, so

\tau_j^2=3\lambda_j^2-\lambda_j^2=2\lambda_j^2.

This gives the Gaussian plug-in standard error $\sqrt{2}\hat\lambda_j/\sqrt n$, and the corresponding interval replaces $s_{q,j}$ by $\sqrt{2}\hat\lambda_j$. Gaussian data justify that variance formula. Normal quantile calibration can also be asymptotically justified for non-Gaussian data, but then the standard error must account for the actual fourth moment, for example through $s_{q,j}$. The central limit theorem, rather than Gaussian observations themselves, supplies the asymptotic normal shape when the stated moment and simplicity conditions hold.

Worked Example: Gaussian Plug-In Interval

Suppose the estimated leading eigenvalue is $\hat\lambda_1=8.7$, the sample size is $n=100$, and Gaussian calibration is appropriate. Construct a pointwise 95% interval, using $z_{0.975}=1.96$.

The estimated standard error is $\sqrt{2}\hat\lambda_1/\sqrt n=\sqrt{2(8.7)^2/100}$. Since $8.7^2=75.69$, this is $\sqrt{1.5138}\approx1.2304$. The margin of error is $1.96(1.2304)\approx2.4115$. Therefore the interval is $[8.7-2.4115,\;8.7+2.4115]$, or approximately $[6.2885,\;11.1115]$.

This calculation uses the Gaussian fourth-moment formula. If the observations are not Gaussian and their projected fourth moment is not known to match the Gaussian value, this particular standard error need not be appropriate.

Worked Example: A Fourth-Moment-Based Standard Error

Suppose observations are not assumed Gaussian. For a particular simple eigenvalue, the sample eigenvalue is $4.2$, $n=100$, and the sample variance of the squared projected scores is $s_{q,j}^2=14.4$. Find the estimated standard error and a pointwise 95% interval.

The robust plug-in standard error is $s_{q,j}/\sqrt n=\sqrt{14.4}/10\approx3.7947/10=0.3795$. The margin is $1.96(0.3795)\approx0.7438$. The resulting interval is $[4.2-0.7438,\;4.2+0.7438]$, or approximately $[3.4562,\;4.9438]$.

The number $14.4$ estimates the variance of the squared projected score, not the variance of the score itself. Replacing it with $2(4.2)^2=35.28$ would impose the Gaussian fourth-moment formula, which the information in this example does not warrant.

Positivity, Simplicity, and Interpretation

A covariance eigenvalue is nonnegative, but a symmetric Wald interval is not constrained to be. If its lower endpoint is negative, that does not mean the population eigenvalue is negative; it signals that the symmetric approximation is a poor representation near the boundary or that the estimate is imprecise. One possible alternative for a positive eigenvalue is a log-scale interval. Applying the delta method to $\log\hat\lambda_j$ gives approximate standard error $s_{q,j}/(\sqrt n\,\hat\lambda_j)$, and exponentiating the endpoints yields

\left[ \hat\lambda_j\exp\left\{-z_{1-\alpha/2}\frac{s_{q,j}}{\sqrt n\,\hat\lambda_j}\right\}, \quad \hat\lambda_j\exp\left\{z_{1-\alpha/2}\frac{s_{q,j}}{\sqrt n\,\hat\lambda_j}\right\} \right].

This interval is positive, but the transformation does not remove the need for adequate sample size or a simple eigenvalue.

Simplicity is important because the first-order scalar perturbation used here applies to an eigenvalue separated from its neighbors. At a repeated eigenvalue, individual eigenvalues within the tied group do not have the same regular, separated-eigenvalue approximation. The interval above should not be used as though that condition held. Likewise, a small eigengap can make finite-sample estimates sensitive even when the population eigenvalue is technically simple.

Worked Example: Comparing Wald and Log-Scale Intervals

Suppose a positive simple eigenvalue has estimate $\hat\lambda_j=1.2$, estimated standard error $0.3$, and a 95% normal critical value of $1.96$. Compare the symmetric Wald interval with the log-scale interval.

The Wald margin is $1.96(0.3)=0.588$, so the symmetric interval is $[1.2-0.588,\;1.2+0.588]=[0.612,\;1.788]$.

For the log-scale version, the estimated standard error on the log scale is $0.3/1.2=0.25$. Its margin is $1.96(0.25)=0.49$. Exponentiating the endpoints gives $[1.2e^{-0.49},\;1.2e^{0.49}]\approx[0.735,\;1.959]$. Both intervals are positive in this example; the log-scale interval is asymmetric around $1.2$. Neither form changes the pointwise nature of the inference.

Common Mistakes

Using the Gaussian standard error by default. The formula $\sqrt{2}\hat\lambda_j/\sqrt n$ relies on the Gaussian fourth-moment value. For general data, estimate the variance of the squared projected scores instead.
Confusing variance with standard error. The asymptotic variance of $\hat\lambda_j$ is approximately $\tau_j^2/n$; its standard error is approximately $\tau_j/\sqrt n$.
Treating pointwise coverage as simultaneous coverage. A 95% interval for one specified eigenvalue does not automatically give 95% coverage for a collection of eigenvalues together.
Ignoring the simple-eigenvalue condition. A repeated eigenvalue does not meet the regularity condition behind this scalar interval.
Interpreting a negative Wald endpoint as a possible population covariance eigenvalue. Covariance eigenvalues are nonnegative; a negative endpoint reflects the approximation, not an admissible parameter value.

Key takeaway: For a positive simple population eigenvalue with finite fourth moments, estimate uncertainty from the variability of squared projected observations and use a normal critical value for a pointwise asymptotic interval. Use the Gaussian variance formula only when its fourth-moment assumption is justified.

Check Your Understanding

Use the assumptions and formulas in this tutorial to answer the following questions.

Why does the asymptotic variance of a simple sample eigenvalue depend on a fourth moment of the projected observations?
What condition on the population eigenvalue is needed for the scalar interval derived here, and why does a repeated eigenvalue violate it?
For non-Gaussian data, what sample quantity estimates the asymptotic variance factor $\tau_j^2$?
What additional distributional fact justifies replacing that estimate with $2\hat\lambda_j^2$?
Does a pointwise 95% interval for one eigenvalue imply 95% simultaneous coverage for several eigenvalues? Explain briefly.