Local Alternatives and Power in PCA Dimension Tests

Why Local Alternatives Matter

A PCA dimension test asks whether the covariance structure is adequately described by a specified number of principal components. A test may have high power against a fixed departure from its null model as the sample size grows, but that does not describe how it behaves against departures that shrink with the sample size. Local alternatives make this question precise: they let the departure approach the null at rate $n^{-1/2}$, the scale at which sampling fluctuations in a covariance matrix are typically visible.

We study a Gaussian likelihood-ratio test for a common residual eigenvalue. The null says that after the first $q$ components, the remaining $m=p-q$ eigenvalues are equal. A local alternative perturbs the covariance by $n^{-1/2}$ times a fixed matrix. The main result is that only the traceless part of the perturbation within the residual eigenspace contributes to the limiting power. A common shift of all residual eigenvalues does not distinguish the alternative from the null.

Definition: Let $X_1,\ldots,X_n$ be independent $p$-dimensional observations from $N_p(\mu,\Sigma_n)$, with fixed $p$. The null model for a specified dimension $q$ consists of covariance matrices whose $q$ leading eigenvalues exceed a common residual eigenvalue $\sigma^2>0$, with multiplicity $m=p-q$. In coordinates aligned with the population eigenspaces, write $$ \Sigma_0= \begin{pmatrix} A&0\\ 0&\sigma^2 I_m \end{pmatrix}, $$ where $A$ is a $q\times q$ positive-definite matrix and every eigenvalue of $A$ exceeds $\sigma^2$. A root-$n$ local alternative is a sequence $$ \Sigma_n=\Sigma_0+n^{-1/2}H, $$ where $H$ is a fixed symmetric matrix. For all sufficiently large $n$, $\Sigma_n$ is positive definite.

The eigenvalue gap between $A$ and $\sigma^2 I_m$ ensures that the residual eigenspace remains separated from the leading eigenspace. The matrix $H$ can contain changes to the leading block, cross-block changes, and changes to the residual block. The test does not respond equally to all of them.

The Likelihood-Ratio Statistic

Let $S$ be the sample covariance matrix, computed after subtracting the sample mean. Write its ordered eigenvalues as $l_1\geq\cdots\geq l_p$. Under the unrestricted covariance model, the maximum-likelihood eigenvalues are $l_1,\ldots,l_p$. Under the null, the fitted leading eigenvalues remain $l_1,\ldots,l_q$, while the common residual eigenvalue is the average of the last $m$ sample eigenvalues. The likelihood-ratio statistic, with larger values indicating departure from the null, is

T_n=n\left\{m\log\left(\frac{1}{m}\sum_{j=q+1}^{p}l_j\right) -\sum_{j=q+1}^{p}\log l_j\right\}.

This expression is nonnegative by the arithmetic-geometric mean inequality. Its value is zero when all $m$ residual sample eigenvalues are equal. It is unchanged if all of them are multiplied by the same positive constant, which already hints that a common residual-scale change is not what this test detects.

Theorem: Under the Gaussian model above, suppose $p$ and $q$ are fixed and the population gap between the eigenvalues of $A$ and $\sigma^2$ is positive. Under the null, $T_n\xrightarrow{d}\chi^2_d$, where $d=m(m+1)/2-1$. Under $\Sigma_n=\Sigma_0+n^{-1/2}H$, it converges to a noncentral chi-square distribution with the same degrees of freedom and noncentrality parameter $$ \delta=\frac{1}{2\sigma^4}\operatorname{tr}\left[ \left(H_{22}-\frac{\operatorname{tr}(H_{22})}{m}I_m\right)^2 \right]. $$ Here $H_{22}$ is the residual $m\times m$ block of $H$. A level-$\alpha$ test that rejects for $T_n>\chi^2_{d,1-\alpha}$ therefore has limiting local power $$ P\left\{\chi^2_d(\delta)>\chi^2_{d,1-\alpha}\right\}. $$

Deriving the Limiting Distribution

We give the derivation, including the effect of estimating the leading eigenspace. In the population coordinates of $\Sigma_0$, the residual block of $S$ has the expansion

S_{22}=\sigma^2 I_m+n^{-1/2}(H_{22}+G_n)+o_p(n^{-1/2}),

where $G_n$ converges in distribution to a centered symmetric Gaussian matrix $G$. This follows from the covariance-matrix central limit theorem; under the Gaussian model its entries satisfy

\operatorname{Cov}(G_{ij},G_{kl}) =\sigma^4(\mathbf{1}_{i=k}\mathbf{1}_{j=l}+\mathbf{1}_{i=l}\mathbf{1}_{j=k}).

Estimating the residual eigenspace does not add another first-order term here. A perturbation of size $n^{-1/2}$ in the cross-block changes residual eigenvalues only at order $n^{-1}$, because the leading and residual population eigenvalues are separated by a fixed positive gap. Equivalently, first-order perturbation of the repeated residual eigenvalue cluster is governed by the residual block $H_{22}+G_n$. The eigenvalue-cluster limit is the appropriate one: individual eigenvectors within a repeated eigenspace need not have a unique limit.

Let $Z=H_{22}+G$, and let $\xi_1,\ldots,\xi_m$ be its eigenvalues. The last $m$ sample eigenvalues thus have first-order deviations $\xi_j$ from $\sigma^2$. Expanding the logarithms in the likelihood-ratio statistic uses $\log(\sigma^2+x)=\log\sigma^2+x/\sigma^2-x^2/(2\sigma^4)+o(x^2)$. The linear terms cancel between the log of the average and the average of the logs. Consequently,

T_n\xrightarrow{d} \frac{1}{2\sigma^4}\sum_{j=1}^m(\xi_j-\bar{\xi})^2 = \frac{1}{2\sigma^4}\operatorname{tr} \left[\left(Z-\frac{\operatorname{tr}(Z)}{m}I_m\right)^2\right], \qquad \bar{\xi}=\frac{1}{m}\sum_{j=1}^m\xi_j.

The trace identity holds because $Z$ is symmetric and its eigenvalues are $\xi_1,\ldots,\xi_m$. Subtracting the trace removes the common residual shift. To identify the distribution, express the centered symmetric matrix in an orthonormal basis of the space of symmetric trace-zero $m\times m$ matrices. That space has dimension $m(m+1)/2-1$. By the covariance formula for $G$, each coordinate has variance $2\sigma^4$, and the coordinates are independent. The displayed quadratic form is therefore the sum of squares of $d$ independent unit-variance normal variables, with squared mean shifts summing to $\delta$. This is $\chi^2_d(\delta)$. When $H_{22}$ is proportional to $I_m$, all the shifts vanish, giving the central chi-square limit. This proves the theorem. $\square$

Worked Examples

Worked Example: A Two-Dimensional Residual Split

Suppose $p=3$, $q=1$, and the null covariance is $\operatorname{diag}(9,4,4)$. Consider the local alternative $\Sigma_n=\Sigma_0+n^{-1/2}H$, where $H_{22}=\operatorname{diag}(2,-2)$. Find the limiting distribution and the limiting power at level $0.05$.

Here $m=2$, $\sigma^2=4$, and $d=2(3)/2-1=2$. The trace of $H_{22}$ is zero, so its traceless part is $H_{22}$ itself. Since $\sigma^4=16$,

\delta=\frac{\operatorname{tr}(H_{22}^2)}{2\sigma^4} =\frac{2^2+(-2)^2}{2(16)} =\frac{8}{32} =\frac14.

Thus $T_n$ converges to $\chi^2_2(0.25)$. The central $5\%$ critical value is approximately $5.991$, so the limiting power is $P\{\chi^2_2(0.25)>5.991\}$, approximately $0.069$. The departure is detectable in the limit, but the power is only modestly above the test size.

Worked Example: A Residual Shift That Is Not Detectable

Let $m=3$, $\sigma^2=2$, and let the residual perturbation be $H_{22}=3I_3$. This changes all three residual eigenvalues by the same amount at order $n^{-1/2}$. Calculate the noncentrality parameter.

The average diagonal shift is $\operatorname{tr}(H_{22})/3=9/3=3$. Hence

H_{22}-\frac{\operatorname{tr}(H_{22})}{3}I_3 =3I_3-3I_3=0, \qquad \delta=\frac{1}{2(2^2)}\operatorname{tr}(0^2)=0.

The limiting statistic is central $\chi^2_5$, because $d=3(4)/2-1=5$. The local limiting power is therefore $0.05$ for a level-$0.05$ test. This is not a failure to detect an eigenvalue change: the alternative changes the common residual scale, which is allowed under the null. It does not split the residual eigenvalues.

Worked Example: Separating Nuisance Changes From Eigenvalue Splitting

Take $p=3$, $q=1$, and $\Sigma_0=\operatorname{diag}(9,4,4)$. Suppose

H= \begin{pmatrix} 3&2&-1\\ 2&0&1\\ -1&1&2 \end{pmatrix}.

The leading-block change is $3$, and the cross-block changes are $2$ and $-1$. The residual block is $H_{22}=\begin{pmatrix}0&1\\1&2\end{pmatrix}$, with trace $2$. Its traceless part is

H_{22}-\frac{2}{2}I_2= \begin{pmatrix}-1&1\\1&1\end{pmatrix}, \qquad \operatorname{tr}\left[ \begin{pmatrix}-1&1\\1&1\end{pmatrix}^2 \right]=4.

Since $m=2$, $d=2$, and since $\sigma^2=4$, $\sigma^4=16$. Thus $\delta=4/(2\cdot16)=1/8$, and the limiting statistic is $\chi^2_2(1/8)$. The leading-block and cross-block entries of $H$ do not appear in this calculation: they change nuisance features of the covariance, not the residual eigenvalue equality being tested.

Interpreting Local Power

The theorem describes local asymptotic power: the limiting rejection probability along alternatives whose size decreases as $n^{-1/2}$. A fixed nonzero residual split corresponds to a departure that does not shrink; it is not described by a fixed noncentrality parameter and can lead to power tending to one. Conversely, a departure smaller than the sampling-fluctuation scale may be difficult to distinguish from the null.

For local alternatives, power is determined by the size of the residual traceless perturbation relative to $\sigma^2$, as measured by $\delta$. A larger split increases $\delta$ and raises the limiting rejection probability. The trace is removed because the null estimates a common residual eigenvalue; the leading and cross-block changes are removed because they lie in nuisance directions of the covariance model. This separation is why the raw size of $H$, by itself, does not determine power.

Common pitfall: It is incorrect to calculate the noncentrality parameter from $\operatorname{tr}(H_{22}^2)$ without removing its mean eigenvalue shift. For example, $H_{22}=cI_m$ has $\operatorname{tr}(H_{22}^2)>0$ when $c\ne0$, but its traceless part is zero, so its noncentrality parameter is zero. The correction is to subtract $\{\operatorname{tr}(H_{22})/m\}I_m$ first. The test detects residual eigenvalue inequality, not a shared change in residual scale.

Check Your Understanding

Use the residual-block formula to answer each question.

For $m=4$, what are the degrees of freedom of the limiting likelihood-ratio statistic?
Why does a perturbation proportional to $I_m$ have zero local noncentrality?
If $H_{22}$ is traceless, how does the expression for $\delta$ simplify?
In the worked example with $H_{22}=\begin{pmatrix}0&1\\1&2\end{pmatrix}$, why do the cross-block entries of $H$ not enter $\delta$?
What does a fixed nonzero $\delta$ say about power along the root-$n$ local alternative, and how does that differ from a fixed covariance departure?