Wald Tests for Equality of Population Eigenvalues

From Eigenvalue Estimates to Equality Hypotheses

A question such as whether two population eigenvalues are equal is a restriction on a vector of parameters. Wald testing turns that restriction into a quadratic measure of how far the estimated parameters lie from the null. The construction is straightforward once the joint asymptotic covariance is known, but its validity depends on a condition that is especially important here: the estimator must have the stated regular asymptotic distribution under the null hypothesis.

For covariance-based principal component analysis (PCA), sample eigenvalues associated with distinct population eigenvalues have a joint asymptotic normal distribution under suitable moment conditions. At an equality null, however, the relevant population eigenvalues are tied. The distinct-eigenvalue limit then does not apply. We will first set out the Wald construction and its hypotheses, then show how to test a genuine equality without relying on an invalid distinct-eigenvalue calibration.

Definition: Let $\theta\in\mathbb{R}^q$ be a parameter vector, and let $\hat{\theta}$ estimate it. A linear restriction has the form $C\theta=b$, where $C$ is an $r\times q$ matrix and $b\in\mathbb{R}^r$. Its Wald statistic is the squared estimated distance from $C\hat{\theta}$ to $b$, measured using an estimate of the covariance of $C\hat{\theta}$. If $C$ has row rank $r$, the proposed statistic is $$ W_n=n(C\hat{\theta}-b)^\mathsf{T} \left(C\hat{V}C^\mathsf{T}\right)^{-1} (C\hat{\theta}-b), $$ where $\hat{V}$ estimates the asymptotic covariance matrix of $\sqrt n(\hat{\theta}-\theta)$.

The matrix $C$ encodes the restrictions, not the data. For eigenvalues $\lambda_1,\ldots,\lambda_q$, equality of $\lambda_i$ and $\lambda_j$ is the contrast $\lambda_i-\lambda_j=0$, represented by a row with $1$ in position $i$, $-1$ in position $j$, and zeros elsewhere. For several restrictions, the rank of $C$, rather than the number of written equations, determines the number of independent restrictions.

The Wald Limit and Its Conditions

The chi-square calibration is a theorem about an estimator’s behavior under the null. It does not follow merely from writing down a quadratic form. A convenient sufficient set of conditions is: $C$ has full row rank $r$; under $H_0:C\theta=b$, $\sqrt n(\hat{\theta}-\theta)\xrightarrow{d}N(0,V)$; $V$ is finite and positive definite; and $\hat{V}\xrightarrow{p}V$. These conditions imply that $S=CV C^\mathsf{T}$ is positive definite.

Theorem: Under the conditions just stated, $W_n\xrightarrow{d}\chi^2_r$ under $H_0:C\theta=b$.

Proof. Define $Z_n=\sqrt n(C\hat{\theta}-b)$. Under the null, $C\theta=b$, so $$ Z_n=C\sqrt n(\hat{\theta}-\theta). $$ By the assumed asymptotic normality of $\hat{\theta}$ and the continuous mapping theorem, $Z_n\xrightarrow{d}Z\sim N(0,S)$, where $S=CVC^\mathsf{T}$. Consistency of $\hat V$ gives $C\hat V C^\mathsf{T}\xrightarrow{p}S$. Since $S$ is positive definite, matrix inversion is continuous at $S$, and therefore $(C\hat V C^\mathsf{T})^{-1}\xrightarrow{p}S^{-1}$. Slutsky’s theorem now gives $$ W_n=Z_n^\mathsf{T}(C\hat V C^\mathsf{T})^{-1}Z_n \xrightarrow{d} Z^\mathsf{T}S^{-1}Z. $$ Writing $Z=S^{1/2}U$, where $U\sim N(0,I_r)$, the limit is $U^\mathsf{T}U$, which has a $\chi^2_r$ distribution. $\square$

In PCA, a standard input to this construction is the joint limit for sample eigenvalues corresponding to distinct population eigenvalues. For independent observations $X_1,\ldots,X_n\in\mathbb{R}^p$ with mean $\mu$, covariance $\Sigma$, and finite fourth moments, let $\lambda_i$ be a simple eigenvalue of $\Sigma$ with unit eigenvector $u_i$. The first-order influence for its sample eigenvalue is $\{u_i^\mathsf{T}(X-\mu)\}^2-\lambda_i$. Thus the covariance between the limiting coordinates for two simple eigenvalues is $$ V_{ij}=\operatorname{Cov}\left(\{u_i^\mathsf{T}(X-\mu)\}^2,\{u_j^\mathsf{T}(X-\mu)\}^2\right). $$ This distinct-eigenvalue result can supply $V$ when its assumptions hold. It must not be used as though it gave a normal limit at an equality null.

Building Contrast Matrices

For $q$ eigenvalues, let $e_i$ denote the $i$th coordinate vector in $\mathbb{R}^q$. A single equality $\lambda_i=\lambda_j$ uses the row $(e_i-e_j)^\mathsf{T}$. To require $m$ selected eigenvalues to be all equal, choose one as a reference and compare each of the other $m-1$ eigenvalues to it. These $m-1$ contrasts are independent. Writing every pairwise equality instead creates redundant rows, so the resulting matrix is rank deficient and cannot be inverted in the ordinary Wald formula.

Worked Example: Three Eigenvalues Constrained to Be Equal

Suppose the restrictions are $\lambda_1=\lambda_2$ and $\lambda_2=\lambda_3$, for a three-coordinate parameter vector $\lambda=(\lambda_1,\lambda_2,\lambda_3)^\mathsf{T}$. Taking $\lambda_1$ as the reference gives $$ C= \begin{pmatrix} -1&1&0\\ -1&0&1 \end{pmatrix}, \qquad C\lambda= \begin{pmatrix} \lambda_2-\lambda_1\\ \lambda_3-\lambda_1 \end{pmatrix}. $$ The two rows are linearly independent, so the hypothesis has rank $r=2$.

The third pairwise equation $\lambda_3-\lambda_2=0$ adds no independent restriction: its contrast row is the second row minus the first. Appending it would give a singular $CVC^\mathsf{T}$, even if $V$ is positive definite. Use an independent set of contrasts, or reduce a redundant set to a full-row-rank basis, before applying the ordinary Wald formula.

If the eigenvalue estimates really do satisfy the theorem’s null assumptions, a consistent estimate $\hat V$ yields an estimated contrast covariance $C\hat V C^\mathsf{T}$, and the statistic is compared with a $\chi^2_r$ reference distribution. In practice, verifying those assumptions is not a technical afterthought: at a null asserting equality of population eigenvalues, it changes the conclusion.

Why Tied Eigenvalues Need Special Treatment

When two population eigenvalues are equal, their individual eigenvectors are not uniquely defined within their shared eigenspace. More importantly for testing, ordered sample eigenvalues in that cluster do not generally have the ordinary joint normal limit obtained by perturbing simple eigenvalues. Their first-order fluctuations are the ordered eigenvalues of a random symmetric perturbation of the tied block. A difference of two ordered eigenvalues is therefore generally nonnegative and has a non-normal limit.

Consequently, substituting a covariance matrix derived for distinct eigenvalues into a contrast Wald statistic does not establish a chi-square test of equality. The theorem above still holds, but its null asymptotic normality condition has not been met. A valid approach is to express the equality as smooth restrictions on covariance coordinates and construct the Wald statistic from those coordinates, when such a parameterization is available.

Worked Example: A Naive Contrast at a Tie Is Not Chi-Square

Consider a two-dimensional covariance matrix whose null value is $\Sigma=\lambda I_2$, with $\lambda>0$. Under standard finite-fourth-moment conditions, the first-order sample covariance perturbation within this tied eigenspace is a symmetric random matrix $G$. The two ordered sample eigenvalues have first-order fluctuations equal to the ordered eigenvalues of $G$, say $\gamma_1\geq\gamma_2$.

The scaled sample eigenvalue difference therefore has limit $\gamma_1-\gamma_2$, which is always nonnegative. A nondegenerate centered normal random variable takes negative values with positive probability, so this limit cannot be the centered normal contrast required by the usual one-degree-of-freedom Wald theorem. A chi-square calibration based on the distinct-eigenvalue covariance formula is not justified at this null.

This is not a failure of Wald statistics in general. It is a failure to use a regular parameterization and null limit for this particular contrast. In the same two-dimensional setting, equality can be written directly in terms of covariance entries, as the next example demonstrates.

Worked Example: A Valid Wald Test for a Bivariate Covariance Matrix

Let $\Sigma=\begin{pmatrix}a&c\\c&d\end{pmatrix}$ be positive definite. Its two eigenvalues are equal exactly when $(a-d)^2+4c^2=0$, which is equivalent to the two smooth restrictions $g(\Sigma)=(a-d,c)^\mathsf{T}=0$. These are covariance-coordinate restrictions, not a contrast between separately ordered eigenvalue estimates.

Assume independent bivariate normal observations with unknown mean and null covariance $\lambda I_2$, where $\lambda>0$. Let $S_{11},S_{22},S_{12}$ be the entries of the sample covariance matrix, using divisor $n$. Under the null, $$ \sqrt n \begin{pmatrix} S_{11}-\lambda\\ S_{22}-\lambda\\ S_{12} \end{pmatrix} \xrightarrow{d} N\left( \begin{pmatrix}0\\0\\0\end{pmatrix}, \begin{pmatrix} 2\lambda^2&0&0\\ 0&2\lambda^2&0\\ 0&0&\lambda^2 \end{pmatrix} \right). $$ The same first-order limit holds when the mean is estimated. Applying the linear contrast that forms $g$ gives asymptotic covariance $\operatorname{diag}(4\lambda^2,\lambda^2)$ for $\sqrt n(S_{11}-S_{22},S_{12})^\mathsf{T}$.

Estimate $\lambda$ under the null by $\hat\lambda=(S_{11}+S_{22})/2$, which converges in probability to $\lambda$. The Wald statistic is $$ W_n=n\left\{\frac{(S_{11}-S_{22})^2}{4\hat\lambda^2} +\frac{S_{12}^2}{\hat\lambda^2}\right\}. $$ The estimated covariance is positive definite with probability tending to one because $\lambda>0$. The two restrictions have rank two, so the Wald theorem gives $W_n\xrightarrow{d}\chi^2_2$ under the null. This is a valid equality test because the restrictions are smooth in the covariance entries and the stated joint normal limit holds under the null.

Interpretation and Common Pitfalls

A Wald statistic is large when the estimated restrictions are far from zero relative to their estimated sampling variability. Its reference degrees of freedom count independent restrictions, not the number of parameters being estimated. For any application, state the null, give a full-row-rank contrast or smooth restriction map, specify the asymptotic covariance estimate, and justify the null limit used for calibration.

Using a distinct-eigenvalue limit at a tie. The reasoning “sample eigenvalues are asymptotically normal, so their difference is normal under equality” overlooks that the distinct-eigenvalue condition fails under the null. Use a null-valid parameterization and limit, or a method with an appropriate nonstandard calibration.
Counting redundant equations as separate restrictions. Pairwise equalities can be linearly dependent. Check the rank of the restriction matrix and use independent contrasts.
Ignoring covariance between estimates. The variance of a difference depends on covariance terms. Use $C\hat V C^\mathsf{T}$, not just the separate diagonal variances.
Assuming a quadratic statistic is automatically chi-square. The chi-square result requires asymptotic normality under the null, covariance consistency, and a nonsingular restriction covariance.

Key takeaway: Contrast matrices organize eigenvalue equality hypotheses, and a Wald statistic has a chi-square limit when its estimator is jointly asymptotically normal under the null. Exact eigenvalue ties violate the distinct-eigenvalue premise, so the null calibration must be justified using a null-valid formulation.

Check Your Understanding

Use the definitions and conditions above to answer the following.

Write a full-row-rank contrast matrix for $\lambda_2=\lambda_4$ and $\lambda_3=\lambda_4$ in a four-eigenvalue vector.
Why does adding every pairwise equality among four parameters produce a rank-deficient contrast matrix?
State the assumptions needed for the usual Wald statistic to converge to $\chi^2_r$.
Why does a distinct-population-eigenvalue limit not, by itself, justify a chi-square test of equality?
For a bivariate covariance matrix, express equality of the two eigenvalues as restrictions on its entries. How many independent restrictions are there?