Power Along Covariance Eigenvectors

Why Direction Matters

A multivariate mean test does not respond only to the length of a mean shift. It also responds to the direction of that shift relative to the covariance matrix. If two populations have the same mean-shift length, the shift that lies along a high-variance direction is less pronounced against the background variation than one along a low-variance direction. Covariance-standardized noncentrality makes this comparison precise.

We consider independent observations \(X_1,\ldots,X_n\) from a \(p\)-variate normal distribution \(N_p(\mu,\Sigma)\), where \(\Sigma\) is positive definite. The null hypothesis is \(H_0:\mu=0\). Write the spectral decomposition of the covariance matrix using orthonormal eigenvectors \(v_1,\ldots,v_p\) and positive eigenvalues \(\eta_1,\ldots,\eta_p\):

The eigenvalue \(\eta_j\) is the variance of an observation projected onto \(v_j\). We compare shifts of length \(d>0\) along these directions, so that \(\mu=dv_j\). The key quantity for both the known-covariance and one-sample Hotelling tests is the noncentrality parameter

This is a squared, covariance-standardized effect multiplied by sample size. It is not the ordinary squared Euclidean length \(d^2=\mu^\mathsf{T}\mu\).

Deriving the Eigenvector Comparison

Because \(v_j\) is an eigenvector of \(\Sigma\), it is also an eigenvector of \(\Sigma^{-1}\), with eigenvalue \(1/\eta_j\). Consequently, for \(\mu=dv_j\),

The last equality uses the unit length of \(v_j\). This yields the direction comparison directly: if \(\eta_a<\eta_b\), then \(nd^2/\eta_a>nd^2/\eta_b\). To translate this ordering into a power ordering, we need the fact that power for the tests considered here increases with the noncentrality parameter.

Why Larger Noncentrality Means Greater Power

Under the normal model with known \(\Sigma\), the statistic \(n\bar X^\mathsf{T}\Sigma^{-1}\bar X\) has a noncentral chi-square distribution with \(p\) degrees of freedom and noncentrality \(\Delta\) under the alternative. The level-\(\alpha\) test rejects when this statistic exceeds the central \(\chi^2_p\) upper-\(\alpha\) critical value. For estimated covariance, let \(S\) be the sample covariance matrix. When \(n>p\), the one-sample Hotelling statistic has the exact calibration

Here \(F_{p,n-p}(\Delta)\) denotes the noncentral \(F\) distribution with the indicated numerator and denominator degrees of freedom and noncentrality \(\Delta\). These exact distributions assume independent multivariate normal observations and positive-definite \(\Sigma\); the Hotelling form also requires \(n>p\).

A proof follows from a mixture representation. If \(Y\) has a noncentral chi-square distribution with \(\nu\) degrees of freedom and noncentrality \(\Delta\), its moment-generating function is

Set \(K\sim\operatorname{Poisson}(\Delta/2)\). The mixture that first draws \(K\) and then draws a central \(\chi^2_{\nu+2K}\) variable has moment-generating function

This is the same moment-generating function, so the distributions agree. Now take \(\Delta_2>\Delta_1\ge0\). Couple the corresponding Poisson counts as \(K_2=K_1+J\), where \(K_1\sim\operatorname{Poisson}(\Delta_1/2)\), \(J\sim\operatorname{Poisson}((\Delta_2-\Delta_1)/2)\), and \(K_1,J\) are independent. Then \(K_2\ge K_1\). A central chi-square variable with \(r+2\) degrees of freedom can be represented as an independent \(\chi^2_r\) variable plus a \(\chi^2_2\) variable. Increasing the degrees of freedom in this coupling therefore increases the variable, so the probability of exceeding any fixed positive cutoff cannot decrease. This proves the power ordering for the known-covariance test.

For the noncentral \(F\) distribution, its numerator is a noncentral chi-square variable divided by its degrees of freedom, and its denominator is an independent central chi-square variable divided by its degrees of freedom. Couple the numerators using the preceding argument and keep the same positive denominator in both cases. The \(F\) statistic is then no smaller under \(\Delta_2\) than under \(\Delta_1\), so its upper-tail rejection probability cannot decrease. Hence the estimated-covariance test has the same nondecreasing power ordering. \(\square\)

Shifts That Are Not Eigenvectors

An arbitrary unit direction \(u\) can be written as \(u=\sum_{j=1}^p a_jv_j\), where \(\sum_j a_j^2=1\). For a shift \(\mu=du\), the noncentrality is a weighted combination of the eigenvalue-specific contributions:

Since the weights \(a_j^2\) are nonnegative and sum to one, this quantity lies between \(nd^2/\eta_{\max}\) and \(nd^2/\eta_{\min}\), where \(\eta_{\max}\) and \(\eta_{\min}\) are the largest and smallest eigenvalues. The eigenvector directions attain these bounds.

Interpretation and a Common Pitfall

The ordering is about equal Euclidean shifts. A shift along a high-variance eigenvector is harder to detect because its size is small relative to the standard deviation in that direction. If the shifts instead have equal covariance-standardized length, they have equal noncentrality and hence equal power for these tests. For example, setting the shift length along \(v_j\) to \(d_j=c\sqrt{\eta_j}\) gives \(\Delta=nc^2\), independent of \(j\).

A common mistake is to argue that greater variance in a direction must make a mean shift easier to detect because the data vary more in that direction. That reasoning confuses spread with signal. For a fixed shift, greater spread makes the signal-to-noise ratio smaller: the contribution to noncentrality is \(nd^2/\eta_j\), not \(nd^2\eta_j\). Another mistake is to compare raw shift lengths while ignoring their directions. A Euclidean distance alone does not determine power unless the covariance structure is also accounted for.