Why Dimension Changes the Test
Hotelling’s \(T^2\) test compares a multivariate sample mean with a specified mean while accounting for covariance among coordinates. Its defining feature is the inverse sample covariance: the test measures the mean difference after rescaling in directions of high or low variability. This works only if the sample covariance has an inverse. When the dimension is large relative to the number of observations, that condition can fail outright; even before it does, the null scale of the statistic can differ substantially from the familiar fixed-dimension approximation.
The distinction is important. Under a multivariate normal model, the classical finite-sample F calibration remains exact whenever the sample covariance is invertible. What breaks down as dimension grows is the usual fixed-dimension chi-square calibration, and at dimension at least the sample size the ordinary statistic itself is undefined.
Here \(p\) is the number of coordinates measured on each observation. In the classical one-sample setting, the exact distributional result assumes independent observations from \(N_p(\boldsymbol\mu,\boldsymbol\Sigma)\), with \(\boldsymbol\Sigma\) positive definite. The null hypothesis also needs to hold for the stated null distribution.
Why the Sample Covariance Becomes Singular
Centering the observations imposes a linear constraint: the centered observation vectors sum to zero. Thus there can be at most \(n-1\) linearly independent centered vectors, regardless of how many coordinates each observation has. The covariance matrix is built from these vectors, so its rank cannot exceed \(n-1\).
To verify the rank bound, form the \(n\times p\) matrix \(\mathbf A\) whose \(i\)th row is \((\mathbf X_i-\bar{\mathbf X})^\mathsf{T}\). Then \(\mathbf S=\mathbf A^\mathsf{T}\mathbf A/(n-1)\). The rows of \(\mathbf A\) sum to zero, so \(\operatorname{rank}(\mathbf A)\le n-1\); also, \(\operatorname{rank}(\mathbf A)\le p\). Since \(\operatorname{rank}(\mathbf A^\mathsf{T}\mathbf A)=\operatorname{rank}(\mathbf A)\), the claimed bound follows. When \(p\ge n\), this rank is strictly less than \(p\), so \(\mathbf S\) cannot have an inverse. \(\square\)
Worked Example: Six Observations and Six Coordinates
Suppose \(n=6\) patients are measured on \(p=6\) biomarkers. Each centered patient vector has six coordinates, but the six centered vectors sum to the zero vector. Therefore, at most five can be linearly independent.
The \(6\times6\) centered data matrix has rank at most \(5\), and its cross-product \(\mathbf A^\mathsf{T}\mathbf A\) has rank at most \(5\). Consequently, the \(6\times6\) sample covariance matrix is singular. The problem is not that a particular numerical algorithm has failed: the inverse required by the definition of \(T^2\) does not exist.
This argument applies for any \(p\ge6\) with these six observations. It is a rank constraint from centering, not a claim that the population covariance is singular.
Exact F Calibration Under Normality
For \(p<n\), the sample covariance can be invertible. Under multivariate normal sampling, the null distribution of \(T^2\) is then related exactly to an F distribution. This finite-sample result is distinct from approximating \(T^2\) by a chi-square variable.
A derivation makes the assumptions and degrees of freedom visible. Under the null, whiten and rescale the sample mean difference to obtain \(\mathbf Y=\sqrt n\,\boldsymbol\Sigma^{-1/2}(\bar{\mathbf X}-\boldsymbol\mu_0)\sim N_p(\mathbf0,\mathbf I_p)\). Let \(\mathbf W=(n-1)\boldsymbol\Sigma^{-1/2}\mathbf S\boldsymbol\Sigma^{-1/2}\). Normal-sample theory gives \(\mathbf W\sim\operatorname{Wishart}_p(\mathbf I_p,n-1)\), independently of \(\mathbf Y\), and \(T^2=(n-1)\mathbf Y^\mathsf{T}\mathbf W^{-1}\mathbf Y\).
Set \(\nu=n-1\). Rotational invariance of the standard normal vector and the identity-scale Wishart matrix lets us rotate the direction of \(\mathbf Y\) to the first coordinate without changing the distribution of \(\mathbf W\). Thus \(\mathbf Y^\mathsf{T}\mathbf W^{-1}\mathbf Y\) has the distribution of \(\|\mathbf Y\|^2(\mathbf W^{-1})_{11}\). The first factor has a chi-square distribution with \(p\) degrees of freedom. For the second, partition \(\mathbf W\) into its first row and column and the remaining block. The block-inverse formula gives \((\mathbf W^{-1})_{11}=1/(W_{11}-\mathbf W_{12}\mathbf W_{22}^{-1}\mathbf W_{21})\).
The denominator is the residual sum of squares from projecting the first Gaussian column of a \(\nu\times p\) representation of \(\mathbf W\) onto the other \(p-1\) columns. Conditional on those columns, it is the squared length of a standard Gaussian projection onto a subspace of dimension \(\nu-p+1\). It therefore has a chi-square distribution with \(\nu-p+1=n-p\) degrees of freedom. It is independent of \(\|\mathbf Y\|^2\), since \(\mathbf Y\) is independent of \(\mathbf W\). Hence \(\mathbf Y^\mathsf{T}\mathbf W^{-1}\mathbf Y\) has the distribution of \(\chi^2_p/\chi^2_{n-p}\), with independent numerator and denominator. Rewriting this ratio in F form gives \(\mathbf Y^\mathsf{T}\mathbf W^{-1}\mathbf Y\sim \frac{p}{n-p}F_{p,n-p}\). Multiplication by \(n-1\) proves the theorem. \(\square\)
Worked Example: Using the Exact Calibration
Suppose \(n=10\) observations are sampled from a multivariate normal population with \(p=3\) coordinates, and the null mean is being tested. Since \(10>3\), the sample covariance is invertible with probability one under this model. The theorem gives \(\frac{10-3}{3(10-1)}T^2=\frac{7}{27}T^2\sim F_{3,7}\) under the null.
For a level-\(\alpha\) upper-tail test, reject when \(\frac{7}{27}T^2\) exceeds the upper \(1-\alpha\) quantile of \(F_{3,7}\). Equivalently, reject when \(T^2\) exceeds \(27/7\) times that quantile. The finite-sample reference distribution is \(F_{3,7}\), not a chi-square distribution with three degrees of freedom. The exact calibration relies on normal sampling and \(n>p\).
Why Fixed-Dimension Chi-Square Calibration Breaks Down
When \(p\) is fixed and \(n\) grows, the scaling factor in the exact F result approaches the one that leads to a chi-square approximation. But if \(p\) grows as a substantial fraction of \(n\), the sample covariance inverse changes the null scale. The following derivation shows the effect under the same normal model, where the exact distribution is available.
To prove the first convergence, write the exact distribution as \(T^2=\frac{p(n-1)}{n-p}F\), where \(F=(U/p)/(V/(n-p))\), \(U\sim\chi^2_p\), \(V\sim\chi^2_{n-p}\), and \(U,V\) are independent. A chi-square variable with \(k\) degrees of freedom has mean \(k\) and variance \(2k\), so Chebyshev’s inequality gives \(P(|U/p-1|>\varepsilon)\le 2/(p\varepsilon^2)\to0\). Likewise \(V/(n-p)\xrightarrow{p}1\), because \(n-p\to\infty\). The denominator therefore converges to one and is bounded away from zero with probability tending to one; it follows that \(F\xrightarrow{p}1\). Finally, \(\frac{T^2}{p}=\frac{n-1}{n-p}F\xrightarrow{p}1/(1-c)\). For comparison, if \(Q\sim\chi^2_p\), the same variance calculation gives \(Q/p\xrightarrow{p}1\). Since \(c>0\), these limiting scales differ. \(\square\)
Worked Example: Dimension Half the Sample Size
Consider a sequence of normal-model tests with \(n=200\) observations and \(p=100\) coordinates, or with both quantities increasing while \(p/n\) stays near \(1/2\). The exact null relation at \(n=200,p=100\) is \(\frac{100}{100\cdot199}T^2=\frac{1}{199}T^2\sim F_{100,100}\).
In the growing-dimension regime with ratio \(1/2\), the result above says \(T^2/p\) is near \(1/(1-1/2)=2\) under the null. A chi-square reference with \(p=100\) instead has its value divided by \(100\) near \(1\). Thus a chi-square cutoff is on the wrong scale: as the dimensions grow with this ratio, a test using that fixed-dimension calibration can reject with probability tending to one under the null. The exact F calibration remains valid here because \(p<n\) and normality is assumed.
What Fails—and What Does Not
The phrase “Hotelling’s \(T^2\) fails” needs qualification. There are two different issues:
- At \(p\ge n\): the ordinary sample covariance is singular, so the statistic using its inverse is not defined.
- When \(p/n\) stays positive but below one: the statistic is defined, but its null scale is not the fixed-dimension chi-square scale. Under normality, the exact F calibration accounts for this finite-sample effect.
- Without normality: the exact F result does not follow from the stated theorem. A chi-square approximation justified for fixed \(p\) cannot simply be assumed to remain valid when \(p\) grows with \(n\).
Even when \(p<n\), a covariance matrix can be difficult to estimate accurately when the dimension is close to the sample size. Inverting a matrix magnifies errors in directions associated with small eigenvalues. This is a numerical and statistical concern distinct from exact singularity: a matrix may be invertible but poorly conditioned. The Gaussian F result still provides exact null calibration in its setting, but it does not make covariance inversion stable or guarantee useful power against every alternative.
Common Pitfall
Correction: The theorem is exact for every \(n>p\) under its multivariate normal assumptions, including when \(p/n\) is close to one. The problem in that regime is that fixed-dimension chi-square calibration is not appropriate, and covariance inversion can be unstable. At \(p\ge n\), the ordinary statistic cannot be computed because the covariance matrix is singular.
The practical lesson is to check the dimension-to-sample-size relationship before interpreting a Hotelling test. First verify \(p<n\), then identify which null calibration is being used and whether its assumptions apply. A chi-square approximation inherited from a fixed-\(p\) argument is not interchangeable with the exact finite-sample F calibration.
Check Your Understanding
Answer each question by connecting the matrix rank or null calibration to the setting described.
- Why does centering \(n\) observations limit the rank of their sample covariance to at most \(n-1\)?
- For \(p=8\) and \(n=8\), can the ordinary one-sample Hotelling statistic be computed? Explain.
- State the exact null distribution of the scaled \(T^2\) statistic under multivariate normal sampling when \(n>p\).
- If \(p/n\to c\in(0,1)\), what is the probability limit of \(T^2/p\) under the normal null?
- Does a large dimension-to-sample-size ratio by itself invalidate the exact F calibration? What conditions still matter?