Hotelling's T-Squared with Unknown Covariance

Why Estimate Covariance in a Mean Test?

A test of a multivariate mean asks whether a population mean vector equals a specified vector. When the population covariance matrix is known, the displacement between the sample mean and the specified mean can be standardized using that matrix. In practice, the covariance is often unknown and must be estimated from the same sample. Hotelling’s \(T^2\) is the resulting one-sample statistic: it measures the sample mean’s squared displacement from the specified value relative to the observed covariance.

The statistic is a single joint measure. It accounts for the scales of the variables and their correlations, rather than testing each coordinate separately. Its construction is simple, but its use depends on an important dimension condition: the sample covariance must be invertible. Exact finite-sample inference also relies on a sampling model, which must be kept distinct from the algebraic definition of the statistic.

Definition and Conditions

The factor \(n\) is part of the definition. The covariance matrix of the sample mean, when the population covariance is \(\Sigma\), is \(\Sigma/n\); accordingly, the factor \(n\) scales the squared mean displacement to the scale of a single observation’s covariance. The matrix inverse combines all coordinates, including their associations.

For this tutorial’s exact finite-sample testing interpretation, assume \(X_i\sim N_p(\mu,\Sigma)\) independently, where \(\Sigma\) is positive definite. The statistic itself can be calculated for other data whenever \(S\) is invertible, but the Gaussian exact calibration does not automatically apply to nonnormal data. The Gaussian model and the null hypothesis \(H_0:\mu=\mu_0\) specify the setting in which the statistic is used for exact inference.

Why the Sample-Size Condition Is Necessary

Arrange the centered observations as the rows of an \(n\)-by-\(p\) matrix \(Y\), with row \(i\) equal to \((X_i-\bar X)'\). Then \(S=Y'Y/(n-1)\). Since the rows of \(Y\) sum to zero, they are linearly dependent whenever \(n>0\), and \(\operatorname{rank}(Y)\le n-1\). Also, \(Y\) has only \(p\) columns. Therefore $$ \operatorname{rank}(S)=\operatorname{rank}(Y'Y)=\operatorname{rank}(Y)\le \min(p,n-1). $$ If \(n\le p\), this upper bound is at most \(p-1\), so \(S\) cannot have rank \(p\) and cannot be inverted.

For the sufficiency claim under the stated Gaussian model, choose an \(n\)-by-\((n-1)\) matrix \(Q\) whose columns are an orthonormal basis for the subspace perpendicular to the all-ones vector. The matrix \(Q'Y\) has \(n-1\) rows and \(p\) columns; under Gaussian sampling, its rows are independent \(N_p(0,\Sigma)\) vectors. If \(n-1\ge p\), these rows have a joint density, and the first \(p\) rows form a nonsingular \(p\)-by-\(p\) matrix with probability one. Indeed, its determinant is a nonzero polynomial in continuously distributed entries, whose zero set has probability zero. Hence \(Q'Y\), and therefore \(Y\), has rank \(p\) with probability one. This proves the stated condition. \(\square\)

Worked Calculations

Using \(T^2\) to Test a Specified Mean

The one-sample null and alternative are $$ H_0:\mu=\mu_0 \qquad\text{versus}\qquad H_A:\mu\ne\mu_0. $$ Under the Gaussian model and \(n>p\), compute \(T^2\) from the observed sample. Large values indicate that the sample mean is far from \(\mu_0\) relative to the estimated covariance, and therefore provide evidence against \(H_0\). For a test of size \(\alpha\), reject when \(T^2\) exceeds the upper-tail critical value from its null distribution for the given \(n\) and \(p\). The exact conversion to an \(F\) reference distribution is developed in the subsequent calibration material; it should not be replaced by an unqualified chi-square reference in a finite sample.

This decision rule uses the joint pattern of deviations. For instance, if the variables are correlated, a displacement along a direction in which observations naturally vary substantially is treated differently from an equally long displacement in a direction with little natural variation. This is why the inverse covariance matrix appears in the quadratic form.

Common Mistakes and Practical Interpretation

• Using the wrong covariance divisor. The definition here uses \(n-1\), the unbiased sample covariance. Replacing it with a covariance divided by \(n\) changes the statistic.
• Forgetting the factor \(n\). The statistic is \(n(\bar X-\mu_0)'S^{-1}(\bar X-\mu_0)\), not just the quadratic form without \(n\).
• Checking only that there are observations. For \(p\) variables, one needs \(n>p\), and with arbitrary data one must also check that the centered observations span all \(p\) dimensions.
• Testing coordinates one at a time and calling that the same procedure. Separate tests do not, by themselves, provide the same joint test; they do not use the full covariance structure in a single statistic.
• Applying a Gaussian exact calibration without its assumptions. The formula for \(T^2\) may be calculable for nonnormal observations, but its exact finite-sample reference distribution requires the specified Gaussian sampling model.

The practical value of Hotelling’s statistic is that a vector-valued mean hypothesis becomes one covariance-adjusted test. Its computation requires the sample mean, the unbiased sample covariance, and an invertible covariance estimate. Its interpretation then depends on the null distribution appropriate to the model and the sample dimensions.