Deriving Hotelling's F Calibration

Why the Calibration Is Not a Chi-Square Approximation

When the covariance matrix is known, a Gaussian sample mean can be standardized using that fixed matrix. With an unknown covariance matrix, Hotelling's statistic instead uses the sample covariance, which is random. The exact null calibration must therefore account for the variability introduced by estimating covariance. Under multivariate normal sampling, the sample mean and sample covariance are independent, and the covariance estimate has a Wishart distribution. These facts lead to an exact finite-sample \(F\) distribution, not merely a large-sample approximation.

We derive that result for testing a specified mean vector. The essential step is to find the distribution of a quadratic form involving the inverse of a Wishart matrix. Its denominator degrees of freedom depend on both the Wishart degrees of freedom and the dimension of the vector.

Setup and Exact Calibration

The exact result below concerns the distribution under \(H_0\), so throughout the derivation we take \(\mu=\mu_0\). The formula uses the convention that \(F_{r,s}\) is the distribution of \((U/r)/(V/s)\), where \(U\) and \(V\) are independent chi-square random variables with \(r\) and \(s\) degrees of freedom.

The scaling constant and both degrees of freedom matter. We now derive them from the Gaussian and Wishart sampling laws rather than treating the transformation as a formula to memorize.

The Wishart Quadratic-Form Argument

Under the null, let \(A=\sum_{i=1}^n(X_i-\bar X)(X_i-\bar X)'=(n-1)S\). The Gaussian sampling laws give \(\bar X\sim N_p(\mu_0,\Sigma/n)\), \(A\sim\operatorname{Wishart}_p(\Sigma,n-1)\), and independence of \(\bar X\) and \(A\). These are the mean and covariance sampling laws used in the earlier material on sampling laws of the sample mean and covariance.

Whiten both random quantities by the symmetric positive definite matrix \(\Sigma^{-1/2}\), and put \( Z=\sqrt n\,\Sigma^{-1/2}(\bar X-\mu_0) \) and \( W=\Sigma^{-1/2}A\Sigma^{-1/2} \). Then \(Z\sim N_p(0,I_p)\), \(W\sim\operatorname{Wishart}_p(I_p,\nu)\) for \(\nu=n-1\), and \(Z\) and \(W\) are independent. Substitution into \(T^2\) gives \( T^2=(n-1)Z'W^{-1}Z \). It remains to derive the law of \(Z'W^{-1}Z\).

Proof. A Wishart matrix with identity scale and \(\nu\) degrees of freedom can be represented as \(W=G'G\), where \(G\) is a \(\nu\) by \(p\) matrix of independent standard normal entries. Write \(G=[g_1,G_2]\), separating its first column, and partition \(W\) conformably. The Schur complement of the \(G_2'G_2\) block is \( Q=g_1'(I-P_{G_2})g_1 \), where \(P_{G_2}=G_2(G_2'G_2)^{-1}G_2'\) is the orthogonal projection onto the columns of \(G_2\). When \(p=1\), take \(P_{G_2}=0\).

The columns of \(G_2\) have full rank with probability one. Conditional on \(G_2\), \(I-P_{G_2}\) is a projection of rank \(\nu-(p-1)=\nu-p+1\), and \(g_1\) is an independent standard Gaussian vector. In an orthonormal basis for this projection, \(Q\) is the sum of squares of \(\nu-p+1\) independent standard normal variables. Thus \(Q\sim\chi^2_{\nu-p+1}\). The block inverse formula gives \((W^{-1})_{11}=Q^{-1}\).

For any fixed unit vector \(u\), orthogonal invariance of \(W\) implies that \(u'W^{-1}u\) has the same distribution as \((W^{-1})_{11}\). Write \(Z=Ru\), where \(R^2=Z'Z\). Then \(R^2\sim\chi^2_p\). Conditional on any nonzero value of \(Z\), the distribution of \(u'W^{-1}u\) is the same, namely that of \(1/V\) for \(V\sim\chi^2_{\nu-p+1}\). This conditional distribution does not depend on \(Z\); consequently \(R^2\) and \(u'W^{-1}u\) are independent. Therefore \( Z'W^{-1}Z=R^2u'W^{-1}u \) has the distribution of \(U/V\), with independent \(U\sim\chi^2_p\) and \(V\sim\chi^2_{\nu-p+1}\). By the definition of the \(F\) distribution, \( U/V=\frac{p}{\nu-p+1}\frac{U/p}{V/(\nu-p+1)} \), which proves the lemma. \(\square\)

Deriving Hotelling's \(F\) Transformation

Apply the lemma with \(\nu=n-1\). The denominator degrees of freedom in the lemma become \(\nu-p+1=(n-1)-p+1=n-p\). Hence \( Z'W^{-1}Z\sim\frac{p}{n-p}F_{p,n-p} \). Because \(T^2=(n-1)Z'W^{-1}Z\), multiplying by \(n-p\) and dividing by \(p(n-1)\) yields the theorem's stated calibration.

This derivation also explains the degrees of freedom. The numerator \(p\) comes from the squared length of a \(p\)-dimensional standard Gaussian vector. The denominator \(n-p\) comes from the residual projection after accounting for the other \(p-1\) columns in the Wishart representation. Estimating covariance therefore changes the reference distribution from the known-covariance chi-square calibration.

Common Mistakes and Why the Result Matters

A frequent error is to use \((n-1)/n\) as the calibration multiplier, or to assign \(n-1\) denominator degrees of freedom. The sample covariance is defined with divisor \(n-1\), but the denominator degrees of freedom for the \(F\) law are \(n-p\), not \(n-1\). The distinction comes from the dimension-dependent Wishart inverse calculation; it is not a discretionary correction.

Another mistake is to treat the known-covariance chi-square result as exact when the covariance is estimated. Under the Gaussian null, the mean and covariance are independent, but independence alone does not make the covariance fixed. Its Wishart variation produces the denominator chi-square component, and therefore the finite-sample \(F\) law. The exact calibration requires all the stated conditions: independent multivariate normal observations, positive definite \(\Sigma\), the null mean, and \(n>p\). Without these conditions, this derivation does not establish the same finite-sample reference distribution.