Hotelling's T-Squared Sampling Distribution

Why a Multivariate Mean Needs a Joint Statistic

For a vector-valued observation, a discrepancy between a sample mean and a hypothesized mean has several coordinates. Testing each coordinate separately ignores how the coordinates vary together; the sample covariance matrix supplies the scale and dependence needed for a joint comparison. Hotelling’s $T^2$ statistic measures that discrepancy as a squared distance in the geometry determined by the covariance matrix.

Its central sampling result is an exact finite-sample statement, not a large-sample approximation. Under multivariate normal sampling, the sample mean and sample covariance are independent, and the latter has a Wishart distribution. Together these facts turn the quadratic form into a scaled $F$ random variable.

Definition: Let $X_1,\ldots,X_n$ be independent and identically distributed $p$-dimensional random vectors with distribution $N_p(\mu,\Sigma)$, where $\Sigma$ is positive definite. Define $$ \bar X=\frac{1}{n}\sum_{i=1}^n X_i,\qquad S=\frac{1}{n-1}\sum_{i=1}^n (X_i-\bar X)(X_i-\bar X)'. $$ For a specified mean vector $\mu_0$, Hotelling’s statistic is $$ T^2(\mu_0)=n(\bar X-\mu_0)'S^{-1}(\bar X-\mu_0). $$

The inverse $S^{-1}$ exists with probability one when $n>p$ under these assumptions. The factor $n$ reflects the sampling variance of the mean: $\operatorname{Var}(\bar X)=\Sigma/n$. The quadratic form also accounts for correlations among coordinates, so a discrepancy along a high-variance direction is treated differently from one along a low-variance direction.

The Exact Sampling Distribution

The result below concerns the statistic when the specified vector is the true mean. Write $\nu=n-1$. The standard multivariate-normal sampling results give $$ Y=\sqrt{n}\,\Sigma^{-1/2}(\bar X-\mu)\sim N_p(0,I_p), \qquad W=\nu\Sigma^{-1/2}S\Sigma^{-1/2}\sim\operatorname{Wishart}_p(I_p,\nu), $$ with $Y$ and $W$ independent. Here the Wishart degrees of freedom are $\nu$, corresponding to the $n-1$ residual degrees of freedom in $S$.

Theorem: Under the stated independent $N_p(\mu,\Sigma)$ sampling model, if $n>p$, then $$ \frac{n-p}{p(n-1)}T^2(\mu)\sim F_{p,n-p}. $$ Equivalently, $T^2(\mu)$ has the distribution of $\dfrac{p(n-1)}{n-p}F_{p,n-p}$.

We derive the result by first finding the distribution of a normal quadratic form involving an independent Wishart matrix. The chi-square ratio that appears in the derivation also makes the degrees of freedom and the scaling transparent.

Lemma: If $Y\sim N_p(0,I_p)$, $W\sim\operatorname{Wishart}_p(I_p,\nu)$, independently, and $\nu\ge p$, then $$ Y'W^{-1}Y\ \overset{d}{=}\ \frac{U}{V}, $$ where $U\sim\chi^2_p$ and $V\sim\chi^2_{\nu-p+1}$ are independent.

Proof. First consider the $(1,1)$ entry of $W^{-1}$. A Wishart matrix with identity scale has the representation $W=G'G$, where $G$ is a $\nu\times p$ matrix of independent standard normal entries. Write $G=[g_1,G_2]$, with $g_1$ the first column. Since $\nu\ge p$, $G_2$ has rank $p-1$ with probability one when $p>1$; when $p=1$, take $G_2$ to be empty. The block inverse formula gives $$ \frac{1}{(W^{-1})_{11}} =g_1'(I-P_{G_2})g_1, $$ where $P_{G_2}$ is the orthogonal projection onto the column space of $G_2$, and is the zero matrix in the empty-column case. Conditional on $G_2$, $I-P_{G_2}$ is a projection of rank $\nu-p+1$, and $g_1$ is an independent standard normal vector. Thus the right side has a $\chi^2_{\nu-p+1}$ distribution. This proves that $(W^{-1})_{11}$ has the distribution of the reciprocal of such a chi-square variable.

Now condition on $Y=y\ne0$. Choose an orthogonal matrix $O_y$ such that $O_y y=\|y\|e_1$, where $e_1$ is the first coordinate vector. Orthogonal invariance of the identity-scale Wishart distribution implies that $O_y W O_y'$ has the same distribution as $W$. Therefore, conditional on $Y=y$, the quadratic form $Y'W^{-1}Y$ has the distribution of $\|y\|^2/V$, where $V\sim\chi^2_{\nu-p+1}$. This conditional distribution of $V$ does not depend on $y$; consequently $V$ is independent of $\|Y\|^2$. Finally, $\|Y\|^2\sim\chi^2_p$. Since $P(Y=0)=0$, these statements establish the claimed ratio distribution. $\square$

To apply the lemma, rewrite $S=\Sigma^{1/2}W\Sigma^{1/2}/\nu$ and $\bar X-\mu=\Sigma^{1/2}Y/\sqrt n$. Substitution gives $$ T^2(\mu) =n\left(\frac{\Sigma^{1/2}Y}{\sqrt n}\right)' \left(\frac{\Sigma^{1/2}W\Sigma^{1/2}}{\nu}\right)^{-1} \left(\frac{\Sigma^{1/2}Y}{\sqrt n}\right) =\nu Y'W^{-1}Y. $$ The lemma therefore gives $T^2(\mu)\overset{d}{=}\nu U/V$, with independent $U\sim\chi^2_p$ and $V\sim\chi^2_{\nu-p+1}$. By definition, $(U/p)/(V/(\nu-p+1))$ has an $F_{p,\nu-p+1}$ distribution. Since $\nu-p+1=n-p$, multiplying by the required constant yields $$ \frac{n-p}{p(n-1)}T^2(\mu) =\frac{n-p}{p\nu}\left(\nu\frac{U}{V}\right) =\frac{U/p}{V/(n-p)} \sim F_{p,n-p}. $$ This proves the theorem. $\square$

Worked Calculations

Worked Example: A Two-Dimensional Mean Test

Suppose $n=10$, $p=2$, and the null mean is $\mu_0$. The observed difference is $\bar x-\mu_0=(1,2)'$, and the unbiased sample covariance is $$ S=\begin{pmatrix}4&1\\1&9\end{pmatrix}. $$ Calculate the statistic and its exact upper-tail probability under the null.

The determinant of $S$ is $4(9)-1(1)=35>0$, and $$ S^{-1}=\frac{1}{35}\begin{pmatrix}9&-1\\-1&4\end{pmatrix}. $$ Thus $$ (\bar x-\mu_0)'S^{-1}(\bar x-\mu_0) =\frac{1}{35}\bigl(9-2(2)+4(4)\bigr) =\frac{21}{35}=0.6, \qquad T^2=10(0.6)=6. $$ The exact $F$ statistic is $$ \frac{10-2}{2(10-1)}T^2=\frac{4}{9}(6)=\frac{8}{3}, $$ with $2$ and $8$ degrees of freedom. For $F_{2,8}$, the upper-tail probability at $x$ is $(1+x/4)^{-4}$. Hence the exact null upper-tail probability is $$ P(F_{2,8}\ge 8/3)=\left(1+\frac{8/3}{4}\right)^{-4} =\left(\frac{5}{3}\right)^{-4} =\frac{81}{625}\approx0.1296. $$ The data do not give a small upper-tail probability against this null mean.

Worked Example: An Exact Confidence Ellipsoid

Suppose $n=12$, $p=2$, $\bar x=(3,5)'$, and $S=\begin{pmatrix}4&0\\0&1\end{pmatrix}$. Find the form of an exact 95% confidence region for $\mu$.

For any candidate mean $m$, the theorem gives $$ \frac{n-p}{p(n-1)}\,n(\bar X-m)'S^{-1}(\bar X-m)\sim F_{p,n-p} $$ when $m$ is the true mean. Let $F_{2,10;0.95}$ denote the 95th percentile of the $F_{2,10}$ distribution. Inverting this pivot gives the region $$ \left\{m:\ 12(\bar x-m)'S^{-1}(\bar x-m) \le \frac{2(11)}{10}F_{2,10;0.95}\right\}. $$ Because $S^{-1}=\begin{pmatrix}1/4&0\\0&1\end{pmatrix}$, this is $$ \left\{m:\ 12\left(\frac{(m_1-3)^2}{4}+(m_2-5)^2\right) \le 2.2F_{2,10;0.95}\right\}. $$ The different coefficients on the two squared coordinate differences reflect the different sample variances. Under the model, this region contains the true mean with probability exactly $0.95$.

Worked Example: The One-Dimensional Case

Let $p=1$, $n=8$, and suppose the observed sample mean is $0.5$ above the null mean, with sample standard deviation $s=1.2$. Compute $T^2$ and verify its relationship to the usual one-sample $t$ statistic.

In one dimension, $S=s^2=1.44$. Therefore $$ T^2=8\frac{(0.5)^2}{1.44} =\frac{2}{1.44} =\frac{25}{18}\approx1.389. $$ The usual one-sample statistic is $t=\sqrt{8}(0.5)/1.2$, so $$ t^2=\frac{8(0.25)}{1.44}=\frac{25}{18}=T^2. $$ The Hotelling conversion gives $$ \frac{8-1}{1(8-1)}T^2=T^2\sim F_{1,7}. $$ This agrees with the fact that the square of a $t$ random variable with 7 degrees of freedom has an $F_{1,7}$ distribution.

Conditions, Interpretation, and Common Pitfalls

The finite-sample $F$ law depends on the complete model: independent observations from one multivariate normal distribution, positive-definite population covariance, and $n>p$. It is not an exact distribution-free result. Normality supplies the Wishart sample covariance and its independence from the sample mean; changing the sampling assumptions can change the distribution.

A frequent error is to use the right quadratic form with the wrong degrees-of-freedom multiplier. With the unbiased covariance $S$, the conversion is $(n-p)T^2/[p(n-1)]$, not $T^2/p$. For example, the denominator $n-1$ arises because $S$ uses $n-1$ residual degrees of freedom, while the numerator degrees of freedom of the resulting $F$ law are $p$.

Do not substitute a different covariance convention silently. If $C=n^{-1}\sum_i(X_i-\bar X)(X_i-\bar X)'$, then $C=(n-1)S/n$; a statistic formed with $C^{-1}$ is $n/(n-1)$ times the statistic defined here.
Do not replace the estimated covariance with the known population covariance. With known $\Sigma$, the standardized mean quadratic form has a chi-square distribution; estimating covariance produces the scaled $F$ law.
Do not use the inverse when $n\le p$. In that setting the sample covariance is singular with probability one in the stated model, so this statistic is not defined by the formula above.

Hotelling’s statistic is useful because it gives a single joint measure of mean discrepancy and an exact calibration when the model assumptions hold. Its confidence region is an ellipsoid rather than a collection of separate coordinate intervals, preserving the covariance structure in the data. That joint calibration is precisely what separate univariate tests do not provide.

Key takeaway: With the unbiased sample covariance and $n>p$, the one-sample Hotelling statistic satisfies $\dfrac{n-p}{p(n-1)}T^2(\mu)\sim F_{p,n-p}$. Check the covariance convention and both degrees of freedom before using this exact calibration.

Check Your Understanding

Use the definitions and derivation above to answer these questions.

What assumptions and sample-size condition are needed for the exact Hotelling $T^2$ distribution?
If $n=15$ and $p=3$, what are the two degrees of freedom of the $F$ distribution, and what multiplier converts $T^2$ to that $F$ statistic?
Why does the Wishart inverse contribute a chi-square variable with $\nu-p+1$ degrees of freedom in the derivation?
For $p=1$, how does Hotelling’s statistic relate to the square of the one-sample $t$ statistic?
How does defining the sample covariance with divisor $n$, rather than $n-1$, change the quadratic statistic?

Hotelling's T-Squared Sampling Distribution

What You'll Learn