Power of the Pooled Two-Sample Hotelling Test

Why Power Depends on a Multivariate Difference

A test of equality between two mean vectors can miss a real difference, especially when the difference is small relative to the groups’ variability or the sample sizes are limited. Its power is the probability of rejecting equality when a particular alternative is true. For the pooled two-sample Hotelling test, exact power is available under multivariate normal sampling with a common covariance matrix. The alternative enters through a single noncentrality parameter.

This parameter combines the size and direction of the mean difference, measured relative to the covariance, with the effective sample size. The test’s degrees of freedom also matter: they determine the critical value against which the test statistic is compared.

Test Statistic and Power Ingredients

Definition: Let $X_{1,1},\ldots,X_{1,n_1}$ and $X_{2,1},\ldots,X_{2,n_2}$ be two independent samples from $p$-variate normal distributions with means $\mu_1,\mu_2$ and a common positive-definite covariance matrix $\Sigma$. Write $N=n_1+n_2$, $\bar X_1,\bar X_2$ for the sample mean vectors, and $$ S_p=\frac{(n_1-1)S_1+(n_2-1)S_2}{N-2} $$ for the pooled sample covariance matrix. The pooled two-sample Hotelling statistic is $$ T^2=\frac{n_1n_2}{N}(\bar X_1-\bar X_2)'S_p^{-1}(\bar X_1-\bar X_2). $$ For testing $H_0:\mu_1-\mu_2=0$, its usual exact calibration is $$ F_{\mathrm{obs}}=\frac{N-p-1}{p(N-2)}T^2. $$

Assume $N>p+1$, so the denominator degrees of freedom below are positive. At significance level $\alpha$, let $c_\alpha$ denote the $1-\alpha$ quantile of the central $F$ distribution with $p$ and $N-p-1$ degrees of freedom. The test rejects when $F_{\mathrm{obs}}>c_\alpha$.

Definition: For a specified alternative $\delta=\mu_1-\mu_2$, define the noncentrality parameter $$ \lambda=\frac{n_1n_2}{N}\delta'\Sigma^{-1}\delta. $$ The noncentral $F$ distribution with degrees of freedom $p,\nu$ and noncentrality $\lambda$, denoted $F_{p,\nu}(\lambda)$, is the distribution of $(U/p)/(V/\nu)$, where $U$ and $V$ are independent, $U$ has a noncentral chi-square distribution with $p$ degrees of freedom and noncentrality $\lambda$, and $V$ has a central chi-square distribution with $\nu$ degrees of freedom.

The quadratic form $\delta'\Sigma^{-1}\delta$ measures the squared size of the difference in covariance-standardized units. A direction with high variance contributes less than an equally large difference in a low-variance direction. The multiplier $n_1n_2/N$ is the effective sample size for the difference of two independent sample means.

Exact Noncentral F Law

Theorem: Under the independent multivariate normal, common-covariance assumptions above, if $\delta=\mu_1-\mu_2$, then $$ \frac{N-p-1}{p(N-2)}T^2\sim F_{p,N-p-1}(\lambda), \qquad \lambda=\frac{n_1n_2}{N}\delta'\Sigma^{-1}\delta. $$ Consequently, the exact power at level $\alpha$ for this specified alternative is $$ \operatorname{Power}(\delta) =P\!\left\{F_{p,N-p-1}(\lambda)>c_\alpha\right\} =1-F_{p,N-p-1,\lambda}(c_\alpha), $$ where $F_{p,\nu,\lambda}$ is the noncentral $F$ cumulative distribution function.

Derivation. Put $m=N-2$ and $a=n_1n_2/N$. Under the stated assumptions, the sample mean difference is normal with mean $\delta$ and covariance $(1/n_1+1/n_2)\Sigma=\Sigma/a$. The pooled covariance is independent of that difference, and $mS_p$ has a Wishart distribution with scale matrix $\Sigma$ and $m$ degrees of freedom. Whiten both quantities by $\Sigma$, and define $$ Z=\sqrt{a}\,\Sigma^{-1/2}(\bar X_1-\bar X_2), \qquad W=\Sigma^{-1/2}(mS_p)\Sigma^{-1/2}. $$ Then $Z$ is normal with identity covariance and mean $\sqrt{a}\Sigma^{-1/2}\delta$, while $W$ is an independent Wishart matrix with identity scale and $m$ degrees of freedom. Thus $Z'Z$ is noncentral chi-square with $p$ degrees of freedom and noncentrality $a\delta'\Sigma^{-1}\delta=\lambda$.

For completeness, the inverse-Wishart quadratic form supplies the denominator degrees of freedom. Conditional on any nonzero $Z=z$, orthogonal invariance of $W$ allows a rotation taking $z/\lVert z\rVert$ to the first coordinate vector. Partition $W$ into its first row and column and the remaining $p-1$ coordinates. The reciprocal of the first diagonal element of $W^{-1}$ is the Schur complement of the lower-right block. Representing $W=G'G$ with $G$ an $m$-by-$p$ matrix of independent standard normal entries shows that this Schur complement is the residual sum of squares after projection onto $p-1$ columns of $G$. Its rank is $m-p+1$, so it has a chi-square distribution with $\nu=m-p+1=N-p-1$ degrees of freedom. Its conditional distribution does not depend on $z$, hence this chi-square variable is independent of $Z$. It follows that $Z'W^{-1}Z=(Z'Z)/V$ for an independent $V\sim\chi^2_\nu$.

Since $S_p=\Sigma^{1/2}W\Sigma^{1/2}/m$, the statistic satisfies $T^2=mZ'W^{-1}Z$. Therefore $$ \frac{\nu}{pm}T^2 =\frac{(Z'Z)/p}{V/\nu}, $$ which is precisely the defined noncentral $F$ distribution. Substituting $m=N-2$ and the expression for $\lambda$ proves the distributional statement. The power formula follows because the test rejects exactly when this statistic exceeds $c_\alpha$. $\square$

This is an exact result, not a large-sample approximation. Its exactness depends on the normal model and common covariance matrix; the formula does not automatically apply if those assumptions fail.

Worked Power Calculations

Worked Example: Computing Noncentrality From a Mean Difference

Suppose $p=2$, $n_1=8$, $n_2=12$, the common covariance is $\Sigma=\begin{pmatrix}4&0\\0&1\end{pmatrix}$, and the specified mean difference is $\delta=(1,1)'$. Find the noncentrality parameter and express the power at level $0.05$.

First, the effective sample size is $$ \frac{n_1n_2}{N}=\frac{8\cdot12}{20}=4.8. $$ The inverse covariance is $\Sigma^{-1}=\begin{pmatrix}1/4&0\\0&1\end{pmatrix}$, so $$ \delta'\Sigma^{-1}\delta =(1,1)\begin{pmatrix}1/4&0\\0&1\end{pmatrix}\begin{pmatrix}1\\1\end{pmatrix} =\frac14+1=1.25. $$ Thus $\lambda=4.8(1.25)=6$. Here $N-p-1=17$, so the exact power is $$ P\!\left\{F_{2,17}(6)>F_{2,17;0.95}\right\}. $$ This expression is the noncentral $F$ upper-tail probability evaluated at the central $F$ critical value. The first coordinate of the difference is down-weighted because its variance is $4$, whereas the second coordinate has variance $1$.

In general, a numerical power calculation evaluates the noncentral $F$ upper tail at the central critical value. A useful closed-form case provides a check on both the parameterization and the arithmetic.

Result: If $p=2$, $\nu=2$, and the critical value is $c>0$, then $$ P\{F_{2,2}(\lambda)>c\} =1-\frac{c}{c+1}\exp\!\left(-\frac{\lambda}{2(c+1)}\right). $$

Derivation. By the definition of a noncentral chi-square distribution, $U\sim\chi^2_2(\lambda)$ can be represented as a mixture: $U\mid J\sim\chi^2_{2+2J}$, where $J$ has a Poisson distribution with mean $\lambda/2$. Also $V\sim\chi^2_2$ is exponential with mean $2$. Since $F=(U/2)/(V/2)=U/V$, conditioning on $U$ gives $$ P(F>c\mid U)=P(V<U/c\mid U)=1-\exp\!\left(-\frac{U}{2c}\right). $$ For a chi-square variable with $k$ degrees of freedom, its Laplace transform gives $E[\exp(-U/(2c))]=(1+1/c)^{-k/2}$. Averaging over $J$ therefore yields $$ E\!\left[\exp\!\left(-\frac{U}{2c}\right)\right] =\left(1+\frac1c\right)^{-1} \exp\!\left\{\frac{\lambda}{2}\left[\left(1+\frac1c\right)^{-1}-1\right]\right\} =\frac{c}{c+1}\exp\!\left(-\frac{\lambda}{2(c+1)}\right). $$ Taking one minus this expectation proves the result. $\square$

Worked Example: Exact Power for a Small Two-Group Design

Let $p=2$, $n_1=2$, $n_2=3$, so $N=5$ and $N-p-1=2$. At $\alpha=0.05$, the central $F_{2,2}$ upper-tail probability at $c$ is $1/(1+c)$; setting it equal to $0.05$ gives $c=19$. For this critical value, the closed-form result is $$ \operatorname{Power}(\lambda) =1-\frac{19}{20}\exp\!\left(-\frac{\lambda}{40}\right). $$

For $\lambda=10$, $\exp(-10/40)=e^{-0.25}\approx0.778801$. Hence $$ \operatorname{Power}(10) =1-0.95e^{-0.25} \approx1-0.739861 =0.260139. $$ For comparison, when $\lambda=2$, $e^{-2/40}=e^{-0.05}\approx0.951229425$, and $$ \operatorname{Power}(2) =1-0.95e^{-0.05} \approx1-0.903667953 =0.096332047. $$ Both powers exceed the significance level, as they should for these nonzero noncentralities, and the larger noncentrality has greater power.

For example, with $\Sigma=I_2$, the noncentrality is $(6/5)\delta'\delta$. Taking $\delta=(5/\sqrt{3},0)'$ gives $\delta'\delta=25/3$ and $\lambda=(6/5)(25/3)=10$, verifying the first calculation’s compatibility with a specified mean difference.

Worked Example: Noncentrality Needed for a Target Power

In the same $p=2$, $n_1=2$, $n_2=3$, level-$0.05$ setting, find the noncentrality required for power $0.80$. Starting with the exact formula, $$ 0.80=1-0.95e^{-\lambda/40}, \qquad e^{-\lambda/40}=\frac{0.20}{0.95}=\frac4{19}. $$ Taking logarithms gives $$ \lambda=-40\log(4/19)\approx62.3258. $$ If $\Sigma=I_2$ and the difference is placed along the first coordinate, then $\lambda=(6/5)\delta'\delta$. Thus the required squared difference in that coordinate is $$ \delta'\delta=\frac{5}{6}\lambda\approx51.9382, $$ or a difference of approximately $\sqrt{51.9382}\approx7.207$. This is a large separation relative to the small effective sample size, illustrating why a high target power can require a substantial mean difference in a very small study.

Using and Interpreting the Power

For a general design, the calculation follows a direct sequence: obtain the degrees of freedom $p$ and $N-p-1$; compute $\lambda$ from the specified $\delta$, $\Sigma$, $n_1$, and $n_2$; find the central critical value at the chosen $\alpha$; then evaluate the noncentral $F$ upper tail. Statistical software typically provides this last step through a noncentral $F$ survival function. Supplying a central $F$ distribution instead would calculate the null rejection probability, not power under the specified alternative.

Power is specific to the alternative. A single number does not summarize performance for every possible mean difference: changing the direction of $\delta$, the covariance, or either sample size can change $\lambda$, the degrees of freedom, or both. When reporting power, state the dimension, group sizes, significance level, assumed common covariance, and mean difference (or its standardized specification).

Common Mistakes

Using the wrong effective sample size. The multiplier is $n_1n_2/(n_1+n_2)$, not $n_1+n_2$ or either group size alone.
Omitting covariance standardization. The mean difference enters as $\delta'\Sigma^{-1}\delta$, not simply as the unweighted squared Euclidean distance unless $\Sigma=I$.
Using a noncentral critical value. The rejection threshold is the central $F$ quantile under the null. The noncentral distribution is used to evaluate the rejection probability under the alternative.
Confusing noncentrality conventions. Here $\lambda$ is the noncentrality of the chi-square numerator in the noncentral $F$; it is not its square root.
Rounding intermediate calculations too early. For example, $0.95e^{-0.25}=0.739860744$, so the corresponding power is $0.260139256$, approximately $0.2601$. Also, $0.95e^{-0.05}=0.903667953$, giving power $0.096332047$, approximately $0.0963$.

Key takeaway: Under independent multivariate normal sampling with a common positive-definite covariance matrix, exact power is the noncentral $F$ upper-tail probability at the central null critical value, with $\lambda=(n_1n_2/N)\delta'\Sigma^{-1}\delta$.

Check Your Understanding

Answer each question using the assumptions and definitions stated above.

For $p=3$, $n_1=15$, and $n_2=10$, what are the numerator and denominator degrees of freedom of the test’s $F$ statistic?
Why does a difference in a high-variance direction contribute less to $\lambda$ than an equally sized difference in a low-variance direction?
For a specified alternative, which distribution supplies the critical value, and which distribution supplies the power calculation?
With $p=2$, $\nu=2$, and critical value $c=19$, write the exact power as a function of $\lambda$.
Name two assumptions needed for the exact noncentral $F$ calibration described here.