Sample-Size Planning for Hotelling's Test

Planning with an Estimated Covariance Matrix

When the covariance matrix is known, a mean-vector test can use it directly to standardize the observed difference. In practice, it is often estimated from the same sample. That estimation changes the finite-sample distribution of the test statistic, so a sample size calculated using a known-covariance test need not deliver its nominal power for Hotelling’s test.

For exact planning, specify a meaningful alternative mean, a covariance matrix (or an equivalent standardized effect), the significance level, and the target power. The unknown-covariance one-sample Hotelling test then gives an exact finite-sample power calculation under multivariate normal sampling.

Model, Effect Size, and Feasibility

Definition: Let $X_1,\ldots,X_n$ be independent and identically distributed $p$-dimensional random vectors with $X_i\sim N_p(\mu,\Sigma)$, where $\Sigma$ is positive definite. For testing $H_0:\mu=\mu_0$, define the squared Mahalanobis effect size at an alternative mean $\mu$ by $$ \delta^2=(\mu-\mu_0)^\top\Sigma^{-1}(\mu-\mu_0). $$ For a specified significance level $\alpha$, target power $1-\beta$, and effect size $\delta^2>0$, sample-size planning seeks the smallest feasible integer $n$ whose test power is at least $1-\beta$.

The effect size is dimensionless: it measures the mean difference after accounting for the covariance structure. It is not enough to specify the raw coordinate differences unless the covariance is also specified. A difference that looks large in a highly variable direction may represent a modest standardized effect; a smaller difference along a low-variance direction can represent a substantial one.

The sample covariance matrix has rank at most $n-1$, because deviations from the sample mean sum to zero. It therefore cannot be invertible unless $n-1\ge p$, or equivalently $n>p$. Under the stated Gaussian model, when $n>p$, the sample covariance is positive definite with probability one. Thus the ordinary inverse-covariance Hotelling test requires at least $p+1$ observations.

Exact Power from the Noncentral F Distribution

Write $\bar X$ for the sample mean and $S$ for the sample covariance matrix, using divisor $n-1$. Hotelling’s statistic and its exact F calibration are

T^2=n(\bar X-\mu_0)^\top S^{-1}(\bar X-\mu_0), \qquad F_{\mathrm{obs}}=\frac{n-p}{p(n-1)}T^2.

Under $H_0$, $F_{\mathrm{obs}}$ has a central $F$ distribution with $p$ and $n-p$ degrees of freedom. Under a fixed alternative mean $\mu$, it has a noncentral $F$ distribution with those same degrees of freedom and noncentrality parameter $n\delta^2$. This finite-sample result requires independent multivariate normal observations, positive-definite $\Sigma$, and $n>p$.

Theorem: Under the model above, the level-$\alpha$ upper-tail Hotelling test rejects $H_0$ when $$ F_{\mathrm{obs}}>c_n, \qquad c_n=F_{p,n-p;\,1-\alpha}, $$ where $F_{p,n-p;\,1-\alpha}$ is the $1-\alpha$ quantile of the central $F_{p,n-p}$ distribution. At an alternative with squared Mahalanobis effect $\delta^2$, its power is $$ \pi(n;\delta^2)= P\!\left\{F_{p,n-p}(n\delta^2)>F_{p,n-p;\,1-\alpha}\right\}. $$

Here $F_{p,n-p}(\lambda)$ denotes a noncentral F random variable with numerator degrees of freedom $p$, denominator degrees of freedom $n-p$, and noncentrality $\lambda$. One definition is the ratio $(U/p)/(V/(n-p))$, where $U$ and $V$ are independent, $U$ has a noncentral chi-square distribution with $p$ degrees of freedom and noncentrality $\lambda$, and $V$ has a central chi-square distribution with $n-p$ degrees of freedom.

To see why the theorem gives the power, the test rejects precisely when its observed statistic exceeds the central-null critical value $c_n$. At the specified alternative, the statistic has the stated noncentral F distribution. Therefore the probability of rejection is its upper-tail probability beyond $c_n$, which is the displayed formula. In particular, the noncentrality is $n\delta^2$, not merely $\delta^2$: increasing the sample size increases the separation of the sample mean from the null value in standardized units.

The Integer Sample-Size Rule

A target power $1-\beta$ is met at sample size $n$ exactly when

\pi(n;\delta^2)\ge 1-\beta.

The planning rule is to evaluate this inequality for feasible integer values $n=p+1,p+2,\ldots$, using the central F quantile for each candidate and the noncentrality $n\delta^2$. The smallest integer passing the comparison is the planned sample size. Each candidate has different denominator degrees of freedom and a different critical value, so the degrees of freedom must be recalculated at every $n$.

Result: If $\delta^2>0$, $0<\alpha<1$, and the multivariate normal model holds, a finite feasible sample size can attain any target power strictly below one.

For justification, as $n$ grows, the law of large numbers gives $\bar X\to\mu$ and $S\to\Sigma$ in probability. Since $\Sigma$ is positive definite, inversion is continuous at $\Sigma$; consequently $$ \frac{T^2}{n} =(\bar X-\mu_0)^\top S^{-1}(\bar X-\mu_0) \xrightarrow{p}\delta^2. $$ Thus $F_{\mathrm{obs}}/n$ converges in probability to $\delta^2/p>0$. Meanwhile, the central critical value $F_{p,n-p;\,1-\alpha}$ converges to the finite $1-\alpha$ quantile of $\chi^2_p/p$ as $n$ grows. The rejection probability therefore tends to one. This argument guarantees eventual attainment; it does not replace the finite-sample calculation for choosing $n$. If $\delta^2=0$, the power is $\alpha$, so a target greater than $\alpha$ cannot be attained at the null mean. $\square$

Worked Planning Examples

Worked Example: Computing the Effect Size from a Covariance Matrix

Suppose $p=2$, $\mu_0=(0,0)^\top$, the planned alternative is $\mu=(1,1)^\top$, and $$ \Sigma=\begin{pmatrix}4&0\\0&1\end{pmatrix}. $$ Calculate the effect size, then express the exact power at a candidate sample size $n=20$, using $\alpha=0.05$.

The inverse covariance matrix is $$ \Sigma^{-1}=\begin{pmatrix}1/4&0\\0&1\end{pmatrix}. $$ The mean difference is $d=\mu-\mu_0=(1,1)^\top$, so $$ \delta^2=d^\top\Sigma^{-1}d =(1,1)\begin{pmatrix}1/4&0\\0&1\end{pmatrix} \begin{pmatrix}1\\1\end{pmatrix} =\frac14+1=\frac54. $$ The candidate is feasible because $20>2$. Its noncentrality is $n\delta^2=20(5/4)=25$, and its F degrees of freedom are $2$ and $20-2=18$. Therefore its exact power is $$ P\!\left\{F_{2,18}(25)>F_{2,18;\,0.95}\right\}. $$ To compare this candidate with a target such as $0.90$, evaluate this noncentral F upper-tail probability and check whether it is at least $0.90$. This calculation uses both coordinates and their variances; using the unstandardized squared distance $1^2+1^2=2$ as the noncentrality per observation would be incorrect.

Worked Example: Searching Feasible Integers for a Three-Variable Test

A study will test a three-dimensional mean vector at $\alpha=0.05$, aiming for power $0.80$ at an alternative with $\delta^2=0.40$. Formulate the exact integer search and show how to check a candidate $n=30$.

Since $p=3$, feasible sample sizes begin at $n=4$. For candidate $n=30$, the denominator degrees of freedom are $30-3=27$, and the noncentrality is $$ \lambda=30(0.40)=12. $$ The candidate power is therefore $$ \pi(30;0.40)= P\!\left\{F_{3,27}(12)>F_{3,27;\,0.95}\right\}. $$ The test meets the target at $n=30$ if and only if this probability is at least $0.80$. To find the minimum, perform the same calculation for every integer from $4$ through $29$ and select the first candidate whose power reaches $0.80$; if none does, continue above $30$. A numerical implementation should calculate each central critical value with its own denominator degrees of freedom, then evaluate the matching noncentral F tail. Checking only the noncentrality $12$ would not complete the power calculation.

Worked Example: Rescaling a Study Without Changing Its Effect Size

Consider $p=2$, a mean difference $d=(2,1)^\top$, and covariance matrix $$ \Sigma=\begin{pmatrix}4&0\\0&1\end{pmatrix}. $$ Now multiply both the mean difference and the covariance by a common measurement conversion factor: let $d^\star=10d=(20,10)^\top$ and $\Sigma^\star=100\Sigma$. Verify the effect size and state what it implies for planning at any fixed $n$, $\alpha$, and target power.

First, $$ \delta^2=d^\top\Sigma^{-1}d =(2,1)\begin{pmatrix}1/4&0\\0&1\end{pmatrix} \begin{pmatrix}2\\1\end{pmatrix} =1+1=2. $$ Since $(\Sigma^\star)^{-1}=(100\Sigma)^{-1}=\Sigma^{-1}/100$, $$ (\delta^\star)^2 =(10d)^\top\frac{\Sigma^{-1}}{100}(10d) =d^\top\Sigma^{-1}d =2. $$ The noncentrality $n\delta^2=2n$, degrees of freedom, critical value, and power are unchanged. A pure change of measurement units therefore does not alter the planned sample size. A change to the covariance relative to the mean difference, however, can change $\delta^2$ and hence the power.

Practical Inputs and a Common Pitfall

A planning calculation is only as useful as its inputs. The alternative should be a scientifically meaningful mean difference, and its effect size must reflect a plausible covariance matrix for the study population. If $\Sigma$ is uncertain, it is sensible to examine power over several plausible covariance or effect-size values. A smaller $\delta^2$ generally represents a harder alternative to detect, but the actual calculation should still use the noncentral F distribution for each candidate sample size.

Common mistake: Using the known-covariance test’s power calculation for a study that will estimate covariance. That substitutes a different test distribution and omits the finite-sample cost of estimating $\Sigma$. For Hotelling’s test, use $p$ and $n-p$ degrees of freedom, the central F critical value at each candidate $n$, and noncentrality $n\delta^2$. Also enforce $n>p$; a nominal target power cannot make an inverse sample covariance available when the sample size is too small.

Key takeaway: For a one-sample Hotelling test with unknown covariance, plan with the exact noncentral F tail probability. Calculate the covariance-standardized effect, search feasible integer sample sizes, and recompute the critical value and denominator degrees of freedom at each candidate.

Check Your Understanding

Use the exact finite-sample planning rule to answer these questions.

Why must the sample size satisfy $n>p$ for the ordinary one-sample Hotelling test?
For a fixed alternative, what is the noncentrality parameter at sample size $n$?
What degrees of freedom and critical value are used to calculate power at a candidate sample size?
How does multiplying both the mean difference and covariance matrix by the same positive scale factor affect the Mahalanobis effect size?
Why is it not sufficient to find a sample size whose noncentrality parameter exceeds a chosen threshold?