Sample-Size Planning with Known Covariance

Why Sample-Size Planning Uses the Alternative Distribution

A test’s significance level controls its probability of rejection under the null hypothesis; sample-size planning asks a different question: how likely is the test to reject when a specified alternative is true? With a known covariance matrix, this question has an exact answer in terms of a noncentral chi-square distribution. The practical task is to evaluate that power for successive integer sample sizes and select the first one that reaches the target.

Assume $X_1,\ldots,X_n$ are independent and identically distributed $p$-dimensional normal random vectors with mean vector $\mu$ and known positive-definite covariance matrix $\Sigma$. Consider testing $H_0:\mu=\mu_0$ against $H_1:\mu\ne\mu_0$ at significance level $\alpha$, where $0<\alpha<1$. The alternative used for planning is a specified mean vector $\mu_1\ne\mu_0$. This is a planning calculation for that shift, not a guarantee of power for every possible alternative.

Definition: Let $\delta=\mu_1-\mu_0$ be the planned mean shift. Its squared Mahalanobis distance under $\Sigma$ is $$ \Delta^2=\delta^{\mathsf T}\Sigma^{-1}\delta. $$ The known-covariance test statistic is $$ Q_n=n(\bar X-\mu_0)^{\mathsf T}\Sigma^{-1}(\bar X-\mu_0), $$ where $\bar X=n^{-1}\sum_{i=1}^n X_i$. Reject $H_0$ when $Q_n>\chi^2_{p,1-\alpha}$, the $1-\alpha$ quantile of the central chi-square distribution with $p$ degrees of freedom.

The statistic uses the covariance matrix to standardize the discrepancy between the observed sample mean and the null mean. Under $H_0$, $Q_n$ has a central chi-square distribution with $p$ degrees of freedom, so this cutoff gives a test of size $\alpha$. Under the planned alternative, its distribution shifts away from zero. The size of that shift is determined by $n\Delta^2$, the noncentrality parameter.

Deriving the Power Formula

For a multivariate normal sample, $\bar X\sim N_p(\mu,\Sigma/n)$. Under the planned alternative, whiten and scale the sample mean: $$ Z_n=\sqrt n\,\Sigma^{-1/2}(\bar X-\mu_0) \sim N_p\!\left(\sqrt n\,\Sigma^{-1/2}\delta,I_p\right). $$ The statistic is $Q_n=Z_n^{\mathsf T}Z_n$. By definition, the sum of squares of the coordinates of a normal vector with identity covariance and mean vector $m$ has a noncentral chi-square distribution with $p$ degrees of freedom and noncentrality parameter $m^{\mathsf T}m$. Here that parameter is $n\Delta^2$.

Theorem: Under the stated independent normal sampling model, positive-definite known $\Sigma$, and planned mean $\mu_1$, the exact power at sample size $n$ is $$ \pi(n)=P_{\mu_1}(Q_n>\chi^2_{p,1-\alpha}) =P\!\left(\chi^2_p(n\Delta^2)>\chi^2_{p,1-\alpha}\right). $$ Here $\chi^2_p(\lambda)$ denotes a noncentral chi-square random variable with $p$ degrees of freedom and noncentrality parameter $\lambda$.

This result follows directly from the distribution of $Z_n$: its squared norm has the stated noncentral chi-square law, and the rejection event is that this squared norm exceeds the central-null cutoff. Thus, once $p$, $\alpha$, and $\Delta^2$ are specified, the power at any candidate $n$ is determined. The unknown direction of the shift does not enter separately: for this test, shifts with the same Mahalanobis distance have the same power.

Worked Example: A Two-Dimensional Shift

Suppose $p=2$, $\alpha=0.05$, and the planned shift satisfies $\Delta^2=1$. The central chi-square cutoff is $\chi^2_{2,0.95}\approx5.9915$. Find the smallest sample size giving power at least $0.80$.

At $n=9$, the noncentrality parameter is $\lambda=9$. Evaluating the noncentral chi-square upper tail gives $$ \pi(9)=P\!\left(\chi^2_2(9)>5.9915\right)\approx0.7707. $$ This is below $0.80$.

At $n=10$, $\lambda=10$, so $$ \pi(10)=P\!\left(\chi^2_2(10)>5.9915\right)\approx0.8154. $$ This exceeds $0.80$. Power increases with $n$ when $\Delta^2>0$, as proved below. Therefore $n=10$ is the smallest sample size meeting the target.

The noncentrality parameter is not itself the power: $\lambda=10$ must be evaluated against both the degrees of freedom and the rejection cutoff. Reporting $\lambda$ alone does not establish that the target has been reached.

Evaluating Power and Finding the Minimum

Many numerical libraries evaluate a noncentral chi-square tail directly. Another useful representation expresses that tail as a weighted sum of central chi-square tails. If $J$ has a Poisson distribution with mean $\lambda/2$, then a noncentral chi-square variable with $p$ degrees of freedom and noncentrality $\lambda$ is a mixture of central chi-square variables with degrees of freedom $p+2J$.

Result: For $c>0$ and $\lambda\ge0$, $$ P\!\left(\chi^2_p(\lambda)>c\right) =\sum_{j=0}^{\infty} e^{-\lambda/2}\frac{(\lambda/2)^j}{j!} P\!\left(\chi^2_{p+2j}>c\right). $$

To verify the mixture, let $W$ have the distribution on the right, formed by first drawing $J\sim\operatorname{Poisson}(\lambda/2)$ and then drawing $W\mid J=j\sim\chi^2_{p+2j}$. For $t<1/2$, its moment-generating function is $$ E[e^{tW}] =(1-2t)^{-p/2}E[(1-2t)^{-J}] =(1-2t)^{-p/2} \exp\!\left\{\frac{\lambda}{2}\left((1-2t)^{-1}-1\right)\right\}. $$ Since $(1-2t)^{-1}-1=2t/(1-2t)$, this becomes $$ E[e^{tW}]=(1-2t)^{-p/2}\exp\!\left(\frac{\lambda t}{1-2t}\right), $$ the moment-generating function of $\chi^2_p(\lambda)$. Both functions exist in a neighbourhood of zero, so they identify the same distribution. Taking the probability of exceeding $c$ gives the displayed sum. In practice, the Poisson weights allow accurate numerical evaluation by truncating the sum once the remaining Poisson probability is negligible.

The mixture also proves the monotonicity needed for an integer search. A central chi-square variable with $p+2j$ degrees of freedom can be represented as $U+\sum_{k=1}^{j}V_k$, where $U\sim\chi^2_p$ and the independent $V_k\sim\chi^2_2$ are nonnegative. If $\lambda_2\ge\lambda_1$, couple $J_1\sim\operatorname{Poisson}(\lambda_1/2)$ and $K\sim\operatorname{Poisson}((\lambda_2-\lambda_1)/2)$ independently, and set $J_2=J_1+K$. Then $J_2\sim\operatorname{Poisson}(\lambda_2/2)$ and $J_2\ge J_1$. Using the same $U$ and the same initial $V_k$ terms in both representations makes the variable with noncentrality $\lambda_2$ at least as large as the one with noncentrality $\lambda_1$. Its upper-tail probability therefore cannot decrease. Since $n\Delta^2$ increases with $n$ when $\Delta^2>0$, $\pi(n)$ is nondecreasing in the sample size. $\square$

For a requested power $\pi_0$ with $\alpha<\pi_0<1$, compute the cutoff once, then evaluate $\pi(n)$ for successive positive integers until $\pi(n)\ge\pi_0$. Monotonicity ensures that the first successful integer is the minimum. If $\Delta^2>0$, such an integer exists: under the alternative, $Q_n$ has the representation $\|Z+\sqrt n\,b\|^2$, with $Z\sim N_p(0,I_p)$ and a fixed $b\ne0$. The inequality $$ \|Z+\sqrt n\,b\|\ge\sqrt n\,\|b\|-\|Z\| $$ shows that $Q_n$ tends to infinity in probability, because $\|Z\|$ is finite with probability one. Consequently the probability of exceeding any fixed cutoff tends to one. If $\Delta^2=0$, power remains $\alpha$ for every $n$, so a target greater than $\alpha$ cannot be reached.

Worked Example: Checking the Poisson Mixture

Continue with $p=2$, $\alpha=0.05$, $\Delta^2=1$, and $n=10$. Then $\lambda=10$, so the mixture index has $J\sim\operatorname{Poisson}(5)$. Since $p+2J=2+2J$, the power can be evaluated as $$ \pi(10)=\sum_{j=0}^{\infty} e^{-5}\frac{5^j}{j!} P\!\left(\chi^2_{2+2j}>5.9915\right)\approx0.8154. $$

For example, the first two mixture weights are $P(J=0)=e^{-5}\approx0.00674$ and $P(J=1)=5e^{-5}\approx0.03369$. The corresponding central chi-square tails are approximately $0.0500$ for 2 degrees of freedom and $0.1998$ for 4 degrees of freedom. Their contributions to the power are therefore approximately $0.00674(0.0500)=0.00034$ and $0.03369(0.1998)=0.00673$. Continuing the weighted sum over $j$ gives the stated total. The small first contribution is expected: under the mixture, most of the probability mass is not concentrated at $J=0$.

Worked Example: A One-Dimensional Planning Calculation

For a univariate normal mean with known standard deviation, let the standardized planned shift be $\Delta=0.5$, use a two-sided test with $\alpha=0.05$, and require power at least $0.80$. Here $p=1$, $\Delta^2=0.25$, and the rejection cutoff is $\chi^2_{1,0.95}\approx3.8415$, whose square root is approximately $1.9600$.

For one degree of freedom, the noncentral chi-square statistic can be written as $(Z+\sqrt n\,\Delta)^2$, where $Z\sim N(0,1)$. Thus the exact power is $$ \pi(n)=\Phi(\sqrt n\,\Delta-1.9600)+\Phi(-\sqrt n\,\Delta-1.9600), $$ where $\Phi$ is the standard normal distribution function. At $n=31$, $\sqrt n\,\Delta=\sqrt{31}/2\approx2.7839$, giving $$ \pi(31)=\Phi(0.8239)+\Phi(-4.7439)\approx0.7950. $$ The second term is negligible at the shown precision, and the target is not reached.

At $n=32$, $\sqrt n\,\Delta=\sqrt{32}/2\approx2.8284$, so $$ \pi(32)=\Phi(0.8684)+\Phi(-4.7884)\approx0.8074. $$ Since power is nondecreasing in $n$, $n=32$ is the minimum. This example shows why rounding a continuous approximation upward is a useful starting point, but the exact integer candidates should still be checked with the power function.

What the Calculation Depends On

The distance $\Delta^2=\delta^{\mathsf T}\Sigma^{-1}\delta$ combines the proposed shift and the covariance structure. A large raw difference in a high-variance direction may correspond to a modest standardized effect; a smaller difference in a low-variance direction may correspond to a larger one. Planning therefore requires specifying the shift in the original measurement units and a covariance matrix on those same variables, then computing the standardized distance.

This calculation assumes that $\Sigma$ is known and that the observations follow the stated multivariate normal model. Replacing $\Sigma$ by a sample covariance changes the test statistic’s calibration and its power distribution; the known-covariance calculation should not be used as if it were the exact sample-size calculation for that different procedure. In applications where the covariance is estimated from prior data, the known-covariance result can serve as a benchmark, but the planning method must match the test that will actually be used.

A useful planning report states $p$, $\alpha$, the target power, the shift $\delta$, the covariance matrix $\Sigma$, the resulting $\Delta^2$, and the selected integer $n$. Sensitivity checks over plausible values of $\delta$ and $\Sigma$ are informative because the answer can change substantially when either planning input changes.

Common Pitfall

Using the central chi-square tail under the alternative. The central distribution calibrates the rejection threshold under $H_0$; it does not give power at a nonzero shift. Under $\mu_1$, use the noncentral distribution with $\lambda=n\Delta^2$.
Choosing a sample size from the noncentrality parameter alone. The same $\lambda$ can yield different power for different $p$, $\alpha$, and target cutoff. Evaluate the upper tail with the correct degrees of freedom and critical value.
Rounding without checking adjacent integers. The minimum is an integer defined by the power inequality. Evaluate the candidate and the preceding integer, or search upward from the smallest allowable sample size.
Forgetting that the shift is covariance-standardized. The components of $\delta$ cannot be compared in isolation when their variances and correlations differ. Compute $\delta^{\mathsf T}\Sigma^{-1}\delta$.

Key takeaway: With known positive-definite covariance, the exact power at sample size $n$ is a noncentral chi-square upper-tail probability with noncentrality $n\Delta^2$. For a nonzero planned shift, evaluate this power over integer sample sizes and select the first one meeting the target.

Check Your Understanding

Use the model and notation defined above to answer the following.

What parameters determine the noncentrality parameter at sample size $n$?
Why is the central chi-square quantile used for the rejection cutoff, while the power calculation uses a noncentral chi-square distribution?
If two planned mean shifts have the same $\Delta^2$, what does the known-covariance test’s power formula imply about their powers?
How does the Poisson mixture represent the noncentral chi-square upper tail?
What happens to the possibility of reaching a target power greater than $\alpha$ when the planned shift is zero?