Power and Sample Size for Multiple Correlation Tests

Why Power Requires an Alternative

A test of zero multiple correlation asks whether a response is linearly associated with a set of predictors. Its significance level controls the probability of rejecting the null hypothesis when it is true. Power answers a different question: if the population association has a specified size, how likely is the test to detect it? A sample-size calculation reverses that question, finding how many observations are needed to reach a chosen power.

For this calculation, the relevant effect-size input is the population multiple coefficient of determination $R^2$, together with the number of predictors. The noncentral F distribution connects those inputs to power. It is essential to distinguish its exact role under a fixed-design regression model from the common approximation used to plan a study from a population $R^2$.

Test Statistic and Noncentrality

Definition: Consider the regression model $Y=\alpha\mathbf{1}+X_c\beta+\varepsilon$, where $Y$ is an $n$-vector, $X_c$ is a centered $n$-by-$p$ predictor matrix of full column rank, and $\varepsilon\sim N(0,\sigma^2 I_n)$. The test of zero multiple correlation is the test $H_0:\beta=0$ against $H_1:\beta\ne 0$. With an intercept and $n>p+1$, its statistic is $$ F=\frac{SSR/p}{SSE/(n-p-1)} =\frac{R_s^2/p}{(1-R_s^2)/(n-p-1)}, $$ where $SSR=Y^{\mathsf T}HY$, $SSE=Y^{\mathsf T}MY$, $H$ projects onto the column space of $X_c$, $M=I_n-\mathbf{1}\mathbf{1}^{\mathsf T}/n-H$, and $R_s^2=SSR/(SSR+SSE)$ is the sample coefficient of determination.

Under the null, $F$ has a central F distribution with $p$ and $\nu=n-p-1$ degrees of freedom. Under a fixed alternative and fixed design, it has a noncentral F distribution with those degrees of freedom and noncentrality

\lambda=\frac{\beta^{\mathsf T}X_c^{\mathsf T}X_c\beta}{\sigma^2}.

Here is the projection argument establishing these distributions. The vectors $\mathbf{1}$, the columns of $X_c$, and the residual space for $M$ are mutually orthogonal. In an orthonormal basis for the predictor space, $(X_c\beta+H\varepsilon)/\sigma$ has independent unit-variance normal coordinates. Its mean vector has squared length $\beta^{\mathsf T}X_c^{\mathsf T}X_c\beta/\sigma^2=\lambda$, so its squared norm $SSR/\sigma^2$ has a noncentral chi-square distribution with $p$ degrees of freedom and noncentrality $\lambda$. In an orthonormal basis for the residual space, $M\varepsilon/\sigma$ has $\nu$ independent standard normal coordinates, so $SSE/\sigma^2$ has a central chi-square distribution with $\nu$ degrees of freedom. The two projected error vectors are independent because their joint distribution is normal and their covariance is zero. Dividing the two independent chi-square quantities by their degrees of freedom gives the stated F distributions. Under $H_0$, $\beta=0$, hence $\lambda=0$. $\square$

The noncentrality for a fixed design is exact. For planning from a population effect size, suppose predictors have covariance matrix $\Sigma_X$, and the outcome follows a linear model with independent normal error variance $\sigma^2$. Then

R^2=\frac{\beta^{\mathsf T}\Sigma_X\beta} {\beta^{\mathsf T}\Sigma_X\beta+\sigma^2}, \qquad \frac{R^2}{1-R^2}=\frac{\beta^{\mathsf T}\Sigma_X\beta}{\sigma^2}.

If the observed predictor cross-product matrix is approximately $n\Sigma_X$, substitution into the fixed-design expression gives the conventional planning approximation

\lambda\approx n\frac{R^2}{1-R^2}.

This is the noncentrality convention used in the calculations below. It is a planning approximation, not an exact unconditional statement for every random-predictor sample: with random predictors, the conditional noncentrality depends on the realized $X_c^{\mathsf T}X_c$.

Computing Power

Theorem: For a prespecified significance level $\alpha$, predictor count $p$, and sample size $n>p+1$, let $c=F_{p,n-p-1;1-\alpha}$ be the $1-\alpha$ quantile of the central F distribution. Using a noncentrality $\lambda$, the power is $$ \pi(n,\lambda)=P\{F_{p,n-p-1}(\lambda)>c\} =1-G_{p,n-p-1,\lambda}(c), $$ where $G_{p,n-p-1,\lambda}$ is the noncentral F cumulative distribution function.

The rejection region is $F>c$, since the test is an upper-tail test. Under the alternative, $F$ has the noncentral distribution derived above; therefore the probability of falling in the rejection region is exactly its upper-tail probability. This proves the power expression.

The noncentral F distribution can be evaluated through a useful mixture representation. If $J$ has a Poisson distribution with mean $\lambda/2$, then

G_{p,\nu,\lambda}(c) =\sum_{j=0}^{\infty}P(J=j)\,G_{p+2j,\nu,0}(c).

Thus the noncentral tail can be computed as a weighted sum of central F tails. In practice, statistical software evaluates this sum or an equivalent numerical representation. For population-effect planning, insert $\lambda\approx nR^2/(1-R^2)$ and $\nu=n-p-1$. Both degrees of freedom and noncentrality change with $n$, so power must be recalculated for each candidate sample size.

Worked Example: Power for a Specified Population R-Squared

A study plans $n=80$ observations and tests a multiple correlation based on $p=3$ predictors. Use $\alpha=0.05$ and suppose the population $R^2$ is $0.15$. Find the approximate power.

First, the denominator degrees of freedom are $\nu=80-3-1=76$. The planning noncentrality is $$ \lambda\approx 80\frac{0.15}{1-0.15} =80\frac{0.15}{0.85} =\frac{12}{0.85} \approx 14.12. $$

The central critical value is $F_{3,76;0.95}$, approximately $2.73$. The power is therefore the upper tail of $F_{3,76}(14.12)$ beyond $2.73$. Evaluating that noncentral F tail gives power of approximately $0.88$ (the precise last digits depend on numerical precision). Under the stated planning assumptions, the test has roughly an 88% chance of rejecting zero multiple correlation when population $R^2=0.15$.

Finding a Required Sample Size

To plan for target power $1-\beta$, choose $\alpha$, $p$, and the smallest scientifically meaningful population $R^2$. For each integer $n>p+1$, calculate the critical value, the denominator degrees of freedom, and the approximate noncentrality. The required sample size is the smallest $n$ for which the resulting power is at least $1-\beta$. Integer search is usually simpler and safer than trying to rearrange the noncentral F tail equation, which has no general closed-form solution.

Specify the design and target.
Set the predictor count $p$, significance level $\alpha$, target power, and the population $R^2$ worth detecting.

Evaluate candidate sizes.
For each $n>p+1$, use $\nu=n-p-1$ and $\lambda\approx nR^2/(1-R^2)$, then find $F_{p,\nu;1-\alpha}$.

Check the noncentral tail.
Compute $P\{F_{p,\nu}(\lambda)>F_{p,\nu;1-\alpha}\}$. Increase $n$ until the target is met, and verify that the immediately smaller integer does not meet it.

Worked Example: Sample Size for 80% Power

Suppose a study will use $p=4$ predictors, test at $\alpha=0.05$, and aims for 80% power to detect population $R^2=0.20$. The effect-size ratio is $$ \frac{R^2}{1-R^2}=\frac{0.20}{0.80}=0.25, \qquad \lambda\approx 0.25n. $$

For each candidate $n$, use $\nu=n-5$. For example, at $n=46$, $\nu=41$ and $\lambda\approx 0.25(46)=11.50$. The central 5% upper-tail critical value is $F_{4,41;0.95}$, approximately $2.61$; this candidate does not meet the 80% power target. Evaluating the noncentral F tail for each integer sample size places the first candidate meeting the target at $n=53$, with power of approximately $0.804$; $n=52$ remains below 80% power. A final design should use full-precision software output and, if the target is binding, allow a modest buffer for attrition or model departures.

Worked Example: A Smaller Study Has Limited Power

A planned study has $n=40$, $p=2$, and $\alpha=0.05$. Assess its power for population $R^2=0.10$.

The denominator degrees of freedom are $40-2-1=37$. The noncentrality approximation is $$ \lambda\approx 40\frac{0.10}{0.90} =\frac{4}{0.90} \approx 4.44. $$ The central critical value is $F_{2,37;0.95}$, approximately $3.25$. The power calculation is the upper tail of $F_{2,37}(4.44)$ above $3.25$, which is approximately $0.43$. Thus, even if $R^2=0.10$, this design would fail to reject the null in more than half of repeated studies. A nonsignificant result from this design would be weak evidence against an association of that size.

Interpretation and Common Pitfalls

Power is conditional on the effect size and assumptions used in the calculation. A design’s power is not one fixed property: it changes with the population $R^2$, number of predictors, significance level, and sample size. If the true association is smaller than the planning value, actual power will be lower. The population $R^2$ should therefore represent a meaningful effect, not merely an optimistic estimate from a small pilot study.

Confusing sample and population $R^2$. The planning input is a hypothesized population effect. The observed sample $R_s^2$ is random and should not be substituted as though known in advance.
Using the wrong degrees of freedom. With an intercept and $p$ predictors, the denominator degrees of freedom are $n-p-1$, not $n-p$.
Treating the planning noncentrality as universally exact. The exact fixed-design quantity is $\beta^{\mathsf T}X_c^{\mathsf T}X_c\beta/\sigma^2$. The expression $nR^2/(1-R^2)$ uses a population covariance approximation and should be identified as such.
Reading power backward. Power is the probability of rejecting under a specified alternative. A high probability of nonrejection does not show that the null is true, and a low-powered nonsignificant result does not rule out the planned effect.

Key takeaway: For a zero multiple-correlation test, calculate the central F cutoff under the null and the noncentral F tail under a specified alternative. For population-based planning, use $\lambda\approx nR^2/(1-R^2)$, retain $\nu=n-p-1$, and report the assumptions behind that approximation.

Check Your Understanding

Use the definitions and calculations above to answer these questions.

For $n=60$ and $p=5$, what are the numerator and denominator degrees of freedom for the test?
Under the planning approximation, calculate $\lambda$ for $n=50$ and population $R^2=0.12$.
Why is $\beta^{\mathsf T}X_c^{\mathsf T}X_c\beta/\sigma^2$ exact for a fixed design, while $nR^2/(1-R^2)$ is generally described as an approximation?
In a sample-size search, why must the central critical value be recalculated for each candidate $n$?
A study has low power for its prespecified effect size and obtains a nonsignificant result. What can it conclude, and what should it avoid concluding?