Power for F Tests and One-Way ANOVA

Why F-Test Power Depends on the Alternative

An F test compares a measure of variation explained by a model with variation left unexplained. Under a null hypothesis, the ratio is calibrated to follow a central F distribution. Under an alternative, the numerator tends to be larger, and the test statistic follows a noncentral F distribution. The amount of noncentrality records how far the alternative is from the null relative to the error variation.

This distinction matters in power calculations: the null distribution determines the rejection threshold, while the alternative distribution determines the probability of crossing it. The same framework applies to one-way ANOVA and to many tests of restrictions in a normal linear model.

One-Way ANOVA: Model, Degrees of Freedom, and Noncentrality

Suppose there are \(k\) independent groups. In group \(i\), observations \(Y_{ij}\), for \(j=1,\ldots,n_i\), are independent normal variables with mean \(\mu_i\) and a common variance \(\sigma^2\). Observations from different groups are independent as well. Write \(N=\sum_{i=1}^k n_i\). The one-way ANOVA null hypothesis is \(H_0:\mu_1=\cdots=\mu_k\); the upper-tail F test rejects when the between-group mean square is sufficiently large relative to the within-group mean square.

The weighted mean \(\bar{\mu}_w\) is the mean across all observations implied by the group means; it is not generally the unweighted average of the group means. The noncentrality parameter grows when group means are farther apart, when sample sizes give more information about those differences, or when the common within-group variance is smaller. If the equal-variance or independence assumptions are not reasonable, this standard noncentral F calculation need not describe the test’s power.

Why the Noncentral F Appears

The numerator in one-way ANOVA is a sum of squared differences between group sample means and the overall sample mean. Under normal sampling, its scaled form has a chi-square distribution. When the group means differ, those squared differences are centered around different population means rather than a common mean; the resulting chi-square is noncentral. The within-group sum of squares, after division by \(\sigma^2\), remains central chi-square because it measures deviations around each group’s own mean. These two components are independent under the model.

More explicitly, if \(SS_{\text{between}}\) and \(SS_{\text{within}}\) are the between-group and within-group sums of squares, then under the specified alternative, $$ \frac{SS_{\text{between}}}{\sigma^2}\sim\chi^2_{k-1}(\lambda), \qquad \frac{SS_{\text{within}}}{\sigma^2}\sim\chi^2_{N-k}. $$ The two mean squares divide these sums by \(k-1\) and \(N-k\), respectively. Their ratio therefore has the noncentral F distribution defined above. This is why power uses the noncentral F tail, but the cutoff still comes from the central F distribution under \(H_0\).

Calculating ANOVA Power

For a planned upper-tail ANOVA test, specify the significance level, group sizes, common variance, and the group means under the alternative of interest. Then calculate the weighted grand mean, \(\lambda\), and both degrees of freedom. Find the central F critical value for the chosen significance level; finally, evaluate the upper tail of the corresponding noncentral F distribution at that cutoff.

Tests of Restrictions as Variance-Ratio Tests

One-way ANOVA is a special case of a broader class of F tests for restrictions in a normal linear model. For example, a model may compare a restricted fit with a full fit. Under the assumptions of independent normal errors with common variance, the statistic is the reduction in residual sum of squares per restriction, divided by the full model’s residual mean square. Under the null restrictions it has a central F distribution; under a specified alternative it has a noncentral F distribution.

For \(q\) linear restrictions \(R\beta=r\), with design matrix \(X\), coefficient vector \(\beta\), and error variance \(\sigma^2\), the noncentrality parameter is $$ \lambda=\frac{(R\beta-r)^\mathsf{T} \left[R(X^\mathsf{T}X)^{-1}R^\mathsf{T}\right]^{-1} (R\beta-r)}{\sigma^2}. $$ The numerator degrees of freedom are \(q\), and the denominator degrees of freedom are \(n-p\), where \(n\) is the number of observations and \(p\) is the number of fitted coefficients in the full model. This form makes the connection to variance ratios explicit: the F statistic compares variation attributable to the tested restrictions with residual variation. The assumptions include a full-rank model and an invertible restriction covariance matrix.

A Necessary Distinction: Comparing Two Population Variances

Not every statistic called an F test has a noncentral F distribution under its alternative. For two independent normal populations with sample variances \(S_1^2\) and \(S_2^2\), the ratio \(S_1^2/S_2^2\), when scaled by the population variance ratio \(\sigma_1^2/\sigma_2^2\), follows a central F distribution with the two sample-variance degrees of freedom. Under a specified variance ratio, the unscaled statistic is a scaled central F variable. It is not generally noncentral F.

The noncentral F calculation in this tutorial applies when the tested numerator is noncentral chi-square under the alternative, as in ANOVA and normal linear-model tests of restrictions. Applying a noncentrality parameter to the ordinary two-sample test of equality of population variances is a common modeling error. Identify the sampling distribution under the stated alternative before calculating power.

Numerical Evaluation and Exam Traps

Noncentral F probabilities are usually evaluated with a noncentral F table or statistical software. One useful representation expresses the noncentral F upper tail as a Poisson-weighted sum of central F upper tails: $$ P(F_{\nu_1,\nu_2}(\lambda)>c) = \sum_{j=0}^{\infty} e^{-\lambda/2}\frac{(\lambda/2)^j}{j!} P(F_{\nu_1+2j,\nu_2}(0)>c). $$ The weights are the probabilities for a Poisson variable with mean \(\lambda/2\). This identity explains how the alternative distribution differs from the central one and provides a computational route; in practice, software evaluates the sum or an equivalent numerical method.

Several checks prevent common errors:

• Use the correct distribution for each stage. Obtain the rejection cutoff from the central F null distribution, then evaluate power with the noncentral F alternative distribution.
• Keep the ANOVA denominator degrees of freedom straight. They are \(N-k\), not \(N-1\), because each group mean is estimated.
• Use the weighted mean for unequal group sizes. The grand mean implied by the alternative weights each group mean by its sample size.
• Divide the weighted squared deviations by the common variance. A larger variance reduces \(\lambda\), and hence generally reduces power for a fixed design and effect.
• Do not assume every variance-ratio statistic is noncentral F under the alternative. The exact distribution depends on the model and the tested hypothesis.

Power is the probability that a specified design rejects the null under a particular alternative. It is not a property of the test alone: changing the group sizes, mean pattern, error variance, significance level, or model assumptions can change it. The noncentrality parameter gives a compact summary of the alternative’s signal relative to error variation, while the degrees of freedom describe the test and its residual information.