Power Analysis for ANOVA Designs

Introduction

ANOVA is used when a study compares mean outcomes across multiple groups. A power analysis answers a different but closely related question: how many observations are needed for the planned ANOVA test to have a high probability of detecting the effect that the study considers important?

For a one-way ANOVA with \(k\) groups, the primary hypothesis is the omnibus test:

H_0:\mu_1=\mu_2=\cdots=\mu_k\qquad\text{vs.}\qquad H_A:\text{at least one group mean differs}

The analysis compares between-group variation with within-group variation. Power therefore depends on the magnitude of the differences among the group means, the common within-group variability, the number of groups, the sample size, and the significance level.

Key idea: ANOVA power is not determined by the largest pairwise difference alone. It is driven by the overall pattern of group means relative to the residual standard deviation. Cohen's \(f\) summarizes that pattern for many standard power calculations.

What Is Power?

Power is the probability that the planned statistical test rejects the null hypothesis when a specified alternative is true.

\operatorname{Power}=P\!\left(F>F_{1-\alpha;\,df_1,df_2}\mid H_A\right)

A common planning target is 80% power, although 90% or higher may be justified when a missed effect would have important scientific or clinical consequences. The type I error is commonly set at 5%.

Power is conditional on an assumed effect size. Saying that a study has “80% power” is incomplete unless the effect size and other design assumptions are specified.

The One-Way ANOVA F Statistic

For a one-way ANOVA, total variability is decomposed into between-group and within-group components:

SS_{\mathrm{Total}}=SS_{\mathrm{Between}}+SS_{\mathrm{Within}}

The corresponding mean squares are:

MS_{\mathrm{Between}}=\frac{SS_{\mathrm{Between}}}{k-1}\qquad\text{and}\qquad MS_{\mathrm{Within}}=\frac{SS_{\mathrm{Within}}}{N-k}

and the test statistic is:

F=\frac{MS_{\mathrm{Between}}}{MS_{\mathrm{Within}}}

Under the null hypothesis, the F statistic follows a central F distribution with \(k-1\) and \(N-k\) degrees of freedom. Under a specified alternative, its distribution is noncentral F.

ANOVA Power and Cohen's \(f\)

For a balanced one-way ANOVA, let \(\mu_i\) denote the anticipated mean in group \(i\), \(\mu\) the grand mean, and \(\sigma\) the common within-group standard deviation. Cohen's effect size is:

f=\frac{\sqrt{\displaystyle\frac{1}{k}\sum_{i=1}^{k}(\mu_i-\mu)^2}}{\sigma}

For unequal group proportions \(w_i\), use the weighted form:

f=\frac{\sqrt{\displaystyle\sum_{i=1}^{k}w_i(\mu_i-\mu)^2}}{\sigma},\qquad\sum_{i=1}^{k}w_i=1

Cohen's \(f\)	Common description
0.10	Small
0.25	Medium
0.40	Large

These labels are conventions, not clinical thresholds. In applied research, derive \(f\) from meaningful mean differences and a defensible estimate of within-group variability whenever possible.

Planning principle: Do not select \(f=0.25\) simply because it is called a “medium” effect. Ask what group differences matter scientifically and what residual standard deviation is realistically expected.

The Noncentral F Distribution

For a balanced one-way ANOVA with total sample size \(N\), the noncentrality parameter is:

\lambda=Nf^2

The degrees of freedom are:

df_1=k-1,\qquad df_2=N-k

The level-\(\alpha\) critical value is:

F_{\mathrm{crit}}=F^{-1}_{1-\alpha}(df_1,df_2)

Power is the upper-tail probability of the noncentral F distribution beyond that critical value:

\operatorname{Power}=P\!\left(F_{df_1,df_2,\lambda}>F_{\mathrm{crit}}\right)

A Worked Example: Four Parallel Groups

Suppose a study compares four treatment groups using a continuous endpoint. The investigators plan a one-way ANOVA with:

Planning parameter	Value
Number of groups	4
Allocation	Equal
Type I error, \(\alpha\)	0.05
Target power	0.80
Cohen's \(f\)	0.25

The question is: what total sample size is required?

Step 1: Determine the Degrees of Freedom

With \(k=4\) groups:

\[df_1=k-1=4-1=3\]

If the total sample size is \(N\), then:

\[df_2=N-k=N-4\]

Step 2: Determine the Noncentrality Parameter

For \(f=0.25\):

\lambda=N(0.25)^2=0.0625N

As \(N\) increases, \(\lambda\) increases and the alternative distribution moves farther into the rejection region.

Step 3: Search for the Smallest Feasible \(N\)

Set \(df_1\): \(k-1\).

Set \(df_2\): \(N-k\).

Calculate \(\lambda\): \(Nf^2\).

Find the critical F value: the \(1-\alpha\) quantile of the central F distribution.

Calculate power: the noncentral-F probability above the critical value.

Choose the smallest feasible \(N\) that reaches the target power.

For this example, the first integer total sample size reaching at least 80% power is \(N=179\). With four equal-sized groups, the operational sample size is rounded upward to:

N=180\qquad\Rightarrow\qquad n_i=45\text{ per group}

At \(N=180\), the power is approximately 0.804, or 80.4%, under the assumed effect size and one-way ANOVA model.

Quantity	Value
Total \(N\)	180
Patients per group	45
\(df_1\)	3
\(df_2\)	176
\(f\)	0.25
\(\lambda\)	11.25
Power	Approximately 80.4%

Important: The mathematical minimum and the operational sample size need not be identical. When equal allocation across four groups is required, \(N=179\) is not feasible; \(N=180\) gives 45 participants per group and retains at least the target power.

Deriving \(f\) From Actual Means

Suppose the scientific planning exercise starts with anticipated group means of 90, 95, 100, and 105, with a common within-group standard deviation of 20.

\mu=\frac{90+95+100+105}{4}=97.5

f=\frac{\sqrt{\frac{(90-97.5)^2+(95-97.5)^2+(100-97.5)^2+(105-97.5)^2}{4}}}{20}\approx0.280

Thus, this mean pattern corresponds to approximately \(f=0.28\). That value can be used directly in the power calculation.

Effect Size, Variability, and Sample Size

The formula for \(f\) makes the planning trade-off clear. If mean differences stay fixed but the within-group standard deviation decreases, \(f\) becomes larger and fewer observations are needed. If the same means occur in a noisier population, \(f\) becomes smaller and more observations may be required.

Planning scenario	Effect size	Sample-size consequence
Smaller mean separation	Smaller \(f\)	More participants
Same means, larger SD	Smaller \(f\)	More participants
Same means, smaller SD	Larger \(f\)	Fewer participants
Larger meaningful separation	Larger \(f\)	Fewer participants

Clinical interpretation: Underestimating residual variability can make the calculated sample size seriously optimistic.

How the Number of Groups Affects Power

Adding groups changes the numerator degrees of freedom and changes how a fixed total sample is distributed. There is no universal rule that more groups automatically means more or less power.

For example, with three groups \(df_1=2\), while with five groups \(df_1=4\). Both the critical F value and the alternative distribution change when the number of groups changes.

Unequal Allocation

If group proportions are \(w_1,\ldots,w_k\), the effect size can be written as:

f^2=\frac{\displaystyle\sum_{i=1}^{k}w_i(\mu_i-\mu)^2}{\sigma^2}

Unequal allocation can reduce efficiency compared with balanced allocation when the total sample size is fixed. The actual power calculation should therefore use the intended allocation.

Power for a Factorial ANOVA

In a factorial ANOVA, the investigator may be interested in several distinct effects: main effects and interactions. These hypotheses can have different effect sizes and degrees of freedom.

For a \(2\times3\) factorial design:

Factor A has \(2-1=1\) degree of freedom.
Factor B has \(3-1=2\) degrees of freedom.
The A-by-B interaction has \((2-1)(3-1)=2\) degrees of freedom.

df_A=a-1,\qquad df_B=b-1,\qquad df_{A\times B}=(a-1)(b-1)

If the interaction is the primary scientific question, sample-size planning should be driven by the anticipated interaction effect rather than by a large main effect.

Common mistake: A study can have adequate power for a main effect but inadequate power for a smaller interaction. Identify the primary hypothesis before calculating sample size.

Repeated-Measures ANOVA

Repeated-measures designs require additional assumptions because observations from the same participant are correlated. Power can depend on within-subject correlation, covariance structure, the number of repeated measurements, and corrections for violations of sphericity.

Therefore, the simple one-way relationship \(\lambda=Nf^2\) should not be applied blindly to a repeated-measures design. Dedicated software or simulation is preferable when within-subject correlation is important.

ANOVA and Specific Contrasts

The omnibus ANOVA asks whether at least one group mean differs. A significant omnibus test does not identify which groups differ.

If the study has a prespecified primary contrast, such as treatment versus placebo or a dose-response contrast, power for that contrast may be more relevant than omnibus ANOVA power. Multiplicity adjustments for several pairwise comparisons can also change the required sample size.

Allowing for Dropout

If \(N_{analysis}\) evaluable participants are required and the anticipated attrition fraction is \(d\), a simple enrollment inflation is:

N_{enroll}=\frac{N_{analysis}}{1-d}

For 180 evaluable participants and 10% attrition:

N_{enroll}=\frac{180}{1-0.10}=200

With four equal groups, that corresponds to 50 enrolled participants per group.

Do not double-count inflation: Check whether the software already incorporates attrition before applying an additional multiplier.

R Implementation

The pwr package provides a convenient one-way ANOVA calculation:

library(pwr)

pwr.anova.test(
  k = 4,
  f = 0.25,
  sig.level = 0.05,
  power = 0.80
)

The returned sample size is per group. Round upward and, when needed, choose a value compatible with the intended allocation.

A direct noncentral-F calculation makes the mechanism explicit:

k <- 4
N <- 180
f <- 0.25
alpha <- 0.05

df1 <- k - 1
df2 <- N - k
lambda <- N * f^2

Fcrit <- qf(1 - alpha, df1, df2)

power <- 1 - pf(
  Fcrit,
  df1,
  df2,
  ncp = lambda
)

power

SAS Implementation

SAS provides power procedures for ANOVA planning. A typical setup is:

proc power;
   onewayanova
      test = overall
      alpha = 0.05
      power = 0.80
      npergroup = .
      effect = 0.25
      ngroups = 4;
run;

The exact syntax and supported options should be checked against the SAS version in use. Retain the procedure output and assumptions in the statistical documentation.

Simulation as a Validation Tool

For a standard one-way ANOVA, an analytical noncentral-F calculation is usually sufficient. Simulation becomes useful when the planned analysis includes features not represented by the simple model.

Non-normal outcomes
Heteroscedastic variances
Unequal group sizes
Covariate adjustment
Repeated measurements
Missing-data mechanisms
Complex contrasts
Nonstandard test statistics

\widehat{\operatorname{Power}}=\frac{\text{number of simulated studies with rejection}}{\text{number of simulated studies}}

Sensitivity Analysis

Because the assumed effect size is often the largest uncertainty, examine several plausible values rather than relying on one number.

Assumed \(f\)	Description	Planning implication
0.10	Small	Often much larger sample
0.20	Smaller-than-medium	Useful sensitivity scenario
0.25	Medium benchmark	Common reference scenario
0.30	Moderately large	Smaller sample than \(f=0.25\)
0.40	Large	Often substantially smaller sample

A strong planning table reports required sample size for several plausible effect sizes and, when appropriate, both 80% and 90% power.

Common Mistakes

Planning from a vague “medium effect.” Use meaningful means and variability whenever possible.
Confusing omnibus power with contrast power. These are different hypotheses.
Ignoring allocation. Unequal allocation can reduce efficiency.
Using the one-way formula for repeated measures. Correlation and covariance matter.
Ignoring interactions. Main-effect power does not guarantee interaction power.
Using an optimistic standard deviation. Underestimating variability can underpower the study.
Failing to account for attrition. Analysis and enrollment sample sizes are not necessarily the same.
Rounding in the wrong direction. Round upward and respect the allocation plan.
Reporting only the final sample size. Document the effect size, alpha, power, allocation, and variance assumptions.
Assuming 80% power applies to every possible effect. Power is conditional on the specified alternative.

A Practical ANOVA Power Workflow

Define the primary hypothesis: omnibus F test, main effect, interaction, or specific contrast.

Specify the number of groups or factor levels and planned allocation.

Specify scientifically meaningful anticipated group means.

Specify a defensible within-group standard deviation or residual variance.

Convert the assumptions to Cohen's \(f\), when appropriate.

Specify type I error and target power.

Calculate power using the appropriate noncentral-F method or validated software.

Choose the smallest feasible sample size that reaches the target power.

Round for allocation and inflate for expected attrition when appropriate.

Perform sensitivity analyses over plausible effect sizes and variability assumptions.

Validate complex designs with simulation when analytical assumptions are inadequate.

Document assumptions, software, version, method, and final enrollment target in the protocol and SAP.

What Should Be Reported in a Statistical Analysis Plan?

A reproducible ANOVA sample-size section should identify the statistical question and all assumptions that determine power.

Primary endpoint and analysis population
ANOVA model and primary hypothesis
Number of groups or factor levels
Planned allocation
Anticipated group means or standardized effect size
Assumed within-group standard deviation or residual variance
Cohen's \(f\), when used
Type I error
Target power
Required evaluable sample size
Attrition or dropout inflation
Final planned enrollment
Software and version
Method used for the power calculation
Sensitivity analyses and alternative assumptions

Worked Example Summary

Component	Planning value
Design	One-way between-subjects ANOVA
Number of groups	4
Allocation	Equal
Cohen's \(f\)	0.25
Type I error	0.05
Target power	0.80
Mathematical minimum total \(N\)	179
Operational total \(N\)	180
Per-group \(N\)	45
Numerator df	3
Denominator df	176
Noncentrality parameter	11.25
Achieved power	Approximately 80.4%
10% attrition-adjusted enrollment	200 total, or 50 per group

The Most Important Concept

The most important point is that ANOVA power is fundamentally a relationship between effect size, variability, sample size, and the statistical decision rule.

For a standard balanced one-way ANOVA, Cohen's \(f\) summarizes separation of the anticipated group means relative to the common within-group standard deviation, and the noncentral F distribution translates that effect into power.

A sound design therefore does more than report “80% power.” It documents what effect is expected, why that effect is meaningful, what variability is assumed, which hypothesis is primary, how sample size was calculated, how allocation and attrition were handled, and how sensitive the result is to alternative assumptions.

Bottom line: For a conventional one-way ANOVA, specify the number of groups, Cohen's \(f\), alpha, and target power, then evaluate the noncentral F distribution over candidate sample sizes. In applied clinical research, the most important work often occurs before the calculation: choosing clinically meaningful group differences and a defensible estimate of within-group variability. For factorial, repeated-measures, covariate-adjusted, or otherwise complex designs, incorporate the primary hypothesis and design structure explicitly.

References

Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences. 2nd ed. Lawrence Erlbaum Associates.
Cohen, J. (1992). A power primer. Psychological Bulletin, 112(1), 155–159.
Montgomery, D.C. Design and Analysis of Experiments. Wiley.
Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39, 175–191.