Allocation Imbalance and One-Way F-Test Power

Why Allocation Matters for Power

A one-way ANOVA compares several treatment means using the same overall test, but its ability to detect a real difference depends partly on how many observations are assigned to each treatment. When the total sample size is fixed, changing the group sizes changes the weight each treatment mean receives in the test. The effect on power depends on how those weights line up with the particular pattern of treatment means being considered.

The comparison in this tutorial holds the total sample size and the common error variance fixed. That lets us isolate the role of allocation. It also leads to an important qualification: unequal allocation does not invariably reduce power for every specified alternative. It can reduce or increase the noncentrality parameter, depending on which treatment means receive the larger groups.

Model, Test, and Noncentrality Parameter

Suppose there are \(k\) fixed treatments, with \(n_i\) independent observations in treatment \(i\), for \(i=1,\ldots,k\). The total sample size is \(N=\sum_{i=1}^k n_i\). The one-way fixed-effects model is

Y_{ij}=\mu_i+\varepsilon_{ij}, \qquad \varepsilon_{ij}\overset{\text{ind}}{\sim}N(0,\sigma^2), \quad i=1,\ldots,k,\quad j=1,\ldots,n_i.

Here \(\mu_i\) is the fixed mean response for treatment \(i\). The model assumes independent errors, normally distributed errors, and a common variance \(\sigma^2\) across treatments. For the usual test to be valid, the observations must also correspond to independent experimental units under the design. The allocation is treated as fixed for the power calculation.

The ANOVA F test tests \(H_0:\mu_1=\cdots=\mu_k\) against the alternative that not all treatment means are equal. Its numerator and denominator degrees of freedom are \(k-1\) and \(N-k\), respectively. Under a specified alternative, its noncentrality parameter is

Definition: For the one-way fixed-effects F test, the noncentrality parameter is

\lambda=\frac{\sum_{i=1}^k n_i(\mu_i-\bar{\mu}_w)^2}{\sigma^2}, \qquad \bar{\mu}_w=\frac{\sum_{i=1}^k n_i\mu_i}{N}.

The quantity \(\bar{\mu}_w\) is the mean of the treatment means weighted by their group sizes. The noncentrality parameter is dimensionless; it measures the treatment separation relative to the error variance, taking the allocation into account.

Under the alternative, the test statistic has a noncentral F distribution with \(k-1\) and \(N-k\) degrees of freedom and noncentrality parameter \(\lambda\). At significance level \(\alpha\), its power is

P\left(F_{k-1,N-k,\lambda}> F_{1-\alpha;\,k-1,N-k}\right),

where \(F_{1-\alpha;\,k-1,N-k}\) is the upper-tail critical value from the central F distribution. With \(N\), \(k\), and \(\alpha\) fixed, the degrees of freedom and critical value stay the same as allocation changes. The change in power then comes through \(\lambda\).

Why the Weighted Mean Appears

The treatment sum of squares can be written as \(\sum_i n_i(\bar{Y}_i-\bar{Y}_{..})^2\), where \(\bar{Y}_i\) is the sample mean in treatment \(i\) and \(\bar{Y}_{..}\) is the overall sample mean. Under a specified alternative, the expected treatment means are \(\mu_i\), and the expected overall mean is their allocation-weighted mean \(\bar{\mu}_w\). Thus the allocation-weighted squared separation of the treatment means, divided by \(\sigma^2\), gives the noncentrality parameter above.

An equivalent form makes the pairwise differences explicit:

\sum_{i=1}^k n_i(\mu_i-\bar{\mu}_w)^2 = \frac{1}{N}\sum_{i<j}n_i n_j(\mu_i-\mu_j)^2.

To verify the identity, expand the left side:

\sum_i n_i\mu_i^2 -\frac{\left(\sum_i n_i\mu_i\right)^2}{N}.

Expanding the right side gives \[ \frac{1}{N}\sum_{i<j}n_i n_j(\mu_i^2+\mu_j^2-2\mu_i\mu_j). \] Each \(n_i\mu_i^2\) appears with total coefficient \((N-n_i)/N\), while the cross-product terms contribute \(-2\sum_{i<j}n_i n_j\mu_i\mu_j/N\). Combining these terms yields \(\sum_i n_i\mu_i^2-(\sum_i n_i\mu_i)^2/N\), the expanded left side. The identity shows that allocation affects the contribution from each pair of treatment means, not just the overall spread of the means without regard to group sizes.

Worked Comparisons at a Fixed Total Sample Size

Worked Example: Imbalance Lowers Power for a Gradual Pattern

Consider three treatments with specified means \(0\), \(1\), and \(2\) response units, common error variance \(\sigma^2=4\) response-units squared, and total sample size \(N=18\). Compare balanced allocation \((6,6,6)\) with allocation \((3,6,9)\). The model assumptions are taken to hold in both designs.

For the balanced allocation, the weighted mean is \[ \bar{\mu}_w=\frac{6(0)+6(1)+6(2)}{18}=1. \] The weighted squared separation is \[ 6(0-1)^2+6(1-1)^2+6(2-1)^2=6+0+6=12. \] Therefore, \[ \lambda_{\text{balanced}}=\frac{12}{4}=3. \]

For allocation \((3,6,9)\), the weighted mean is \[ \bar{\mu}_w=\frac{3(0)+6(1)+9(2)}{18}=\frac{24}{18}=\frac{4}{3}. \] The weighted squared separation is \[ 3\left(0-\frac{4}{3}\right)^2+ 6\left(1-\frac{4}{3}\right)^2+ 9\left(2-\frac{4}{3}\right)^2 =\frac{48+6+36}{9}=10. \] Thus \(\lambda_{\text{unequal}}=10/4=2.5\).

Both designs have \(k-1=2\) numerator degrees of freedom and \(N-k=15\) denominator degrees of freedom. At the same \(\alpha\), they therefore use the same critical value. Because \(2.5<3\), the unequal allocation has lower power for this specified pattern of means. The model assumptions, total sample size, and variance were held fixed, so this comparison isolates the allocation effect.

Worked Example: Imbalance Raises Power for a Particular Alternative

Now suppose the treatment means are \(0\), \(0\), and \(2\) response units, with \(\sigma^2=4\) response-units squared and \(N=18\). Compare balanced allocation \((6,6,6)\) with \((4,4,10)\). Again assume independent normal errors with the same variance in all groups.

With equal group sizes, the weighted mean is \(2/3\). The weighted squared separation is \[ 6\left(0-\frac{2}{3}\right)^2+ 6\left(0-\frac{2}{3}\right)^2+ 6\left(2-\frac{2}{3}\right)^2 =\frac{24+24+96}{9}=16. \] Therefore \(\lambda_{\text{balanced}}=16/4=4\).

For allocation \((4,4,10)\), the weighted mean is \[ \bar{\mu}_w=\frac{4(0)+4(0)+10(2)}{18}=\frac{10}{9}. \] The individual contributions to the weighted squared separation are \[ 4\left(0-\frac{10}{9}\right)^2=\frac{400}{81},\qquad 4\left(0-\frac{10}{9}\right)^2=\frac{400}{81},\qquad 10\left(2-\frac{10}{9}\right)^2=\frac{640}{81}. \] Their sum is \[ \frac{400+400+640}{81}=\frac{1440}{81}=\frac{160}{9}\approx17.78. \] Consequently, \[ \lambda_{\text{unequal}}= \frac{160/9}{4}=\frac{40}{9}\approx4.444. \]

The alternative places one treatment mean apart from two equal means, and this allocation puts more observations in the treatment with the distinct mean. That weighting raises \(\lambda\) relative to equal allocation. Since both designs again have \(df=2\) and \(df=15\), the unequal allocation has greater power for this particular alternative. This does not establish that it is better for other possible patterns of treatment means.

Turning Noncentrality into Power

A larger noncentrality parameter means the alternative distribution of the F statistic is shifted toward larger values. For fixed degrees of freedom, the probability of exceeding the same upper-tail critical value increases with \(\lambda\). One justification uses the representation of a noncentral chi-square variable: a noncentral chi-square with \(r\) degrees of freedom and parameter \(\lambda\) is a mixture of central chi-square distributions with \(r+2J\) degrees of freedom, where \(J\) has a Poisson distribution with mean \(\lambda/2\). Increasing \(\lambda\) increases \(J\) in stochastic order; adding independent squared normal terms increases the numerator. The denominator of the noncentral F statistic has the same central chi-square distribution for the designs being compared. Thus, with degrees of freedom fixed, increasing \(\lambda\) increases the upper-tail rejection probability.

This result lets us compare power in the examples without calculating a separate noncentral-F tail probability for each design. If a numerical power value is needed, evaluate the stated noncentral-F probability using the corresponding \(\lambda\), degrees of freedom, and significance level. The comparison still requires a specified alternative: without treatment means and an error variance, allocation alone does not determine power.

Worked Example: Comparing Power Under a Second Fixed-N Plan

Suppose three treatments have means \(0\), \(1\), and \(2\) minutes, common error variance \(\sigma^2=2.25\) minutes squared, and \(N=24\). Compare \((8,8,8)\) with \((4,8,12)\). Assume the one-way model’s independence, normality, and equal-variance conditions hold.

For \((8,8,8)\), \(\bar{\mu}_w=1\) minute, so the weighted squared separation is \[ 8(0-1)^2+8(1-1)^2+8(2-1)^2=16\text{ minutes}^2. \] Thus \[ \lambda_{\text{balanced}}=\frac{16}{2.25}=\frac{64}{9}\approx7.111. \]

For \((4,8,12)\), the weighted mean is \[ \bar{\mu}_w=\frac{4(0)+8(1)+12(2)}{24}=\frac{32}{24}=\frac{4}{3}\text{ minutes}. \] The weighted squared separation is \[ 4\left(0-\frac{4}{3}\right)^2+ 8\left(1-\frac{4}{3}\right)^2+ 12\left(2-\frac{4}{3}\right)^2 =\frac{64+8+48}{9}=\frac{40}{3}\text{ minutes}^2. \] Hence \[ \lambda_{\text{unequal}}=\frac{40/3}{2.25} =\frac{160}{27}\approx5.926. \]

Both designs have \(df_1=2\) and \(df_2=21\). At \(\alpha=0.05\), the balanced design’s power is \(P(F_{2,21,64/9}>F_{0.95;2,21})\), while the unequal design’s power is \(P(F_{2,21,160/27}>F_{0.95;2,21})\). Since \(64/9>160/27\), the balanced allocation has higher power for these means and this variance. The units cancel in each ratio that forms \(\lambda\).

How to Interpret an Allocation Comparison

When \(N\) is fixed, every allocation in a comparison has the same residual degrees of freedom \(N-k\), provided every treatment is represented. If the number of treatments and \(\alpha\) are also fixed, the critical value is unchanged. The remaining comparison is through \(\lambda\), which depends jointly on the group sizes, treatment means, and error variance. Multiplying all mean differences by the same factor multiplies \(\lambda\) by the square of that factor; doubling \(\sigma\) divides \(\lambda\) by four.

For equal group sizes \(n_i=N/k\), the weighted mean is the ordinary average of the treatment means. This symmetry can be useful when planning a design, especially if no particular contrast or alternative pattern is privileged. But equal allocation is not a universal maximizer of power for every single specified alternative. The second worked example shows how concentrating observations in a treatment whose mean differs from the others can raise the omnibus test’s noncentrality parameter. Conversely, an allocation that puts relatively few observations in a treatment important to the alternative can reduce it.

An omnibus F test asks whether the means are all equal, not which specific means differ. Therefore, an allocation that is favorable for one mean pattern may not be favorable for another. A power calculation should state the alternative means and the common variance, not simply say that an allocation is “balanced” or “imbalanced.”

Common Mistake

Wrong reasoning: “Unequal group sizes always reduce the power of the one-way F test.”
Correction: With total sample size fixed, compare \(\lambda=\sum_i n_i(\mu_i-\bar{\mu}_w)^2/\sigma^2\) under the specified alternative. Imbalance lowers power when it lowers this quantity, and can raise power when it increases it. The direction depends on which treatments receive the larger groups and on the pattern of means.
Another pitfall: Comparing allocations while also changing total sample size, error variance, or the alternative means does not isolate the effect of allocation. Hold these quantities fixed when making that comparison.

Key takeaway: For a one-way fixed-effects F test, allocation affects power through the noncentrality parameter. With the same total sample size, degrees of freedom, and variance, compare the allocation-weighted spread of the specified treatment means. Imbalance is not automatically harmful or beneficial.

Check Your Understanding

Use the model and definitions above to answer these questions.

For a one-way design with \(k\) groups and total sample size \(N\), what are the F test’s numerator and denominator degrees of freedom?
Why does the weighted mean \(\bar{\mu}_w\) depend on the group sizes?
For fixed \(N\), \(k\), and \(\alpha\), what quantity must be compared to determine which allocation has higher power under a specified alternative?
Explain why unequal allocation can raise power when one treatment mean differs from the other means.
List the equal-variance model assumptions that are needed for the noncentral-F power calculation used here.