Degrees of Freedom in One-Way ANOVA

Why Degrees of Freedom Matter in One-Way ANOVA

In one-way ANOVA, the observed variation is divided into variation between groups and variation within groups. Each part has its own degrees of freedom, and those values depend on the number of groups and observations. Getting them right is important for organizing an ANOVA calculation and understanding which information each part of the variation represents.

The key idea is that degrees of freedom count how many values can vary independently once required constraints have been imposed. When deviations are measured from a mean, they cannot vary freely: their sum must be zero. In ANOVA, there is one such constraint for the total data, one among the group means, and one within each group.

Definition: The degrees of freedom for a sum of squared deviations count the independent deviations that remain after the constraints defining those deviations are applied. In a one-way ANOVA with \(k\) nonempty groups and \(N\) total observations, the total, between-group, and within-group degrees of freedom are \(N-1\), \(k-1\), and \(N-k\), respectively.

Suppose the data contain \(k\) groups. Group \(i\) has \(n_i\) observations, and every observation belongs to exactly one group. Thus,

N=\sum_{i=1}^{k} n_i

Here, each group must contain at least one observation, so \(n_i\geq 1\). The formulas below count the degrees of freedom associated with the total sum of squares, the between-group sum of squares, and the within-group sum of squares. They do not require the groups to be the same size.

The Three Degrees of Freedom

The total sum of squares measures the squared deviations of all observations from the grand mean. There are \(N\) such deviations, but they must sum to zero. Once \(N-1\) deviations are chosen, the final one is fixed by that constraint. Therefore, the total degrees of freedom are \(N-1\).

df_{\text{total}}=N-1

The between-group sum of squares describes how the \(k\) group means vary around the grand mean. Those means do not vary independently: their deviations from the grand mean, weighted by the group sizes, must sum to zero. That is one constraint on \(k\) group means, leaving \(k-1\) degrees of freedom.

df_{\text{between}}=k-1

The within-group sum of squares describes the deviations of observations from their own group means. Within each group, the deviations must sum to zero. Group \(i\) has \(n_i\) deviations and one constraint, so it contributes \(n_i-1\) degrees of freedom. Adding across groups gives

df_{\text{within}}=\sum_{i=1}^{k}(n_i-1) =\sum_{i=1}^{k}n_i-k =N-k

These counts describe the structure of the sums of squares. They do not change if some observed values happen to be equal, or if all the group means happen to match. Such patterns can make a particular sum of squares small or even zero, but they do not remove the constraints that determine its degrees of freedom.

Why the Degrees of Freedom Add

In one-way ANOVA, the total variation separates into between-group variation and within-group variation. The earlier material on ANOVA variation decomposition establishes the corresponding identity for the sums of squares. The degrees of freedom follow the same partition: the total centered data have \(N-1\) independent directions of variation, divided into \(k-1\) between-group directions and \(N-k\) within-group directions.

df_{\text{between}}+df_{\text{within}} =(k-1)+(N-k) =N-1 =df_{\text{total}}

A useful way to justify the partition is to consider the \(N\) observation deviations from the grand mean. They form a vector whose entries sum to zero, so they lie in a space of dimension \(N-1\). Within-group deviations make up a space of dimension \(N-k\): each of the \(k\) groups has its own zero-sum constraint. The remaining variation is constant within each group and describes differences among the group means. Those group-level deviations have one weighted zero-sum constraint, leaving dimension \(k-1\). These two kinds of variation are orthogonal, so their dimensions add to the dimension of the total centered data.

Worked Examples

Worked Example: Three Groups With Unequal Sizes

Suppose a hypothetical experiment records response values for three groups. Group A has observations \(4, 6, 5, 7\); Group B has \(8, 9, 7, 10, 6\); and Group C has \(3, 5, 4\). Find the total, between-group, and within-group degrees of freedom.

There are \(k=3\) groups. Their sizes are \(n_1=4\), \(n_2=5\), and \(n_3=3\), so the total number of observations is

$$ N=4+5+3=12 $$

The total degrees of freedom are

df_{\text{total}}=N-1=12-1=11

The between-group degrees of freedom are

df_{\text{between}}=k-1=3-1=2

The within-group degrees of freedom can be calculated from the group sizes:

df_{\text{within}}=(4-1)+(5-1)+(3-1)=3+4+2=9

Equivalently, \(df_{\text{within}}=N-k=12-3=9\). The check is

$$ 2+9=11 $$

Thus, the degrees of freedom are 2 between groups, 9 within groups, and 11 in total. The unequal group sizes affect the within-group calculation through the values \(n_i-1\), but the compact formula \(N-k\) gives the same result.

Worked Example: Calculating Within-Group Degrees of Freedom by Group

A hypothetical study compares four groups with sample sizes \(2\), \(6\), \(3\), and \(5\). Calculate the three degrees of freedom and explain the within-group count.

There are \(k=4\) groups, and

$$ N=2+6+3+5=16 $$

For the total variation,

df_{\text{total}}=N-1=16-1=15

For the between-group variation,

df_{\text{between}}=k-1=4-1=3

Within the groups, the four contributions are \(2-1=1\), \(6-1=5\), \(3-1=2\), and \(5-1=4\). Therefore,

df_{\text{within}}=1+5+2+4=12

Using the total and group count gives the same value:

df_{\text{within}}=N-k=16-4=12

The complete check is \(3+12=15\). The group of size 2 contributes only 1 within-group degree of freedom because its two deviations from its group mean must sum to zero; after one is specified, the other is determined.

Worked Example: Finding the Number of Groups From Degrees of Freedom

A hypothetical ANOVA has \(N=18\) observations and a reported within-group degrees of freedom of \(14\). Assuming every group is nonempty, find the number of groups and the other two degrees of freedom.

The within-group formula is \(df_{\text{within}}=N-k\). Substitute the known values and solve for \(k\):

$$ 14=18-k $$

Subtracting 14 from 18 gives

$$ k=18-14=4 $$

The between-group degrees of freedom are then

df_{\text{between}}=k-1=4-1=3

The total degrees of freedom are

df_{\text{total}}=N-1=18-1=17

The values are consistent because \(3+14=17\). This check is useful when reading or completing an ANOVA calculation: the between and within degrees of freedom must add to the total.

What the Counts Do—and Do Not—Depend On

The formulas require the number of observations and the number of nonempty groups, not the numerical values of the responses. For example, the total degrees of freedom remain \(N-1\) even if the observations are all identical. In that case the total sum of squares is zero, but the total deviations still have the same sum-to-zero constraint.

Likewise, if every group has the same sample mean, the between-group sum of squares is zero, but its degrees of freedom remain \(k-1\). The degrees of freedom describe the number of independent directions available in the data structure; the sum of squares describes how much observed variation lies in those directions.

Counting degrees of freedom is an algebraic feature of this ANOVA setup. Normality and equal population variances are not needed to count them. Those conditions concern the assumptions for particular inferential procedures, not the arithmetic that produces \(N-1\), \(k-1\), and \(N-k\).

Common Mistake: Using the Wrong Count

A frequent error is to use the number of groups \(k\) in place of the total number of observations \(N\) when calculating within-group degrees of freedom. Another is to say that the between-group degrees of freedom equal \(k\), overlooking the constraint on the group means.

For example, with \(N=16\) observations in \(k=4\) groups, the within-group degrees of freedom are \(16-4=12\), not \(16-1\) and not \(4-1\). The between-group degrees of freedom are \(4-1=3\). The final check, \(12+3=15=16-1\), catches either mix-up.

Key takeaway: For \(N\) observations in \(k\) nonempty groups, \(df_{\text{total}}=N-1\), \(df_{\text{between}}=k-1\), and \(df_{\text{within}}=N-k\). The component degrees of freedom add because the total centered variation is partitioned into between-group and within-group variation.

Check Your Understanding

Use the group counts and constraints above to answer the following questions.

A one-way ANOVA has \(N=25\) observations in \(k=5\) nonempty groups. Find the total, between-group, and within-group degrees of freedom, and verify that they add.
Three groups have sample sizes \(4\), \(4\), and \(7\). Calculate the within-group degrees of freedom by adding the group contributions and by using \(N-k\).
Why does calculating deviations from a mean impose one constraint on their sum?
If every group mean is numerically identical, do the between-group degrees of freedom become zero? Explain.
A report gives \(df_{\text{total}}=20\) and \(df_{\text{between}}=3\). Find \(N\), \(k\), and \(df_{\text{within}}\), assuming the standard one-way ANOVA setup.