Mean Squares and the F Statistic

From Sums of Squares to Mean Squares

In one-way analysis of variance, the total observed variation is divided into variation between the group means and variation within the groups. Those sums of squares describe amounts of variation, but their magnitudes depend on how many independent pieces of information contribute to each amount. Dividing each sum of squares by its degrees of freedom produces a mean square, a variation measure adjusted for that information.

The F statistic compares the between-group mean square with the within-group mean square. Its construction is mechanical, but the details matter: the two sums of squares have different degrees of freedom, and the ratio has a specific order. The statistic summarizes how large the between-group variation is relative to variation among observations within their groups.

Definition: Suppose there are $k$ groups, with $n_j$ observations in group $j$, for $j=1,\ldots,k$, and $N=\sum_{j=1}^k n_j$ observations altogether. Let $\bar{x}_j$ be the mean of group $j$, and let $\bar{x}$ be the grand mean of all $N$ observations. The between-group and within-group sums of squares are $$ SS_B=\sum_{j=1}^k n_j(\bar{x}_j-\bar{x})^2, \qquad SS_W=\sum_{j=1}^k\sum_{i=1}^{n_j}(x_{ij}-\bar{x}_j)^2. $$ The corresponding mean squares and the one-way ANOVA F statistic are $$ MS_B=\frac{SS_B}{k-1}, \qquad MS_W=\frac{SS_W}{N-k}, \qquad F=\frac{MS_B}{MS_W}. $$

Here $SS_B$ measures how far the group means lie from the grand mean, with each squared difference weighted by that group’s size. $SS_W$ measures how far observations lie from their own group mean. Both sums of squares are nonnegative. The degrees of freedom are $k-1$ for the between-group sum of squares and $N-k$ for the within-group sum of squares.

Why These Degrees of Freedom?

The degrees of freedom count how many independent deviations remain after the relevant mean or means have been estimated. For the group means, there are $k$ deviations from the grand mean, $\bar{x}_j-\bar{x}$. These deviations cannot vary freely: their size-weighted sum is zero.

\sum_{j=1}^k n_j(\bar{x}_j-\bar{x}) = \sum_{j=1}^k n_j\bar{x}_j-\bar{x}\sum_{j=1}^k n_j = N\bar{x}-N\bar{x} = 0.

That one constraint leaves $k-1$ independent between-group deviations. Within group $j$, the $n_j$ residuals $x_{ij}-\bar{x}_j$ also sum to zero, because the deviations from a group’s sample mean always sum to zero. This leaves $n_j-1$ independent within-group deviations in that group. Adding across groups gives

\sum_{j=1}^k(n_j-1)=\sum_{j=1}^k n_j-k=N-k.

This explains why $k-1$ and $N-k$, rather than $k$ and $N$, are the denominators in the mean squares. It also agrees with the total degrees of freedom $N-1$: the between-group and within-group degrees of freedom add to $(k-1)+(N-k)=N-1$.

Calculating and Reading the F Ratio

Once the two sums of squares and their degrees of freedom are available, calculate each mean square separately, then divide the between-group mean square by the within-group mean square. The ratio is unitless because both mean squares use the same squared measurement units. A ratio greater than 1 means that the between-group mean square exceeds the within-group mean square; a ratio below 1 means it is smaller.

In an ANOVA analysis, a relatively large F statistic can indicate that observed group differences are large compared with the within-group variation. The ratio alone does not give a probability or a conclusion about statistical significance. Such an inferential interpretation requires the model and reference distribution for the test; here the focus is on forming and understanding the statistic.

Worked Example: Calculate Mean Squares From Raw Observations

A hypothetical analyst compares processing times, in hours, for three methods used on a set of claims. The recorded times are Method A: 4, 6, 5, 5; Method B: 7, 9, 8, 8; and Method C: 10, 12, 11, 11. Calculate the two mean squares and the F statistic.

Method. Find the group means and grand mean, calculate each sum of squares, use the correct degrees of freedom, and form $MS_B/MS_W$.

Group means. Method A has mean $(4+6+5+5)/4=20/4=5$. Method B has mean $(7+9+8+8)/4=32/4=8$. Method C has mean $(10+12+11+11)/4=44/4=11$. The grand mean is $(20+32+44)/12=96/12=8$.

Between-group sum of squares. Each group has size 4, so

$$ SS_B=4(5-8)^2+4(8-8)^2+4(11-8)^2 =4(9)+4(0)+4(9)=72. $$

Within-group sum of squares. For Method A, the squared deviations from 5 sum to $1+1+0+0=2$. For Method B, the squared deviations from 8 sum to $1+1+0+0=2$. For Method C, the squared deviations from 11 sum to $1+1+0+0=2$. Thus $SS_W=2+2+2=6$.

There are $k=3$ groups and $N=12$ observations. Therefore, the between-group degrees of freedom are $3-1=2$, and the within-group degrees of freedom are $12-3=9$. The mean squares are

MS_B=\frac{72}{2}=36, \qquad MS_W=\frac{6}{9}=\frac{2}{3}.

The F statistic is

F=\frac{36}{2/3}=36\left(\frac{3}{2}\right)=54.

Check. The total sum of squares is $SS_B+SS_W=72+6=78$. Directly, the observations’ squared deviations from the grand mean 8 sum to $16+4+9+9+1+1+0+0+4+16+9+9=78$. The two parts agree. The calculation avoids the common mistake of using the number of observations as a mean-square denominator.

Unequal Group Sizes and Summary Information

The same formulas apply when group sizes differ. The between-group sum of squares must weight each group’s squared distance from the grand mean by its own size. If group means, group sizes, and within-group sums of squares are provided, the raw observations are not needed to calculate the mean squares.

Worked Example: Use Group Summaries With Unequal Sizes

A hypothetical service team compares the time required to process a particular type of request under three procedures. The summary information is: Procedure A has $n_1=2$, mean 1, and within-group sum of squares 2; Procedure B has $n_2=4$, mean 3, and within-group sum of squares 6; Procedure C has $n_3=2$, mean 5, and within-group sum of squares 4. Calculate $MS_B$, $MS_W$, and F.

Method. First calculate the weighted grand mean, then use the group-size-weighted between sum of squares and add the given within-group sums of squares.

The total number of observations is $N=2+4+2=8$. The weighted grand mean is

\bar{x}=\frac{2(1)+4(3)+2(5)}{8} =\frac{2+12+10}{8} =\frac{24}{8}=3.

The between-group sum of squares is

$$ SS_B=2(1-3)^2+4(3-3)^2+2(5-3)^2 =2(4)+4(0)+2(4)=16. $$

The within-group sum of squares is $SS_W=2+6+4=12$. With $k=3$, the degrees of freedom are $k-1=2$ between groups and $N-k=8-3=5$ within groups. Hence

MS_B=\frac{16}{2}=8, \qquad MS_W=\frac{12}{5}=2.4, \qquad F=\frac{8}{2.4}=\frac{10}{3}=3.333\ldots.

Rounded to four decimal places, $F=3.3333$. Check. The weighted group deviations satisfy $2(1-3)+4(3-3)+2(5-3)=-4+0+4=0$, as they must about the grand mean. The calculation avoids treating the three group means as if each represented the same number of observations.

A Complete Summary-Based Calculation

Mean squares can also be calculated when the sums of squares are already supplied. The key is still to identify $k$ and $N$ before choosing the denominators. Consider a hypothetical comparison of four rating methods, with $N=20$, $SS_B=45$, and $SS_W=64$.

Worked Example: Identify the Correct Denominators

Calculate the mean squares and F statistic for the four-method comparison, then state what the ratio means as a comparison of variation.

Method. Use $k-1$ for the between-group degrees of freedom and $N-k$ for the within-group degrees of freedom. Here $k=4$ and $N=20$, so

df_B=4-1=3, \qquad df_W=20-4=16.

Divide each sum of squares by its corresponding degrees of freedom:

MS_B=\frac{45}{3}=15, \qquad MS_W=\frac{64}{16}=4.

Therefore,

F=\frac{15}{4}=3.75.

Interpretation. The between-group mean square is 3.75 times the within-group mean square. This describes the relative sizes of the two adjusted variation measures; by itself, it is not a p-value or a test decision. Check. The degrees of freedom add to $3+16=19=N-1$, confirming the partition’s total degrees of freedom.

Common Mistakes to Avoid

A frequent error is to divide by the number of groups or total observations rather than by the appropriate degrees of freedom. Another is to reverse the F ratio. Both errors can produce a numerical answer, but neither gives the one-way ANOVA statistic.

Common mistake: In the first worked example, an incorrect calculation might use $MS_B=72/3=24$ and $MS_W=6/12=0.5$, giving $F=24/0.5=48$. The denominators $3$ and $12$ are counts, not degrees of freedom. The correction is $MS_B=72/(3-1)=36$ and $MS_W=6/(12-3)=2/3$, so $F=36/(2/3)=54$. Also keep the ratio in the stated order, $MS_B/MS_W$, rather than its reciprocal.

When checking a calculation, verify each of the following before interpreting the ratio:

The between-group degrees of freedom are $k-1$, not $k$.
The within-group degrees of freedom are $N-k$, not $N$ or $N-1$.
The mean square uses the sum of squares with the matching degrees of freedom.
The F statistic is between-group mean square divided by within-group mean square.
The between-group sum of squares uses group-size weights when group sizes are unequal.

Key takeaway: Calculate $MS_B=SS_B/(k-1)$ and $MS_W=SS_W/(N-k)$, then form $F=MS_B/MS_W$. The F ratio compares between-group variation with within-group variation; its inferential interpretation depends on the model used for the ANOVA test.

Check Your Understanding

For each problem, calculate the requested mean squares and F statistic. Answers are provided for checking.

A study has $k=3$ groups and $N=15$ observations, with $SS_B=40$ and $SS_W=36$. Find both degrees of freedom, both mean squares, and F. (Answer: $df_B=2$, $df_W=12$, $MS_B=20$, $MS_W=3$, $F=6.6667$.)
Three groups have sizes 2, 4, and 2, and respective means 1, 3, and 5. The within-group sum of squares is 20 in total. Find the grand mean, $SS_B$, both mean squares, and F. (Answer: $\bar{x}=3$, $SS_B=16$, $MS_B=8$, $MS_W=4$, $F=2$.)
An ANOVA has $k=5$, $N=25$, total sum of squares $SS_T=100$, and $SS_B=28$. Find $SS_W$, the degrees of freedom, the mean squares, and F. (Answer: $SS_W=72$, $df_B=4$, $df_W=20$, $MS_B=7$, $MS_W=3.6$, $F=1.9444$.)
For four groups and $N=16$, suppose $SS_B=18$ and $SS_W=24$. Calculate the correct F ratio and state whether it is above or below 1. (Answer: $df_B=3$, $df_W=12$, $MS_B=6$, $MS_W=2$, $F=3$, above 1.)
For three groups and $N=12$, an analyst reports $SS_B=54$ and $SS_W=45$. Calculate F and explain what the ratio compares, without making a significance claim. (Answer: $df_B=2$, $df_W=9$, $MS_B=27$, $MS_W=5$, $F=5.4$; the between-group mean square is 5.4 times the within-group mean square.)

What You'll Learn