Common Errors in ANOVA Sums of Squares

Why Small ANOVA Errors Matter

In one-way ANOVA, the sums of squares describe different sources of variation. A small mistake—such as averaging group means without accounting for group sizes—can change the result, even when every individual observation was recorded correctly. Other common errors come from using the wrong center for a deviation or assigning degrees of freedom to the wrong source of variation.

This tutorial focuses on checking those calculations. The setting is the one-way model with \(k\) groups, where group \(j\) contains \(n_j\) observations \(Y_{ij}\), for \(i=1,\ldots,n_j\). The total sample size is \(N=\sum_{j=1}^k n_j\), the group sample mean is \(\bar{Y}_j\), and the grand mean is \(\bar{Y}_{..}\). For the usual inferential one-way ANOVA, observations are independent and the errors are normally distributed with a common variance across groups. The arithmetic identities among sums of squares hold regardless of those distributional assumptions.

Definition: The total sum of squares measures variation of observations around the grand mean: \(SS_T=\sum_{j=1}^k\sum_{i=1}^{n_j}(Y_{ij}-\bar{Y}_{..})^2\). The between-groups sum of squares measures variation of group means around the grand mean, weighted by group sizes: \(SS_B=\sum_{j=1}^k n_j(\bar{Y}_j-\bar{Y}_{..})^2\). The within-groups sum of squares measures variation of observations around their own group means: \(SS_W=\sum_{j=1}^k\sum_{i=1}^{n_j}(Y_{ij}-\bar{Y}_j)^2\). In one-way ANOVA, \(SS_T=SS_B+SS_W\).

Common Error 1: Treating Unequal Groups as Equally Sized

The grand mean is the mean of all \(N\) observations. When group sizes differ, it is a weighted average of the group means: larger groups contribute more observations and therefore have more weight. An unweighted average of the group means gives each group equal influence, regardless of how many observations it contains.

\bar{Y}_{..} =\frac{\sum_{j=1}^k n_j\bar{Y}_j}{N} \qquad\text{whereas}\qquad \frac{1}{k}\sum_{j=1}^k\bar{Y}_j \text{ is generally not the grand mean.}

Worked Example: Correcting an Unweighted Grand Mean

Suppose three hypothetical groups have scores, in points, of A: 2 and 4; B: 6, 7, and 8; and C: 9, 10, 12, and 13. Their sample sizes are 2, 3, and 4, and their means are 3, 7, and 11 points. Calculate the between-groups sum of squares. Then compare it with the result from incorrectly centering the group means at their unweighted average.

There are \(N=2+3+4=9\) observations. The grand mean is the total of all scores divided by 9:

\bar{Y}_{..} =\frac{2(3)+3(7)+4(11)}{9} =\frac{6+21+44}{9} =\frac{71}{9} \approx 7.889\text{ points}.

Use this grand mean in the between-groups formula. Each squared deviation of a group mean is multiplied by that group’s size:

\begin{aligned} SS_B &=2\left(3-\frac{71}{9}\right)^2 +3\left(7-\frac{71}{9}\right)^2 +4\left(11-\frac{71}{9}\right)^2\\ &=2\left(\frac{-44}{9}\right)^2 +3\left(\frac{-8}{9}\right)^2 +4\left(\frac{28}{9}\right)^2\\ &=\frac{3872+192+3136}{81} =\frac{7200}{81} \approx 88.89\text{ points}^2. \end{aligned}

The unweighted average of the group means is \((3+7+11)/3=7\) points. If 7 is incorrectly used as the center, the calculation is:

2(3-7)^2+3(7-7)^2+4(11-7)^2 =2(16)+3(0)+4(16) =96\text{ points}^2.

The arithmetic gives 96, but that is not the between-groups sum of squares around the grand mean. It is the weighted squared deviation around 7, which is not the mean of all nine observations. The difference is not a rounding issue: using the incorrect center adds \(N(7-\bar{Y}_{..})^2=9(8/9)^2=64/9\approx7.111\) points² to the correct value, and \(88.89+7.111=96.00\) points².

Common Error 2: Mixing Within- and Between-Groups Deviations

The two sums of squares use different centers. For \(SS_B\), compare each group mean with the grand mean. For \(SS_W\), compare each observation with its own group mean. Using the grand mean for every observation computes total variation, not within-groups variation. Using individual observations in the between-groups formula also changes what is being measured.

Worked Example: Choosing the Right Center for Each Sum of Squares

Consider two hypothetical groups of scores, in points: A has 2, 4, and 6; B has 8, 10, and 12. Find the within- and between-groups sums of squares, and check their relationship to the total sum of squares.

The group means are \(\bar{Y}_A=4\) and \(\bar{Y}_B=10\), and the grand mean is \((2+4+6+8+10+12)/6=7\) points. For within-groups variation, center observations at their own group means:

\begin{aligned} SS_W &=[(2-4)^2+(4-4)^2+(6-4)^2]\\ &\quad+[(8-10)^2+(10-10)^2+(12-10)^2]\\ &=(4+0+4)+(4+0+4) =16\text{ points}^2. \end{aligned}

For between-groups variation, center the two group means at the grand mean and weight each squared deviation by its group size:

SS_B=3(4-7)^2+3(10-7)^2 =3(9)+3(9) =54\text{ points}^2.

As a check, calculate total variation around the grand mean. The squared deviations are 25, 9, 1, 1, 9, and 25, so \(SS_T=70\) points². This agrees with the decomposition: \(SS_B+SS_W=54+16=70\) points². If the observations had instead been squared around 7 and called “within,” that calculation would have produced \(SS_T=70\), not \(SS_W=16\). The center determines which source of variation is being measured.

Why the Decomposition Works

The split between within- and between-groups variation follows by expressing each observation’s deviation from the grand mean as two parts: its deviation from its own group mean and that group mean’s deviation from the grand mean. When the square is expanded, the cross-product terms sum to zero within each group.

\begin{aligned} Y_{ij}-\bar{Y}_{..} &=(Y_{ij}-\bar{Y}_j)+(\bar{Y}_j-\bar{Y}_{..}),\\ SS_T &=\sum_j\sum_i(Y_{ij}-\bar{Y}_j)^2 +\sum_j n_j(\bar{Y}_j-\bar{Y}_{..})^2\\ &\quad+2\sum_j(\bar{Y}_j-\bar{Y}_{..}) \sum_i(Y_{ij}-\bar{Y}_j). \end{aligned}

For every group \(j\), the deviations from its sample mean sum to zero: \(\sum_i(Y_{ij}-\bar{Y}_j)=0\). Thus the last, cross-product term vanishes. The first term is \(SS_W\), and the second is \(SS_B\), giving \(SS_T=SS_W+SS_B\). This identity is a useful diagnostic: if the calculated totals do not agree, check the means, centers, weights, and arithmetic.

Common Error 3: Assigning the Wrong Degrees of Freedom

Degrees of freedom count the independent pieces of information left after constraints are imposed. In one-way ANOVA, total deviations from the grand mean have one constraint: they sum to zero. Within each group, deviations from that group’s mean sum to zero, giving one constraint per group. The between-groups deviations of the \(k\) group means also satisfy one weighted constraint, because their weighted deviations sum to zero.

Worked Example: Checking Degrees of Freedom With Unequal Group Sizes

Return to the three hypothetical groups with sizes 2, 3, and 4. There are \(k=3\) groups and \(N=9\) observations. Determine the degrees of freedom for total, between-groups, and within-groups variation.

There are nine observations, and their deviations from the grand mean must sum to zero. Therefore, total degrees of freedom are \(N-1=9-1=8\). For between-groups variation, there are three group means, but their weighted deviations from the grand mean satisfy one constraint, \(\sum_j n_j(\bar{Y}_j-\bar{Y}_{..})=0\). Thus \(df_B=k-1=3-1=2\).

For within-groups variation, each group’s residuals must sum to zero. Those are three constraints across the nine observations, so \(df_W=N-k=9-3=6\). The degrees of freedom add up as they should: \(df_B+df_W=2+6=8=df_T\). Using \(N-1\) for within-groups degrees of freedom, or using \(k-1\) for total degrees of freedom, would assign the constraints to the wrong source.

A Practical Error-Checking Routine

Before accepting a one-way ANOVA sum-of-squares calculation, check the following items in order. They are especially useful when group sizes are unequal, because unweighted shortcuts can look plausible while targeting the wrong quantity.

Check the group summaries.
Confirm each \(n_j\) and \(\bar{Y}_j\) from the observations, and verify that \(N=\sum_j n_j\).

Recompute the grand mean.
Use \(\bar{Y}_{..}=\sum_j n_j\bar{Y}_j/N\), not the unweighted average of group means unless all group sizes are equal.

Match each deviation to its source.
Use observations minus their group mean for \(SS_W\); use group means minus the grand mean for \(SS_B\).

Check the identities and constraints.
Verify \(SS_T=SS_B+SS_W\) and \(df_T=df_B+df_W=(N-1)\).

Common Mistakes and Their Corrections

Using an unweighted mean of group means. “There are three groups, so each mean gets one-third of the weight.” This is only appropriate for the grand mean when group sizes are equal. Otherwise, weight each group mean by \(n_j\).
Centering within-group observations at the grand mean. That measures total variation. For \(SS_W\), each observation must be centered at its own group mean.
Leaving out group-size weights in \(SS_B\). Each group mean represents \(n_j\) observations, so its squared deviation contributes \(n_j(\bar{Y}_j-\bar{Y}_{..})^2\), not just \((\bar{Y}_j-\bar{Y}_{..})^2\).
Using a degrees-of-freedom formula without identifying constraints. The correct values are \(df_B=k-1\), \(df_W=N-k\), and \(df_T=N-1\). Check that the first two sum to the third.
Assuming a matching total proves every part is right. Errors can sometimes offset one another. Check the centers, weights, and degrees of freedom separately, as well as the final decomposition.

Key takeaway: Keep the centers and weights straight: observations are compared with their own group mean for \(SS_W\), and group means are compared with the weighted grand mean for \(SS_B\). Then check \(SS_T=SS_B+SS_W\) and \(df_T=df_B+df_W=N-1\).

Check Your Understanding

Try these questions using the definitions and checks in this tutorial.

Two groups have sizes 4 and 8 and means 5 and 9. Write the formula for their grand mean and calculate it. Why is the average of 5 and 9 not the grand mean?
For an observation in group A, should the within-groups deviation be measured from group A’s mean or from the grand mean? Explain what the other choice measures.
A one-way ANOVA has \(N=20\) observations in \(k=4\) groups. Find \(df_B\), \(df_W\), and \(df_T\), and verify that the degrees of freedom add correctly.
Suppose a calculation gives \(SS_T=40\), \(SS_B=28\), and \(SS_W=15\). What check fails, and what should be reviewed before accepting the calculations?
Why does the between-groups formula multiply each squared group-mean deviation by that group’s sample size?