The Total Sum of Squares Decomposition

Why the Total Sum of Squares Splits Into Two Parts

In a one-way analysis of variance, each observation differs from the overall average for two possible reasons: it belongs to a group whose average differs from the overall average, and it differs from its own group’s average. The total sum of squares decomposition separates these sources of squared variation exactly. It is an algebraic identity, and it is the foundation for organizing variation in a one-way ANOVA.

The identity applies to a set of numerical observations divided into groups. No normality, equal-variance, or independence assumption is needed for the identity itself. Those assumptions concern statistical inference using ANOVA, not this arithmetic partition.

Definition: Suppose there are \(k\) groups. Group \(j\) contains \(n_j\) observations \(y_{1j},\ldots,y_{n_jj}\), with group mean \(\bar{y}_{j\cdot}\). The total number of observations is \(N=\sum_{j=1}^{k}n_j\), and the grand mean is \(\bar{y}_{\cdot\cdot}=\frac{1}{N}\sum_{j=1}^{k}\sum_{i=1}^{n_j}y_{ij}\). The total sum of squares, between-group sum of squares, and within-group sum of squares are, respectively,

SS_T=\sum_{j=1}^{k}\sum_{i=1}^{n_j}(y_{ij}-\bar{y}_{\cdot\cdot})^2,\qquad SS_B=\sum_{j=1}^{k}n_j(\bar{y}_{j\cdot}-\bar{y}_{\cdot\cdot})^2,\qquad SS_W=\sum_{j=1}^{k}\sum_{i=1}^{n_j}(y_{ij}-\bar{y}_{j\cdot})^2.

Here \(SS_T\) measures total squared deviation from the grand mean, \(SS_B\) measures squared deviation of group means from the grand mean with each group mean weighted by its size, and \(SS_W\) measures squared deviation of observations from their own group means.

The Decomposition and Why It Holds

For an observation in group \(j\), insert and subtract that group’s mean. Its deviation from the grand mean is the sum of a within-group deviation and a between-group deviation:

y_{ij}-\bar{y}_{\cdot\cdot} =(y_{ij}-\bar{y}_{j\cdot})+(\bar{y}_{j\cdot}-\bar{y}_{\cdot\cdot}).

Squaring and summing gives a within-group squared term, a between-group squared term, and a cross-product. The cross-product is zero because deviations from a group’s own mean sum to zero within every group.

\begin{aligned} SS_T &=\sum_{j=1}^{k}\sum_{i=1}^{n_j} \left[(y_{ij}-\bar{y}_{j\cdot})+(\bar{y}_{j\cdot}-\bar{y}_{\cdot\cdot})\right]^2\\ &=\sum_{j=1}^{k}\sum_{i=1}^{n_j}(y_{ij}-\bar{y}_{j\cdot})^2 +\sum_{j=1}^{k}\sum_{i=1}^{n_j}(\bar{y}_{j\cdot}-\bar{y}_{\cdot\cdot})^2\\ &\quad+2\sum_{j=1}^{k}(\bar{y}_{j\cdot}-\bar{y}_{\cdot\cdot}) \sum_{i=1}^{n_j}(y_{ij}-\bar{y}_{j\cdot}). \end{aligned}

For each group \(j\), the inner sum in the last line is \(\sum_{i=1}^{n_j}(y_{ij}-\bar{y}_{j\cdot})=\sum_i y_{ij}-n_j\bar{y}_{j\cdot}=0\), by the definition of the group mean. Thus the entire cross-product vanishes. In the second term, the same squared group-mean deviation appears \(n_j\) times, once for each observation in group \(j\). Therefore the term equals \(SS_B\), while the first term equals \(SS_W\).

Theorem: For any finite collection of numerical observations partitioned into groups, using the observation-weighted grand mean and the usual group means,

$$ SS_T=SS_B+SS_W. $$

The equality is exact, apart from possible rounding in displayed intermediate calculations.

The weighting in \(SS_B\) matters: a group mean represents \(n_j\) observations, so its squared distance from the grand mean contributes \(n_j\) times. The grand mean itself is also observation-weighted. When group sizes are unequal, it is generally not the simple average of the group means.

Worked Applications

Worked Example: Verify the Partition From Raw Observations

A hypothetical laboratory records the number of minutes required to complete a calibration task. Team A has observations 2 and 4 minutes; Team B has observations 6 and 8 minutes. Calculate \(SS_T\), \(SS_B\), and \(SS_W\), and verify the decomposition.

Method. Find the two group means and the observation-weighted grand mean, calculate each sum of squares, and compare the total with the two components.

The group means are \(\bar{y}_{A\cdot}=(2+4)/2=3\) and \(\bar{y}_{B\cdot}=(6+8)/2=7\). There are \(N=4\) observations, so the grand mean is \(\bar{y}_{\cdot\cdot}=(2+4+6+8)/4=5\).

The total sum of squares is

$$ SS_T=(2-5)^2+(4-5)^2+(6-5)^2+(8-5)^2 =9+1+1+9=20. $$

For the between-group sum of squares, each group has size 2:

$$ SS_B=2(3-5)^2+2(7-5)^2=2(4)+2(4)=16. $$

Within Team A, the squared deviations are \((2-3)^2=1\) and \((4-3)^2=1\). Within Team B, they are \((6-7)^2=1\) and \((8-7)^2=1\). Hence \(SS_W=1+1+1+1=4\). The check is \(SS_B+SS_W=16+4=20=SS_T\).

Common mistake avoided: using the group means as if they were the four observations would omit the within-group variation. The direct total uses all four deviations from the grand mean.

Worked Example: Unequal Group Sizes and the Weighted Grand Mean

A hypothetical service center compares resolution times, in hours, for two work teams. Team A has observations 4, 5, and 6; Team B has observations 8 and 10. Find all three sums of squares and verify the identity.

Method. Use all five observations to find the grand mean; then use group sizes as weights for the between-group sum.

Team A’s mean is \(\bar{y}_{A\cdot}=(4+5+6)/3=5\). Team B’s mean is \(\bar{y}_{B\cdot}=(8+10)/2=9\). The grand mean is \(\bar{y}_{\cdot\cdot}=(4+5+6+8+10)/5=33/5=6.6\).

The between-group sum of squares is

$$ SS_B=3(5-6.6)^2+2(9-6.6)^2 =3(2.56)+2(5.76)=7.68+11.52=19.20. $$

The within-group sum is

$$ SS_W=[(4-5)^2+(5-5)^2+(6-5)^2]+[(8-9)^2+(10-9)^2] =2+2=4. $$

For a direct check, the total sum is

\begin{aligned} SS_T &=(4-6.6)^2+(5-6.6)^2+(6-6.6)^2+(8-6.6)^2+(10-6.6)^2\\ &=6.76+2.56+0.36+1.96+11.56=23.20. \end{aligned}

The components agree: \(SS_B+SS_W=19.20+4=23.20=SS_T\).

Common mistake avoided: taking the unweighted average of the two group means gives \((5+9)/2=7\), not the grand mean of 6.6. Team A has three observations and Team B has two, so their means must not receive equal weight when forming the grand mean.

Worked Example: Recover Total Variation From Group Summaries

A hypothetical process study has three batches. Batch 1 contains 2 observations with mean 10 and within-group sum of squares 2. Batch 2 contains 3 observations with mean 13 and within-group sum of squares 4. Batch 3 contains 1 observation with mean 16 and within-group sum of squares 0. Find the grand mean and the three sums of squares, and check the partition.

Method. Recover the grand mean from the group sizes and group means, calculate the between-group component, and add the supplied within-group components.

The total size is \(N=2+3+1=6\). The sum of all observations, obtained from the group totals, is \(2(10)+3(13)+1(16)=20+39+16=75\). Thus \(\bar{y}_{\cdot\cdot}=75/6=12.5\).

The between-group sum of squares is

\begin{aligned} SS_B &=2(10-12.5)^2+3(13-12.5)^2+1(16-12.5)^2\\ &=2(6.25)+3(0.25)+1(12.25)\\ &=12.5+0.75+12.25=25.5. \end{aligned}

The within-group sum is \(SS_W=2+4+0=6\). By the decomposition, \(SS_T=SS_B+SS_W=25.5+6=31.5\). The check is consistent: the total squared variation is the between-group component plus the within-group component, with neither component omitted.

Common mistake avoided: the third batch’s within-group sum of zero does not mean that the batch contributes zero to total variation. Its single observation equals its own group mean, but that mean is 3.5 units above the grand mean and contributes \(1(3.5)^2=12.25\) to \(SS_B\).

Common Errors and Interpretation

The decomposition is useful because it distinguishes variation among group averages from variation among observations within the same group. It does not say that either component is zero, nor does it by itself establish that group means differ statistically. It is an exact accounting identity for the observed data.

Omitting the group-size weight. The between-group contribution is \(n_j(\bar{y}_{j\cdot}-\bar{y}_{\cdot\cdot})^2\), not merely \((\bar{y}_{j\cdot}-\bar{y}_{\cdot\cdot})^2\).
Using an unweighted grand mean. The grand mean is calculated from all observations. An unweighted average of group means works only when all group sizes are equal.
Mixing the two centers. Total deviations are measured from the grand mean; within-group deviations are measured from each group’s own mean.
Assuming the identity requires a probability model. The equality follows from centering and algebra, so it holds for any finite grouped data set. A probability model is relevant to later inference, not to the partition.

Key takeaway: Write each observation’s deviation from the grand mean as its deviation from its group mean plus the group mean’s deviation from the grand mean. The cross-product sums to zero within every group, leaving \(SS_T=SS_W+SS_B\).

Check Your Understanding

For each problem, identify the appropriate group means and grand mean before checking the sum-of-squares partition.

Group A has observations 3 and 5, and Group B has observations 7 and 9. Find \(SS_W\), \(SS_B\), and \(SS_T\). (Answer: \(SS_W=4,\ SS_B=16,\ SS_T=20\).)
Two groups have sizes 2 and 4, means 5 and 8, and combined within-group sum of squares 9. Find the grand mean and \(SS_T\). (Answer: grand mean \(7,\ SS_T=21\).)
Group A has observations 1 and 3; Group B has observations 4, 6, and 8. Find \(SS_W\), \(SS_B\), and \(SS_T\). (Answer: \(SS_W=10,\ SS_B=19.2,\ SS_T=29.2\).)
Two groups have sizes 3 and 2 and means 10 and 14. Their combined within-group sum of squares is 8. Find the grand mean, \(SS_B\), and \(SS_T\). (Answer: grand mean \(11.6,\ SS_B=19.2,\ SS_T=27.2\).)
One group has size 1 and mean 0; another has size 3 and mean 10. Find the grand mean and \(SS_B\). Explain why averaging the two group means directly gives the wrong grand mean. (Answer: grand mean \(7.5,\ SS_B=75\); the direct average \(5\) incorrectly gives the groups equal weight despite unequal sizes.)

What You'll Learn