Total Sum of Squares

Why Measure Overall Variation?

When a response is recorded for several individuals, the mean gives one central value, but it does not tell us whether the observations lie close to that value or are widely spread. The total sum of squares is a way to summarize this overall variation: it adds the squared distance of every observed response from the grand mean.

In one-way ANOVA, the observations are arranged in groups, but the total sum of squares is calculated across all the responses together. It describes their overall spread around the grand mean, without yet separating variation associated with differences among group means from variation among observations within groups.

Definition: Suppose there are $k$ groups, with response $x_{ij}$ for observation $j$ in group $i$. Let $n_i$ be the number of observations in group $i$, and let $N=\sum_{i=1}^{k}n_i$ be the total number of observations. The grand mean is the mean of all $N$ responses. The total sum of squares is $$ SS_{\text{Total}}=\sum_{i=1}^{k}\sum_{j=1}^{n_i}(x_{ij}-\bar{x}_{\text{grand}})^2. $$

The grand mean is the weighted mean of the group means: each group mean contributes in proportion to the number of observations in that group. Equivalently, it is the sum of all individual responses divided by $N$. This matters whenever the groups have different sizes; averaging the group means equally does not generally give the grand mean.

Each deviation $x_{ij}-\bar{x}_{\text{grand}}$ can be positive or negative. Squaring makes every contribution nonnegative, so observations above and below the grand mean both contribute to the total. A response exactly equal to the grand mean contributes zero.

Calculating From the Deviations

The direct calculation follows the definition: find the grand mean, subtract it from every observation, square each deviation, and add the squared deviations. The groups help organize the observations, but every response is compared with the same grand mean.

Worked Example: Direct Calculation With Equal Group Sizes

Suppose two hypothetical groups have response values, in score points, of 4, 6, 8 and 5, 7, 9. Calculate the total sum of squares.

There are $N=6$ observations, and their sum is $4+6+8+5+7+9=39$. Thus the grand mean is $$ \bar{x}_{\text{grand}}=\frac{39}{6}=6.5 \text{ score points}. $$

Subtract 6.5 from each response, square the deviations, and add: $$ SS_{\text{Total}}=(4-6.5)^2+(6-6.5)^2+(8-6.5)^2+(5-6.5)^2+(7-6.5)^2+(9-6.5)^2 $$ $$ =6.25+0.25+2.25+2.25+0.25+6.25=17.5 \text{ score points}^2. $$

The total sum of squares is 17.5 score points squared. It summarizes the squared deviations of all six scores from 6.5; it is not an average deviation and is not itself measured in score points.

A useful check is that the signed deviations from the grand mean sum to zero: $$ \sum_{i=1}^{k}\sum_{j=1}^{n_i}(x_{ij}-\bar{x}_{\text{grand}})=0. $$ This follows because the grand mean is the mean of all the responses. In the example, the deviations are $-2.5,-0.5,1.5,-1.5,0.5,2.5$, which sum to zero. The squared deviations do not cancel, which is why their sum can measure spread.

Unequal Group Sizes

If group sizes differ, calculate the grand mean from the full set of observations, or weight each group mean by its group size. Then use that one grand mean in every squared deviation. Do not give a small group mean the same weight as a large group mean unless the group sizes happen to be equal.

Worked Example: Unequal Groups and a Weighted Grand Mean

Consider three hypothetical groups of response values: group A has 2 and 4, group B has 5, 7, and 9, and group C has 8. Calculate the total sum of squares.

There are $N=2+3+1=6$ responses. Their total is $2+4+5+7+9+8=35$, so $$ \bar{x}_{\text{grand}}=\frac{35}{6}=5.8333\ldots . $$

For comparison, the group means are 3, 7, and 8. Their unweighted average is $(3+7+8)/3=6$, which is not the grand mean: the groups have different sizes. Weighting each group mean by its size gives $$ \bar{x}_{\text{grand}}=\frac{2(3)+3(7)+1(8)}{6}=\frac{35}{6}. $$

Now square each observation’s deviation from $35/6$: $$ SS_{\text{Total}}=\left(2-\frac{35}{6}\right)^2+\left(4-\frac{35}{6}\right)^2+\left(5-\frac{35}{6}\right)^2+\left(7-\frac{35}{6}\right)^2+\left(9-\frac{35}{6}\right)^2+\left(8-\frac{35}{6}\right)^2 $$ $$ =\frac{529+121+25+49+361+169}{36} =\frac{1254}{36} =34.8333\ldots . $$

The total sum of squares is approximately 34.83 in squared response units. Every response contributes once, including the two observations in group A and the single observation in group C.

A Computational Shortcut

The definition is usually the clearest way to understand what is being measured. For arithmetic, an equivalent formula can be quicker, especially when there are many observations. The shortcut uses the sum of the squared responses and the grand total.

Equivalent formula: If the $N$ observed responses are $x_1,\ldots,x_N$, and $G=\sum_{\ell=1}^{N}x_\ell$ is their grand total, then $$ SS_{\text{Total}}=\sum_{\ell=1}^{N}x_\ell^2-\frac{G^2}{N} =\sum_{\ell=1}^{N}x_\ell^2-N\bar{x}_{\text{grand}}^2. $$

This identity follows by expanding the definition and using the fact that the sum of all responses is $N\bar{x}_{\text{grand}}$:

\begin{aligned} SS_{\text{Total}} &=\sum_{\ell=1}^{N}(x_\ell-\bar{x}_{\text{grand}})^2\\ &=\sum_{\ell=1}^{N}x_\ell^2-2\bar{x}_{\text{grand}}\sum_{\ell=1}^{N}x_\ell +\sum_{\ell=1}^{N}\bar{x}_{\text{grand}}^2\\ &=\sum_{\ell=1}^{N}x_\ell^2 -2\bar{x}_{\text{grand}}(N\bar{x}_{\text{grand}}) +N\bar{x}_{\text{grand}}^2\\ &=\sum_{\ell=1}^{N}x_\ell^2-N\bar{x}_{\text{grand}}^2\\ &=\sum_{\ell=1}^{N}x_\ell^2-\frac{G^2}{N}. \end{aligned}

The second line has $N$ copies of $\bar{x}_{\text{grand}}^2$, because the same grand mean is used for each of the $N$ observations. The last equality uses $\bar{x}_{\text{grand}}=G/N$. The shortcut and the direct definition are mathematically identical, though rounding intermediate values can make their reported answers differ slightly.

Worked Example: Using the Computational Formula

A hypothetical laboratory records five measurements of a response concentration, in milligrams per litre: 12, 15, 15, 18, and 20. Find the total sum of squares using the computational formula.

The grand total is $G=12+15+15+18+20=80$, and $N=5$. The sum of squared responses is $$ \sum_{\ell=1}^{5}x_\ell^2=12^2+15^2+15^2+18^2+20^2 =144+225+225+324+400=1318. $$

Therefore, $$ SS_{\text{Total}}=1318-\frac{80^2}{5} =1318-\frac{6400}{5} =1318-1280 =38 \text{ (mg/L)}^2. $$

As a check, the grand mean is $80/5=16$ mg/L, so direct calculation gives $$ (12-16)^2+(15-16)^2+(15-16)^2+(18-16)^2+(20-16)^2 =16+1+1+4+16=38 \text{ (mg/L)}^2. $$ Both methods give the same total sum of squares.

Interpreting the Total Sum of Squares

The total sum of squares is a measure of overall variation around the grand mean. A value of zero means every observed response equals the grand mean, and therefore all the responses are identical. As the responses lie farther from the grand mean, their squared deviations contribute more, so the total sum of squares increases.

The units are the square of the response units. If a response is measured in milligrams per litre, the total sum of squares is in $(\text{mg/L})^2$. It is not the total of the original responses, and it is not in the original response units. It is also a sum rather than an average, so its size can depend on how many observations are included.

Changing the measurement scale changes the numerical value of the total sum of squares. If every response is increased by the same constant, its deviation from the grand mean stays unchanged, so the total sum of squares stays the same. If every response is multiplied by a constant $a$, each deviation is multiplied by $a$, and each squared deviation by $a^2$. Thus the total sum of squares is multiplied by $a^2$. These properties are useful when checking whether a result is sensible.

The calculation is descriptive: it uses the observed responses and their grand mean. It does not by itself require a probability model or establish that group means differ. In one-way ANOVA, this overall variation is important because the analysis examines how it relates to the grouping structure. The total sum of squares alone does not answer the ANOVA question.

Common Mistakes

Using the wrong center. The definition uses the grand mean of all observations. With unequal group sizes, an unweighted average of group means may not be that grand mean.
Adding signed deviations. Deviations above and below the mean cancel when summed. The definition squares each deviation before adding.
Forgetting to square. Adding absolute distances or adding the deviations themselves does not calculate the total sum of squares.
Calling it a variance or reporting the wrong units. The total sum of squares is a sum in squared response units, not an average in the original units.
Using deviations from a different center. Each response must be compared with the grand mean for this quantity. Changing the center changes the calculation.

Key takeaway: Calculate the grand mean from all observations, subtract it from each response, square each deviation, and add. The result is the overall sum of squared deviations from the grand mean, expressed in squared response units.

Check Your Understanding

Try these questions without looking back.

Four hypothetical responses are 3, 5, 7, and 9. Calculate their grand mean and total sum of squares.
Two groups have means 4 and 10, with 2 and 6 observations, respectively. What is the grand mean in terms of the group means and group sizes?
Why can the unweighted average of group means fail to equal the grand mean?
If every response is increased by 5 units, what happens to the total sum of squares? Explain using deviations from the grand mean.
A response is measured in centimetres. What are the units of its total sum of squares?

What You'll Learn