Total Variation in the Response Variable

Total Variation Starts With a Baseline

In “Residuals Mixed Practice Set,” residuals described how far individual observed responses were from the values predicted by a fitted line. Before asking how well a regression line predicts, it helps to describe how much the response values vary in the first place. A natural reference point is the response mean, \(\bar{y}\).

For each observation, \(y-\bar{y}\) is its deviation from the mean. Some deviations are positive and some are negative; their signed sum is zero. To measure the overall size of these deviations without allowing positive and negative values to cancel, square them and add them. This sum is called the total variation in the response, or the total sum of squares.

Here \(y_i\) is the response for observation \(i\), \(\bar{y}\) is the mean of all \(n\) responses, and \(SST\) is a single nonnegative number. It is zero only when every response equals the mean. Because deviations are squared, \(SST\) is measured in squared response units: for example, minutes squared or score-points squared.

The mean provides a useful baseline because it is the constant prediction that minimizes the sum of squared prediction errors when no predictor is used. Predicting \(\bar{y}\) for every case gives prediction errors \(y_i-\bar{y}\). Their squared sum is exactly \(SST\). A regression line uses \(x\) to make different predictions for different cases; it aims to account for some of the response’s departures from this mean baseline. The next tutorial develops what it means for a regression to account for variability.

Calculating the Total Sum of Squares

To calculate \(SST\), find the response mean, subtract it from each response, square each deviation, and add the squares. Keep the deviations’ signs during subtraction, even though squaring makes each final contribution nonnegative.

This calculation is different from finding the range or adding absolute deviations. The range uses only the largest and smallest values, while \(SST\) uses every observation. Squaring also makes larger deviations contribute disproportionately more. A response far from the mean can therefore contribute much more to \(SST\) than a response only a little above or below the mean.

How \(SST\) Relates to the Response Standard Deviation

The total sum of squares is connected to the sample standard deviation \(s_y\) of the response. The sample variance \(s_y^2\) is the sum of squared deviations divided by \(n-1\). Thus, if \(s_y\) is known, \(SST\) can be recovered by multiplying \(s_y^2\) by \(n-1\).

The denominator is \(n-1\) for the sample variance; it is not part of the definition of \(SST\). The standard deviation \(s_y\) is in the original response units, whereas \(SST\) is in squared response units. For the practice-time data, \(n=5\) and \(SST=38\text{ minutes}^2\), so \(s_y=\sqrt{38/4}=\sqrt{9.5}\approx3.08\) minutes. Reversing the calculation gives \((5-1)(3.08)^2\approx38\text{ minutes}^2\), allowing for rounding.

This relationship offers a convenient check on a calculation, but the two quantities answer different questions. \(s_y\) gives a summary of the typical distance of responses from their mean in response units. \(SST\) is the total of all squared distances and depends on the number of observations as well as their spread.

Total Variation Depends on the Response Units

Changing the response’s measurement units changes the numerical size of its deviations from the mean, and therefore changes \(SST\). The underlying observations have not become more or less spread out in a practical sense; only their numerical scale has changed. Since \(SST\) squares deviations, a change in the size of the unit has a squared effect on its numerical value.

How the Baseline Connects to a Regression Line

The mean-only prediction is a useful reference for regression. In a least-squares regression line with an intercept, the fitted predictions vary with \(x\), while their residuals measure the differences between observed and predicted responses. As established in “The Idea of the Least-Squares Criterion,” the least-squares line is chosen to minimize the sum of squared residuals among candidate lines.

A horizontal line at \(\bar{y}\) is one possible line with an intercept. Its squared errors add to \(SST\). Because the least-squares regression line chooses a line with the smallest sum of squared residuals, its SSE cannot be greater than the mean-only line’s \(SST\). This comparison gives a baseline for judging the fitted line. It does not, by itself, tell us what proportion of variability the line accounts for; that interpretation belongs in “What Variability Explained Means.”

Common Mistakes and AP Exam Tips

• Subtracting the wrong center. \(SST\) uses deviations from \(\bar{y}\), the response mean, not deviations from a particular predicted value or from zero.
• Adding signed deviations instead of squared deviations. The signed deviations from the mean sum to zero. For total variation, square each deviation before adding.
• Calling \(SST\) the variance or standard deviation. \(SST\) is a sum of squares. The sample variance is \(SST/(n-1)\), and the sample standard deviation is its square root.
• Reporting original response units. If \(y\) is measured in minutes, \(SST\) is in minutes squared. The sum is not a prediction or a typical number of minutes.
• Reversing the effect of a unit conversion. A larger measurement unit gives smaller numerical values and deviations. If the unit is twice as large, deviations are halved and \(SST\) is one quarter as large—not four times as large.
• Calling the baseline comparison “variability explained.” \(SST\) measures total variation around the mean. Comparing it with SSE sets up a later interpretation, but the comparison alone is not a complete explanation of the proportion or meaning of variability explained.

For full-credit communication, identify the response mean as the reference point, show the squared deviations or a clear equivalent calculation, and report \(SST\) in squared response units. If comparing the result with a regression line, state what each sum measures: \(SST\) is variation around the mean, while SSE is the sum of squared residuals around the fitted line.