From Total Variation to What a Line Accounts For
In “Total Variation in the Response Variable,” you saw that \(SST\) measures the squared departures of observed responses from their mean. That mean is a baseline prediction: it gives the same predicted response, \(\bar{y}\), to every observation. A regression line instead uses \(x\) to make different predictions, \(\hat{y}\). Comparing those predictions with the mean baseline helps describe how much of the response’s variation the line accounts for.
The comparison involves two kinds of departures. The fitted values \(\hat{y}_i\) may vary above and below \(\bar{y}\), reflecting the part of the response’s variation associated with the line. Each observed response may also differ from its fitted value; that difference is the residual \(y_i-\hat{y}_i\), as covered in “Defining a Residual as Observed Minus Predicted.” Those residuals represent variation left around the line.
Both quantities are in squared response units, just like \(SST\). For a least-squares line with an intercept, they divide the total variation:
This identity is a useful way to make the idea precise. The fitted values’ squared departures from \(\bar{y}\) describe variation represented by the line; the squared residuals describe the remaining departures of observed responses from their fitted values. The least-squares properties make the two parts add to \(SST\). Thus, a smaller SSE than the mean baseline’s \(SST\) means the line has reduced the total squared prediction error.
This is a description of how the line fits the observed data, not evidence that changing \(x\) causes changes in \(y\). “Accounts for” refers to the line’s statistical representation of variation. It does not establish a cause-and-effect relationship.
Reading the Comparison
The mean-only baseline has squared prediction errors \(y_i-\bar{y}\), whose sum is \(SST\). The fitted line has squared prediction errors \(y_i-\hat{y}_i\), whose sum is \(SSE\). The amount by which the line reduces the total squared error is \(SST-SSE\), equal to the fitted values’ squared departures from \(\bar{y}\).
The amounts are measured in squared response units, so they are not themselves percentages or typical errors. A later tutorial will define a standardized summary that expresses the accounted-for amount as a fraction of total variation. For now, focus on what each sum measures and how the sums compare.
Worked Example: Compare a Line With the Mean Baseline
A fictional robotics club records four practice hours, \(x\), and corresponding quiz scores, \(y\), in points: \((1,2),(2,6),(3,2),(4,6)\). The least-squares line is \(\hat{y}=2+0.8x\). Compare the total variation with the variation accounted for by the line and the variation left in its residuals.
State. Find \(SST\), \(SSE\), and the squared departures of the fitted values from \(\bar{y}\). Use the comparison to describe how much variation the line accounts for.
Plan. Calculate the response mean and its squared deviations to find \(SST\). Use the given line to find each fitted value and residual, then square and add the residuals for \(SSE\). Finally, compare the fitted values with the mean. All sums will be in squared points.
Do. The response mean is \(\bar{y}=(2+6+2+6)/4=16/4=4\) points. The deviations from the mean are \(-2,2,-2,2\), so
The fitted values from \(\hat{y}=2+0.8x\) are \(2.8,3.6,4.4,5.2\) points. Subtracting each fitted value from its observed response gives residuals \(-0.8,2.4,-2.4,0.8\) points. Therefore,
The fitted values’ deviations from the mean of 4 are \(-1.2,-0.4,0.4,1.2\) points. Their squared sum is \((-1.2)^2+(-0.4)^2+0.4^2+1.2^2=1.44+0.16+0.16+1.44=3.2\text{ points}^2\).
Conclude. The decomposition checks: \(16=3.2+12.8\). The line accounts for \(3.2\text{ points}^2\) of the total \(16\text{ points}^2\) in this squared-deviation comparison, while \(12.8\text{ points}^2\) remains in the residuals. The line reduces the baseline’s squared error by \(16-12.8=3.2\text{ points}^2\). The horizontal line at 4 is the mean-only baseline, not the least-squares line for these data.
Why the Two Parts Add to the Total
For each observation, the departure from the response mean can be split into a fitted-value departure and a residual:
Squaring and adding this equation might seem to produce the decomposition immediately, but in general the square of a sum includes a cross-product term. For a least-squares line with an intercept, the cross-products between fitted-value departures and residuals sum to zero. That least-squares property is why the total squared departures split exactly into the two sums shown above. It is not a general identity for any arbitrary line.
This distinction matters: a line chosen by eye, or another line that is not the least-squares line, may not have the same decomposition. In the earlier tutorials on the least-squares criterion and residuals, you learned that the least-squares line minimizes SSE and that residuals are observed minus predicted. Here those ideas combine to compare the line’s error with the mean-only baseline.
Worked Example: Separate Accounted-for and Remaining Variation
A fictional greenhouse tracks daily light exposure \(x\), in hours, and seedling growth \(y\), in centimeters, for five observations: \((1,2),(2,5),(3,5),(4,8),(5,10)\). Find the total variation, the variation represented by the least-squares line, and the variation left around it.
State. Compare the three squared-deviation amounts for seedling growth.
Plan. Find the response mean and \(SST\). Calculate the least-squares slope using the sum of products of deviations divided by the sum of squared \(x\)-deviations, then find the intercept. Use the resulting fitted values to calculate SSE and their squared departures from the response mean.
Do. The response mean is \(\bar{y}=(2+5+5+8+10)/5=30/5=6\) cm. The response deviations are \(-4,-1,-1,2,4\), giving
The predictor mean is \(\bar{x}=3\) hours. The sum of products of deviations is \((-2)(-4)+(-1)(-1)+(0)(-1)+(1)(2)+(2)(4)=8+1+0+2+8=19\). The sum of squared predictor deviations is \(4+1+0+1+4=10\). Thus \(b=19/10=1.9\) centimeters per hour, and \(a=\bar{y}-b\bar{x}=6-(1.9)(3)=0.3\) centimeters. The fitted line is \(\hat{y}=0.3+1.9x\).
The fitted values are \(2.2,4.1,6.0,7.9,9.8\) cm. The residuals are \(-0.2,0.9,-1.0,0.1,0.2\) cm, so
The fitted-value deviations from \(\bar{y}=6\) are \(-3.8,-1.9,0,1.9,3.8\) cm. Their squared sum is \(14.44+3.61+0+3.61+14.44=36.10\text{ cm}^2\).
Conclude. The amounts add correctly: \(38=36.10+1.90\). The line’s fitted values account for \(36.10\text{ cm}^2\) of the \(38\text{ cm}^2\) total variation, and \(1.90\text{ cm}^2\) remains as squared residual variation. This describes the fit for these observations; it does not show that light exposure alone caused the growth.
When a Line Accounts for Little
A fitted line does not necessarily account for much variation just because it slopes upward or downward. The slope describes the direction and predicted change in \(y\) for a one-unit increase in \(x\); the comparison of \(SST\) with SSE describes how much the line improves on the mean-only baseline overall. A visible trend can coexist with considerable scatter around the line.
Worked Example: A Shallow Trend With Residual Variation
A fictional community group records the number of weekly volunteer visits \(x\) and a satisfaction score \(y\), in points: \((1,5),(2,7),(3,5),(4,7)\). Find the least-squares line and compare the variation it accounts for with the variation left around it.
State. Determine the baseline total variation, the fitted line’s SSE, and its accounted-for variation.
Plan. Find \(\bar{x}\) and \(\bar{y}\), calculate the least-squares slope and intercept, then use fitted values and residuals to obtain the two parts of the variation comparison.
Do. The means are \(\bar{x}=2.5\) visits and \(\bar{y}=6\) points. The \(y\)-deviations are \(-1,1,-1,1\), so \(SST=1+1+1+1=4\text{ points}^2\). The sum of products of deviations is \((-1.5)(-1)+(-0.5)(1)+(0.5)(-1)+(1.5)(1)=1.5-0.5-0.5+1.5=2\). The sum of squared \(x\)-deviations is \(2.25+0.25+0.25+2.25=5\). Therefore, \(b=2/5=0.4\) points per visit and \(a=6-(0.4)(2.5)=5\) points. The line is \(\hat{y}=5+0.4x\).
Its fitted values are \(5.4,5.8,6.2,6.6\) points, and its residuals are \(-0.4,1.2,-1.2,0.4\) points. Thus, \(SSE=0.16+1.44+1.44+0.16=3.20\text{ points}^2\). The fitted values’ departures from 6 are \(-0.6,-0.2,0.2,0.6\), whose squared sum is \(0.36+0.04+0.04+0.36=0.80\text{ points}^2\).
Conclude. The total \(4\text{ points}^2\) divides into \(0.80\text{ points}^2\) represented by the fitted values and \(3.20\text{ points}^2\) left in residuals. Although the line slopes upward, it accounts for relatively little of the response’s total variation compared with what remains around the line.
Common Mistakes and AP Exam Tips
- Calling \(SSE\) the accounted-for variation. SSE is the variation left in residuals. The accounted-for amount is \(\sum(\hat{y}_i-\bar{y})^2\), which equals \(SST-SSE\) for the least-squares line with an intercept.
- Using the wrong baseline. The total variation \(SST\) is measured around \(\bar{y}\), not around zero or an individual fitted value.
- Assuming every line gives the decomposition. The exact sum holds for a least-squares line with an intercept. Do not apply it automatically to an arbitrary line.
- Mixing squared units with response units. If \(y\) is measured in centimeters, \(SST\), SSE, and accounted-for variation are in square centimeters. They are not typical errors in centimeters.
- Claiming causation. Saying a line accounts for variation describes a statistical fit. It does not prove that the predictor causes the response.
- Calling an amount a percentage. A squared-deviation amount has squared units. To report a proportion or percentage, use the specific standardized measure introduced in the next tutorial.
For full-credit communication, identify the mean-only baseline, name which sum describes the fitted values and which describes residuals, and compare them in context with squared response units. When a question asks what the line accounts for, do not merely state that SSE is small; explain that the line’s fitted values vary around \(\bar{y}\) and that residual variation remains.
Check Your Understanding
Answer in context and include squared response units when describing sums of squares.
- In your own words, distinguish the variation represented by fitted values from the variation measured by SSE.
- A data set has \(SST=50\text{ points}^2\) and \(SSE=18\text{ points}^2\) for its least-squares line with an intercept. How much variation is represented by the fitted values?
- Why is the response mean a useful baseline when describing how much variation a regression line accounts for?
- Can a line have a nonzero slope and still account for relatively little variation? Explain.
- Why does a line accounting for variation not by itself prove that the predictor causes the response?