Visual Fit and Least Squares
A scatterplot can suggest a line that follows the overall pattern of the points. But a line drawn by eye is a visual estimate: different people might sketch slightly different lines, and the graph’s scale can affect what looks like a good fit. The least-squares regression line has a more precise definition. As you learned in “The Idea of the Least-Squares Criterion,” it is the line that minimizes the sum of squared residuals, or SSE, for the observed data.
In this tutorial, we will compare a hand-drawn candidate line with a least-squares line by calculating both lines’ SSE values. The key is to use the same observations and the same calculation for each line. A visual impression can help us notice a pattern, but the SSE comparison measures the candidates according to the squared-residual criterion.
The least-squares line has the smallest SSE among all possible lines for those observations. A hand-drawn line is one candidate among them. If its SSE is greater than the least-squares line’s SSE, that calculation shows how far its squared-error fit falls short for the data. If the two SSE values are equal, the lines tie by this criterion; a visual judgment alone does not establish which line is the least-squares line.
A Consistent Comparison Method
For each observed \(x\), substitute that value into a candidate line to get \(\hat{y}\). Then calculate the residual \(y-\hat{y}\), square it, and add the squared residuals. Repeat the same steps for the other line. Keeping the signed residual in the table before squaring makes it easier to check the arithmetic.
Identify which equation is the hand-drawn candidate and which is the least-squares regression line.
Use the same \(x\)-values for both lines; do not compare predictions at different locations.
For every case, calculate \(y-\hat{y}\), then calculate \((y-\hat{y})^2\).
The smaller SSE indicates the smaller total squared residuals for these observations. The least-squares line is the one that minimizes SSE over all lines.
An equation makes a visual comparison calculable. Do not estimate a line’s SSE just by looking at the scatterplot, and do not compare one line’s SSE with another line’s sum of absolute residuals. As emphasized in “Why Squared Residuals Not Absolute Values,” those are different criteria.
Worked Example: A Line That Looks Plausible but Has Larger SSE
Suppose an invented hydroponic setup records an input setting \(x\) and a plant-growth response \(y\), in centimeters. The four observed points are \((1,2)\), \((2,3)\), \((3,5)\), and \((4,6)\). A student sketches the candidate line \(\hat{y}=1+1.2x\). For comparison, the least-squares line for these data is \(\hat{y}=0.5+1.4x\). We will evaluate both equations at every observed input setting.
Worked Example: Calculate SSE for Two Candidate Lines
For the hand-drawn line \(\hat{y}=1+1.2x\), the predictions at \(x=1,2,3,4\) are \(2.2,3.4,4.6,5.8\). Subtract each prediction from its observed response to get the residuals \(y-\hat{y}\):
| \(x\) | Observed \(y\) | Hand-drawn prediction | Residual | Squared residual |
|---|---|---|---|---|
| 1 | 2 | 2.2 | \(-0.2\) | 0.04 |
| 2 | 3 | 3.4 | \(-0.4\) | 0.16 |
| 3 | 5 | 4.6 | 0.4 | 0.16 |
| 4 | 6 | 5.8 | 0.2 | 0.04 |
The hand-drawn line’s SSE is:
For the least-squares line \(\hat{y}=0.5+1.4x\), the predictions are \(1.9,3.3,4.7,6.1\). The residuals are \(0.1,-0.3,0.3,-0.1\), so:
For the same four observations, the least-squares line has the smaller SSE: \(0.20\) compared with \(0.40\) for the hand-drawn line. The candidate line may look plausible, but it has twice the total squared residuals in this example. Because \(y\) is measured in centimeters, both SSE values are in square centimeters.
Worked Example: Compare Lines for a Negative Association
The comparison works the same way whether the trend rises or falls. Imagine a fictional set of observations relating a practice duration \(x\), in hours, to a task-completion time \(y\), in minutes. The data are \((1,8)\), \((2,6)\), \((3,7)\), and \((4,3)\). A student’s hand-drawn line is \(\hat{y}=10-1.5x\); the least-squares line for these points is \(\hat{y}=9.5-1.4x\).
Worked Example: Check Each Residual Before Adding
For the hand-drawn line, the predictions are \(8.5,7,5.5,4\). The residuals are observed minus predicted: \(-0.5,-1,1.5,-1\). Thus:
For the least-squares line, the predictions are \(8.1,6.7,5.3,3.9\). The residuals are \(-0.1,-0.7,1.7,-0.9\). Its SSE is:
The least-squares line again has the smaller SSE, although the difference between \(4.50\) and \(4.20\) is not large. Both candidate lines slope downward, but having a similar direction or appearing close on a graph does not make their SSE values identical. The calculation supports a specific comparison: for these observations, one line has a smaller total squared residual than the other.
Worked Example: A Visually Close Line Can Be Close by SSE
A hand-drawn line does not have to be far from the least-squares line. For an invented sensor example, the observed pairs are \((0,1)\), \((1,4)\), \((2,4)\), and \((3,7)\). Suppose a student sketches \(\hat{y}=1+2x\), while the least-squares line for the observations is \(\hat{y}=1.3+1.8x\). The equations are similar, so we should not assume the SSE values are identical; we can calculate them.
Worked Example: Quantify a Small Difference in Fit
The hand-drawn line predicts \(1,3,5,7\). Its residuals are \(0,1,-1,0\), giving:
The least-squares line predicts \(1.3,3.1,4.9,6.7\). Its residuals are \(-0.3,0.9,-0.9,0.3\), so:
The least-squares line’s SSE is \(0.20\) lower for these observations. This example shows why the comparison should be based on calculated totals rather than a claim that one line “looks perfect.” The hand-drawn line is close, but the least-squares line still has the smaller SSE.
What the Comparison Does and Does Not Show
When you calculate SSE for two lines, you establish which of those two has the lower total squared residuals for the stated data. The comparison by itself does not show that the hand-drawn line is the best-fitting line among every possible line. The least-squares line has that minimizing property by definition, as discussed in “The Idea of the Least-Squares Criterion.”
It is also useful to keep the role of a graph in perspective. A scatterplot can help you judge whether a straight-line model seems reasonable and whether any points sit far from a candidate line. But a drawing is not a precise equation, and a graph’s scale can make small differences hard to see. Once both equations are stated, the residual calculation gives a direct comparison under the squared-error criterion.
Pay attention to the equation’s precision. If a line is given with rounded coefficients, predictions and SSE calculated from that rounded equation may differ slightly from calculations using the unrounded coefficients. Use the equations as supplied, keep enough digits during calculations, and round the final SSE consistently. Do not round each prediction too early if that could change the comparison.
Common Mistakes and AP Exam Tips
- Comparing lines by appearance alone. “It looks closer” is not an SSE calculation. Show predictions or residuals and add the squared residuals for both lines.
- Using different observations for the two totals. A fair comparison uses the same paired data and the same observed \(x\)-values for each candidate line.
- Adding residuals instead of squared residuals. Positive and negative residuals can cancel. SSE requires squaring each residual before adding.
- Reversing the residual convention without noticing. Use observed \(y\) minus predicted \(\hat{y}\), the convention used in the earlier tutorials. Squaring removes the sign in the final contribution, but writing the residual correctly helps catch mistakes.
- Claiming a two-line comparison proves a global minimum. If line A has a smaller SSE than line B, say that A beats B by the squared-residual criterion. The broader claim that the least-squares line minimizes SSE over all lines comes from its definition, not from checking just two candidates.
- Mixing up SSE and another error total. Compare SSE with SSE. A smaller sum of absolute residuals does not establish a smaller SSE.
- Ignoring units. If the response is measured in minutes, residuals are in minutes and SSE is in minutes squared. The squared units follow from squaring the residuals.
A strong AP response identifies the two lines, uses the same observed cases to calculate each SSE, and states exactly what the comparison establishes. For example: “For these observations, the least-squares line has an SSE of \(4.20\), which is less than the hand-drawn line’s SSE of \(4.50\). Therefore, it has the smaller total squared residuals.” That is more precise than simply calling one line “better.”
Check Your Understanding
Use observed minus predicted for each residual, and compare lines using SSE.
- A candidate line has residuals \(1,-2,1\). Calculate its SSE.
- For four observations, a hand-drawn line has SSE \(3.6\) and the least-squares line has SSE \(3.1\). What conclusion can you make about these two lines for those observations?
- Why is it not enough to say that a hand-drawn line “looks closer” on a scatterplot?
- If the response is measured in seconds, what are the units of each residual and of SSE?
- If a line has a smaller SSE than one other candidate line, does that comparison alone establish that it minimizes SSE among all lines? Explain.