Sign of a Residual: Over- and Underprediction

What a Residual’s Sign Tells You

In “Calculating a Residual by Hand,” you found a residual by subtracting the predicted response from the observed response. The result is not just a measure of how far a point is from the fitted line: its sign tells you which side of the line the point is on, and whether the line’s prediction was too low or too high for that observation.

For an observed case, \(y\) is the actual response and \(\hat{y}\) is the response predicted by the regression line at that case’s \(x\)-value. The residual is \(y-\hat{y}\). Because the subtraction is observed minus predicted, compare those two response values to determine the sign.

The geometric explanation follows directly from a scatterplot. At a particular predictor value \(x\), the regression line has height \(\hat{y}\), while the data point has height \(y\). The residual \(y-\hat{y}\) is the signed vertical difference between those heights. If the point is higher, \(y\) is greater and the difference is positive. If the point is lower, \(y\) is smaller and the difference is negative.

“Underprediction” means the line’s predicted response is less than the observed response for that case. “Overprediction” means its predicted response is greater than the observed response. These words describe the direction of the prediction error relative to the observation; they do not say whether a larger response is good or bad. For example, underpredicting a wait time means the model predicted fewer minutes than the person actually waited.

Read the Sign at the Same Predictor Value

To compare an observation with a line, use the line’s predicted response at the observation’s own \(x\)-value. The comparison is vertical: the point and the line are considered at the same horizontal position. Comparing a point with the line’s height at some other \(x\)-value does not determine that point’s residual.

The size and sign answer different questions. The sign tells which side of the line the point is on; the absolute value tells the vertical distance from the line in response units. For instance, a residual of \(-6\) and a residual of \(+6\) have the same distance from the line, but they lie on opposite sides and indicate opposite prediction directions.

A point’s predictor value does not determine its residual sign. Points to the left or right of the mean can each be above or below the line. Likewise, a positive slope does not mean that every residual is positive, and a negative slope does not mean that every residual is negative. The slope describes the direction of the fitted line; the residual sign describes the position of an individual observed point relative to that line.

Worked Example: Underpredicting a Trailhead Wait

Notice that the positive residual does not mean the line has a positive slope, even though this example’s slope happens to be positive. The residual’s sign comes from comparing this observation with the line’s prediction, not from the coefficient of \(x\).

Worked Example: Overpredicting Lantern Charge

Here the regression line has a negative slope, but the residual is negative for a separate reason: this particular observed charge is below the fitted value. Another lantern at a different operating time could have a positive residual even when evaluated against this same downward-sloping line.

Worked Example: A Point on the Line

Common Mistakes and AP Exam Tips

A full-credit explanation connects the arithmetic to both the point’s position and the prediction in context. Give the residual sign, say above or below the line, and identify whether the line underpredicted or overpredicted. Use the response variable and its units when describing the difference.

• Reversing the subtraction. The residual is observed minus predicted, \(y-\hat{y}\). Reversing it flips the sign and leads to the opposite conclusion about the point’s position.
• Calling a positive residual an overprediction. If \(y-\hat{y}\) is positive, the observed value is larger than the prediction. The line was too low, so it underpredicted.
• Confusing residual sign with slope sign. The slope describes the fitted line’s direction, not the side of the line on which every point lies. Use the observed and predicted responses for the specific case.
• Comparing at different predictor values. A residual measures the vertical difference between the observation and the line at that observation’s \(x\). Do not compare the point with a prediction made for another \(x\).
• Using “above” or “below” without naming the line. State that the point is above or below the regression line. “The response is high” is less precise because it does not say what it is high relative to.
• Dropping units or giving an unclear prediction statement. A residual has the response variable’s units. Write, for example, “The line underpredicted the wait by 4 minutes,” rather than only “It was off by 4.”
• Treating zero as a positive or negative residual. A residual of zero means the observed response equals its prediction. The point is on the line, with no overprediction or underprediction for that observation.

A useful final check is to ask whether the sign agrees with the comparison: if the observation is greater than the prediction, the residual must be positive; if it is smaller, the residual must be negative. When a residual is already calculated, adding it to the prediction should recover the observed response. These checks help catch a reversed subtraction before you write your interpretation.