Reading the Sign of a Residual

Why the Sign Matters

A fitted regression line gives a predicted response for each predictor value. Comparing that fitted value with the response actually observed for the same case tells us the direction of the discrepancy. The sign of the residual gives this direction: it indicates whether the observed point lies above, below, or exactly on the fitted line.

For example, if a model predicts a quiz score of 72 points and a student’s observed score is 76 points, the observed response is greater than the fitted value. The residual is positive. If the observed score is 68 points instead, the residual is negative. This interpretation is about the position of an observation relative to a model’s fitted value; by itself, it does not say whether the model is good overall or why the difference occurred.

Reading the Sign Geometrically

In a scatterplot, the horizontal axis represents the predictor and the vertical axis represents the response. At a case’s predictor value, compare the observed point’s vertical position with the fitted line. A point above the line has a response greater than the fitted value, so its residual is positive. A point below the line has a response smaller than the fitted value, so its residual is negative. A point on the line has a residual of zero.

The residual is a vertical difference, measured in response units. It is not the shortest distance from the point to the line, which could run diagonally. The subtraction order matters: observed minus fitted. Reversing the order would reverse every sign and would no longer follow the residual definition.

This schematic scatterplot shows observations (filled circles, \(\bullet\)) and fitted values (open circles, \(\circ\)) for the line \(\hat{y}=x+2\). The fitted line is the straight line through the open circles. At \(x=1\), the observation is above its fitted value; at \(x=2\), it is below; at \(x=3\), it is below; and at \(x=4\), it is above. Where a filled and open circle occupy the same position, the observation and fitted value would coincide; there is no such case in this particular schematic.

The vertical comparisons in the plot make the sign visible, while calculation gives its numerical size. For instance, at \(x=2\), the fitted value is 4 and the observed response is 3, so the residual is \(3-4=-1\). It is one response unit below the line.

Worked Examples

Why the Sign Interpretation Follows

The geometric rule is a direct consequence of the definition, not a separate assumption about the data. At a fixed predictor value, the fitted line’s vertical coordinate is \(\hat{y}_i\), and the observed point’s vertical coordinate is \(y_i\). Their signed vertical difference is observed coordinate minus fitted coordinate.

Thus, for every case, the algebraic sign and the graphical position agree. The units also follow directly: subtracting two response values measured in the same units leaves a difference in those units. For a response measured in points, residuals are in points; for a response measured in millimeters of mercury, residuals are in mmHg.

There are no probability assumptions needed to determine the sign of a calculated residual. The essential requirements are that the observed response and fitted value refer to the same case, use the same response scale, and be compared using the definition \(y_i-\hat{y}_i\). Assumptions may matter for other regression goals, but they are not needed for this arithmetic and geometric interpretation.

What a Residual’s Sign Does and Does Not Say

The sign tells the direction of the observed response relative to the fitted value. The residual’s magnitude, \(\lvert e_i\rvert\), tells how far apart the two values are vertically. A residual of \(+1\) and a residual of \(-1\) have equal magnitudes but opposite directions. A residual of \(+10\) has a larger vertical discrepancy than one of \(+1\), assuming the same response units and context.

A positive residual does not mean the response is “good,” and a negative residual does not mean it is “bad.” Those judgments depend on the application and cannot be read from the sign alone. Nor does one positive or negative residual show that the fitted line systematically overpredicts or underpredicts across a whole data set. It describes one case’s response relative to its own fitted value.

A residual is calculated from an observed response and its fitted value. It is not itself a probability, a measure of causal effect, or a guarantee of what another case’s response will be. In particular, a point above the line says only that its observed response is greater than the fitted value at that predictor value. The plot and subtraction do not identify the reason for the difference.

Common Mistake: Reversing the Subtraction

A frequent error is to calculate fitted minus observed and then interpret that result using the usual residual rules. For the 76-point score with fitted value 72, that reversed calculation gives \(72-76=-4\), which might incorrectly be described as a negative residual. But the residual is defined as observed minus fitted, so the correct calculation is \(76-72=4\): positive, with the point above the line.

A second, related mistake is to call every positive residual an “overprediction.” The fitted value is below the observed response in that case, so the model has underpredicted that observation. A negative residual means the fitted value is above the observed response, so the model has overpredicted it for that case. A useful check is to say the comparison in words: “observed minus fitted.” If the verbal description and the sign disagree, check the subtraction order.