What a Straight-Line Fit Leaves Unexplained

What the Line Summarizes—and What It Does Not

A simple linear regression line gives a compact description of how a response tends to change as a predictor changes. But a fitted value on the line is not necessarily the observed response for any particular case. Individual observations can fall above or below it, and the differences can matter: they show where the line misses and whether those misses look like random scatter or a systematic pattern.

For each case, compare its observed response with the response fitted at that case’s predictor value. The difference is called a residual. Looking at these differences in a residual plot helps separate the broad straight-line trend from case-to-case departures that the line does not represent.

A residual is a vertical difference on a scatterplot: the observed response minus the point on the fitted line at the same predictor value. A positive residual means the observation is above the line; a negative residual means it is below. Its units are the units of the response. A residual of 2 minutes, for example, means the observed time is 2 minutes greater than its fitted value for that case.

The equation \(y_i=\hat{y}_i+e_i\) is a bookkeeping identity, not a claim that the fitted value and residual have separate physical causes. The line gives the fitted part of the response, while the residual records the difference for that observed case. A residual by itself does not say why the case differs from the line.

Comparing Observed and Fitted Responses

Suppose the response is a student’s score on a practice assessment and the predictor is hours spent studying. The fitted value is the score given by the estimated line at a student’s study time. The residual is the student’s actual score minus that fitted score. A residual of \(-3\) points indicates an observed score 3 points below the line’s fitted score; it does not mean the student lost 3 points because of studying.

In a least-squares fit with an intercept, the line is chosen to minimize the sum of squared residuals among straight lines. This does not mean every residual is small, nor that the line explains every difference between cases. It means that, among the candidate lines, this one has the smallest total squared vertical discrepancy for the observed data. The residual plot helps us inspect the pattern of what remains.

How to Read a Residual Plot

A residual plot places predictor values \(x_i\) on the horizontal axis and residuals \(e_i\) on the vertical axis. The horizontal reference line at residual zero represents exact agreement between observed and fitted responses. Points above it have positive residuals; points below it have negative residuals. The vertical scale uses the response variable’s units.

The purpose is not simply to see whether points lie close to zero. Instead, look for how their positions change across the predictor range. A roughly patternless cloud around zero is consistent with the straight line having captured the main systematic trend in the observed data. A curve or other organized shape indicates that the departures are not just scattered without structure: the fitted straight line has missed a feature of the data.

This is a descriptive check of the fit, not a formal test. The zero reference is useful because the sign and size of every residual are read relative to it. Also notice the scale: residuals varying by a few response units tell a different practical story from residuals varying by hundreds, even if both plots have a similar shape.

Variation Among Cases at the Same Predictor Value

The fitted line assigns the same fitted response to every case with the same predictor value. The observed responses need not be identical. A residual plot can make this especially clear when predictor values repeat: cases at one \(x\) value appear vertically above or below the zero line according to their own residuals. The fit summarizes the line’s response at that \(x\), not every individual response.

What Residuals Do—and Do Not—Tell Us

Residual plots matter because a numerical equation can look precise while hiding a poor description of the data’s structure. The plot provides a direct way to ask whether the straight-line summary leaves a recognizable pattern behind. If it does, then the line alone is not an adequate summary of all the observed variation, even if its slope and fitted values have been calculated correctly.

A residual is also not the same as an unobserved model error. The residual is calculated from the sample after estimating the line; it is observable for each case. An error term is a theoretical component in a model, and its value is not directly observed. For the descriptive task here, residuals are the quantities we can calculate and plot.

A patternless residual plot does not prove that the relationship is truly linear or that predictions will be accurate in a new setting. It only indicates that this plot does not show a clear systematic pattern in the observed residuals. Likewise, a visible pattern is evidence about the limitations of the straight-line summary, not proof of why the pattern arose. Context and the quality of the data remain important.

Common Mistake: Treating the Fitted Value as the Actual Response

A frequent error is to report \(\hat{y}_i\) as though it were the observed \(y_i\), or to call a residual “the amount caused by” the predictor. The fitted response is the value on the estimated line. The residual is the signed difference between the observed and fitted responses. Neither statement identifies the cause of a case’s departure from the line.

For instance, if a fitted score is 70 points and an observed score is 66 points, the residual is \(66-70=-4\) points. Correctly, the observed score is 4 points below the fitted score. It would be incorrect to conclude that the predictor caused a four-point loss. The residual is a comparison to the fitted line, not a causal explanation.