Reading Residuals Versus Fitted Values

Why Plot Residuals Against Fitted Values?

A regression equation gives a fitted response for each observation, but a good-looking equation or a high correlation alone does not show whether the model describes the data adequately. A residual plot makes the model’s errors visible. In particular, plotting residuals against fitted values helps reveal whether errors appear to follow an unstructured pattern around zero or instead show systematic features.

The horizontal axis is the fitted value, the response predicted by the regression equation for an observed case. The vertical axis is the residual, the observed response minus that fitted value. Because fitted values are in the response’s units, both axes use the response scale, even though they represent different quantities.

Definition: For observation \(i\), let \(y_i\) be its observed response and \(\hat{y}_i\) its fitted response from a regression model. Its residual is \(e_i=y_i-\hat{y}_i\). A residual-versus-fitted plot places the point \((\hat{y}_i,e_i)\) for every observation, usually with a horizontal reference line at \(e_i=0\).

A positive residual means the observed response is above the model’s prediction; a negative residual means it is below. The distance from zero indicates the size of the prediction error, measured in response units. The plot is not a second regression equation. It is a diagnostic display for examining the errors left after fitting the model.

What Pattern Is Expected?

If a linear model captures the systematic relationship between predictor and response, and the error variability is reasonably similar across the fitted range, the residual-versus-fitted plot should look like a roughly horizontal band centered at zero. “Random” here means that the points show no clear, systematic shape; it does not mean that every point is exactly on zero or that positive and negative residuals alternate perfectly.

The band need not be narrow for the model’s structure to be reasonable. Its width reflects how far observed responses tend to fall from their fitted values, while its shape is the main focus of this plot. A broad but patternless band and a narrow curved band raise different questions: the first may reflect substantial prediction error, while the second suggests that a straight-line form may be missing structure.

Residual plots are diagnostic evidence, not a mechanical pass-or-fail test. A visible pattern can suggest a problem, but the plot alone usually does not identify its cause. Likewise, a plot without an obvious pattern cannot prove that every modeling assumption is satisfied. In particular, this display uses fitted values rather than observation order, so it is not designed to assess whether errors depend on the order in which data were collected.

Worked Example: Constructing a Residual-versus-Fitted Plot

Suppose five hypothetical observations have predictor values \(x=1,2,3,4,5\) and responses \(y=11,15,16,19,19\). A least-squares line with an intercept is \(\hat{y}=10+2x\). Calculate each fitted value and residual, then describe the plotted points.

For each case, substitute its predictor value into the line and subtract the fitted response from the observed response:

\begin{aligned} x=1 &: \quad \hat{y}=10+2(1)=12,\quad e=11-12=-1,\\ x=2 &: \quad \hat{y}=10+2(2)=14,\quad e=15-14=1,\\ x=3 &: \quad \hat{y}=10+2(3)=16,\quad e=16-16=0,\\ x=4 &: \quad \hat{y}=10+2(4)=18,\quad e=19-18=1,\\ x=5 &: \quad \hat{y}=10+2(5)=20,\quad e=19-20=-1. \end{aligned}

The plotted coordinates are \((12,-1),(14,1),(16,0),(18,1),(20,-1)\). The horizontal reference line is at zero. These points fall both above and below it, with no clear trend in this small example. The residuals are in the response’s units; if \(y\) were measured in minutes, then each residual would also be in minutes.

The axis choice matters. A point at \((14,1)\) means that the model’s fitted response is 14 response units and the observed response is 1 unit higher than that prediction. It does not mean that the predictor equals 14.

Why There Is No Overall Linear Trend

For ordinary least squares with an intercept, the residuals have special algebraic properties. These properties explain why a residual-versus-fitted plot should not be judged by asking whether a straight-line trend remains. The least-squares fit already makes the residuals have zero overall sum and zero overall linear association with the fitted values. That fact is guaranteed by the fitting calculation; it is not evidence by itself that the model is adequate.

To see why, let the fitted line be \(\hat{y}_i=b_0+b_1x_i\), where \(b_0\) and \(b_1\) are chosen to minimize the sum of squared residuals. Write that sum as \(S=\sum_i(y_i-b_0-b_1x_i)^2=\sum_i e_i^2\). At the least-squares minimum, its derivatives with respect to \(b_0\) and \(b_1\) are zero:

\frac{\partial S}{\partial b_0}=-2\sum_i e_i=0 \quad\Longrightarrow\quad \sum_i e_i=0,

\frac{\partial S}{\partial b_1}=-2\sum_i x_i e_i=0 \quad\Longrightarrow\quad \sum_i x_i e_i=0.

Since every fitted value is \(b_0+b_1x_i\), these two equalities also give:

\sum_i \hat{y}_i e_i =\sum_i(b_0+b_1x_i)e_i =b_0\sum_i e_i+b_1\sum_i x_i e_i =0.

Thus the residuals have zero sum and zero cross-product with the fitted values. With an intercept, the residuals have mean zero and no overall linear trend against the fitted values. However, they can still form a curve, change their spread, or show clusters. Those patterns are nonlinear or otherwise structured features, and the least-squares identities do not eliminate them.

Reading Patterns in the Plot

Start by locating the zero line, then scan from lower to higher fitted values. Ask whether the points stay near the line on average, whether their vertical spread looks broadly comparable, and whether the cloud has a recognizable shape. A curved or wavelike pattern can indicate that the fitted linear relationship does not capture the response’s systematic behavior. A spread that grows or shrinks across the horizontal axis can indicate that the error variability is not constant. Clusters may suggest that different groups or conditions behave differently. These are clues to investigate, not conclusions about a cause.

An isolated point far above or below the band has a relatively large residual: its observed response differs substantially from its fitted response. That is a reason to check the observation and its context. A large residual alone does not establish why the case differs, and the residual-versus-fitted plot does not tell whether that case has unusual predictor values.

Worked Example: Recognizing a Curved Pattern

Consider hypothetical data with \(x=1,2,3,4,5\), responses \(y=10,10,12,16,22\), and fitted line \(\hat{y}=5+3x\). Calculate the fitted values and residuals, then assess the residual pattern.

The fitted values are \(8,11,14,17,20\). Subtracting each fitted value from its observed response gives:

\begin{aligned} e_1&=10-8=2, & e_2&=10-11=-1,\\ e_3&=12-14=-2, & e_4&=16-17=-1,\\ e_5&=22-20=2. \end{aligned}

The plotted points are \((8,2),(11,-1),(14,-2),(17,-1),(20,2)\). Residuals are positive at both ends and negative in the middle, producing a U-shaped pattern rather than an unstructured horizontal band. This suggests that a straight-line fit may be missing systematic curvature. The plot flags a possible inadequacy; it does not, by itself, determine what alternative model should be used.

Worked Example: Noticing a Change in Vertical Spread

In another hypothetical data set, suppose \(x=1,2,3,4,5,6\), the fitted line is \(\hat{y}=5+2x\), and the observed responses are \(6,10,10,14,17,15\). Calculate the residuals and interpret their spread across fitted values.

The fitted values are \(7,9,11,13,15,17\). The residual calculation for all six cases is:

\begin{aligned} e_1&=6-7=-1, & e_2&=10-9=1,\\ e_3&=10-11=-1, & e_4&=14-13=1,\\ e_5&=17-15=2, & e_6&=15-17=-2. \end{aligned}

The first four points have residuals of \(1\) unit in magnitude, while the last two have residuals of \(2\) units in magnitude. In a plot, the later points would sit farther from zero than the earlier ones. This small example gives a hint of increasing vertical spread, which is worth checking in a larger data set. It would be too strong to claim that this pattern proves unequal variability; a few points can vary by chance, and the display is a diagnostic rather than a formal verdict.

A Practical Reading Sequence

A consistent reading sequence prevents the most common interpretation errors. First confirm that the vertical axis contains residuals and the horizontal axis contains fitted responses. Then use the zero line to determine which observations are overpredicted or underpredicted. Finally, inspect the overall shape and spread rather than focusing only on whether individual points are positive or negative.

Identify the axes.
Check that each point represents a fitted response paired with its observed-minus-fitted residual.

Read the zero line.
Residuals above zero are positive; residuals below zero are negative. Their vertical distances are response-scale errors.

Scan for structure.
Look for curvature, changing spread, clusters, or isolated large residuals. Describe what is visible before suggesting what it might mean.

Why the Plot Matters—and a Common Mistake

A regression line summarizes a relationship with a compact equation, but the residuals show what that summary leaves behind. A residual-versus-fitted plot can reveal that predictions are systematically too high in one part of the response range and too low in another, or that prediction errors vary noticeably in size across the range. Finding such a pattern matters because predictions and model-based conclusions may be misleading when the model’s structure does not match the data well.

A common mistake is to treat “the residuals sum to zero” or “there is no straight-line trend” as proof that the fit is good. For least squares with an intercept, those features follow from how the line is calculated. A U-shaped pattern can still have residuals summing to zero and no overall linear trend. The correction is to inspect the full cloud for structure, not just its average or linear association.

Another mistake is to call the horizontal axis the predictor. In this display it is the fitted response. State what the axes actually show, describe the pattern in context, and present any model concern as a possibility supported by the plot—not as a proven explanation.

Key takeaway: An adequate linear fit is generally consistent with residuals scattered in a patternless, roughly horizontal band around zero. Curvature, changing spread, clusters, or large individual residuals can flag concerns, but the plot is a diagnostic clue rather than proof.

Check Your Understanding

Use the definitions and diagnostic ideas in this tutorial to answer these questions.

An observation has observed response 42 units and fitted response 39 units. What is its residual, and where would its point lie relative to the zero line?
In a residual-versus-fitted plot, what does a point with coordinates \((25,-3)\) mean in context of the axes?
Why does the absence of an overall straight-line trend in least-squares residuals not establish that the model is adequate?
A plot shows residuals mostly below zero in the middle of the fitted range and mostly above zero at both ends. What pattern is present, and what might it suggest?
What is one thing a residual-versus-fitted plot is not designed to assess?