Why Plot Residuals Against a Predictor?
A single error metric can summarize predictive performance, but it cannot show where along a predictor’s range errors tend to occur. A model might have a small overall average error because positive and negative residuals cancel, even while it consistently overpredicts for one range of a continuous predictor and underpredicts for another. A residual plot helps reveal that structure.
For example, consider predictions of a numeric outcome using a person’s age, a property’s size, or a vehicle’s annual mileage. Plotting the residual for each observation against that predictor lets us ask whether the model’s errors appear randomly scattered around zero, or whether they change systematically as the predictor increases.
With this convention, a positive residual means the observed outcome exceeded the prediction: the model underpredicted. A negative residual means the prediction exceeded the observed outcome: the model overpredicted. The sign convention matters; other definitions are possible, but the interpretation must match the one actually used.
What to Look for in the Plot
A useful first check is whether residuals are scattered on both sides of zero without an obvious shape as \(x\) changes. This is not a demand for every point to be close to zero, nor does it prove that a model is correct. It is a check for visible systematic structure that may be hidden by a single summary such as RMSE.
- A rising or falling pattern: residuals tend to become more positive or more negative as \(x\) increases. This may indicate that predictions miss a trend associated with \(x\).
- Curvature: residuals tend to be positive at one end and negative in the middle, or the reverse. A straight-line relationship may not capture a curved relationship between the outcome and \(x\).
- A range of systematic overprediction or underprediction: residuals in a particular range tend to fall on one side of zero. The sign identifies the direction of the error.
- No clear pattern: points appear on both sides of zero without a consistent trend or shape. This is reassuring about this particular diagnostic, but does not establish that the model will perform well in every respect.
In practice, plot a horizontal zero line and consider a smooth summary of the residuals across \(x\), as well as the individual points. A smooth that stays above or below zero for a substantial range can make a pattern easier to see; it should not replace examination of the data. Sparse observations at an endpoint, for example, make the apparent shape there less certain.
For predictions intended for new cases, examine residuals on data not used to fit the model when such data are available. Training data can still be useful for diagnosing a fitted relationship, but a pattern seen only in training residuals may not carry over to new observations. The plot describes an association between residuals and the predictor; by itself, it does not identify the cause of that association.
Worked Examples
Worked Example: Detecting a Rising Residual Pattern
A hypothetical insurer evaluates a model predicting an amount paid per claim using annual vehicle mileage, measured in thousands of miles. On a validation sample, five claims have the following observed and predicted amounts, in dollars. Calculate the residuals and interpret the pattern.
| Mileage (thousands) | Observed \(y_i\) | Predicted \(\hat{y}_i\) |
|---|---|---|
| 2 | 410 | 430 |
| 4 | 430 | 440 |
| 6 | 470 | 450 |
| 8 | 510 | 460 |
| 10 | 550 | 470 |
Method. Subtract the prediction from the observed amount for each claim. The calculations are \(410-430=-20\), \(430-440=-10\), \(470-450=20\), \(510-460=50\), and \(550-470=80\) dollars. Thus, the plotted points have residuals \(-20,-10,20,50,80\) at increasing mileage values.
The residuals rise as mileage increases: the first two are negative, while the last three are positive and larger. As a simple check of the contrast, the average residual for the first two observations is \((-20-10)/2=-15\) dollars. For the last two it is \((50+80)/2=65\) dollars. This is a clear increasing pattern in these five observations. The model overpredicts at the lower mileage values and underpredicts at the higher values.
Common mistake avoided. Calling the positive residuals overpredictions reverses the convention. Since \(e_i=y_i-\hat{y}_i\), a positive value means the observed amount is higher than the prediction. This small sample flags a pattern to investigate; it does not establish its cause or how it will behave in another sample.
Worked Example: Recognizing Curvature Despite a Small Mean Residual
A hypothetical analyst predicts repair costs from a continuous equipment-age measure, where \(x\) is age in years. The following five validation observations have costs and predictions in dollars. Determine whether their residuals show a pattern.
| Age \(x\) (years) | Observed \(y_i\) | Predicted \(\hat{y}_i\) | Residual \(y_i-\hat{y}_i\) |
|---|---|---|---|
| 1 | 112 | 100 | 12 |
| 2 | 106 | 110 | -4 |
| 3 | 112 | 120 | -8 |
| 4 | 126 | 130 | -4 |
| 5 | 152 | 140 | 12 |
Method. The row-by-row subtractions are \(112-100=12\), \(106-110=-4\), \(112-120=-8\), \(126-130=-4\), and \(152-140=12\). The residuals are positive at both ends and negative at the three middle ages, giving a U-shaped pattern.
The endpoint average is \((12+12)/2=12\) dollars. The middle average is \((-4-8-4)/3=-16/3\), or approximately \(-5.33\) dollars. Across all observations, the residual sum is \(12-4-8-4+12=8\) dollars, so the overall mean is \(8/5=1.6\) dollars. That small overall mean does not remove the visible curvature: the model underpredicts at the endpoints and overpredicts in the middle.
Common mistake avoided. Concluding that there is no meaningful pattern because the overall mean residual is close to zero ignores cancellation. Positive errors at the endpoints offset negative errors in the middle. The residual plot retains the information that the average discards.
Worked Example: Locating an Overprediction Range
A hypothetical planning team predicts a numeric service cost from customer age. For nine validation records, predicted costs are shown in dollars. Calculate the residuals and use local averages to describe where predictions tend to be high or low.
| Age \(x\) | Observed \(y_i\) | Predicted \(\hat{y}_i\) | Residual \(e_i\) |
|---|---|---|---|
| 20 | 306 | 300 | 6 |
| 25 | 314 | 310 | 4 |
| 30 | 321 | 320 | 1 |
| 35 | 327 | 330 | -3 |
| 40 | 335 | 340 | -5 |
| 45 | 346 | 350 | -4 |
| 50 | 358 | 360 | -2 |
| 55 | 375 | 370 | 5 |
| 60 | 388 | 380 | 8 |
Method. For each row, subtract predicted cost from observed cost; for instance, \(306-300=6\) and \(327-330=-3\). The full residual sequence is \(6,4,1,-3,-5,-4,-2,5,8\). To summarize the pattern without relying on a single point, compare pre-specified ranges: ages 20–30, 35–50, and 55–60.
The average residual for ages 20–30 is \((6+4+1)/3=11/3\), or approximately \(3.67\) dollars. For ages 35–50 it is \((-3-5-4-2)/4=-14/4=-3.50\) dollars. For ages 55–60 it is \((5+8)/2=6.50\) dollars. The model tends to underpredict in the lower and upper ranges and overpredict in the middle range. The sequence suggests curvature rather than a single steadily rising or falling trend.
Common mistake avoided. A negative residual does not mean the actual cost was negative; it means only that the prediction was higher than the observed cost. Also, the local averages describe these validation observations, not a proven causal effect of age. With only a few records in each range, the pattern should be treated as a diagnostic signal, not a precise estimate of performance for all customers.
Why a Pattern Is Informative
The basic justification follows directly from the definition of residual. Suppose a model produces a prediction function \(\hat{f}(x)\), so that \(\hat{y}_i=\hat{f}(x_i)\). At a given predictor value \(x\), the expected residual is
The equality holds because, once \(X=x\) is specified, \(\hat{f}(x)\) is a fixed number. Therefore, if the prediction matches the conditional mean \(E[Y\mid X=x]\) throughout the predictor’s range, the expected residual at each such \(x\) is zero. A sustained positive or negative pattern suggests that the predictions differ from the conditional mean in that range. A curved residual pattern suggests that this difference itself changes with \(x\).
This is a diagnostic relationship, not a guarantee that any observed pattern is real or will persist. Residuals vary from observation to observation, and a finite validation sample may show an accidental shape. Use the plot to identify a question about the model, then assess whether the pattern is consistent with the modeling objective and data. Do not infer a cause from the plot alone.
A Practical Reading Process
Identify the observed outcome, prediction, continuous predictor, and units. Calculate residuals as observed minus predicted so the sign has a clear interpretation.
Put \(x_i\) on the horizontal axis and \(e_i\) on the vertical axis. Add a zero reference line and keep the predictor’s scale visible.
State whether residuals rise, fall, curve, or tend to stay on one side of zero in a range. Translate positive values to underprediction and negative values to overprediction.
Look at the points, sample density, and any local summaries. Do not treat a few endpoint observations or a smooth line as conclusive evidence.
In R, after calculating a residual column using the same convention, a basic diagnostic can be made with a call such as plot(validation$x, validation$residual). Label both axes with the variable names and units, and add a horizontal line at zero. The plot should use the validation observations whose predictions are being assessed; changing the data set without changing the interpretation can lead to misleading conclusions.
Common Interpretation Traps
- Reversing the sign. With \(e_i=y_i-\hat{y}_i\), positive means underprediction and negative means overprediction.
- Relying only on an overall average. Positive and negative residuals can cancel, concealing a trend or curved pattern.
- Calling every wiggle a model defect. A small sample can produce irregular patterns by chance. Describe what is visible and qualify the strength of the evidence.
- Confusing association with explanation. A residual pattern against a predictor points to a possible model mismatch; it does not establish why the mismatch occurs.
- Ignoring scale and units. A residual of 5 dollars and a residual of 5 thousand dollars have different meanings. State the units before judging practical importance.
Check Your Understanding
For each problem, use residual \(e_i=y_i-\hat{y}_i\) and interpret positive residuals as underprediction.
- A model predicts an outcome of 80 when the observed value is 74 at \(x=3\), and predicts 90 when the observed value is 96 at \(x=9\). Calculate both residuals and state the direction of prediction error at each value of \(x\). (Answer: \(-6\) at \(x=3\), so overprediction; \(+6\) at \(x=9\), so underprediction.)
- At continuous predictor values \(x=1,2,3,4\), residuals are \(8,2,-2,-8\). Describe the pattern and the prediction direction at the low and high ends. (Answer: A decreasing pattern; underprediction at the low end and overprediction at the high end.)
- At \(x=1,2,3,4,5\), the residuals are \(9,-3,-6,-3,9\). Calculate the average residual for the endpoints and for the three middle observations. Interpret the result. (Answer: Endpoint average \(9\); middle average \(-4\); underprediction at the endpoints and overprediction in the middle, consistent with a U-shaped pattern.)
- For \(x=10,20,30,40,50,60\), residuals are \(4,2,-2,-4,3,5\). Calculate the average for \(x=10,20\), for \(x=30,40\), and for \(x=50,60\). What pattern do these local summaries suggest? (Answer: \(3,-3,4\), respectively; underprediction in the low and high ranges and overprediction in the middle range, suggesting curvature.)
- A validation set has residuals that are positive at low predictor values and negative at high values, but the overall mean residual is zero. Explain why the mean does not rule out a systematic pattern and state the prediction direction at each end. (Answer: Positive and negative residuals can cancel in the overall mean; the model underpredicts at low values and overpredicts at high values.)