Why Plot Residuals Against Fitted Values?
A regression line summarizes the modeled response at each predictor value. Its fitted values are the responses predicted by that line, while residuals record how far the observed responses lie above or below those predictions. Plotting residuals against fitted values makes it easier to see whether the errors look like a roughly patternless band or show a systematic feature that the line has not captured.
This plot is a diagnostic, not a pass-or-fail test. A curve, widening band, or isolated point is a reason to investigate the model and data; the picture alone does not establish the cause. The central reference is the horizontal line at residual zero: points above it have observed responses greater than their fitted values, and points below it have observed responses smaller than their fitted values.
In simple linear regression, the fitted response has the form \(\hat{y}_i=b_0+b_1x_i\), where \(b_0\) and \(b_1\) are the estimated intercept and slope. The residual is measured in the same units as the response. For example, if the response is a claim amount in dollars, the residual is also in dollars. The horizontal axis is not the original predictor; it shows the model's fitted responses.
What a Patternless Plot Looks Like
If a straight-line model is suitable for the mean response and the error spread is reasonably stable, residuals should generally scatter around zero without a clear shape. They will not usually sit exactly on zero, and small runs of positive or negative residuals can occur by chance. The question is whether the overall arrangement suggests a systematic structure that the fitted model has missed.
The least-squares fitting process itself imposes useful balance conditions. In a regression with an intercept, residuals sum to zero, and their products with the fitted values sum to zero. These facts do not guarantee that a plot will look patternless; they explain why the plot is centered overall and why systematic visual shapes deserve attention.
Three Visual Patterns to Recognize
Read the plot by looking first for a broad shape rather than judging one point in isolation. Three useful patterns are curvature, changing vertical spread, and an isolated residual far from the rest. Each raises a different question about the model.
- Curvature: residuals tend to be positive in one fitted-value region and negative in another, or they trace a bend such as a U-shape or upside-down U. This can suggest that a straight line does not adequately describe the mean response across the observed range.
- Changing spread: the vertical scatter becomes wider or narrower as fitted values change. This can suggest that the error variance is not constant. The plot raises a concern; it does not by itself prove why the spread changes.
- An isolated residual: one point lies much farther above or below zero than the others. It marks an observation whose response differs substantially from its fitted response. It may be valid, may reflect a recording issue, or may have another explanation that requires investigation.
Do not confuse these readings. A broad curve is a feature shared by several observations, whereas an isolated residual is a single conspicuous vertical departure. Likewise, a widening band concerns the amount of scatter at different fitted values, not merely whether the residuals are positive or negative.
Worked Example: A Curved Pattern Despite a Balanced Fit
A hypothetical analyst relates a device's measured output \(y\) to an operating setting \(x\). The five observations are \((-2,10),(-1,9),(0,10),(1,13),(2,18)\). Fit a least-squares straight line and inspect the residual-versus-fitted pattern. The method is to calculate the least-squares coefficients, then compute fitted values and residuals observation by observation.
The predictor mean is \(\bar{x}=(-2-1+0+1+2)/5=0\), and the response mean is \(\bar{y}=(10+9+10+13+18)/5=60/5=12\). The sum of squared predictor deviations is \(S_{xx}=(-2)^2+(-1)^2+0^2+1^2+2^2=10\). The cross-product sum is \(S_{xy}=(-2)(10)+(-1)(9)+0(10)+1(13)+2(18)=-20-9+0+13+36=20\).
Thus the slope is \(b_1=S_{xy}/S_{xx}=20/10=2\), and the intercept is \(b_0=\bar{y}-b_1\bar{x}=12-2(0)=12\). The fitted line is \(\hat{y}=12+2x\). The complete calculations are:
| \(x\) | \(y\) | \(\hat{y}=12+2x\) | \(e=y-\hat{y}\) |
|---|---|---|---|
| -2 | 10 | 8 | 2 |
| -1 | 9 | 10 | -1 |
| 0 | 10 | 12 | -2 |
| 1 | 13 | 14 | -1 |
| 2 | 18 | 16 | 2 |
The plotted pairs are \((8,2),(10,-1),(12,-2),(14,-1),(16,2)\). Residuals are positive at both ends and negative through the middle, forming a U-shaped pattern. The line is balanced in the least-squares sense: the residuals sum to \(2-1-2-1+2=0\), and their products with \(x\) sum to \((-2)(2)+(-1)(-1)+0(-2)+1(-1)+2(2)=0\). Still, the plot suggests that the response's mean pattern may bend rather than follow a straight line.
Common mistake to avoid: Seeing residuals sum to zero and concluding that the model fits adequately. Zero sum is a least-squares balance condition, not evidence against curvature. The plotted sequence of signs reveals the pattern that the sum alone hides.
Worked Example: A Wider Residual Band at Higher Fitted Values
A hypothetical service team models completion time \(y\), in minutes, using workload level \(x\). The observations are \((1,11),(2,15),(3,14),(4,20),(5,23),(6,19)\). Calculate the fitted line and residuals, then decide what the plot suggests. The method is to use the least-squares summary quantities and plot each fitted-residual pair.
Here \(\bar{x}=(1+2+3+4+5+6)/6=3.5\), and \(\bar{y}=(11+15+14+20+23+19)/6=102/6=17\). The sum of squared predictor deviations is \(S_{xx}=(-2.5)^2+(-1.5)^2+(-0.5)^2+(0.5)^2+(1.5)^2+(2.5)^2=17.5\). Also, \(\sum xy=11+30+42+80+115+114=392\), so \(S_{xy}=\sum xy-6\bar{x}\bar{y}=392-6(3.5)(17)=392-357=35\).
The slope is \(b_1=35/17.5=2\), and the intercept is \(b_0=17-2(3.5)=10\). Therefore \(\hat{y}=10+2x\). The fitted values and residuals are:
| \(x\) | \(y\) | \(\hat{y}=10+2x\) | \(e=y-\hat{y}\) |
|---|---|---|---|
| 1 | 11 | 12 | -1 |
| 2 | 15 | 14 | 1 |
| 3 | 14 | 16 | -2 |
| 4 | 20 | 18 | 2 |
| 5 | 23 | 20 | 3 |
| 6 | 19 | 22 | -3 |
The plotted pairs are \((12,-1),(14,1),(16,-2),(18,2),(20,3),(22,-3)\). The residual magnitudes are \(1,1,2,2,3,3\) as fitted values increase. This is a widening band and a possible warning that the error spread changes with the level of the fitted response. With only six observations, the plot is limited evidence; it does not establish a general variance pattern.
Check: The residuals sum to \(-1+1-2+2+3-3=0\). Their products with \(x\) sum to \(-1+2-6+8+15-18=0\), consistent with the computed least-squares line. Common mistake to avoid: Calling every run of positive or negative residuals changing variance. Here the relevant visual feature is the increasing size of the vertical departures, not their signs alone.
Worked Example: One Isolated Residual
A hypothetical analyst predicts the output of five test units from a centered setting \(x\). The observations are \((-2,13),(-1,16),(0,24),(1,22),(2,25)\). Fit the line and determine whether the residual plot shows a broad pattern or one unusually distant response. The method is again least squares followed by a direct residual check.
The predictor mean is \(\bar{x}=0\), and the response mean is \(\bar{y}=(13+16+24+22+25)/5=100/5=20\). The predictor sum of squares is \(S_{xx}=4+1+0+1+4=10\). The cross-product sum is \(S_{xy}=(-2)(13)+(-1)(16)+0(24)+1(22)+2(25)=-26-16+0+22+50=30\).
Thus \(b_1=30/10=3\) and \(b_0=20-3(0)=20\), giving \(\hat{y}=20+3x\). The fitted values for \(x=-2,-1,0,1,2\) are \(14,17,20,23,26\). Subtracting each fitted value from its observed response gives residuals \(13-14=-1\), \(16-17=-1\), \(24-20=4\), \(22-23=-1\), and \(25-26=-1\).
The residual plot contains four points at \(-1\) and one at \(4\). The point \((20,4)\) is isolated vertically relative to the others. It deserves a check of the original record and context, but the plot alone cannot determine whether the observation is erroneous or explain why its response differs from the fitted value.
Check: The residuals sum to \(-1-1+4-1-1=0\), and their products with \(x\) sum to \(2+1+0-1-2=0\). Common mistake to avoid: Automatically deleting the isolated point. First verify the data and investigate the context; a valid unusual observation should not be removed just because it looks inconvenient.
How to Make a Careful Interpretation
A useful reading proceeds in a consistent order. First check that the horizontal axis really is fitted response and that the vertical axis is observed minus fitted. Then locate the zero line and describe the broad pattern in context. Avoid diagnosing from one visual feature without considering how many observations there are and how much random scatter is present.
Confirm that residuals are \(y_i-\hat{y}_i\), with zero marked. A reversed residual definition reverses the vertical signs but not the basic presence of a pattern.
Describe a bend, a widening or narrowing band, or a point far from the rest. State what is visible before proposing an explanation.
For example, say that curvature may indicate an inadequate straight-line mean pattern, or that changing spread may warrant checking the variance assumption. Do not claim the plot proves a cause.
Check the observations and study context. A visual warning can guide further assessment, but it is not by itself a reason to discard a point or select a new model.
The distinction between an unusual residual and an influential observation is especially important. A large vertical departure describes how far an observed response is from its fitted value. It does not, by itself, tell how much the fitted line would change if that observation were omitted. That is a separate question requiring other diagnostic ideas.
Similarly, this plot is not a complete check of every regression assumption. It is useful for spotting mean-pattern and spread concerns and for finding isolated vertical departures. It does not establish that residuals have a normal distribution, nor does it replace checking the data and study design. A visual pattern is a prompt for reasoned follow-up, not a verdict.
Check Your Understanding
For each question, calculate or interpret the residual information given before deciding what the plot suggests.
- A fitted model gives values \(15,20,25,30\), and the observed responses are \(16,19,26,29\). Find the four residuals and state whether these values alone show a clear systematic shape. (Answer: Residuals \(1,-1,1,-1\); they alternate around zero, with no clear curve or changing spread evident from these four points.)
- A residual-versus-fitted plot has residuals \(-3,1,3,1,-3\) at fitted values \(10,20,30,40,50\), respectively. Identify the visual pattern and its cautious implication. (Answer: An upside-down U-shaped pattern; it may indicate that a straight-line mean pattern is inadequate.)
- For fitted values \(8,12,16,20,24,28\), residuals are \(-1,1,-2,2,4,-4\). What feature should be noted, and what should not be concluded from this small plot? (Answer: The vertical spread appears larger at higher fitted values, suggesting a possible changing-variance concern; six points do not prove the cause or establish a general pattern.)
- A fitted line is \(\hat{y}=5+4x\). For \(x=1,2,3,4,5,6\), observed responses are \(8,14,15,23,28,26\). Calculate fitted values and residuals, then describe the main visual suggestion. (Answer: Fitted values \(9,13,17,21,25,29\); residuals \(-1,1,-2,2,3,-3\); the residual magnitudes increase toward higher fitted values, suggesting a widening band.)
- A residual plot has points near zero throughout, except one point far above zero. Give two sensible next steps and one conclusion that is not justified by the plot alone. (Answer: Check the observation against the source record and investigate its context; the plot alone does not justify deleting it or concluding that it is an error.)