Why Plot Residuals Against a Predictor?
A residual-versus-fitted plot can reveal broad problems in a regression fit, but it does not always show which predictor is associated with a pattern. Plotting residuals against an included quantitative predictor gives a more focused view: does the model leave a systematic pattern as that predictor changes, or does the vertical spread of residuals change? The plot is useful, but it needs careful interpretation when the model includes other predictors.
The key distinction is between a predictor’s raw association with the residuals and its contribution to the fitted model after accounting for other predictors. A raw residual plot is a helpful first check. A component-plus-residual plot can give a more adjusted view of one predictor’s functional form.
Model and Residuals
Consider a multiple linear regression model with a quantitative response \(Y_i\) and predictors \(x_{i1},\ldots,x_{ip}\):
Here \(\beta_0,\ldots,\beta_p\) are model parameters and \(\varepsilon_i\) is the unobserved error for observation \(i\). The model says that the conditional mean of \(Y_i\), given the included predictor values, is the stated linear combination. For the usual diagnostic interpretation, assume this conditional mean is correctly specified, the errors have mean zero, observations are independent, and their variance is constant. A residual plot can help assess some aspects of this model, but it cannot establish that every assumption holds. Normality is not required simply to make or interpret these plots.
After fitting the model by least squares, the fitted value and residual for observation \(i\) are:
Before reading such a plot, identify which model produced its residuals. They should come from the full model being assessed, including the other predictors that are intended to affect the fit. Residuals from a model using only \(x_j\) answer a different question from residuals from a model that also includes, for example, age, treatment group, or another quantitative predictor.
Reading the Raw Residual Plot
If the model’s form for \(x_j\) is adequate and its variance is constant, residuals should generally be scattered on both sides of zero without a systematic change in their typical size as \(x_j\) changes. The goal is not to find a perfectly patternless cloud: individual residuals vary, and small data sets can look irregular by chance. Look for a sustained, interpretable pattern rather than reacting to one or two points.
- Curvature: residuals tend to be positive over one region of \(x_j\) and negative over another, or display a repeated bend. This can suggest that the modeled contribution of \(x_j\) is too simple.
- Changing spread: the vertical scatter becomes noticeably wider or narrower as \(x_j\) changes. This can suggest that the conditional variability is not constant over the predictor’s range.
- No clear pattern: the residuals remain roughly centered around zero with similar spread across the observed range. This is compatible with the model’s form for this diagnostic, but does not prove the model is correct.
There is an important least-squares constraint. When the model includes \(x_j\) as a linear predictor and an intercept, its residuals have zero sample linear association with that predictor:
This constraint does not guarantee a patternless plot. It rules out an overall straight-line trend in the residuals against that included predictor, but a curved pattern can still be present. Likewise, the residuals can have larger spread at some values of \(x_j\) than at others. The absence of a straight-line trend is therefore not sufficient evidence that the predictor’s contribution is well modeled.
Worked Example: Curvature After Including Another Predictor
Suppose a hypothetical data set has eight observations. A least-squares fit using a quantitative predictor \(x\) and a second predictor \(z\) gives \(\hat{y}=10+2x+3z\). The values below are consistent with that fit; the second predictor is included in every fitted value.
| \(x\) | \(z\) | \(y\) | \(\hat{y}=10+2x+3z\) | \(e=y-\hat{y}\) |
|---|---|---|---|---|
| -2 | 0 | 7.5 | 6.0 | 1.5 |
| -2 | 1 | 10.5 | 9.0 | 1.5 |
| -1 | 0 | 6.5 | 8.0 | -1.5 |
| -1 | 1 | 9.5 | 11.0 | -1.5 |
| 1 | 0 | 10.5 | 12.0 | -1.5 |
| 1 | 1 | 13.5 | 15.0 | -1.5 |
| 2 | 0 | 15.5 | 14.0 | 1.5 |
| 2 | 1 | 18.5 | 17.0 | 1.5 |
For the first observation, \(\hat{y}=10+2(-2)+3(0)=6\), so \(e=7.5-6=1.5\). For the observation with \(x=-1,z=1,y=9.5\), \(\hat{y}=10+2(-1)+3(1)=11\), so \(e=9.5-11=-1.5\). The remaining residuals follow by the same calculation.
Against \(x\), the residuals are positive at \(x=-2\) and \(x=2\), and negative at \(x=-1\) and \(x=1\). This repeated U-shaped pattern is evidence of curvature left by the fitted model. The second predictor \(z\) was already included; it does not explain this pattern in these data. In fact, the residuals at each \(x\) value are the same for both values of \(z\). The plot would prompt investigation of whether a curved contribution of \(x\) is more appropriate, not an automatic conclusion that a particular revised model must be used.
Accounting for Other Predictors
A raw residual plot against \(x_j\) is a plot of the full-model residuals, but it does not compare observations that are alike on every other predictor. If the other predictors vary systematically with \(x_j\), the plotted pattern can partly reflect their distribution. This makes it risky to interpret every visible pattern as evidence about \(x_j\) alone.
For a model with additive terms and no interactions, a useful adjusted display is the component-plus-residual plot. For predictor \(x_j\), calculate the component-plus-residual value:
The other fitted predictor contributions cancel algebraically:
Thus the vertical value is the observed response after subtracting the fitted intercept and the fitted contributions of the other predictors. Plotting it against \(x_j\) gives a view of \(x_j\)’s contribution with those other fitted terms removed. If that contribution is well described by a straight line, the plot should show an approximately linear pattern, though noise can obscure it. Curvature can suggest that a linear contribution for \(x_j\) is inadequate.
This plot is an aid, not a guarantee of isolation. It relies on the fitted model’s structure, and patterns can be hard to separate when predictors are strongly related or the data provide little overlap in their values. If the model includes interactions, a predictor’s contribution depends on the values of other predictors; the simple component-plus-residual formula above does not capture that situation.
Worked Example: Comparing Raw and Adjusted Values
In a hypothetical fitted model, \(\hat{y}=5+2x+1.5z\), where \(x\) is the quantitative predictor of interest and \(z\) is another included predictor. Consider two observations with \(x=1\): one has \(z=0,y=9\), and the other has \(z=2,y=12\).
For the first observation, \(\hat{y}=5+2(1)+1.5(0)=7\), so \(e=9-7=2\). Its component-plus-residual value for \(x\) is \(e+\hat{\beta}_x x=2+2(1)=4\).
For the second observation, \(\hat{y}=5+2(1)+1.5(2)=10\), so \(e=12-10=2\). Its component-plus-residual value is also \(e+\hat{\beta}_x x=2+2(1)=4\). Equivalently, subtracting the fitted terms other than \(x\) gives \(9-5-1.5(0)=4\) and \(12-5-1.5(2)=4\).
The raw responses differ by 3 units, but the difference is accounted for by the model’s fitted contribution of \(z\). The adjusted values agree at the same \(x\). This illustrates why a plot of raw response against one predictor can be misleading when other predictors affect the fit. It does not prove the model is correct; it shows how the component-plus-residual calculation separates the fitted contribution of \(x\) from the other fitted terms.
Worked Example: Investigating Changing Spread
Suppose a hypothetical analysis uses 48 observations and fits a model with continuous predictors \(x\) and \(z\). The analyst plots the full-model residuals against \(x\). In a record with \(x=0.5\), \(z=4\), and \(y=22.4\), the fitted model is \(\hat{y}=12+1.8x+0.7z\). The fitted value and residual for this record are:
That individual residual is large and positive, but it is not enough by itself to establish a change in spread. Across the full plot, suppose the residuals remain roughly centered around zero, while their vertical scatter grows as \(x\) increases. For example, residual standard deviations in three successive \(x\)-ranges might be 0.8, 1.7, and 3.1 response units.
The increasing spread is a reason to investigate whether the constant-variance assumption is plausible over the observed range of \(x\). It is not proof that variance changes: the ranges may contain different numbers or types of observations, and visual variation can occur by chance. Because \(z\) also affects the fit, the analyst should check whether its distribution changes across the \(x\)-ranges and inspect an adjusted display where appropriate. The plot does not justify treating the large residual of 6.7 as evidence of changing spread on its own.