Why Make a Plot for Each Predictor?
A multiple regression model can include several predictors, each with its own scale and role. Looking only at residuals versus fitted values can make it difficult to see whether a pattern is concentrated in one predictor. A set of residual-versus-predictor plots makes the diagnostic task more organized: each panel places the same residuals against one predictor, so you can look for systematic structure associated with that predictor.
These panels are useful, but they do not isolate the effect of one predictor while holding all the others fixed. That distinction matters especially when predictors are correlated. A plot can suggest where to investigate; it does not by itself establish why a pattern occurs or prove that a model assumption is violated.
The Model and What the Plots Can Assess
For \(n\) observations and \(p\) predictors, consider the multiple linear regression model
Here \(Y_i\) is the response for observation \(i\), \(x_{ij}\) is its value of predictor \(j\), and \(\varepsilon_i\) is the unobserved error. The fitted value is \(\hat{y}_i\), and the residual is \(e_i=y_i-\hat{y}_i\). The model describes a mean response that is additive and linear in the included predictors.
For the usual regression interpretation, observations should be independent, the conditional mean should be represented adequately by the specified model, and errors should have mean zero and constant variance at each predictor combination. The predictor columns must not be exact linear combinations of one another for the coefficients to be uniquely estimated. Normality is relevant to some inference procedures, but is not required just to draw or inspect these plots. A residual plot can give evidence about patterns in the fitted model; it cannot directly display the unobserved errors.
A panel can show whether residuals appear to vary systematically over a predictor’s observed values. For a continuous predictor, look for curvature, changes in vertical spread, or unusual isolated points. For a categorical predictor, compare the residual distributions across categories, including their centers and spreads. A visible pattern is a reason for closer examination, not an automatic diagnosis. A cloud with no obvious pattern is also limited evidence: the sample may be small, or the predictor may cover too narrow a range to reveal a departure.
A Useful Least-Squares Constraint
There is a special caution for the linear trend in each panel. With an intercept and ordinary least squares, the fitted residuals have zero sample linear association with every included predictor. This is a property of how the coefficients were fitted, not evidence that every aspect of the model is correct.
This constraint does not mean that all residual plots must look like featureless horizontal bands. The zero cross-product is a global linear condition; a nonlinear pattern can still be present. Nor does it mean a predictor-specific panel shows the predictor’s isolated contribution. The plot simply pairs each residual with that observation’s value of one predictor, without adjusting the horizontal coordinate for the other predictors.
Organizing a Set of Panels
When a model has many predictors, consistency makes the display easier to read. Put one predictor in each panel, label it with its name and units, and use the same residual scale and reference line at zero throughout. Use an appropriate horizontal scale for each predictor; forcing variables with very different units onto a shared horizontal scale can obscure their observed ranges. A smooth guide may help reveal broad structure, but it should not replace inspection of the points or be treated as proof.
Group related panels in a predictable order, such as continuous predictors followed by categorical predictors, or by the order in which they enter the model description. Make the category labels legible and show the observed range of continuous predictors. With a large number of predictors, prioritize clear labels and readable panels over squeezing every plot into a tiny grid. Keep track of which patterns recur and which appear in only one panel, while remembering that correlated predictors may produce related patterns.
Worked Example: Reading a Small Set of Predictor Panels
Suppose a hypothetical training program models trainees’ final assessment scores using the number of practice sessions \(x_1\) and an indicator \(x_2\) for the training method (0 for Method A and 1 for Method B). A fitted model is \(\hat{y}=10+2x_1+3x_2\). The following hypothetical observations show how the residuals are calculated and organized.
| Trainee | Sessions \(x_1\) | Method \(x_2\) | Score \(y\) | Fitted score \(\hat{y}\) | Residual \(e=y-\hat{y}\) |
|---|---|---|---|---|---|
| 1 | 1 | 0 | 13 | 12 | 1 |
| 2 | 2 | 0 | 12 | 14 | -2 |
| 3 | 3 | 0 | 17 | 16 | 1 |
| 4 | 4 | 1 | 20 | 21 | -1 |
| 5 | 5 | 1 | 25 | 23 | 2 |
| 6 | 6 | 1 | 24 | 25 | -1 |
For trainee 5, the fitted score is \(10+2(5)+3(1)=23\), so the residual is \(25-23=2\) score points. The other rows are calculated in the same way. In the sessions panel, plot the pairs \((1,1),(2,-2),(3,1),(4,-1),(5,2),(6,-1)\). In the method panel, plot residuals \((1,-2,1)\) for Method A and \((-1,2,-1)\) for Method B.
These six residuals alternate around zero, with no clear sustained curve in the sessions panel. Each method’s residual mean is also zero: for Method A, \((1-2+1)/3=0\); for Method B, \((-1+2-1)/3=0\). In this example, that group-mean result is consistent with the least-squares fit including an indicator for method. It does not establish that the model is suitable in general: there are only three observations per method, and the plots cannot show whether patterns would appear in a larger sample.
Interpret Correlated Predictors Carefully
When two predictors tend to vary together, a residual-versus-one-predictor panel can reflect the way both predictors occur in the data. The horizontal coordinate is one predictor’s observed value, but the residual was calculated from a model that includes all the predictors. Therefore, a curve in that panel is not automatically evidence that the plotted predictor alone needs a curved term. The other predictor’s pattern, the combinations of predictor values represented in the sample, or a more complex feature of the model may also matter.
A useful reading habit is to check the predictor-predictor relationships and the observed combinations before assigning a cause to a residual pattern. Ask whether the apparent pattern is supported across the range or driven by a few observations, and whether it appears in panels for related predictors. State what is visible first; keep explanations tentative unless additional analysis supports them.
Worked Example: A Curved Pattern With Strongly Related Predictors
In a hypothetical model of daily energy use, two predictors are outdoor temperature and cooling-system operating time. The sample correlation between these predictors is \(r=0.94\), and a residual plot against temperature shows a U-shaped pattern. For a simple summary, the mean residuals in four consecutive temperature bins are \(1.0,-1.0,-1.0,\) and \(1.0\) kilowatt-hours.
The average of these four bin means is \((1.0-1.0-1.0+1.0)/4=0\) kilowatt-hours. The means are lower in the middle bins and higher at both ends, which describes the U-shape rather than a steady straight-line trend. The pattern merits attention, but the value \(r=0.94\) warns that temperatures and operating times are closely related in these data.
The plot alone does not tell us whether the apparent curvature is attributable to temperature, operating time, or their combined representation in the model. The cautious conclusion is that the fitted model leaves a curved residual pattern across the observed temperature bins, in a setting where the two predictors are strongly related. The panel is a diagnostic signal to investigate, not a stand-alone determination of which predictor or model component is responsible.
Limited Ranges and Uneven Coverage
A plot only describes the predictor values actually observed. If a predictor spans a narrow range, a curved relationship may look nearly straight within that range; values outside it cannot be assessed from the plot. Even within the range, sparse regions and gaps matter. A smooth guide drawn through a few points should not be mistaken for strong evidence about the relationship.
In multiple regression, coverage of combinations matters as well as coverage of each predictor separately. A predictor may span a broad range overall, but only a narrow range of it may occur at particular values of another correlated predictor. A panel that collapses all those combinations onto one horizontal axis does not show that limitation directly. Check the data coverage before making broad claims from the panel.
Worked Example: A Predictor Observed Over a Narrow Range
A hypothetical clinic model relates follow-up recovery score to patient age and four other predictors. The intended application concerns adults aged 20 to 80, but the observations used to fit the model have ages from 42 to 58 years. The age residual panel shows points scattered around zero without obvious curvature over those 16 years.
The observed age span is \(58-42=16\) years. The plot supports only a limited statement: within the observed ages of 42 to 58, this panel does not show an obvious residual pattern. It cannot establish that the model behaves adequately from age 20 to 42 or from age 58 to 80, because there are no observations in those ranges. The other predictor panels do not extend the evidence about age beyond its observed range.
Common Mistake: Treating Each Panel as an Isolated Effect
Incorrect reasoning: “The residual plot against predictor \(x_j\) curves, so \(x_j\) must be the cause and should be modeled as a curve.” This conclusion treats a marginal display as if it held the other predictors fixed. That is not what the panel does.
Correction: Describe the observed pattern and its range, then consider how the other predictors relate to the plotted predictor and to one another. A predictor-specific residual plot is one part of model checking, not a causal attribution or a complete test of the model. Likewise, a flat-looking panel is not proof that the model is adequate, particularly with a small sample, correlated predictors, or limited coverage.
The practical value of a many-predictor display is organization: it helps locate patterns, compare scales and categories, and make gaps in coverage visible. Its value depends on reporting what the plots support and not claiming more than they show.
Check Your Understanding
Use the definitions and cautions in this tutorial to answer the following questions.
- In a residual-versus-predictor panel, what goes on the horizontal axis, and what does the vertical axis measure?
- Why does least squares constrain a straight-line residual trend against an included predictor, yet still allow a curved residual pattern?
- A residual panel curves against one predictor that is strongly correlated with another included predictor. What can you conclude from the panel, and what remains uncertain?
- A predictor’s observed values cover only a small portion of the range where a model will be used. What does a featureless residual plot establish, and what does it not establish?
- Why should residual scales and predictor labels be made consistent and clear across a set of many panels?