Why Plot Residuals Against a Predictor?
A generalized linear model (GLM) specifies how predictors relate to the conditional mean of a response. Even if a continuous predictor is included, the model may impose the wrong shape for that relationship. For example, a straight-line effect on the link scale may miss curvature. A residual-versus-predictor plot helps reveal such problems by showing whether residuals vary systematically over the observed values of a predictor.
The plot places the predictor \(x_i\) on the horizontal axis and a residual for observation \(i\) on the vertical axis. A reference line at zero makes it easier to see whether residuals are mostly positive or negative in particular regions. The aim is not to prove that a model is correct from a graph. It is to look for structure that the fitted mean appears not to explain.
For a response \(Y_i\) with fitted conditional mean \(\hat{\mu}_i\), the raw residual is \(y_i-\hat{\mu}_i\). For a GLM, a Pearson residual commonly takes the form
where \(V(\mu)\) is the model’s variance function and \(\hat{\phi}\) is the dispersion estimate when one is used. A deviance residual is another possible choice. The earlier material on choosing GLM residuals distinguishes these residual definitions; whichever is used, identify it and interpret it on its own scale. This tutorial focuses on what a plot against a continuous predictor can reveal about the mean structure.
Why Residuals Should Not Trend with a Predictor
The core justification follows from the definition of the conditional mean. Suppose the specified model correctly represents the mean, so that \(E(Y_i\mid x_i)=\mu_i\). Then
Thus, at a given predictor value, residuals should not systematically tend to be positive or negative. Pearson scaling divides by a positive standard deviation, so it does not change this zero-centered expectation. In an actual fitted model, parameters have been estimated from the same data, and residuals are not independent of the fit. The argument is a diagnostic principle, not a guarantee that every finite sample will look perfectly patternless.
A curve in the plot can indicate that the model’s assumed mean shape is too restrictive. A steady increase or decrease can indicate a missing or poorly specified effect. Other explanations are possible too, including an omitted predictor or interaction, an unusual observation, or a data problem. A plot is a signal to investigate, not a verdict about the cause.
Reading the Main Patterns
| Appearance against the predictor | Possible interpretation | Careful response |
|---|---|---|
| Unstructured scatter around zero | No obvious mean trend is visible for this predictor. | Continue other model checks; a plot cannot establish overall adequacy. |
| U-shaped or inverted U-shaped pattern | A straight-line effect on the link scale may be missing curvature. | Consider a justified nonlinear term or another plausible mean specification. |
| Mostly negative at low values and positive at high values, or the reverse | The fitted mean may change too slowly or too quickly across the predictor range. | Check the functional form and whether the predictor’s effect is specified appropriately. |
| One or a few separated residuals | Could reflect unusual observations, sparse regions, or an observation the model fits poorly. | Check the records and context; do not infer a general curve from isolated points. |
A smooth curve drawn through the residuals can help summarize a pattern, but it does not replace the points. Pay attention to where observations actually occur: apparent behavior in a region with very little data is uncertain, and a fitted relationship should not be judged by extrapolating beyond the observed range. If several predictors are involved, inspect the continuous predictors that matter for the model’s substantive question rather than relying on one plot alone.
Worked Example: A Curved Pattern in a Gaussian Model
A hypothetical analyst models a continuous response \(Y\) using a Gaussian GLM with identity link and a straight-line mean, \(\hat{\mu}_i=10+2x_i\). For this illustration, take the conditional variance to be \(\sigma^2=1\). The observed pairs \((x_i,y_i)\) are \((0,12),(1,11),(2,12),(3,15),(4,20)\). Calculate residuals and assess the plot against \(x\).
Method. For an identity-link Gaussian model, the fitted mean is on the response scale, so calculate each raw residual as \(e_i=y_i-\hat{\mu}_i\). With the stated variance, the Pearson residuals equal these raw residuals because \(\sqrt{\sigma^2}=1\).
At \(x=0\), \(\hat{\mu}=10+2(0)=10\), so \(e=12-10=2\). At \(x=1\), \(\hat{\mu}=12\), so \(e=11-12=-1\). At \(x=2\), \(\hat{\mu}=14\), so \(e=12-14=-2\). At \(x=3\), \(\hat{\mu}=16\), so \(e=15-16=-1\). At \(x=4\), \(\hat{\mu}=18\), so \(e=20-18=2\).
| \(x_i\) | \(y_i\) | \(\hat{\mu}_i\) | \(e_i\) |
|---|---|---|---|
| 0 | 12 | 10 | 2 |
| 1 | 11 | 12 | -1 |
| 2 | 12 | 14 | -2 |
| 3 | 15 | 16 | -1 |
| 4 | 20 | 18 | 2 |
Interpretation and check. The residuals are positive at both ends and negative in the middle, a U-shaped pattern rather than an unstructured cloud. The residuals sum to \(2-1-2-1+2=0\), and their products with \(x\) sum to \(0-1-4-3+8=0\), consistent with the fitted least-squares straight line including an intercept. The pattern still suggests that a straight-line mean may miss curvature. The common mistake is to interpret the zero residual sum as proof of adequate form: the sum can be zero while residuals are systematically arranged across \(x\).
Residuals for Count and Binary Responses
Residuals are on different scales for different response distributions. For a Poisson model with conditional mean \(\mu_i\), the variance is also \(\mu_i\), so the Pearson residual is \((y_i-\hat{\mu}_i)/\sqrt{\hat{\mu}_i}\) when the Poisson dispersion is fixed at one. For a binomial count \(Y_i\) from \(n_i\) trials with fitted success probability \(\hat{p}_i\), the fitted mean is \(n_i\hat{p}_i\) and the variance is \(n_i\hat{p}_i(1-\hat{p}_i)\). These formulas make residual sizes comparable to their model-based standard deviations; they do not make a plot a formal test of the model.
Worked Example: A Poisson Residual Plot with No Clear Trend
A hypothetical model uses a Poisson distribution with dispersion fixed at one and a log link for event counts. For six observations, the fitted means from the model are \(2, 2.5, 3, 3.5, 4,\) and \(4.5\), at predictor values \(x=1,2,3,4,5,6\), respectively. The observed counts are \(1,4,2,5,3,6\). Calculate Pearson residuals and describe what this small plot suggests.
Method. For a Poisson response, \(V(\mu)=\mu\), so calculate \(r_i^P=(y_i-\hat{\mu}_i)/\sqrt{\hat{\mu}_i}\) for each observation.
The six calculations are \((1-2)/\sqrt{2}=-0.7071\); \((4-2.5)/\sqrt{2.5}=1.5/1.5811=0.9487\); \((2-3)/\sqrt{3}=-1/\sqrt{3}=-0.5774\); \((5-3.5)/\sqrt{3.5}=1.5/1.8708=0.8018\); \((3-4)/\sqrt{4}=-0.5000\); and \((6-4.5)/\sqrt{4.5}=1.5/2.1213=0.7071\). Values are rounded to four decimals.
| \(x_i\) | \(y_i\) | \(\hat{\mu}_i\) | \(r_i^P\) |
|---|---|---|---|
| 1 | 1 | 2.0 | -0.7071 |
| 2 | 4 | 2.5 | 0.9487 |
| 3 | 2 | 3.0 | -0.5774 |
| 4 | 5 | 3.5 | 0.8018 |
| 5 | 3 | 4.0 | -0.5000 |
| 6 | 6 | 4.5 | 0.7071 |
Interpretation and check. The signs alternate, and this small set does not show an obvious sustained curve or trend with \(x\). That is not evidence that the model is correct: six observations provide limited diagnostic information, and other diagnostics remain necessary. A common calculation error is to divide by \(\hat{\mu}_i\) rather than by its square root. For example, when \(y=6\) and \(\hat{\mu}=4\), the incorrect calculation gives \((6-4)/4=0.5\), while the Poisson Pearson residual is \(2/\sqrt{4}=1\). The variance is \(\mu\), so its standard deviation is \(\sqrt{\mu}\).
Worked Example: Grouped Binomial Residuals Reveal a Pattern
A hypothetical analyst fits a binomial model to four groups, each containing \(n_i=10\) trials, using a continuous predictor that increases from group 1 to group 4. The fitted probabilities are \(0.20,0.40,0.60,0.80\), while the observed success counts are \(1,4,6,9\). Calculate grouped Pearson residuals and interpret their pattern.
Method. For each group, use \(r_i^P=(y_i-n_i\hat{p}_i)/\sqrt{n_i\hat{p}_i(1-\hat{p}_i)}\). The denominator is the model-based standard deviation of the success count.
For group 1, the expected count is \(10(0.20)=2\), the variance is \(10(0.20)(0.80)=1.6\), and the residual is \((1-2)/\sqrt{1.6}=-0.7906\). For group 2, the expected count is \(4\), the variance is \(10(0.40)(0.60)=2.4\), and the residual is \((4-4)/\sqrt{2.4}=0\). For group 3, the expected count is \(6\), the variance is \(2.4\), and the residual is \((6-6)/\sqrt{2.4}=0\). For group 4, the expected count is \(8\), the variance is \(1.6\), and the residual is \((9-8)/\sqrt{1.6}=0.7906\). Values are rounded to four decimals.
Interpretation and check. Residuals are negative at the low end and positive at the high end, with zero residuals in the middle. The pattern is consistent with underprediction at higher predictor values relative to lower ones, and warrants checking the mean specification. Grouping summarizes the pattern here; it does not mean that grouped residuals should replace individual-level diagnostics in an individual-level analysis. The expected counts and variances used in all four calculations match the stated binomial model.
From a Pattern to a Model Check
A residual pattern suggests a question to investigate, not an automatic instruction to add terms. If the model assumes a linear effect of \(x\) on the link scale and the plot curves, a candidate might include a nonlinear term, such as a quadratic, if that shape is substantively and statistically reasonable. A transformation or another justified specification may also be appropriate. After refitting, inspect the residual plot again and compare the models using suitable criteria. A more complicated model is not automatically better simply because it can follow a small-sample fluctuation.
Use a clearly identified residual type and include a horizontal zero reference.
Check whether residuals tend to be positive or negative in particular regions, or whether a curve or sustained slope appears.
Check the assumed functional form and whether an omitted term, interaction, unusual record, or data issue could explain the pattern.
Assess whether the revised mean structure resolves the pattern without overfitting or relying on unsupported extrapolation.
A frequent exam trap is to treat every pattern in a residual plot as evidence of the same problem. A curve against a predictor is a reason to investigate the mean structure; changing residual spread is a different diagnostic question. Do not declare variance misspecification solely because the cloud looks wider in one part of the plot, and do not ignore a persistent positive-to-negative trend by describing it as “random noise.” Likewise, a few scattered points are not enough to establish a systematic functional-form defect.
Check Your Understanding
For each question, focus on the calculation or pattern stated and distinguish a diagnostic clue from a final model conclusion.
- A Poisson model with dispersion fixed at one gives \(y=7\) and \(\hat{\mu}=4\) for an observation. Calculate its Pearson residual. (Answer: \(1.5000\))
- A Gaussian identity-link model gives an observed response of 18 and a fitted mean of 20. What is the raw residual, and is the observation above or below its fitted mean? (Answer: \(-2\); below the fitted mean)
- A grouped binomial observation has \(n=20\), \(y=8\), and fitted probability \(\hat{p}=0.30\). Calculate its Pearson residual to four decimals. (Answer: \(0.9759\))
- Residuals plotted against increasing \(x\) are mostly positive at both low and high values and mostly negative in the middle. What mean-structure concern does this suggest? (Answer: Possible missing curvature, such as a nonlinear effect; the plot alone does not prove the cause.)
- Residuals are mostly negative for low \(x\) and mostly positive for high \(x\). State one reasonable next diagnostic step and one conclusion that would be too strong. (Answer: Check the predictor’s functional form and refit a justified alternative; it would be too strong to claim that the plot proves a specific nonlinear term is required.)