Residual Plots Against Fitted Means

Why Plot Residuals Against Fitted Means?

A generalized linear model describes a response distribution whose mean depends on predictors. A plot of residuals against fitted means asks whether the observed responses depart from that description in a systematic way across the range of fitted response levels. It is a compact diagnostic: a trend or curve can suggest that the mean structure is inadequate, while a widening band may reflect the variance structure—or a mismatch in it.

For observation \(i\), let \(Y_i\) be the response, \(\hat{\mu}_i\) its fitted mean, and \(\hat{y}_i\) its observed value. The horizontal coordinate in these plots is \(\hat{\mu}_i\); the vertical coordinate is a chosen residual. A horizontal reference at zero is useful for residuals centered at zero. A smooth curve can help reveal broad structure, but it is a visual summary, not a formal test.

The plots answer related but different questions. Response residuals remain in the response’s original units. Pearson residuals divide by a variance-based scale, and deviance residuals express each observation’s contribution relative to the fitted likelihood. Randomized quantile residuals transform the fitted response distribution to a normal scale, with randomization needed for discrete responses. None should be read as if the residuals were independent, identically distributed errors: fitted parameters have been estimated from the same data.

What Patterns Mean

Under a correctly specified mean model, residuals should not show a strong, systematic change in their center across fitted means. A smooth that rises or falls through the zero line, or a curved pattern, signals that the fitted means may be systematically too low in some range and too high in another. This is a reason to investigate the mean specification; the plot alone does not identify the cause.

The vertical spread needs more careful interpretation. For a GLM response with conditional mean \(\mu_i\), the variance is \(\operatorname{Var}(Y_i\mid x_i)=\phi V(\mu_i)\). Consequently, response residuals can naturally have different spreads at different means. Pearson residuals divide by an estimate of that changing standard deviation, so under known parameters their variance is one. This makes Pearson plots useful for seeing whether spread remains after variance standardization, though estimation and finite samples prevent an exact unit-variance guarantee for fitted residuals.

To verify this, subtract the nonrandom conditional mean from \(Y_i\). Linearity of expectation gives \(E[Y_i-\mu_i\mid x_i]=E[Y_i\mid x_i]-\mu_i=0\). Subtracting a constant does not change variance, so \(\operatorname{Var}(Y_i-\mu_i\mid x_i)=\operatorname{Var}(Y_i\mid x_i)=\phi V(\mu_i)\). Dividing by the fixed positive standard deviation gives variance one. For fitted residuals, \(\hat{\mu}_i\) and often \(\hat{\phi}\) are estimated, so this calculation is a reference rather than an exact statement about their joint distribution. \(\square\)

Comparing Scales and Reading Spread

For a Poisson response, \(V(\mu)=\mu\) and \(\phi=1\). The response residual’s standard deviation therefore grows as \(\sqrt{\mu}\), whereas the Pearson residual divides by \(\sqrt{\hat{\mu}}\). The signed Poisson deviance residual is \( \operatorname{sign}(y-\hat{\mu})\sqrt{2\{y\log(y/\hat{\mu})-(y-\hat{\mu})\}}\), with \(y\log(y/\hat{\mu})\) taken as zero when \(y=0\). Near the fitted mean, deviance residuals are approximately on the Pearson scale; farther away, their transformation differs.

The approximation follows by expanding the log-likelihood around its maximum at the mean parameter \(\mu=y\). The first derivative of the log-likelihood with respect to its mean parameter is zero there, and its second derivative, evaluated at the saturated mean \(\mu=y\), is \(-1/(\phi V(y))\) for an exponential-dispersion model. Taylor expansion gives the log-likelihood difference between the saturated mean \(y\) and a nearby mean \(\mu\) as \((y-\mu)^2/(2\phi V(y))\) to second order. Deviance is twice that difference, so its leading term is \((y-\mu)^2/(\phi V(y))\), equivalently \((y-\mu)^2/(\phi V(\mu))\) to leading order as \(\mu\to y\). Taking the signed square root gives the stated local scale, with the factor \(\phi\) included in the residual denominator when dispersion-standardized deviance residuals are used. The approximation is local; it does not claim equality for large discrepancies. \(\square\)

Randomized Quantile Residuals and Isolated Discrepancies

For a continuous fitted response distribution with cumulative distribution function \(F_i\), the quantile residual is \(\Phi^{-1}(F_i(Y_i))\), where \(\Phi\) is the standard normal cumulative distribution function. For a discrete response, \(F_i(Y_i)\) has jumps and is not uniform even when the fitted distribution is correct. Randomized quantile residuals address this by drawing \(U_i\) uniformly between \(F_i(Y_i^-)\) and \(F_i(Y_i)\), then setting \(r_{Qi}=\Phi^{-1}(U_i)\).

To see this, list the possible response values in order. Each value \(y\) occurs with probability \(F(y)-F(y^-)\), exactly the length of the interval assigned to it. Conditional on \(Y=y\), independent uniform \(W\) makes \(U\) uniform on that interval. These intervals partition \((0,1)\), apart from endpoints of probability zero. For any subinterval \(A\subset(0,1)\), the probability that \(U\in A\) is the sum, over the partition intervals, of the length of \(A\)'s intersection with each interval; this sum is the length of \(A\). Hence \(U\) is uniform. Applying the inverse standard normal distribution function gives a standard normal variable. With estimated parameters, this is an ideal reference, not an exact finite-sample guarantee. \(\square\)

Common Mistakes and Practical Reading

• Calling every change in response-residual spread a variance failure. Response residuals inherit the family’s mean-dependent variance. Compare with Pearson or deviance residuals and the variance function before diagnosing a problem.
• Describing the shape inaccurately. A sequence that rises from negative to positive is an increasing trend; it is not U-shaped. Describe what the plotted values actually do.
• Treating a large residual as proof of a bad observation. It identifies a surprising response relative to its fitted mean, but does not distinguish data error, random variation, or model inadequacy.
• Expecting fitted residuals to be exactly independent and standardized. The fitted means were estimated from the responses, so the ideal reference behavior under known parameters is only a guide to interpretation.
• Overinterpreting a smooth line. A smooth can expose broad curvature, but it depends on the data and smoothing choices. Look at the points, scale, and sample size as well.

A useful reading sequence is to inspect the residual center across fitted means, then assess spread on a scale appropriate to the response family, and finally note any isolated large departures. Agreement across residual types strengthens a visual concern; disagreement can reveal that the apparent feature is mainly a consequence of the residual scale. These plots are diagnostic tools, not automatic model-selection rules.