Why the Mean Function Needs Diagnosing
A generalized linear model (GLM) specifies how a response’s conditional mean changes with its predictors. A model can use an appropriate response distribution and link function yet still describe that mean poorly. For example, a straight-line relationship between a continuous predictor and the link-transformed mean may be too restrictive if the relationship bends. Residuals plotted against the predictor can help reveal this problem.
The diagnostic question here is whether the chosen mean functional form adequately represents how the mean varies with a continuous predictor. A residual pattern is evidence to investigate, not proof that one particular alternative is correct. The response distribution, link, other predictors, exposure, and data collection process all matter when interpreting it.
Residual Patterns Across a Predictor
For a GLM with conditional mean \(\mu_i\), variance \(\operatorname{Var}(Y_i\mid\mathbf{x}_i)=\phi V(\mu_i)\), and dispersion parameter \(\phi\), the Pearson residual based on the fitted mean \(\hat{\mu}_i\) is
For a Poisson GLM, \(V(\mu)=\mu\) and the usual model sets \(\phi=1\). For a binomial observation that is a proportion from \(m_i\) trials, \(V(\mu_i)=\mu_i(1-\mu_i)/m_i\), where \(\mu_i\) is the fitted or true proportion as appropriate. These residuals scale the observed-minus-expected difference by a variance estimate; they do not remove all effects of fitting the model.
Under the correctly specified model, residuals plotted against a continuous predictor should generally fluctuate around zero without a systematic shape. A smooth curve through the residuals that rises and falls, or a sequence of binned residual means that is positive at one end and negative in the middle, can suggest that the chosen mean function is inadequate. A fan-shaped spread may instead point toward variance or dispersion issues. Individual residuals also vary randomly, so a few that do not lie on a smooth pattern are not by themselves evidence of curvature.
The distinction between true and fitted means is important. If the true conditional mean is \(\mu_i\), then the Pearson residual centered and scaled using that true mean has conditional expectation zero, provided the variance is finite and nonzero. This follows directly from the definition:
The same exact statement does not generally hold for fitted Pearson residuals: \(\hat{\mu}_i\) is estimated from the data, and estimation affects the residuals. Under a correctly specified model, fitted residuals should still fluctuate around zero overall, but their conditional expectations are not generally exactly zero in a finite sample.
Worked Example: Recognizing Curvature in Poisson Residuals
A hypothetical insurer models claim counts using a Poisson GLM with a log link and an exposure offset. The fitted model includes a continuous risk score \(x\) linearly on the log-mean scale. In a displayed subset of the fitted data, the fitted expected counts and observed counts are as follows. The fit uses the full data set, not only these four displayed observations.
| Risk score \(x\) | Observed count \(y\) | Fitted mean \(\hat{\mu}\) | Pearson residual |
|---|---|---|---|
| 0 | 12 | 10 | \(2/\sqrt{10}=0.6325\) |
| 1 | 18 | 20 | \(-2/\sqrt{20}=-0.4472\) |
| 2 | 36 | 40 | \(-4/\sqrt{40}=-0.6325\) |
| 3 | 88 | 80 | \(8/\sqrt{80}=0.8944\) |
Method. For a Poisson GLM, the Pearson residual is \((y-\hat{\mu})/\sqrt{\hat{\mu}}\). At \(x=0\), the residual is \((12-10)/\sqrt{10}=2/3.1623=0.6325\). At \(x=1\), it is \((18-20)/\sqrt{20}=-2/4.4721=-0.4472\). At \(x=2\), it is \((36-40)/\sqrt{40}=-4/6.3246=-0.6325\). At \(x=3\), it is \((88-80)/\sqrt{80}=8/8.9443=0.8944\).
The signs are positive at both ends and negative in the middle. Across these ordered predictor values, that is a possible curved pattern rather than random scatter around zero. It motivates examining a curved mean specification, for example one that adds \(x^2\). It does not establish that a quadratic is the right form: the subset is small, and the rest of the data and other diagnostics still matter.
Common mistake avoided: Treating each displayed fitted residual as if its conditional expectation must be exactly zero. The zero-expectation derivation applies when centering at the true mean. These fitted residuals are diagnostic estimates affected by model fitting.
Comparing a Linear and Curved Mean
A common extension for a continuous predictor is to include a quadratic term while retaining the linear term. With a link \(g\), a model with other predictors \(z_i\) might be
Here \(\beta_2=0\) reduces the specification to one that is linear in \(x_i\) on the link scale. Including both \(x_i\) and \(x_i^2\) allows the slope on that scale to vary with \(x_i\): it is \(\beta_1+2\beta_2x_i\), holding the other predictors fixed. The curve is on the link scale; the relationship between \(x_i\) and the response mean itself need not be a quadratic curve.
A transformation is another option. Replacing \(x_i\) with \(h(x_i)\) gives \(g(\mu_i)=\beta_0+\beta_1h(x_i)+\cdots\). A log transformation requires \(x_i>0\), while a square-root transformation is defined for \(x_i\geq0\). The substantive meaning and observed range should guide this choice; a transformation should not be selected solely because it makes a plot look smoother.
Worked Example: Comparing Linear and Quadratic Gaussian Fits
A hypothetical study models a continuous response \(Y\) as a Gaussian GLM with identity link and constant variance, using predictor values \(x=0,1,2,3,4\). The observed responses are \(10,14,21,28,39\). Compare a linear mean with a quadratic mean using residual sums of squares. For the model-comparison calculation, assume independent normal errors with a common variance.
Linear fit. The sample means are \(\bar{x}=2\) and \(\bar{y}=22.4\). The centered predictor sum of squares is \(S_{xx}=(-2)^2+(-1)^2+0^2+1^2+2^2=10\). The centered cross-product is \((-2)(-12.4)+(-1)(-8.4)+0(-1.4)+1(5.6)+2(16.6)=72\). Thus the estimated slope is \(72/10=7.2\), and the intercept is \(22.4-7.2(2)=8\). The fitted values are \(8,15.2,22.4,29.6,36.8\), giving residuals \(2,-1.2,-1.4,-1.6,2.2\). Therefore,
Quadratic fit. Regressing on \(1,x,x^2\) gives fitted coefficients \(384/35\), \(13/7\), and \(9/7\), respectively. To check the calculation, the normal-equation matrix is \[ \begin{pmatrix}5&10&30\\10&30&100\\30&100&354\end{pmatrix}, \] and its right-hand side, formed from the five responses, is \((112,332,1030)^{\mathsf T}\). The stated coefficients solve these equations. The resulting fitted values are approximately \(10.9714,14.1143,19.8286,28.1143,38.9714\). The residuals are approximately \(-0.9714,-0.1143,1.1714,-0.1143,0.0286\). Squaring and summing gives
The residual sum of squares falls by \(14.80-2/35=14.74286\). For a formal nested-model comparison under the stated Gaussian assumptions, the quadratic term adds one parameter, and the quadratic model has \(5-3=2\) residual degrees of freedom. Thus \(F=[(14.80-2/35)/1]/[(2/35)/2]=516\). This very large statistic signals a substantial improvement in this small, constructed data set under the assumed model. The quadratic fit has only two residual degrees of freedom, however, so the example is not a general argument for choosing a quadratic. The model should also be checked for plausibility and evaluated with suitable data beyond the observations used to fit it.
Common mistake avoided: Comparing the two fitted residual sums of squares without accounting for the extra parameter. A more flexible model can reduce training error even when the added term does not generalize well.
Transformations and Out-of-Sample Evidence
A transformed predictor can be compared with a linear one by looking for less systematic residual behavior and, where appropriate, by assessing predictions on data not used to fit the models. The following Poisson example uses a held-out set so that the observations shown do not determine the displayed fitted means. A lower residual sum on a small validation set is suggestive, not conclusive.
Worked Example: Assessing a Square-Root Predictor
A hypothetical claim-count model uses a Poisson distribution, a log link, and exposure \(E_i\) as an offset: \(\log(\mu_i)=\log(E_i)+\beta_0+\beta_1x_i\) for the linear model, or \(\log(\mu_i)=\log(E_i)+\beta_0+\beta_1\sqrt{x_i}\) for the transformed model. On three held-out groups, each with exposure \(100\), the observed counts are \(4,7,11\). The linear model predicts means \(4,6,8\), while the square-root model predicts \(4,7,10\).
For the linear model, the Pearson residuals are \(0/\sqrt{4}=0\), \(1/\sqrt{6}=0.4082\), and \(3/\sqrt{8}=1.0607\). Their squared sum is \(0+1/6+9/8=1.2917\).
For the square-root model, the Pearson residuals are \(0/\sqrt{4}=0\), \(0/\sqrt{7}=0\), and \(1/\sqrt{10}=0.3162\). Their squared sum is \(0+0+1/10=0.1000\). The square-root specification has smaller Pearson-residual squared error on these held-out groups, and the increasing positive residuals under the linear specification are reduced.
This comparison favors investigating the transformed specification, but it does not prove that the square-root form is correct. Three held-out groups provide limited evidence, and the models must be compared on the same observations and exposure scale. The offset must remain in the mean model: omitting \(\log(E_i)\) would change the interpretation and could create misleading residual patterns.
Common mistake avoided: Treating a lower residual measure on a small sample as definitive proof. Use residual plots and predictive comparisons as evidence, check the relevant assumptions, and prefer a simpler adequate form when the evidence does not justify added complexity.
A Practical Diagnostic Sequence
When a residual plot against a continuous predictor suggests a pattern, proceed in a way that separates a mean-form concern from other possible explanations.
Use residuals appropriate to the response and variance, such as Pearson residuals, and retain the predictor’s original scale on the horizontal axis. For grouped binary data, inspect appropriately formed bins rather than interpreting each individual binary response as a smooth signal.
Distinguish a smooth trend or repeated sign pattern from random scatter. Consider whether a few unusual observations, leverage, or an uneven predictor distribution could dominate the appearance.
Try a scientifically interpretable transformation or a curved term such as \(x^2\), generally retaining the linear term when adding a quadratic. Do not extrapolate a polynomial beyond the predictor range without justification.
Check whether the residual pattern is reduced and whether predictive performance or an appropriate nested-model comparison supports the added complexity. Keep the response distribution, link, and other model terms in view.
A pattern across \(x\) can arise for reasons other than the functional form of \(x\), including an omitted predictor or interaction, unusual observations, or variance misspecification. The residual plot is therefore a diagnostic prompt, not a stand-alone test that identifies the cause. The earlier material on residuals against individual predictors and binned residual plots provides useful context for reading such displays. This tutorial focuses on the mean’s dependence on a continuous predictor, not on diagnosing a different link function.
Common Exam Traps
- Calling every nonzero fitted residual evidence of misspecification. Individual fitted residuals vary even under a correctly specified model. Look for systematic structure across the predictor.
- Claiming fitted residuals have exact conditional expectation zero. The direct zero-expectation result uses the true mean. Estimation affects fitted residuals.
- Confusing curvature with a link choice. A quadratic term or transformation changes the predictor function. It does not, by itself, change the link.
- Assuming lower training error settles the choice. Added flexibility can improve fit to the data used for estimation. Consider complexity and predictive evidence.
- Ignoring exposure or the variance function. In a count model with exposure, the offset belongs in the mean specification; Pearson residual scaling depends on the variance function.
Check Your Understanding
For each problem, identify the relevant calculation or diagnostic conclusion.
- A Poisson GLM has fitted mean \(10\) for an observation with count \(13\). Calculate its Pearson residual to four decimals. (Answer: \(0.9487\))
- A binomial group has \(m=40\) trials, \(18\) successes, and fitted success probability \(0.4\). Using the Pearson residual for the group count, calculate the residual to four decimals. (Answer: \(2/\sqrt{40(0.4)(0.6)}=0.6455\))
- Across increasing values of a continuous predictor, binned Pearson residual means are positive, then negative, then positive. What is the main mean-form concern, and does this pattern alone prove that a quadratic model is correct? (Answer: It suggests possible curvature in the mean function; no, it motivates investigation but does not establish a quadratic as correct.)
- A Poisson model uses exposure \(E=50\), log link, and \(\log(\mu)=\log(E)+\beta_0+\beta_1\sqrt{x}\). If \(\beta_0=\log(0.02)\), \(\beta_1=0.10\), and \(x=9\), calculate the fitted mean count. (Answer: \(50(0.02)e^{0.10(3)}=1.3499\), to four decimals.)
- A Gaussian identity-link model has \(n=12\) observations. Its linear fit has SSE \(30\); adding one quadratic coefficient gives SSE \(18\). Under independent normal errors with common variance, calculate the nested-model \(F\) statistic, using the larger model’s residual degrees of freedom. (Answer: \([(30-18)/1]/[18/(12-3)]=6\))