Why Examine Residuals for Link Misspecification?
A generalized linear model (GLM) relates the conditional mean response to predictors through a link function. Even when the predictors are suitable, the chosen link can impose the wrong shape on that relationship. The resulting fitted values may be systematically too high in some ranges and too low in others. A residual plot can reveal this pattern because it compares each observed response with the mean predicted by the model.
The diagnostic question is not simply whether residuals are large. Individual residuals vary even when a model is appropriate. Instead, ask whether residuals show a repeatable pattern across predictor values or fitted means. For a binary response, for example, observed success proportions can be compared with fitted probabilities across ranges of the linear predictor. For count data, observed counts or rates can be compared with fitted means while accounting for exposure.
The notation above describes the mean relationship; a GLM also specifies a response distribution and variance structure. For example, a Bernoulli response has conditional variance \(\mu_i(1-\mu_i)\), while a Poisson response has conditional variance \(\mu_i\). Residuals should therefore be interpreted in light of the response distribution, not as though every response had the same variability. Earlier material on standardized GLM residuals and residual patterns for discrete outcomes provides the relevant residual scales.
What a Link-Related Pattern Looks Like
Under a correctly specified conditional mean, the residual has conditional expectation zero: \(E[Y_i-\mu_i\mid\mathbf{x}_i]=0\). The link determines how the mean changes with the linear predictor. If the true mean is \(h(\eta_i)\) but the fitted link implies \(g^{-1}(\eta_i)\), then, at a fixed predictor value, the expected response residual is \(h(\eta_i)-g^{-1}(\eta_i)\). This difference can change direction or magnitude over the predictor range. A plot may therefore show a curved sequence of residuals, or observed averages that sit above the fitted means in some ranges and below them in others.
For a binary response, one practical display is to group observations into ranges of fitted probability or linear predictor. Within each group, compare the observed proportion of successes with the average fitted probability. For a count response, compare the observed total count with the sum of fitted means; with exposure, compare observed and fitted rates using the exposure-weighted means. The groupings summarize a pattern; they do not replace the observation-level diagnostic plots.
Worked Examples
Worked Example: Binary Responses Across Fitted-Probability Ranges
A hypothetical insurer models whether a small-business policy renews using a Bernoulli GLM with a logit link. Assume independent binary responses and, for this diagnostic calculation, treat the fitted probabilities as fixed. Four groups each contain 100 policies. Their fitted probabilities are 0.10, 0.30, 0.60, and 0.85; the observed numbers of renewals are 14, 25, 54, and 92. Compare observed and fitted values and assess the pattern.
Method. For each group, calculate the expected number of renewals as \(n\hat p\), the observed proportion as \(O/n\), and the residual count as \(O-n\hat p\). A useful approximate standardized group residual under independent Bernoulli outcomes is \((O-n\hat p)/\sqrt{n\hat p(1-\hat p)}\). This scale uses the binomial variance and is descriptive here, not a formal test.
Group 1. The expected number is \(100(0.10)=10\). The observed proportion is \(14/100=0.14\), so the residual count is \(14-10=4\). Its standard deviation is \(\sqrt{100(0.10)(0.90)}=\sqrt{9}=3\), giving a standardized residual of \(4/3=1.33\).
Group 2. The expected number is \(100(0.30)=30\). The observed proportion is \(25/100=0.25\), and the residual count is \(25-30=-5\). The standard deviation is \(\sqrt{100(0.30)(0.70)}=\sqrt{21}\approx4.583\), giving \(-5/\sqrt{21}\approx-1.09\).
Group 3. The expected number is \(100(0.60)=60\). The observed proportion is \(54/100=0.54\), and the residual count is \(54-60=-6\). The standard deviation is \(\sqrt{100(0.60)(0.40)}=\sqrt{24}\approx4.899\), giving \(-6/\sqrt{24}\approx-1.22\).
Group 4. The expected number is \(100(0.85)=85\). The observed proportion is \(92/100=0.92\), and the residual count is \(92-85=7\). The standard deviation is \(\sqrt{100(0.85)(0.15)}=\sqrt{12.75}\approx3.571\), giving \(7/\sqrt{12.75}\approx1.96\).
Interpretation and check. Across increasing fitted probabilities, the observed proportions are 0.14, 0.25, 0.54, and 0.92, compared with fitted probabilities 0.10, 0.30, 0.60, and 0.85. The residual signs are positive, negative, negative, positive. This curved pattern is a reason to inspect link adequacy, though the group summaries alone are not conclusive. The residual counts sum to \(4-5-6+7=0\), but that cancellation does not remove the pattern across groups. The method avoids treating a zero overall residual total as evidence that the fit is adequate.
Worked Example: Correctly Weighting Fitted Probabilities Within a Range
A hypothetical renewal model produces a broad predictor-range group with two fitted-probability subgroups. In the first, 80 policies each have fitted probability 0.20 and 19 renew. In the second, 20 policies each have fitted probability 0.80 and 10 renew. Calculate the group’s observed renewal proportion, average fitted probability, and residual. Explain why an unweighted average of the two fitted probabilities is inappropriate.
Method. Aggregate expected counts by summing the fitted probabilities for all observations. The first subgroup contributes \(80(0.20)=16\) fitted renewals. The second contributes \(20(0.80)=16\). Thus the total fitted count is \(16+16=32\) among \(80+20=100\) policies, and the average fitted probability is \(32/100=0.32\).
The observed renewal count is \(19+10=29\), so the observed proportion is \(29/100=0.29\). The residual count is \(29-32=-3\), and the observed-minus-fitted difference in proportions is \(0.29-0.32=-0.03\).
Common mistake and correction. Averaging the two subgroup probabilities without regard to their sizes gives \((0.20+0.80)/2=0.50\). Comparing the observed proportion with that number would produce \(0.29-0.50=-0.21\), which is wrong because the subgroups do not have equal sizes. The correct mean weights each probability by its number of policies: \((80(0.20)+20(0.80))/100=0.32\). The corrected difference is \(-0.03\), not \(-0.21\).
Interpretation and check. This broad group has an observed proportion three percentage points below its correctly aggregated fitted probability. The result is a group-level comparison; it cannot show whether the difference is spread across both subgroups or concentrated in one. The weighted fitted count, 32, and observed count, 29, reproduce the difference of \(-3\) policies.
Worked Example: Count Residuals With Exposure
A hypothetical roadside-assistance study uses a Poisson GLM with a log link and an exposure offset. Assume independent Poisson counts conditional on the predictors and exposures, with variance equal to the mean. Three predictor ranges have exposures of 50, 100, and 100 contract-years. The fitted event rates per contract-year are 0.08, 0.12, and 0.18, while the observed event counts are 7, 10, and 21. Calculate fitted counts, observed rates, and Pearson residuals, then assess the sequence.
Method. With a log link and exposure \(E_i\), the fitted mean count is \(\hat{\mu}_i=E_i\hat{\lambda}_i\), where \(\hat{\lambda}_i\) is the fitted rate. Under the stated Poisson assumption, the Pearson residual for a grouped count is \((O_i-\hat{\mu}_i)/\sqrt{\hat{\mu}_i}\).
Range 1. The fitted count is \(50(0.08)=4\). The observed rate is \(7/50=0.14\). The residual is \(7-4=3\), and the Pearson residual is \(3/\sqrt{4}=3/2=1.50\).
Range 2. The fitted count is \(100(0.12)=12\). The observed rate is \(10/100=0.10\). The residual is \(10-12=-2\), and the Pearson residual is \(-2/\sqrt{12}\approx-0.58\).
Range 3. The fitted count is \(100(0.18)=18\). The observed rate is \(21/100=0.21\). The residual is \(21-18=3\), and the Pearson residual is \(3/\sqrt{18}\approx0.71\).
Interpretation and check. The observed rates are 0.14, 0.10, and 0.21, compared with fitted rates 0.08, 0.12, and 0.18. The Pearson residuals have signs positive, negative, positive. That alternation could indicate a systematic mismatch in the response-scale mean pattern and warrants further inspection, but three ranges do not establish a link problem. The fitted counts sum to \(4+12+18=34\); observed counts sum to \(7+10+21=38\); hence the total residual is \(38-34=4\), matching \(3-2+3=4\). Exposure is essential: comparing raw counts alone would ignore that the second and third ranges each contain twice the exposure of the first.
Why the Residual Comparison Works
The justification follows from the definition of a conditional mean. If the fitted mean is correct, then for a response \(Y\) and predictor vector \(\mathbf{x}\), \(E[Y\mid\mathbf{x}]=\mu(\mathbf{x})\). Therefore, \(E[Y-\mu(\mathbf{x})\mid\mathbf{x}]=E[Y\mid\mathbf{x}]-\mu(\mathbf{x})=0\). Residuals may still be positive or negative for individual observations, but they should not have a persistent conditional direction.
If the true mean instead follows \(h(\eta)\) while the candidate link implies \(g^{-1}(\eta)\), then at a fixed \(\eta\), \(E[Y-\hat{\mu}\mid\eta]\) is approximately \(h(\eta)-g^{-1}(\eta)\), when the fitted linear predictor is close to the relevant value. Binning observations by \(\eta\) or fitted mean estimates this conditional difference by comparing an observed group average with an average fitted mean. A changing difference across groups is the diagnostic signal. The approximation matters: fitted coefficients are estimated from the same data, so fitted residuals are not independent of the estimation procedure and a visual pattern is not an exact test.
For binary data, examine observed proportions against fitted probabilities. For counts, use observed and fitted counts or rates, retaining exposure in the fitted means.
Use ranges of the linear predictor, fitted mean, fitted probability, or a predictor relevant to the mean relationship. Calculate the group observed average and the correspondingly weighted average fitted mean.
Repeated movement from positive to negative residuals, or a systematic curve in observed-versus-fitted averages, is more informative than one unusual group.
Check whether the pattern could instead arise from an incorrect predictor shape, omitted structure, unusual observations, or an unsuitable variance assumption before attributing it to the link.
Common Interpretive Traps
- Calling every residual pattern a link problem. The link is one part of the conditional-mean specification. A pattern may have another source, so the conclusion should be that the mean fit needs investigation.
- Using unweighted group averages. When group sizes differ, average the fitted means over observations, not over subgroup labels. With exposure, fitted counts must include exposure.
- Reading one large residual as a trend. Individual outcomes vary. Link evidence is stronger when a pattern persists across ordered ranges rather than depending on one observation or bin.
- Assuming fitted residuals must sum to zero in every subgroup. Estimation may impose aggregate constraints, but it does not generally make residuals cancel within every predictor or probability range.
- Treating a visual diagnostic as a formal test. Grouping choices and parameter estimation affect the display. Use it as evidence to guide model assessment, not as a stand-alone proof.
Check Your Understanding
For each problem, calculate the requested comparison before interpreting its diagnostic meaning.
- A hypothetical binary GLM has 50 observations in one fitted-probability range, with \(\hat p=0.20\) for each observation and 7 observed successes. Find the fitted count, observed proportion, residual count, and approximate standardized group residual using the Bernoulli variance. (Answer: fitted count \(10\); observed proportion \(0.14\); residual count \(-3\); standardized residual \(-3/\sqrt{8}\approx-1.06\).)
- In a binary-response range, 40 observations have fitted probability 0.25 and 60 have fitted probability 0.50. The observed success counts are 8 and 35, respectively. Find the weighted fitted proportion, observed proportion, and residual count. (Answer: fitted count \(40(0.25)+60(0.50)=40\), fitted proportion \(0.40\); observed proportion \(43/100=0.43\); residual count \(+3\).)
- A Poisson model with exposure offset predicts a rate of 0.08 for 50 exposure units. The observed count is 9. Calculate the fitted count and Pearson residual under the Poisson variance assumption. (Answer: fitted count \(50(0.08)=4\); Pearson residual \((9-4)/\sqrt{4}=2.50\).)
- Across increasing fitted probabilities, a binary model has observed proportions that are above fitted values in the lowest and highest ranges, but below fitted values in the middle ranges. What does this pattern suggest, and what conclusion is not justified from the pattern alone? (Answer: it suggests a systematic mismatch in the conditional-mean shape and merits checking the link; it does not prove that the link, rather than another aspect of the mean or response model, is the cause.)
- A count model has observed counts larger than fitted counts in every range, with no clear change in the size of the discrepancy relative to the means. Is this by itself the distinctive changing residual pattern sought for link evidence? State one other model feature to investigate. (Answer: no; a consistent discrepancy is not by itself a changing pattern across the range. Investigate the mean level or variance specification, among other possible sources.)