Introduction
A fitted GLM can produce plausible predictions and still be a poor description of the data. A coefficient may be difficult to interpret without its scale; residuals may show a pattern the model has not captured; and a small number of observations may have a disproportionate effect on the fitted coefficients. A diagnostic review brings these signals together before results are used or communicated.
This tutorial follows one hypothetical claim-frequency analysis from coefficient interpretation through residual and influence checks. The aim is not to declare a model acceptable or unacceptable from one statistic. It is to identify what the diagnostics suggest, distinguish a signal from a conclusion, and recommend a sensible next check. The methods are aligned with the SOA Exam PA (Predictive Analytics) syllabus.
The Hypothetical Model and What It Assumes
Suppose an insurer models the number of equipment breakdown claims for each policy group. Let \(Y_i\) be the claim count for group \(i\), and let \(e_i\) be its exposure measured in policy-years. A Poisson GLM with a log link and an exposure offset is
Here \(\mu_i=E[Y_i\mid\mathbf{x}_i]\), and \(\log(e_i)\) is an offset: its coefficient is fixed at one. The model assumes that, conditional on the predictors, the count has a Poisson distribution with variance equal to its mean, and that the log of the expected count per unit exposure is linear in the included predictors. The offset also assumes proportionality between expected count and exposure. These are modeling assumptions to assess, not facts guaranteed by choosing a Poisson GLM.
Worked Example: Interpreting Coefficients and a Fitted Mean
Worked Example: Interpreting Coefficients and a Fitted Mean
In a hypothetical claim-frequency model, the fitted log mean is
where \(e\) is exposure in policy-years, \(A\) is equipment age measured in decades, and \(T=1\) for a specified territory and \(T=0\) for the reference territory. Interpret both coefficients and calculate the fitted expected count for a policy group with two policy-years of exposure, \(A=1\), and \(T=1\).
Method. For a log-link model, exponentiating a coefficient gives the multiplicative change in the fitted mean for a one-unit increase in that predictor, holding other predictors fixed. For age, one unit means ten years, not one year.
Age coefficient. The fitted mean is multiplied by \(\exp(0.12)=1.12749685\). Thus a ten-year increase in equipment age corresponds to an estimated \(12.749685\%\) increase in the expected count, holding exposure and territory fixed. To two decimal places, this is a \(12.75\%\) increase.
Territory coefficient. The fitted mean for \(T=1\) is multiplied by \(\exp(0.25)=1.28402542\) relative to the reference territory, holding age and exposure fixed. This is an estimated increase of \(28.402542\%\).
Fitted count. Substitute \(e=2\), \(A=1\), and \(T=1\):
The fitted expected count is \(0.1158\) claims, rounded to four decimal places. The calculation checks: the reference rate is \(0.04\) per policy-year, and the two coefficient effects together multiply that rate by \(\exp(0.37)\); multiplying by two policy-years gives the stated expected count.
Common mistake avoided. Treating \(0.12\) as a \(12\%\) increase is incorrect because the coefficient is on the log-mean scale. The percentage change is \((\exp(0.12)-1)\times100=12.749685\%\), not \(12\%\). The age unit also matters: this is the change per decade.
Residual Patterns: What the Plots Can Tell You
A residual is a difference between an observed response and what the fitted model predicts, expressed on a scale suited to the model. For count GLMs, the Pearson residual for observation \(i\) is
For the Poisson model, \(\widehat{\operatorname{Var}}(Y_i)=\hat{\mu}_i\), so \(r_{Pi}=(y_i-\hat{\mu}_i)/\sqrt{\hat{\mu}_i}\). A positive residual means the observed count exceeds its fitted mean; a negative residual means it is below. Pearson residuals are useful for viewing departures relative to the model’s assumed variability, but a large residual is a reason to investigate, not proof that the observation is erroneous.
A plot of residuals against a predictor can reveal a systematic pattern. For example, residuals that tend to be negative for young equipment, positive for middle-aged equipment, and negative again for old equipment may indicate that a straight-line effect of age on the log-mean is inadequate. A curved smoother can help reveal such structure. A funnel-like increase in residual spread may suggest that variability changes with fitted mean or another predictor. In a count model, however, raw residual spread is naturally affected by the mean, so the pattern must be judged on an appropriate residual scale.
Residuals against fitted values help assess broad calibration and variance patterns; residuals against individual predictors can highlight a predictor-specific shape. Neither plot identifies the correct replacement model by itself. A visible trend should prompt checks such as whether the predictor’s functional form is suitable, whether an omitted structure is plausible, and whether the pattern is driven by only a few observations.
Why Leverage-Adjusted Residuals Are Useful
An observation can affect the fitted model and then appear less unusual because the model has partly adapted to it. Leverage measures how unusual an observation’s predictor values are relative to the design used for fitting. In GLMs, leverage is defined using the model’s weighted fitting structure. A common approximate diagnostic standardization of a Pearson residual is \(r_{Pi}/\sqrt{1-h_{ii}}\), where \(h_{ii}\) is the observation’s leverage.
The justification comes from the local weighted least-squares approximation used in GLM fitting. In a linear model, fitting the response projects it onto the model space, leaving residual variation reduced by the diagonal leverage \(h_{ii}\). The approximate residual variance is therefore proportional to \(1-h_{ii}\). Dividing by its square root adjusts for that reduction. In a GLM the same reasoning applies locally, with weights determined by the fitted mean and variance function; it motivates the approximation, but does not make it an exact universal residual formula.
Worked Example: Reading Two Residual Diagnostics
A diagnostic output for a hypothetical fitted count model gives observation B a Pearson residual of \(3.1\) and leverage \(0.02\). Observation C has a Pearson residual of \(-2.8\) and leverage \(0.09\). Calculate their approximate leverage-adjusted Pearson residuals, then state what should be checked.
For B:
For C:
Both results are rounded to three decimal places. B is a notably positive residual after this adjustment; C is notably negative. The analyst should inspect the records, their fitted values, and the residuals against relevant predictors. The leverage adjustment does not show that either record is invalid, nor does it establish a universal cutoff for action. It gives a more comparable signal of residual size while accounting approximately for leverage.
Common mistake avoided. Do not replace \(1-h_{ii}\) with \(h_{ii}\) in the denominator. The adjustment shown is division by \(\sqrt{1-h_{ii}}\). Also preserve the residual’s sign: C remains negative after division by a positive quantity.
Dispersion and Influence: Signals, Not Verdicts
For a Poisson GLM, the variance assumption is \(\operatorname{Var}(Y_i\mid\mathbf{x}_i)=\mu_i\). One summary of whether the residual variation is larger than this assumption anticipates is the Pearson dispersion estimate:
A value appreciably above one can signal overdispersion: conditional variability may exceed the Poisson variance, or the mean structure may be missing important variation. It can also reflect dependence, unmodeled heterogeneity, or a few unusual records. A value near one is not proof that every assumption holds. The size and context of the data, residual plots, and other checks matter. This summary is a diagnostic signal, not by itself a model-selection decision.
Influence concerns how much a fitted result changes when an observation is omitted. Leverage alone is not influence: an observation may have unusual predictor values but fit the model well. A large residual alone is also not influence: the observation may have little effect on the fitted coefficients. Case-deletion diagnostics combine these ideas by refitting without a case and checking how much estimates or predictions change. A change should trigger investigation of the record and the model’s sensitivity; deletion is not justified merely because it improves a statistic.
Worked Example: Combining Dispersion and Case-Deletion Evidence
A hypothetical Poisson frequency model has Pearson statistic \(X_P^2=162\) on 100 residual degrees of freedom. In a separate case-deletion check, the estimated coefficient for age in decades is \(0.12\) in the full data and \(0.08\) after omitting one policy group. The full-model standard error of that coefficient is \(0.02\). Explain the evidence and recommend follow-up.
Dispersion calculation:
The Pearson dispersion estimate is \(1.62\), above one. This is a signal to investigate whether the Poisson variability and mean structure are adequate. It does not identify the cause. The analyst should examine residual patterns, the observations contributing most to \(X_P^2\), and plausible dependence or unmodeled heterogeneity.
Coefficient sensitivity: The age coefficient changes by \(0.12-0.08=0.04\). Relative to the stated full-model standard error, this is \(0.04/0.02=2\) standard errors. The age rate multiplier for a ten-year increase is \(\exp(0.12)=1.1275\) in the full fit and \(\exp(0.08)=1.0833\) after deletion, to four decimal places. Thus the estimated age effect is meaningfully sensitive to this group.
Recommendation. Inspect the group’s data quality, exposure, outcome, and predictor values; examine its residual and leverage; and verify that the model form is suitable for the relevant range. Report the sensitivity if it affects the intended use. Do not remove the group simply to obtain a smaller dispersion estimate or a more convenient coefficient.
Common mistake avoided. A coefficient changing after deletion does not prove that the observation is erroneous. It demonstrates sensitivity of the fitted result to that case. Validity and model suitability require contextual checks.
A Practical Diagnostic Sequence
A concise review can keep the evidence organized without treating each plot as an isolated pass-or-fail test.
Confirm the link, predictor units, reference levels, and offset. Translate log-link coefficients using exponentiation, with other predictors held fixed.
Look at residuals against fitted values and important predictors. Note systematic curvature, changing spread, clusters, and unusually large positive or negative residuals.
Calculate or inspect relevant summaries, then identify whether a broad pattern or a small number of cases appears to drive them.
Use case-deletion evidence to identify influential cases. Verify records and consider plausible model limitations before deciding how to proceed.
This sequence supports a measured conclusion: state what the diagnostics show, what they do not establish, and what should be checked next. For example, “the Pearson dispersion estimate is above one and several residuals are large; inspect the affected records and assess possible unmodeled variation” is more defensible than declaring the model invalid from one number.
Check Your Understanding
For each original problem, identify the calculation or diagnostic implication before checking the final answer.
- A Poisson GLM with a log link has an age coefficient of \(0.08\) per five-year increase in machinery age. What is the multiplicative change and percentage change in fitted expected count for a five-year increase, holding other predictors fixed? (Answer: \(\exp(0.08)=1.0833\), an increase of \(8.33\%\).)
- A count observation has fitted mean \(10\) and observed count \(18\). Under the Poisson variance assumption, calculate its Pearson residual to three decimal places. (Answer: \((18-10)/\sqrt{10}=2.530\).)
- A frequency model has baseline rate \(0.025\) per exposure unit. A policy group has three exposure units and two predictor contributions of \(0.20\) and \(0.10\) on the log scale. Calculate its fitted expected count to four decimal places. (Answer: \(3(0.025)\exp(0.30)=0.1012\).)
- A Poisson GLM has Pearson statistic \(72\) on \(48\) residual degrees of freedom. Calculate the Pearson dispersion estimate and state what a value above one signals. (Answer: \(72/48=1.50\); it signals possible extra variation or other model inadequacy to investigate, not a definitive diagnosis.)
- An age coefficient is \(0.15\) in a full fit and \(0.11\) after deleting one policy group; the full-fit standard error is \(0.02\). Calculate the coefficient change in standard-error units and state the appropriate interpretation. (Answer: \((0.15-0.11)/0.02=2\) standard errors; the estimate is sensitive to that case, which should be investigated rather than automatically deleted.)