Why Plot Residuals Against Rating Variables?
An overall goodness-of-fit statistic can summarize model performance while concealing where predictions are systematically high or low. A model might fit reasonably overall but overpredict one range of driver ages and underpredict another. Or its predictions might be consistently low for one territory and high for another. Plotting residuals against rating variables helps reveal these patterns and indicates which parts of the model merit investigation.
The goal is diagnostic, not automatic model selection. A trend can point to a missing nonlinear effect, an omitted predictor, or a change in the relationship across segments. It does not, by itself, identify the cause or prove that a particular model revision is appropriate.
Choose and Read a Residual Scale
For an observation with response \(Y\) and fitted mean \(\hat{\mu}\), the response residual is \(Y-\hat{\mu}\). Its size depends on the response scale and may not be comparable across observations with different expected variances. For a Poisson count model, where the conditional mean and variance are both \(\mu\), the Pearson residual is
This tutorial uses that Pearson residual for the Poisson examples. Under the stated Poisson model, its conditional expected value is zero and its conditional variance is approximately one when the fitted mean is treated as fixed. In actual fitted models, parameter estimation affects the residuals; leverage and other adjustments are addressed by the standard residual-diagnostic methods.
Use the same residual definition and scale throughout a plot. For continuous predictors, inspect whether residuals fluctuate around zero or instead show a trend, curve, or changing spread. A smooth line can clarify a broad pattern, but should not be mistaken for a formal test. For rating categories, compare the direction, magnitude, and persistence of errors by category. If the categories are nominal—for example, territories—do not connect them as though adjacent categories represented nearby points on a meaningful numeric scale.
Why a Systematic Pattern Matters
The key reference is zero: positive residuals mean the observed response exceeds the fitted mean, while negative residuals mean the fitted mean exceeds the observation. Under a correctly specified mean model, the conditional mean of the response is the fitted mean, so
For a Poisson response with fixed conditional mean \(\mu\), \(E[Y\mid x]=\mu\) and \(\operatorname{Var}(Y\mid x)=\mu\). Dividing the response residual by \(\sqrt{\mu}\) therefore gives
Thus, if the model is adequate, residuals should not show a persistent mean pattern as a rating variable changes. Random scatter will not be perfectly even in a finite sample; the diagnostic concern is a coherent pattern that is substantial enough to investigate, not every isolated residual above or below zero.
Worked Example: Detecting a Trend Across Driver Ages
A hypothetical insurer fits a Poisson claim-frequency model to five equally exposed age groups. The fitted means are expected claim counts for each group. Calculate the Pearson residuals and assess whether the pattern suggests systematic error by age.
| Age | Observed count \(Y\) | Fitted mean \(\hat{\mu}\) |
|---|---|---|
| 25 | 1 | 2 |
| 35 | 2 | 3 |
| 45 | 4 | 5 |
| 55 | 8 | 7 |
| 65 | 10 | 8 |
Method and assumptions. Treat each count as Poisson conditional on its age group and fitted mean, so its variance equals its mean. Use \(r_P=(Y-\hat{\mu})/\sqrt{\hat{\mu}}\). The fitted means are given; no refitting is required.
Age 25: \(Y-\hat{\mu}=1-2=-1\), so \(r_P=-1/\sqrt{2}=-0.7071\).
Age 35: \(Y-\hat{\mu}=2-3=-1\), so \(r_P=-1/\sqrt{3}=-0.5774\).
Age 45: \(Y-\hat{\mu}=4-5=-1\), so \(r_P=-1/\sqrt{5}=-0.4472\).
Age 55: \(Y-\hat{\mu}=8-7=1\), so \(r_P=1/\sqrt{7}=0.3780\).
Age 65: \(Y-\hat{\mu}=10-8=2\), so \(r_P=2/\sqrt{8}=0.7071\).
The residuals move from negative at younger ages to positive at older ages. A plot would suggest overprediction at the younger end and underprediction at the older end. This directional pattern is a reason to investigate the age effect—for example, whether its form is adequate—not proof that age alone is responsible. Values are rounded to four decimal places only after calculation.
Common mistake avoided: The age-65 residual is \(2/\sqrt{8}\), not \(2/8\). The latter divides by the fitted mean rather than its square root and is not the Poisson Pearson residual.
Curvature and Other Patterns for Continuous Predictors
A roughly increasing or decreasing residual trend may suggest that the modeled relationship changes across the predictor’s range. A curved pattern can signal that a straight-line effect is inadequate on the model’s linear-predictor scale. A sudden shift may indicate a threshold, a recording issue, or an omitted rating distinction. Changing vertical spread may point to a variance or exposure issue, though a residual plot alone cannot diagnose that cause.
These interpretations depend on the model and the predictor. In a generalized linear model, a relationship that is linear in the linear predictor need not be linear in the response mean. Accordingly, assess residual structure against the predictor in light of the chosen link and predictor specification. Do not infer that a response-scale curve is necessarily a defect if it is consistent with the specified link.
Worked Example: Recognizing Curvature in a Poisson Fit
A hypothetical insurer models claim counts across six equally exposed values of a continuous equipment-age predictor. The model gives a fitted mean of four claims in every group. The observed counts are \(8,3,1,1,3,8\), in increasing order of equipment age. Determine the residual pattern and its implication.
Method and assumptions. Assume independent Poisson counts conditional on age, each with mean and variance four. The Pearson residual is \((Y-4)/\sqrt{4}=(Y-4)/2\).
For the first group, \(r_P=(8-4)/2=4/2=2.0000\). For the second, \(r_P=(3-4)/2=-1/2=-0.5000\). For the third, \(r_P=(1-4)/2=-3/2=-1.5000\). The fourth group also has \(r_P=(1-4)/2=-1.5000\). The fifth has \(r_P=(3-4)/2=-0.5000\). The sixth has \(r_P=(8-4)/2=4/2=2.0000\).
The sequence is \(2.0000,-0.5000,-1.5000,-1.5000,-0.5000,2.0000\). It is positive at both ends and negative in the middle, a U-shaped pattern. The observed total is \(8+3+1+1+3+8=24\), equal to the total fitted mean \(6(4)=24\). Thus, aggregate calibration here does not remove the visible pattern across age. A nonlinear age effect is one possible explanation to investigate.
Common mistake avoided: Looking only at total observed versus total expected would give \(24-24=0\), hiding the pattern. A good overall total does not establish that predictions are appropriate across the predictor’s range. Residuals are rounded to four decimal places.
Comparing Residuals Across Rating Categories
For categorical rating variables, plot residuals by category or summarize the pattern in a category-by-category display. A category with residuals predominantly above zero may be underpredicted; one with residuals predominantly below zero may be overpredicted. Check whether the pattern is persistent rather than driven by a single observation, and consider the amount of data and expected claim count in each category.
For a group of independent Poisson counts, their sum is Poisson with mean equal to the sum of their means. If category \(g\) has observed total \(O_g\) and expected total \(E_g\), its aggregate Pearson residual is
This is a useful category-level summary when the Poisson assumptions apply. It does not replace inspection of individual residuals: positive and negative errors within a category can cancel in the total.
Worked Example: Comparing Errors Across Territories
A hypothetical rating model predicts Poisson claim counts for four territories. Within each territory, counts are independent and the expected total is the sum of the fitted means. Use the aggregate Pearson residual to compare observed and expected counts.
| Territory | Observed total \(O_g\) | Expected total \(E_g\) |
|---|---|---|
| A | 18 | 20 |
| B | 24 | 20 |
| C | 39 | 30 |
| D | 21 | 30 |
Territory A: \(O_A-E_A=18-20=-2\), so \(R_A=-2/\sqrt{20}=-0.4472\).
Territory B: \(O_B-E_B=24-20=4\), so \(R_B=4/\sqrt{20}=0.8944\).
Territory C: \(O_C-E_C=39-30=9\), so \(R_C=9/\sqrt{30}=1.6432\).
Territory D: \(O_D-E_D=21-30=-9\), so \(R_D=-9/\sqrt{30}=-1.6432\).
The category summaries show positive error in B and C and negative error in A and D, with the largest magnitudes in C and D. Treat these as signals for further review, not conclusive evidence of a territory effect: differences in exposure, data quality, or model structure may also matter.
Common mistake avoided: For territory B, \(4/20=0.20\) is the relative error, not the Pearson residual. The Pearson residual is \(4/\sqrt{20}=0.8944\). Dividing by \(E_g\) instead of \(\sqrt{E_g}\) changes the diagnostic scale. Residuals are rounded to four decimal places.
Practical Interpretation and Exam Traps
A residual plot is a screening tool. Before proposing a model change, confirm that the residuals correspond to the intended response and prediction unit, that exposure has been handled consistently, and that the plotted predictor values match the model inputs. In frequency models, for example, observed counts must be compared with expected counts for the same exposure basis; comparing a raw count with a fitted rate can create a pattern that is purely a units error.
A residual pattern can also arise from sparse data or an unusual observation rather than a stable rating relationship. Look for persistence across nearby predictor values or multiple observations within a category. For a nominal factor, compare categories rather than interpreting their arbitrary display order as a continuous trend. For a continuous factor, a smooth curve should summarize the broad pattern without being allowed to chase every observation.
- Do not equate zero total error with no structure. Positive and negative residuals can cancel across the full portfolio.
- Do not confuse scales. Response residuals, Pearson residuals, and relative errors answer different questions.
- Do not overinterpret one point. A single large residual can be noise or an unusual case; it does not establish a persistent predictor effect.
- Do not treat a pattern as a prescription. A curve suggests investigating the predictor specification, but does not by itself determine whether to use bins, a polynomial, or a spline.
Check Your Understanding
For the Poisson calculations below, assume the conditional variance equals the fitted mean. Give Pearson residuals to four decimal places where needed.
- A rating cell has observed count 9 and fitted mean 6. Calculate its Pearson residual and state whether the fitted mean is above or below the observed count. (Answer: \(3/\sqrt{6}=1.2247\); the fitted mean is below the observed count.)
- For four increasing values of a continuous predictor, residuals are mostly negative at the low end and mostly positive at the high end. State the direction of prediction error at each end and one possible model feature to investigate. (Answer: The model tends to overpredict at the low end and underpredict at the high end; investigate whether the predictor effect’s form is adequate.)
- A model predicts a mean count of 3 in each of three ordered groups. Observed counts are 2, 3, and 6. Calculate the three Pearson residuals and describe the direction of the pattern. (Answer: \(-1/\sqrt{3}=-0.5774,\ 0,\ 3/\sqrt{3}=1.7321\); residuals rise from negative through zero to positive.)
- A category has observed total 12 and expected total 16. Calculate its aggregate Pearson residual and its observed-to-expected ratio. (Answer: \((12-16)/\sqrt{16}=-1.0000\); the observed-to-expected ratio is \(12/16=0.75\).)
- Two nominal territories have observed and expected totals of 28 and 25, and 18 and 22, respectively. Calculate each aggregate Pearson residual and state why the territories should not be joined by a trend line based on their display order. (Answer: Territory one: \(3/\sqrt{25}=0.6000\); territory two: \(-4/\sqrt{22}=-0.8528\). Their nominal order has no inherent numerical meaning.)