Exposure-Weighted Residual Patterns

Why Exposure Changes a Residual

A frequency model predicts a claim rate, but the data usually record claim counts. A policy observed for one year creates more opportunity for claims than a similar policy observed for one month. Consequently, a raw difference between observed and expected counts cannot be interpreted without considering how much exposure produced it. This is the central issue in exposure-weighted residual analysis.

For a standard frequency model, let \(Y_i\) be the claim count for observation \(i\), \(e_i\) its positive exposure measured in a specified unit, and \(\lambda_i\) the fitted claim rate per unit of exposure. Under a Poisson model, conditional on the predictors and exposure, counts are independent and satisfy \(Y_i\sim\operatorname{Poisson}(e_i\lambda_i)\). The exposure is assumed to represent the amount of claim-producing opportunity, with expected count proportional to exposure. For a log-link model, this proportionality is implemented by including \(\log(e_i)\) as an offset, whose coefficient is fixed at one.

\mu_i=E[Y_i\mid x_i,e_i]=e_i\lambda_i, \qquad \log(\mu_i)=\log(e_i)+x_i^\mathsf{T}\beta.

Here \(\mu_i\) is the fitted expected count and \(x_i^\mathsf{T}\beta\) describes the fitted log rate. Exposure and rate must use compatible units: if \(e_i\) is in vehicle-years, then \(\lambda_i\) is claims per vehicle-year. A zero-exposure record has no claim opportunity and cannot be used as an ordinary positive-exposure observation in this formulation.

Definition: The response residual for observation \(i\) is \(r_i=Y_i-\mu_i\), a discrepancy in claim-count units. Its exposure-adjusted rate residual is \(q_i=(Y_i-\mu_i)/e_i\), a discrepancy in claims per exposure unit. These quantities answer different questions: \(r_i\) measures count error, while \(q_i\) measures rate error.

Why Rates and Counts Behave Differently

The count residual is useful for describing the number of claims above or below expectation, but its scale naturally changes with exposure. Under the Poisson model, the expected count is \(e_i\lambda_i\), so the variance of the count also grows with exposure. The residual per exposure has a different variance. This result explains why rate residuals from small-exposure records are generally more variable than those from large-exposure records.

E\left[\frac{Y_i-\mu_i}{e_i}\,\middle|\,x_i,e_i\right]=0, \qquad \operatorname{Var}\left(\frac{Y_i-\mu_i}{e_i}\,\middle|\,x_i,e_i\right) =\frac{\lambda_i}{e_i}.

To justify the variance result, treat \(e_i\) as fixed and use \(\mu_i=e_i\lambda_i\). The conditional mean of the rate residual is \((E[Y_i\mid x_i,e_i]-\mu_i)/e_i=0\). Its conditional variance is \(\operatorname{Var}(Y_i\mid x_i,e_i)/e_i^2\). For a Poisson variable, the variance equals its mean, so this becomes \(e_i\lambda_i/e_i^2=\lambda_i/e_i\). Thus, a noisy rate residual from a short exposure is not automatically evidence of a poor model; the residual's expected variability depends on exposure.

For comparison, the Poisson Pearson residual divides the count residual by its fitted standard deviation. It adjusts for the count variance increasing with the fitted mean. A rate residual is particularly direct for interpreting calibration in rate units; the Pearson residual is useful when comparing count discrepancies on a variance-adjusted scale. Neither residual should be interpreted without the model and its assumptions.

r_i=Y_i-\mu_i, \qquad q_i=\frac{Y_i-\mu_i}{e_i}, \qquad r_{P,i}=\frac{Y_i-\mu_i}{\sqrt{\mu_i}} \quad\text{(Poisson model).}

Worked Example: Two Count Errors, Two Rate Errors

Hypothetical scenario. A frequency model predicts \(0.10\) claims per vehicle-year for each of two otherwise comparable records. Record A has 10 vehicle-years and 2 claims; Record B has 100 vehicle-years and 14 claims. Calculate their expected counts, response residuals, rate residuals, and Poisson Pearson residuals. Which record has the larger absolute rate discrepancy? The method is to convert the fitted rate to an expected count before forming residuals; this avoids treating raw count errors as rate errors.

Record A. Its fitted expected count is \(\mu_A=10(0.10)=1\). The response residual is \(2-1=1\) claim. The rate residual is \(1/10=0.10\) claims per vehicle-year. Its Pearson residual is \(1/\sqrt{1}=1.0000\).

Record B. Its fitted expected count is \(\mu_B=100(0.10)=10\). The response residual is \(14-10=4\) claims. The rate residual is \(4/100=0.04\) claims per vehicle-year. Its Pearson residual is \(4/\sqrt{10}=1.2649\).

Record B has the larger raw count residual, four claims rather than one, and the larger Pearson residual. But Record A has the larger absolute rate residual: \(0.10>0.04\) claims per vehicle-year. Their observed rates are \(2/10=0.20\) and \(14/100=0.14\), respectively, compared with the fitted rate of \(0.10\). The calculations are consistent: both observed rates exceed the fitted rate, but the smaller-exposure record has the greater rate discrepancy. A single record's rate residual is still subject to sampling variation.

Combining Residuals Across Unequal Exposures

When the question concerns a portfolio or a group of records, add observed and expected counts before comparing rates. For a set \(G\) of observations, the aggregate observed rate is total observed claims divided by total exposure; the aggregate fitted rate is total expected claims divided by total exposure. Their difference equals the total response residual divided by total exposure.

\widehat{\lambda}_{\mathrm{obs},G} =\frac{\sum_{i\in G}Y_i}{\sum_{i\in G}e_i}, \qquad \widehat{\lambda}_{\mathrm{fit},G} =\frac{\sum_{i\in G}\mu_i}{\sum_{i\in G}e_i}, \qquad \widehat{\lambda}_{\mathrm{obs},G}-\widehat{\lambda}_{\mathrm{fit},G} =\frac{\sum_{i\in G}(Y_i-\mu_i)}{\sum_{i\in G}e_i}.

The last expression is also an exposure-weighted average of the individual rate residuals: multiply each \(q_i\) by \(e_i\), add, and divide by total exposure. This identity follows directly because \(e_iq_i=Y_i-\mu_i\). It is not generally equal to the unweighted average of the individual rate residuals. The unweighted average gives a short-exposure record the same influence as a long-exposure record, even though the two represent different amounts of claim opportunity.

An observed-to-expected count ratio can also summarize calibration: \(O/E=\sum_iY_i/\sum_i\mu_i\), provided total expected claims are positive. A ratio above one indicates more observed claims than expected in total; below one indicates fewer. It is a relative count comparison, not a difference in claims per exposure unit. Use the aggregate rate difference when the size of the rate discrepancy is the target.

Worked Example: Aggregating Cell Residuals Correctly

Hypothetical scenario. Three rating cells have exposures of 20, 50, and 30 vehicle-years. Their fitted rates are \(0.04\), \(0.06\), and \(0.08\) claims per vehicle-year, and their observed counts are 1, 2, and 5. Calculate the aggregate observed and fitted rates, their difference, and the observed-to-expected ratio. The method is to aggregate counts and exposure; it avoids averaging cell rates without accounting for their exposure sizes.

Expected counts. For the first cell, \(\mu_1=20(0.04)=0.8\). For the second, \(\mu_2=50(0.06)=3.0\). For the third, \(\mu_3=30(0.08)=2.4\). Total expected claims are \(0.8+3.0+2.4=6.2\), and total exposure is \(20+50+30=100\) vehicle-years.

Aggregate rates. Total observed claims are \(1+2+5=8\). The observed aggregate rate is \(8/100=0.080\) claims per vehicle-year. The fitted aggregate rate is \(6.2/100=0.062\). Their difference is \(0.080-0.062=0.018\) claims per vehicle-year, equivalently a total response residual of \(8-6.2=1.8\) claims divided by 100 vehicle-years.

Check using cell rate residuals. The three rate residuals are \((1-0.8)/20=0.010\), \((2-3.0)/50=-0.020\), and \((5-2.4)/30=0.0867\). Their exposure-weighted average is \([20(0.010)+50(-0.020)+30(0.0867)]/100\), which, using the unrounded third residual \(2.6/30\), is \((0.2-1.0+2.6)/100=0.018\). This agrees with the aggregate rate difference. The unweighted mean of the three cell residuals is approximately \(0.0256\), which does not equal the portfolio rate discrepancy. Finally, \(O/E=8/6.2=1.2903\), indicating 29.03% more observed claims than expected in aggregate. The count difference and the rate difference are both positive and agree with the calculations.

Reading Residual Patterns Against Exposure

A residual plot against exposure can be informative, but the vertical scale matters. Raw response residuals are in claim-count units. Under the Poisson model their variance is \(\mu_i=e_i\lambda_i\); larger-exposure records can therefore have larger-magnitude raw residuals even when the fitted rate is correct. The expected residual remains zero, but equal residual magnitudes do not have equal meaning across exposures.

Rate residuals put discrepancies in comparable rate units, but their variance is \(\lambda_i/e_i\). A scatterplot of rate residual against exposure should therefore not be read as if every point had the same expected spread. Under the model, small-exposure records tend to scatter more widely. A systematic shift in rate residuals across exposure ranges can prompt a check of rate calibration, exposure measurement, or data handling; random scatter alone does not establish a defect. If the model is correct and the Poisson assumptions hold, a Pearson residual is approximately on a common variance scale, although it does not remove every finite-sample or model-fitting complication.

Worked Example: A Pattern That Persists After Dividing by Exposure

Hypothetical scenario. Four records have exposures of 10, 20, 40, and 80 vehicle-years. Each has a fitted rate of \(0.05\) claims per vehicle-year, and the observed counts are 0, 2, 4, and 10. Compare the count residuals and rate residuals, then describe the diagnostic evidence. The method is to calculate expected counts first and inspect the rate-scale discrepancies; this avoids assuming that a larger raw residual alone signals a worse rate fit.

The expected counts are \(10(0.05)=0.5\), \(20(0.05)=1\), \(40(0.05)=2\), and \(80(0.05)=4\). Subtracting these from the observed counts gives response residuals of \(-0.5\), \(1\), \(2\), and \(6\) claims. Dividing each by its exposure gives rate residuals of \(-0.5/10=-0.05\), \(1/20=0.05\), \(2/40=0.05\), and \(6/80=0.075\) claims per vehicle-year.

The observed rates are \(0/10=0\), \(2/20=0.10\), \(4/40=0.10\), and \(10/80=0.125\). The first record is below the fitted rate; the other three are above it, and the later records' rate residuals remain positive. The Pearson residuals are \(-0.5/\sqrt{0.5}=-0.7071\), \(1/\sqrt{1}=1.0000\), \(2/\sqrt{2}=1.4142\), and \(6/\sqrt{4}=3.0000\). The last observation is notably high on this scale.

Across all four records, observed claims total \(16\), expected claims total \(7.5\), and exposure totals \(150\) vehicle-years. Thus the observed aggregate rate is \(16/150=0.1067\), the fitted aggregate rate is \(7.5/150=0.0500\), and their difference is \(8.5/150=0.0567\) claims per vehicle-year. This small hypothetical set shows evidence of an aggregate underprediction and positive residuals in three records, but it does not by itself establish why the pattern occurred. The rate pattern is more relevant than the increase in raw residual size alone; the few observations also limit how strongly a visual pattern can be interpreted.

Common Mistakes and Practical Checks

Comparing raw residuals as though exposure were equal. A four-claim discrepancy on 100 vehicle-years may be a smaller rate error than a one-claim discrepancy on 10 vehicle-years. Divide by exposure when the question is about rate-scale error.
Using an unweighted average of cell rates for a portfolio rate. The overall observed rate is total claims divided by total exposure. Equivalently, cell rates are weighted by their exposures, not given equal weight.
Interpreting every large rate residual as decisive evidence. Rate residual variance decreases with exposure under the Poisson model. Check exposure size and a variance-adjusted residual before deciding whether an individual discrepancy is unusual.
Mixing exposure units. A rate per policy-year multiplied by policy-months gives an incorrect expected count unless the units are converted. Confirm that the offset and rate use matching units.
Assuming the offset solves every data problem. The offset encodes proportional expected count with exposure. If exposure does not measure claim opportunity as intended, or records are dependent in a way the model ignores, the usual diagnostic interpretation may not hold.

Key takeaway: For frequency models, response residuals describe count discrepancies, while rate residuals describe discrepancies per exposure unit. For a portfolio-level rate diagnostic, divide the sum of count residuals by total exposure. Keep exposure units consistent, and interpret residual spread in light of the exposure-dependent variance.

Check Your Understanding

For each problem, calculate on the stated exposure scale and distinguish count error from rate error where requested.

A hypothetical model predicts \(0.12\) claims per vehicle-year. A record has 25 vehicle-years and 4 claims. Find its expected count, response residual, rate residual, and Poisson Pearson residual. (Answer: Expected count \(3\); response residual \(1\) claim; rate residual \(0.040\) claims per vehicle-year; Pearson residual \(1/\sqrt{3}=0.5774\).)
Two hypothetical cells have exposures of 40 and 60 policy-years. Their fitted rates are \(0.05\) and \(0.08\) claims per policy-year, and observed counts are 1 and 8. Find the aggregate observed rate, fitted rate, rate difference, and \(O/E\). (Answer: Observed rate \(0.090\), fitted rate \(0.068\), difference \(0.022\) claims per policy-year, and \(O/E=9/6.8=1.3235\).)
Three records have exposures of 5, 25, and 100 vehicle-years; each has fitted rate \(0.02\) claims per vehicle-year, and observed counts are 0, 1, and 5. Find all three rate residuals and the aggregate rate residual. (Answer: Individual rate residuals \(-0.020\), \(0.020\), and \(0.030\) claims per vehicle-year; aggregate rate residual \(3.4/130=0.0262\) claims per vehicle-year.)
A frequency model has a correct fitted rate and Poisson variance assumptions. Why can response residuals become more spread out as exposure increases, while rate residuals can be more spread out at low exposures? (Answer: Count-residual variance is \(e_i\lambda_i\), which increases with exposure; rate-residual variance is \(\lambda_i/e_i\), which decreases with exposure.)
A record has 24 policy-months of exposure and a fitted annual rate of \(0.06\) claims per policy-year. Find the correct expected count and the erroneous count obtained by multiplying 24 directly by the annual rate. (Answer: Correct exposure is \(24/12=2\) policy-years, so expected count is \(2(0.06)=0.12\); the unit-mismatched calculation gives \(24(0.06)=1.44\), twelve times too large.)

What You'll Learn