From Claim Counts to a Defensible Model
A claim-rate analysis must account for how much exposure each observation represents. Ten claims from 100 policy-years do not indicate the same rate as ten claims from 1,000 policy-years. An exposure offset builds that difference into the model, while weights can represent grouped observations when the response is recorded as a rate. Once the model is specified, candidate GLMs can be compared using fit evidence and a clear account of their assumptions.
The case below is hypothetical. Each row summarizes independent policy groups, with claim counts and earned exposure. The aim is not merely to find the model with the smallest deviance: a defensible choice also checks that the response, offset, and weights match the data structure, and that any extra terms are supported.
This specification assumes that expected claims are proportional to exposure, conditional on the predictors, and that the Poisson mean equals the conditional variance. Exposure must be positive for rows in this rate model. A zero-exposure group supplies no information about a rate and should not be included as an ordinary observation with \(\log(E_j)\). If the exposure unit changes, the offset and rate interpretation must change consistently.
Worked Example: Comparing a Common Rate With Group-Specific Rates
Worked Example: Comparing a Common Rate With Group-Specific Rates
A hypothetical insurer summarizes experience for three regions. The counts are 12, 18, and 36 claims, and the corresponding exposures are 100, 200, and 300 policy-years. Compare a common-rate model with a model that gives each region its own rate.
Method and specification. Use independent Poisson counts with a log link and offset \(\log(E_j)\). The common-rate model has \(\lambda_j=\lambda\); the region model has a separate rate for each region. The maximum-likelihood estimate of a common rate is total claims divided by total exposure:
The common-rate model therefore predicts \(100(0.11)=11\), \(200(0.11)=22\), and \(300(0.11)=33\) claims. The observed region rates are \(12/100=0.12\), \(18/200=0.09\), and \(36/300=0.12\).
For Poisson counts, the deviance is \(D=2\sum_j\{Y_j\log(Y_j/\widehat{\mu}_j)-(Y_j-\widehat{\mu}_j)\}\), with a zero-count term defined by continuity as zero. Here the fitted counts and observed counts each sum to 66, so the linear terms sum to zero. The calculation for the common-rate model is:
The three-rate model fits each cell exactly: its fitted counts are 12, 18, and 36, so its deviance is zero. It uses two additional parameters compared with the common-rate model. The likelihood-ratio statistic is the deviance difference, \(1.129-0=1.129\), compared with a chi-square distribution with 2 degrees of freedom under the usual large-sample conditions. The approximate p-value is 0.569. This is not compelling evidence that the three rates differ, so the more parsimonious common-rate model is reasonable on these data. The region model is saturated for these three observations, however; its perfect fit alone is not evidence that it will predict better.
Check. The common-rate fitted counts sum to \(11+22+33=66\), matching the observed total. The group-specific model's rates reproduce all three observed counts.
Common mistake avoided: averaging the three observed rates equally would give \(0.11\) here by coincidence; in general it does not give the pooled rate. The correct common-rate estimate weights each group's experience through its exposure.
Grouped Rates and Exposure Weights
A grouped data set may store the observed rate \(R_j=Y_j/E_j\) rather than the count. Under a log-link Poisson GLM, fitting those rates with prior weights \(E_j\) gives the same coefficient estimates as fitting counts with \(\log(E_j)\) as an offset, provided the model terms and rows correspond. The weight represents how much exposure supports the rate; it is not a replacement for the offset in a count model.
The reason for the equivalence can be seen directly from the score. For the count model, the log-likelihood terms depending on the coefficients are \(Y_j\log(\lambda_j)-E_j\lambda_j\). Differentiating with respect to \(\boldsymbol{\beta}\), using \(\log(\lambda_j)=\mathbf{x}_j^\mathsf{T}\boldsymbol{\beta}\), gives a contribution \(\mathbf{x}_j(Y_j-E_j\lambda_j)\). For a rate response with weight \(E_j\), the Poisson GLM score contribution is \(E_j\mathbf{x}_j(R_j-\lambda_j)\). Since \(E_jR_j=Y_j\), these contributions are equal. This explains why the coefficient estimates agree; it does not make the two data representations interchangeable for every reported likelihood quantity.
Worked Example: Fitting a Grouped Rate With Exposure Weights
A hypothetical portfolio has two grouped records. Group A has exposure 100 and 10 claims; Group B has exposure 300 and 60 claims. Fit a common rate using the grouped rates and exposure weights, and compare it with the unweighted average of the two rates.
Method. Calculate each rate, then use exposure as the prior weight for a Poisson rate model with an intercept only. The rates are \(R_A=10/100=0.10\) and \(R_B=60/300=0.20\). The weighted intercept-only estimate is:
The unweighted average would be \((0.10+0.20)/2=0.15\), which gives the smaller group the same influence as the group with three times its exposure. The fitted expected counts under the common rate are \(100(0.175)=17.5\) and \(300(0.175)=52.5\), totaling 70, the observed total.
In R, the count representation is specified as a Poisson GLM with claims as the response and \(\log(\text{exposure})\) as the offset. The grouped-rate representation uses the rate as the response and exposure as prior weights. For model comparison by AIC, fit every candidate to the same response representation. The weighted-rate fit is useful for estimating the same coefficients, but its likelihood-based AIC should not be casually compared with an AIC from the count representation.
Check. The weighted fitted rate multiplied by total exposure gives \(0.175(400)=70\), equal to total claims. The unweighted average fails this check because \(0.15(400)=60\).
Common mistake avoided: using weights equal to the number of rows, or treating exposure as a freely estimated predictor, does not encode the intended amount of rate information.
Comparing Nested Models and Justifying a Choice
For nested Poisson models fitted to the same counts and exposures, the smaller model is contained in the larger one when its parameter restrictions are imposed. The likelihood-ratio statistic is the difference in residual deviances, \(D_{\text{small}}-D_{\text{large}}\). With standard regularity conditions and a correctly specified Poisson variance, this is approximately chi-square with degrees of freedom equal to the number of added parameters. The approximation can be unreliable with sparse data or boundary problems, so it should not be treated as an automatic verdict.
AIC can also compare candidate models fit to the same response under a comparable likelihood: \(AIC=-2\ell+2k\), where \(\ell\) is the maximized log-likelihood and \(k\) is the number of estimated parameters. Lower AIC balances fit against complexity. AIC and a nested likelihood-ratio test answer related but different questions; neither substitutes for checking the response definition, offset, data support, and intended use.
Worked Example: Is an Interaction Supported?
A hypothetical fleet portfolio has four equally exposed cells, each with 100 policy-years. The observed counts are 10 for region A/light vehicles, 20 for region A/heavy vehicles, 20 for region B/light vehicles, and 60 for region B/heavy vehicles. Compare a model with region and vehicle main effects against one that also includes their interaction.
Method. Both models use independent Poisson counts and the same offset \(\log(100)\). With equal exposures, fitting the main-effects model gives fitted counts equal to the independence-model estimates calculated from the row and column totals. The row totals are 30 and 80, the column totals are 30 and 80, and the grand total is 110. Thus the fitted counts are \(30(30)/110=90/11\), \(30(80)/110=240/11\), \(80(30)/110=240/11\), and \(80(80)/110=640/11\), or approximately 8.182, 21.818, 21.818, and 58.182.
The interaction model has one additional parameter and fits the four cell counts exactly. Its deviance is zero. For the main-effects model, the linear terms in the deviance sum to zero because observed and fitted totals are both 110. Therefore:
The deviance difference is \(0.745-0=0.745\), with 1 degree of freedom. The approximate p-value is 0.39, so these observations do not provide strong evidence that the interaction improves fit. The main-effects model is a defensible, simpler choice, subject to the modeling purpose and other checks. The interaction model's zero deviance is expected because it has one parameter per cell; it does not establish that the interaction generalizes.
Check. The four fitted main-effect counts sum to \((90+240+240+640)/11=110\), matching the observed total. The comparison uses the same four cells, response, and offset in both models.
Common mistake avoided: comparing raw deviances without accounting for additional parameters, or calling the interaction useful simply because its deviance is lower. A larger nested model cannot fit worse; the question is whether its improvement is persuasive given the added complexity.
A Practical Comparison Sequence
A model comparison is most convincing when each step can be traced to the data and the question being asked.
Confirm that each response is a claim count, or explicitly construct a rate from count divided by positive exposure.
For count data, use \(\log(\mu_j)=\log(E_j)+\mathbf{x}_j^\mathsf{T}\boldsymbol{\beta}\). Check exposure units and keep its coefficient fixed at one.
Keep rows, response representation, and offset consistent. Add terms only when they address a stated modeling question.
Use deviance differences for nested models under suitable conditions, or AIC for likelihoods on the same basis. Also consider parsimony and whether the cells support the proposed terms.
Common Traps and Interpretation
- Using the wrong exposure unit. If exposure is converted from policy-years to policy-months, the offset changes. Expected counts must remain consistent after the rate scale is adjusted; mixing units changes the model meaning.
- Confusing an offset with a weight. The count model uses exposure in its mean through a fixed-coefficient log offset. In a rate-response representation, exposure is a prior weight. These roles are connected but not interchangeable within the same specification.
- Comparing incompatible AIC values. Keep the response representation and likelihood basis consistent across candidate models. In particular, do not assume a weighted-rate AIC is numerically identical to a count-model AIC.
- Using a deviance test without checking the comparison. The models must be nested and based on the same observations and response. The chi-square approximation also depends on regularity and adequate information in the data.
- Choosing the lowest deviance automatically. Adding parameters cannot increase a nested model's maximized likelihood. Consider whether the improvement justifies the complexity and whether the fitted structure is useful for the intended task.
Check Your Understanding
Work each problem by identifying the model structure before calculating or interpreting the result.
- A hypothetical group has 8 claims over 80 policy-years and another has 24 claims over 120 policy-years. Under a common-rate Poisson model with a log-exposure offset, calculate the estimated rate and each group's fitted claim count. (Answer: Rate \(=32/200=0.16\) claims per policy-year; fitted counts are 12.8 and 19.2.)
- Two grouped rates are 0.05 on exposure 100 and 0.15 on exposure 300. Find the intercept-only rate estimate from a Poisson rate model using exposure prior weights. (Answer: \((100(0.05)+300(0.15))/400=0.125\).)
- A nested model has residual deviance 18.4 and 5 estimated parameters; a larger model has residual deviance 12.1 and 8 estimated parameters. State the likelihood-ratio statistic and its degrees of freedom, assuming both fits use the same count data and offset. (Answer: \(18.4-12.1=6.3\), with 3 degrees of freedom.)
- A count model uses exposure measured in policy-years. A data extract instead reports exposure in policy-months. What must be checked before using the new offset? (Answer: Convert consistently: policy-month exposure is 12 times policy-year exposure, so the rate scale and offset must be adjusted together to preserve the same expected counts.)
- A grouped-rate GLM uses exposure as prior weights and has the same predictors as a count GLM with a log-exposure offset. Which result is expected to agree, and what comparison needs extra care? (Answer: Coefficient estimates and weighted deviance agree under the stated setup; AIC comparisons require a consistent response representation and likelihood basis.)