Why Candidate Definitions Matter
A model comparison is informative only when the candidates answer the same ratemaking question. If one candidate predicts claim counts per earned policy-year and another predicts counts per written policy, a difference in predictions may reflect the denominator rather than the model structure. The same problem arises when models use different data periods, eligibility rules, or target populations.
Defining comparable candidates means fixing the inputs and the prediction question before fitting or comparing models. Candidate models may differ in their rating variables or functional forms, but the response, exposure basis, data period, and target population should be aligned unless the purpose of the analysis is specifically to study one of those changes. A comparison on identical validation data, as covered in earlier material on validation, follows the same principle: hold the comparison basis fixed.
Specify the Common Modeling Basis
For a frequency model, let \(N_i\) be the claim count for observation \(i\), and let \(E_i\) be its earned exposure in the chosen unit, such as policy-years. A standard Poisson GLM specification is
Here \(\log(E_i)\) is an offset: its coefficient is fixed at one. The model therefore represents the expected claim count as exposure multiplied by a modeled claim rate. If the model uses a log link, the offset and the response must use consistent exposure units. Changing exposure from policy-years to policy-months changes its numerical value by a factor of 12; the modeled rate must be converted consistently as well.
For a pure-premium model, one possible response is \(Y_i=L_i/E_i\), where \(L_i\) is the loss amount and \(E_i\) is exposure. A GLM for this response can use exposure weights, so that observations with more exposure contribute correspondingly more information. This is not the same specification as modeling aggregate loss with an offset. A candidate comparison must state which response and modeling convention it uses, and apply that convention consistently.
| Specification item | What must be fixed | Question to ask |
|---|---|---|
| Response | Outcome and its definition | Are all candidates predicting the same quantity? |
| Exposure | Unit, earning rule, and treatment of partial periods | Does a unit of exposure mean the same thing everywhere? |
| Data period | Accident, policy, or calendar dates, as appropriate | Do all candidates use the same experience window? |
| Eligibility | Rules for including observations and records | Are any risks missing from one candidate but not another? |
| Target population | Portfolio or fixed mix used for indications | Are predictions summarized over the same mix? |
The response distribution and link should also be held fixed when the purpose is to isolate the effect of a predictor or model structure. If candidates deliberately use different distributions or links, that is a legitimate broader modeling question, but the difference in fit or indication cannot be attributed only to the predictors. Documenting the intended comparison prevents that ambiguity.
Worked Examples
Worked Example: Comparing Frequency Candidates on One Basis
A hypothetical insurer is comparing two annual claim-frequency candidates for eligible personal auto risks during calendar years 2024 and 2025. The training records use earned policy-years. Both candidates use the same eligible risks, claim-count response, period, and Poisson distribution with log link and exposure offset. Candidate A uses territory; Candidate B uses territory and vehicle age. Their fitted annual claim rates for three territory groups are shown below. The target portfolio has fixed shares of 40%, 35%, and 25% in groups 1, 2, and 3. Calculate each candidate's indicated frequency for that target mix.
Method and assumptions. The response is claims per risk record, with exposure \(E_i\) measured in policy-years. Conditional counts are Poisson, with mean \(\mu_i=E_i\lambda_i\), where \(\lambda_i\) is the annual claim rate. No interest or mortality assumption is relevant. The supplied predictions are annual claim rates, not claim counts.
Step 1: Check the basis. Both sets of predictions refer to the same target groups, use claims per policy-year, and apply to the same data period and eligibility definition. The candidate models differ in predictors, which is the intended comparison.
Step 2: Apply the fixed target shares. Candidate A predicts rates of 0.50, 0.45, and 0.55. Its target-mix frequency is \(0.40(0.50)+0.35(0.45)+0.25(0.55)=0.200+0.1575+0.1375=0.495\) claims per policy-year.
Candidate B predicts rates of 0.48, 0.52, and 0.62. Its target-mix frequency is \(0.40(0.48)+0.35(0.52)+0.25(0.62)=0.192+0.182+0.155=0.529\) claims per policy-year.
Step 3: State the comparison. Candidate B's indicated frequency exceeds Candidate A's by \(0.529-0.495=0.034\) claims per policy-year. Relative to Candidate A, this is \(0.034/0.495=0.06869\), or 6.9% when rounded to one decimal place. This calculation compares indications on one fixed target mix; it does not by itself establish which model predicts better.
Common mistake avoided. Using each candidate's own portfolio mix would confound its predicted rates with the mix on which they were summarized. The common target shares make the indicated frequencies comparable.
Worked Example: Aligning Pure-Premium Response and Exposure Weights
A hypothetical property portfolio has two construction groups for the same 2025 accident year. Group A has earned exposure of 12 policy-years and losses of $18,000; Group B has 14 policy-years and losses of $28,000. Two candidate models use the same Gamma response definition, log link, records, and exposure weights. Candidate C predicts annual pure premiums of $1,600 for Group A and $2,000 for Group B. Candidate D predicts $1,700 and $1,900, respectively. A fixed target portfolio contains 10 policy-years of Group A and 15 of Group B. Calculate each candidate's target indicated pure premium and the observed portfolio pure premium.
Method and assumptions. The response is \(Y_i=L_i/E_i\), in dollars per policy-year, and the GLM uses exposure weights \(E_i\). For the illustrative Gamma GLM, responses are positive and the conditional variance is proportional to the square of the mean; predictions are conditional means. Both candidates have the same distribution, link, observation period, and target definition. No interest or mortality assumption is relevant.
Step 1: Verify the observed responses. For Group A, \(Y_A=\$18{,}000/12=\$1{,}500\) per policy-year. For Group B, \(Y_B=\$28{,}000/14=\$2{,}000\) per policy-year. The observed aggregate pure premium is total losses divided by total exposure: \(\$18{,}000+\$28{,}000=\$46{,}000\), and \(12+14=26\) policy-years, giving \(\$46{,}000/26=\$1{,}769.23\) per policy-year, rounded to the nearest cent.
Step 2: Standardize Candidate C. Its predicted total loss for the target mix is \(10(\$1{,}600)+15(\$2{,}000)=\$16{,}000+\$30{,}000=\$46{,}000\). Divide by the target's \(10+15=25\) policy-years: \(\$46{,}000/25=\$1{,}840.00\) per policy-year.
Step 3: Standardize Candidate D. Its predicted total loss is \(10(\$1{,}700)+15(\$1{,}900)=\$17{,}000+\$28{,}500=\$45{,}500\). Thus its target indicated pure premium is \(\$45{,}500/25=\$1{,}820.00\) per policy-year.
Check. Candidate C exceeds Candidate D by \(\$1{,}840-\$1{,}820=\$20.00\) per policy-year on this target mix. The observed pure premium uses the observed exposure mix, whereas the two candidate indications use the specified target mix; those are different summaries with different denominators.
Common mistake avoided. Averaging the two group predictions equally would give neither candidate's target indication, because the target has 10 policy-years in Group A and 15 in Group B. Exposure weighting and dividing by total target exposure preserve the stated target mix.
Why Exposure Weighting Gives the Aggregate Rate
The exposure-weighted average is not an arbitrary convention. Suppose a candidate predicts a rate \(\widehat{\lambda}_g\) for each group \(g\), and the target mix has exposure \(E_g\) in each group. The predicted aggregate count is the sum of predicted counts, \( \sum_g E_g\widehat{\lambda}_g\). An aggregate frequency is that expected count divided by the total exposure, so
The weights \(E_g/\sum_g E_g\) sum to one, so this is a weighted average of group rates. The same reasoning gives the target pure premium when \(\widehat{\lambda}_g\) is replaced by a predicted loss cost per exposure unit. This standardization is useful when candidates' segment predictions are to be summarized for a common portfolio indication.
Worked Example: Detecting a Different Fitting Sample
A hypothetical insurer reviews an intercept-only frequency baseline for eligible equipment policies in 2025. Four territory records have claim counts and earned exposures as follows: North, 2 claims and 4 policy-years; South, 3 and 6; East, 4 and 8; West, 3 and 2. Candidate A has complete territory data. Candidate B has a missing predictor for West and was fitted after dropping that record. Compare the fitted intercept-only annual frequencies and identify a comparable-sample remedy.
Method and assumptions. Each record's claim count is conditionally Poisson, with mean equal to exposure times a common annual rate. Both models use the same log link and exposure offset. The intended comparison is the effect of candidate predictors, not a change in the fitting population.
Step 1: Calculate Candidate A's intercept-only rate. Its total count is \(2+3+4+3=12\) claims, and its total exposure is \(4+6+8+2=20\) policy-years. The fitted rate is \(12/20=0.60\) claims per policy-year.
Step 2: Calculate Candidate B's rate on its actual fitting sample. Dropping West leaves \(2+3+4=9\) claims and \(4+6+8=18\) policy-years. Its fitted rate is \(9/18=0.50\) claims per policy-year. The 0.10 difference is not evidence of a predictor effect: the candidates were fitted to different records.
Step 3: Restore a common basis. One remedy is to fit both candidates on the same 18 policy-years, if restricting the analysis to records with complete predictors is appropriate for the intended target. The common-sample intercept-only baseline is then \(9/18=0.50\) for each. Alternatively, if missingness has a defensible treatment, define it consistently and retain the eligible records in both candidates.
Check and common mistake avoided. The all-record estimate is \(12/20=0.60\), not \(12/18\): West contributes both three claims and two exposure-years. Dividing all claims by only the complete-case exposure would mix numerators and denominators from different samples. A common comparison requires matching both.
Common Comparability Traps
Before interpreting a candidate comparison, record its specification in a compact audit table or model plan. This makes changes in basis visible before they are mistaken for model improvements. In particular, check the following:
- Mixed exposure units. A policy-month is one-twelfth of a policy-year only under a consistent earning convention. Convert both the offset and reported rate to the same basis.
- Different response definitions. Claim counts, claim frequency, severity, aggregate loss, and pure premium are not interchangeable responses. State whether the target is per claim, per exposure, or an aggregate amount.
- Different periods. Calendar-year claims and accident-year claims can include different experience. Align the period and any associated exposure before comparing candidates.
- Unmatched records. Missing predictors, eligibility filters, or data-cleaning rules can change the sample. Use common records or explicitly define and justify a consistent treatment.
- Candidate-specific portfolio mixes. If the goal is to compare indicated rates, apply each model's predictions to the same target exposure distribution. Otherwise, a change in business mix can appear to be a rate change.
- Silent changes in model assumptions. If the response distribution, link, or offset treatment changes, document that as part of the candidate definition rather than attributing the result solely to a predictor change.
A common basis does not require candidates to have identical predictors or identical predictions. It makes the interpretation of differences more precise: after inputs and target are fixed, a difference can be associated with the stated modeling choices, subject to the usual uncertainty and validation considerations.
Check Your Understanding
For each problem, identify the comparable basis or calculate the requested quantity.
- A Poisson frequency model has 18 claims over 30 earned policy-years. Another report gives 18 claims over 360 earned policy-months. What is the annual frequency on either consistent basis? (Answer: 0.600 claims per policy-year, or 0.0500 per policy-month.)
- A pure-premium model has predicted rates of $900 and $1,200 for two groups. The target has 20 and 30 policy-years, respectively. What is the target indicated pure premium? (Answer: \((20(900)+30(1{,}200))/50=\$1{,}080.00\) per policy-year.)
- Two intercept-only candidates use the same Poisson model. One has 15 claims over 25 policy-years; the other has 12 claims over 20 policy-years because records were dropped. What are their fitted rates, and can their difference be attributed to model structure? (Answer: 0.600 and 0.600 claims per policy-year; no, the samples differ, even though the rates happen to match.)
- Candidate A predicts rates of 0.40 and 0.70 for two risk groups; Candidate B predicts 0.50 and 0.60. A common target has exposure shares of 60% and 40%. Calculate both target frequencies and their difference. (Answer: A is \(0.60(0.40)+0.40(0.70)=0.520\); B is \(0.60(0.50)+0.40(0.60)=0.540\); B is higher by 0.020 claims per exposure unit.)
- A model uses 2023–2025 accident-year losses and earned exposure, while its proposed candidate uses 2024–2025 calendar-year counts and written exposure. Name four basis elements that must be aligned before interpreting a difference as a model-choice effect. (Answer: response definition, exposure basis and earning convention, experience period, and eligible records/target population.)