What Does an Added Parameter Need to Earn?
A larger generalized linear model can usually fit its training data at least as well as a nested smaller model. That fact alone does not make the larger model preferable: its extra parameters consume degrees of freedom, may not improve predictions on new experience, and can produce rate indications that are difficult to explain or implement. The relevant question is whether the incremental value of the added complexity is meaningful for the intended ratemaking use.
This assessment is broader than a single statistical criterion. In-sample fit describes how well the model represents the data used to estimate it. Validation performance assesses prediction on data not used for fitting. Segment-level summaries show whether the added structure changes indications in a way that is material, concentrated, or operationally consequential. A model can improve one dimension without improving all three.
For a clean comparison, candidates should use the same response definition, observations, exposure treatment, likelihood, and relevant data preparation. A nested comparison means that the reduced model is contained in the full model: setting the added parameters to particular values recovers the reduced model. These conditions make differences in fit interpretable as consequences of adding parameters rather than of changing the data or modeling setup.
Relating Fit Improvement to Complexity
For two candidates fitted by maximum likelihood to the same data under the same likelihood, let \(D_R\) and \(D_F\) denote the reduced and full model deviances, and let \(k_R\) and \(k_F\) denote their respective numbers of estimated parameters. A smaller deviance indicates a better fit to the estimation data. The full model has an in-sample deviance improvement of \(D_R-D_F\), but its extra parameters are not free.
For candidates with a common likelihood and the same observations, the AIC difference can be written using deviance:
The full model is favored by AIC when this difference is negative. This comparison does not mean that every added parameter must individually clear a fixed deviance threshold; it means that the improvement in fit is weighed against an explicit complexity penalty. BIC uses a different penalty, \(k\ln(n)\), where \(n\) is the number of observations under the convention used for the comparison. Neither criterion, on its own, tells whether a change is useful for ratemaking.
The AIC difference formula follows directly from its definition. For a maximum-likelihood model, \(\operatorname{AIC}=-2\ell+2k\), where \(\ell\) is the maximized log likelihood. For two models on the same data and likelihood, deviance differs from \(-2\ell\) only by a constant shared by both candidates. Subtracting the reduced AIC from the full AIC therefore cancels that constant and gives the displayed formula. The added parameters increase the penalty by \(2(k_F-k_R)\), so improved fit must offset that cost for the full model to have lower AIC.
Worked Example: A Fit Gain That Does Not Offset the AIC Penalty
A hypothetical insurer fits nested Poisson claim-count GLMs with a log link and a log-exposure offset. Conditional on the rating variables, claim counts are assumed independent Poisson observations; both models use the same 240 policies and the same response and offset. The reduced model has 5 estimated parameters and deviance 500.0. The full model adds two parameters and has deviance 491.6. Compare the candidates by AIC.
Method. Apply the full-minus-reduced AIC difference. A negative result favors the full model; a positive result favors the reduced model. The common mistake to avoid is treating a decrease in deviance as proof that the more complex model is preferred.
The deviance change is \(D_F-D_R=491.6-500.0=-8.4\). The parameter-count change is \(k_F-k_R=2\), so the AIC penalty increase is \(2(2)=4\). Thus:
The difference is negative, so AIC favors the full model by 4.4 points. The full model’s 8.4-point deviance improvement exceeds the 4-point penalty in this comparison. This conclusion is specifically an AIC comparison, not proof that the added parameters improve future predictions or produce suitable rates.
Check. If the reduced AIC were 510.0, the full AIC would be \(510.0-4.4=505.6\), consistent with the full model having lower AIC.
Worked Example: When a Smaller Deviance Still Favors the Reduced Model
Consider another hypothetical nested Poisson claim-count comparison, again with a log link, log-exposure offset, independent conditional counts, and the same 240 observations for both candidates. The reduced model has 4 parameters and deviance 320.0. The full model adds 2 parameters and has deviance 311.6. Calculate the full-minus-reduced AIC difference and state the preference.
Method. Combine the deviance change with the penalty for the two added parameters. The trap is to use the deviance change alone, or to subtract the penalty rather than add it to the full-minus-reduced change.
The deviance change is \(311.6-320.0=-8.4\). The parameter difference is \(2\), so the AIC penalty difference is \(2(2)=4\). Therefore:
AIC favors the full model. Its deviance improvement is large enough to offset the penalty. The calculation is identical in form to the first example, but the parameter counts and deviances here are new hypothetical values; the conclusion follows from their difference, not from their absolute scale.
Check. Subtracting the displayed change from the reduced AIC gives a full AIC 4.4 points lower. The sign and interpretation agree.
Test Prediction Separately from Training Fit
A training-fit criterion uses the observations that helped estimate the parameters. It therefore cannot by itself show that added structure predicts better on new experience. For predictive evidence, score both candidates on the same validation observations, using the same response definition, exposure treatment, and scoring rule. A lower validation deviance is better when deviance is the chosen score, but its size should be interpreted in context: the candidate, exposure volume, and segment distribution all affect the result.
A useful comparison records both the overall score difference and its distribution across rating segments. Segment-level score changes can reveal that an overall gain comes from a small portion of the portfolio, or that a candidate improves most segments while materially worsening another. When the observed difference is small relative to the uncertainty in the comparison, it should not be presented as decisive evidence. The validation comparison should not be confused with an AIC calculation: AIC penalizes fitted-model complexity, whereas a validation score evaluates performance on held-out data.
Worked Example: Reading an Overall Validation Gain by Segment
A hypothetical insurer compares two frequency models on the same held-out policies. Both are Poisson GLMs with a log link and log-exposure offset, and the validation deviance is calculated by the same rule. The reduced model’s validation deviances are 900 for Segment A, 800 for Segment B, and 780 for Segment C. The full model adds two parameters; its corresponding values are 890, 790, and 782. The segment exposures are 1,000, 700, and 300, respectively. A pre-established review rule for this hypothetical analysis treats an overall reduction of at least 12 deviance points as potentially material. Assess the result.
Method. First total the paired segment scores, then examine their changes and exposure scale. Do not apply an AIC parameter penalty to validation deviance. The common trap is to report only the aggregate improvement and overlook a segment where performance worsens.
For the reduced model, total validation deviance is \(900+800+780=2{,}480\). For the full model, it is \(890+790+782=2{,}462\). The overall reduction is \(2{,}480-2{,}462=18\) deviance points, which exceeds the stated 12-point review threshold. The total validation exposure is \(1{,}000+700+300=2{,}000\), so the reduction per 100 exposure units is \(18/(2{,}000/100)=0.9\).
By segment, the changes in deviance (full minus reduced) are \(-10\) for A, \(-10\) for B, and \(+2\) for C. Thus A and B improve, while C worsens. Per 100 exposure units, the reductions are \(10/(1{,}000/100)=1.0\) for A and \(10/(700/100)\approx1.43\) for B; C has an increase of \(2/(300/100)\approx0.67\). The results meet the hypothetical overall review threshold but are not uniformly favorable. The added structure merits further review, including the cause and practical importance of the deterioration in C; the aggregate score alone does not settle the decision.
Check. The segment changes sum to \(-10-10+2=-18\), so the full model’s total deviance is 18 points lower, as calculated above.
Translate Model Changes into Segment Indications
A model-selection decision is for ratemaking, so assess how the added parameters alter the indicated values as well as how they change statistical scores. For an exposure-weighted portfolio summary, if segment \(j\) has exposure \(e_j\) and indicated pure premium \(r_j\), its aggregate indicated pure premium is \(\sum_j e_j r_j/\sum_j e_j\). This weighted mean describes the portfolio total; it does not imply that every segment has a similar change.
A useful summary can include the change in each segment’s indication, the exposure behind that segment, and the aggregate change. Small exposure segments can have large percentage changes and still have limited aggregate weight. Conversely, a modest percentage change in a large segment may have a substantial portfolio effect. These summaries help identify whether an apparent improvement in fit yields indications that merit actuarial review. They do not, by themselves, establish that the changes are justified or implementable.
Worked Example: A Small Portfolio Change Can Hide a Large Segment Change
A hypothetical book has three rating segments with exposures of 600, 300, and 100. A reduced model indicates annual pure premiums of $500, $800, and $1,200 for Segments A, B, and C. A nested full model adds one parameter and indicates $510, $760, and $1,320. Calculate each segment change and both exposure-weighted portfolio indications.
Method. Calculate each segment’s dollar and percentage change, then exposure-weight the rates. The mistake to avoid is taking an unweighted average of the three segment indications, which would give the segments equal weight despite their different exposures.
For Segment A, the change is \($510-$500=$10\), or \($10/$500=2\%\). For Segment B, it is \($760-$800=-$40\), or \(-$40/$800=-5\%\). For Segment C, it is \($1{,}320-$1{,}200=$120\), or \($120/$1{,}200=10\%\).
The total exposure is \(600+300+100=1{,}000\). The reduced model’s exposure-weighted indicated amount is:
The full model’s weighted amount is:
The portfolio indication increases by \($666-$660=$6\), or \($6/$660\approx0.009091=0.91\%\). However, the segment movements are +2%, -5%, and +10%. Segment C has only 100 of the 1,000 exposure units, but its 10% change is material enough to inspect rather than dismiss because the portfolio average moved by less than 1%.
Check. The segment-level dollar changes in exposure-weighted amounts are \(600(10)=\$6{,}000\), \(300(-40)=-\$12{,}000\), and \(100(120)=\$12{,}000\). They sum to a net increase of $6,000, which divided by 1,000 exposure units gives the $6 portfolio increase.
A Practical Incremental-Value Review
The evidence should be considered together rather than reduced to a rule that “more fit is always better” or “fewer parameters are always better.” An added parameter can be worthwhile when it provides a credible and repeatable predictive improvement, produces a useful distinction in risk, and leads to indications that can be explained and reviewed. It may be hard to justify when the training-fit gain is small, validation performance is unchanged or inconsistent, and the rate movements are noisy or concentrated without supporting evidence.
Check that the candidate models use the same response, observations, exposure treatment, likelihood, and scoring definitions. For a nested comparison, verify that the reduced model is contained in the full model.
Report the deviance change and, where its assumptions are appropriate, the information-criterion change. State the number of added parameters and interpret the direction of each difference correctly.
Compare candidates on identical validation data and the same score. Describe the scale and segment distribution of the difference, and avoid treating a small uncertain change as decisive.
Summarize exposure-weighted aggregate and segment changes. Look for important movements that an overall mean could conceal, and assess whether the added distinction has a defensible ratemaking interpretation.
A frequent exam trap is to conflate nested-model fit improvement with overall model value. The full model’s deviance cannot increase when it is fitted by maximum likelihood in a nested setting, but this mechanical fit advantage is not a guarantee of better validation performance or more suitable rates. Another trap is to reverse the sign of an information-criterion difference: if the defined difference is full minus reduced, a positive AIC difference favors the reduced candidate.
Check Your Understanding
For each hypothetical comparison, use the stated assumptions and keep fit, prediction, and rate-impact evidence distinct.
- Two nested Poisson GLMs use the same policies and exposure offset. The reduced model has 6 parameters and deviance 410; the full model has 8 parameters and deviance 400. Calculate the full-minus-reduced AIC difference and state which candidate AIC favors. (Answer: \(-10+2(2)=-6\); AIC favors the full model by 6 points.)
- On the same held-out data, a reduced model has validation deviance 1,720 and a full model has 1,713. The total validation exposure is 1,400 units. Calculate the reduction per 100 exposure units, and say whether the full model has lower validation deviance. (Answer: The full model is lower by 7 points; the reduction is \(7/(1{,}400/100)=0.50\) deviance points per 100 exposure units.)
- A reduced model indicates $400 for a segment with 800 exposure units; a full model indicates $420. A second segment has 200 exposure units and indications of $900 and $840, respectively. Calculate the exposure-weighted indications under both models and the aggregate percentage change. (Answer: Reduced \(=(800(400)+200(900))/1{,}000=\$500\); full \(=(800(420)+200(840))/1{,}000=\$504\); increase \(=\$4/\$500=0.80\%\).)
- A nested model adds three parameters and reduces training deviance by 5 points. On identical validation data it has a higher score than the reduced model, while segment indications change by +1% in most segments and +18% in a low-exposure segment. Is there sufficient evidence from these facts alone to conclude that the added parameters earn their complexity? (Answer: No. Training fit improves, but the stated validation score worsens; the concentrated +18% indication also warrants investigation rather than being justified by the small training-deviance reduction.)
- For a common-likelihood comparison with \(n=200\), the full model adds two parameters and reduces deviance by 7. Compute the full-minus-reduced BIC difference using \(D_F-D_R+(k_F-k_R)\ln(n)\), and identify the preferred model. (Answer: \(-7+2\ln(200)\approx-7+10.597=3.597\); BIC favors the reduced model.)