End-To-End Model Selection And Indication Workshop

From Model Choice to Monitored Rates

A GLM analysis is not complete when one candidate has the smallest training deviance. A ratemaking decision must also account for validation performance, estimate stability, and whether the resulting rates can be implemented. The selected model’s predictions then need to be translated into indicated premiums, adjusted if implementation constraints apply, and monitored after introduction.

The hypothetical portfolio below has four rating segments. Exposures are annual units, losses and premiums are in thousands of dollars, and all candidate models use the same response definition and exposure basis. The response is aggregate loss cost, modeled with a log link and an exposure offset. The table’s expected losses are predictions from the candidate ultimately selected; they are not the observed losses.

Compare Candidates Before Indicating Rates

Consider three models fit to the same hypothetical data: M0, a coarse main-effects model; M1, a model with selected nonlinear terms and a controlled interaction; and M2, a more complex model with additional interactions. The dispersion estimate is \(\hat{\phi}=1.5\). For this comparison, use the deviance-based quasi-likelihood score \(D/\hat{\phi}+2k\), where \(D\) is training deviance and \(k\) is the number of estimated parameters. Lower scores are preferred, but the score is only one part of the decision.

Why Validation and Stability Change the Decision

Training fit measures how closely the fitted model describes the data used to estimate it. Holdout or cross-validation performance provides evidence about how well the model may predict data not used for fitting. A flexible model can improve training fit by following random variation as well as signal. If that extra complexity does not generalize, the model may produce weak indications even when its training statistic looks attractive.

Stability asks whether small changes in the data cause large changes in fitted relativities or indicated rates. Instability is especially important when a rating cell has little exposure: a few observations can have disproportionate influence. Stability does not mean every estimate must be identical under resampling. It means the actuary understands the movement, can explain it, and has a defensible plan for any volatile or unusable result.

A useful selection decision therefore considers several kinds of evidence, rather than treating one statistic as an automatic rule:

• Fit: Does the model capture meaningful differences in expected loss?
• Validation: Does predictive performance remain acceptable outside the fitting sample?
• Stability: Do estimated effects and resulting indications withstand reasonable resampling or sensitivity tests?
• Usability: Can the rating structure be explained, administered, and implemented within applicable constraints?

These checks complement, rather than replace, review of the response, exposure offset, link, distributional assumptions, and data. A favorable comparison cannot repair a misdefined response or an incorrect exposure basis.

Translate Predictions Into Premium Indications

For a pure-premium indication, a selected model’s expected losses are converted into required premium using a target loss ratio. The target loss ratio is the proportion of premium allocated to losses under the pricing assumptions. In this workshop it is 65%, applied consistently to every segment. No separate adjustment for trend, expenses, profit, or investment income is being introduced; those assumptions are represented here only through the stated target loss ratio.

Why the Portfolio Change Is Premium-Weighted

The portfolio change follows directly from adding premiums, not from averaging segment percentages equally. Let \(w_j=P_j^{C}/\sum_k P_k^{C}\) be segment \(j\)’s share of current premium. Since \(P_j^{I}=P_j^{C}(1+g_j)\), the total indicated premium is:

Subtract current portfolio premium and divide by that premium. The result is:

Thus the portfolio indication is a current-premium-weighted average of segment indications. In the example, the large negative indication for D does not dominate the portfolio result because D represents only \(300/1{,}800=16.6667\%\) of current premium. The formula applies when segment indications and the portfolio are based on the same premium definitions and are aggregated consistently.

Apply Constraints and Recalculate the Implemented Change

A technically indicated change may not be implemented in full. For this hypothetical exercise, the insurer applies a one-renewal-cycle cap of no more than a 10% increase or decrease to any segment. The constraint is applied to each segment’s indicated percentage change relative to its current premium. It is not a constraint on the total portfolio change.

Monitor Experience After Implementation

Monitoring should test whether the model’s predictions and the implemented rates remain adequate as experience emerges. It should use earned exposure and earned premium for the same period and should allow for the maturity of reported claims. A short-period loss ratio can be distorted by seasonality, large claims, catastrophe losses, or reporting delays, so it is a signal for investigation rather than an automatic instruction to change rates.

Suppose the first quarter after implementation earns one quarter of each segment’s annual exposure: 250, 200, 125, and 50 exposure units. At the implemented premiums, earned premium is one quarter of each annual segment premium: 161.538462, 129.230769, 105.769231, and 67.5 thousand dollars. The total earned premium is 464.038462 thousand dollars. M1’s expected quarterly losses are likewise one quarter of its annual expected losses: 105, 84, 68.75, and 33 thousand dollars, totaling 290.75 thousand dollars.

Build a Monitoring Plan That Can Trigger Investigation

A monitoring plan should specify what is measured, how often it is reviewed, and what level of deviation prompts investigation. For this portfolio, a practical plan could include:

• Actual-to-expected loss by segment: compare losses on a consistent basis with selected-model expected losses, using suitable development or maturity adjustments.
• Earned loss ratio: compare incurred or ultimate loss estimates with earned premium for the same coverage period.
• Exposure and mix: track exposure volume and the distribution of business among segments. A portfolio mix shift can change aggregate results even if segment-level performance is stable.
• Relativity and data checks: look for unexpected changes in rating-variable distributions, coding, claim reporting, and segment assignments.
• Predefined review thresholds: set thresholds for further analysis based on the business context and expected volatility; a threshold is an investigation trigger, not an automatic rate action.

Interpretation should distinguish model error from implementation effects. A segment can show a high actual-to-expected ratio because its model prediction is weak, because claims have not matured evenly, or because the mix within the segment has changed. Separately, a constrained rate can produce a different expected loss ratio from the target loss ratio even if the model’s expected losses are accurate. Monitoring both loss performance and earned premium helps identify which issue may be driving the result.

A complete decision trail records the candidate comparison, reasons for selection, key assumptions, indicated rates, constraints applied, and monitoring actions. This makes the analysis reproducible and helps explain why the implemented portfolio change differs from the unconstrained model indication. This integrated workflow is aligned with the CAS Exam 8 (Advanced Ratemaking) syllabus.