Why Exposure Weighting Matters
A lift table sorts validation observations by a model’s predicted outcome and summarizes performance across ordered groups. It is useful for seeing whether risks assigned higher predicted costs also tend to have higher observed costs. But the table can mislead if its group averages give a small-exposure policy the same influence as a large-exposure policy. For ratemaking validation, the relevant comparison is generally between aggregate outcomes and aggregate predictions over the exposure represented by each group.
The outcome must be defined consistently. For a loss-cost table, let policy \(i\) have exposure \(e_i\), observed loss \(L_i\), and predicted loss cost \(p_i\) per unit of exposure. Here, \(L_i\) is the observed loss for the validation period and \(p_i\) is the model’s predicted loss per exposure unit for that same period and basis. The examples use hypothetical validation data. A frequency table can instead use observed claim counts and predicted claim frequencies.
A lift table often reports each group’s observed and predicted loss cost, along with lift relative to the portfolio average. Define observed lift as the group’s observed loss cost divided by the portfolio’s observed loss cost. Define predicted lift in the same way using predicted loss costs. These lifts describe relative levels: for example, observed lift of 1.5 means the group’s observed loss cost is 1.5 times the portfolio’s. The observed-to-predicted ratio, abbreviated O/P, is a separate comparison: group observed total loss divided by group predicted total loss. O/P near 1 indicates aggregate agreement for that group; it is not the same quantity as lift.
A zero overall observed loss cost makes observed lift undefined. A group with positive exposure but zero observed loss has observed loss cost and observed lift equal to zero when the portfolio has positive observed loss. Its O/P is zero if its predicted total is positive. Do not divide by the group’s observed loss: predicted divided by observed is undefined for that group.
Why the Weighted Average Preserves the Total
The weight is exposure because a predicted loss cost is a rate per exposure unit. Multiplying that rate by exposure gives the prediction on the aggregate-loss scale. Summing those products and dividing by total exposure returns the group’s average predicted cost per exposure unit:
The final expression is a weighted average: the weights are each observation’s share of group exposure, and they sum to one. This is why the weighted average reproduces the aggregate prediction. The unweighted average, \(\sum p_i/n_G\), instead gives every record equal weight and generally does not equal total predicted loss divided by total exposure. The same argument applies to observed loss cost when each observation’s observed rate is \(L_i/e_i\): weighting those rates by exposure gives \(\sum L_i/\sum e_i\).
Constructing and Reading a Lift Table
Use predictions, observed outcomes, and exposures from the same period and with consistent definitions. State whether the outcome is loss cost, claim frequency, or another quantity.
Sort from highest to lowest predicted outcome if the purpose is to inspect high-risk lift. Use a stated rule for tied predictions; tied observations should not be split arbitrarily if that would imply a meaningful ranking within the tie.
Specify whether groups contain approximately equal numbers of records or approximately equal amounts of exposure. For each group, sum exposure, observed outcome, and predicted outcome before calculating averages.
Report group observed and predicted rates, and state whether any reported lift is relative to the observed or predicted portfolio average. Keep O/P distinct from either lift.
Equal-record groups are easy to construct, but they need not have similar exposure. Equal-exposure groups more directly balance the amount of risk represented, though exact boundaries may be impossible when observations have different exposure amounts. If a boundary is approximate, report the actual group exposure. Do not silently split a policy record across groups; if records are intentionally divided into exposure units, explain the rule and ensure the observed outcome and prediction are allocated consistently.
Lift is a descriptive ranking summary, not a complete assessment of a model. A high observed lift in the highest-predicted group may indicate useful ordering, but it does not alone show that predicted amounts are calibrated. Conversely, an O/P different from 1 can occur even when the groups are ordered sensibly. Interpret both the quantities shown and the way the groups were formed.
Worked Example: Unequal Exposures Within Groups
A hypothetical insurer ranks four validation records by predicted loss cost. The first two are the high-prediction group and the last two are the low-prediction group. Calculate exposure-weighted observed and predicted costs, lifts, and O/P. Then compare the weighted results with unweighted averages.
| Record | Group | Exposure | Observed loss | Predicted loss cost |
|---|---|---|---|---|
| A | High | 1 | $900 | $500 |
| B | High | 3 | $1,500 | $400 |
| C | Low | 1 | $0 | $200 |
| D | Low | 5 | $600 | $100 |
Method. Sum exposure, observed loss, and exposure times predicted cost within each group. For the high group, exposure is \(1+3=4\), observed loss is \(900+1{,}500=\$2{,}400\), and predicted loss is \(1(500)+3(400)=\$1{,}700\). Its observed cost is \(2{,}400/4=\$600\), and its predicted cost is \(1{,}700/4=\$425\). Its O/P is \(2{,}400/1{,}700=1.4118\).
For the low group, exposure is \(1+5=6\), observed loss is \(0+600=\$600\), and predicted loss is \(1(200)+5(100)=\$700\). Its observed cost is \(600/6=\$100\), predicted cost is \(700/6=\$116.6667\), and O/P is \(600/700=0.8571\).
Across the portfolio, exposure is \(4+6=10\), observed loss is \(\$3{,}000\), and predicted loss is \(\$2{,}400\). Therefore, portfolio observed cost is \(3{,}000/10=\$300\) and predicted cost is \(2{,}400/10=\$240\). The high group’s observed and predicted lifts are \(600/300=2.000\) and \(425/240=1.771\). The low group’s lifts are \(100/300=0.333\) and \(116.6667/240=0.486\).
As a check, exposure times group cost reproduces each group total: \(4(600)=\$2{,}400\) observed and \(6(100)=\$600\) observed; the predicted totals similarly reproduce \(\$1{,}700\) and \(\$700\). Common mistake: the unweighted mean of the two high-group record-level observed rates is \((900/1+1{,}500/3)/2=\$700\), not \(\$600\). The exposure-weighted result correctly reflects that record B represents three exposure units.
Worked Example: A Group With Zero Claims
A hypothetical frequency validation table has three prediction bands. Each band contains records with the displayed total exposure, observed claim count, and predicted frequency per exposure unit. Calculate the observed and predicted frequency, observed lift, and O/P for each band. Treat the overall observed frequency as positive.
| Band | Exposure | Observed claims | Predicted frequency |
|---|---|---|---|
| Top | 4 | 0 | 0.08 |
| Middle | 3 | 1 | 0.12 |
| Bottom | 3 | 2 | 0.20 |
Method. For frequency, divide claim count by exposure for the observed rate, and multiply predicted frequency by exposure to get predicted claim count. Overall exposure is \(4+3+3=10\), observed claims are \(0+1+2=3\), and predicted claims are \(4(0.08)+3(0.12)+3(0.20)=0.32+0.36+0.60=1.28\). Thus, overall observed frequency is \(3/10=0.3000\) and predicted frequency is \(1.28/10=0.1280\).
The top band has observed frequency \(0/4=0\), predicted frequency \(0.08\), observed lift \(0/0.30=0\), and O/P \(0/(4(0.08))=0\). The middle band has observed frequency \(1/3=0.3333\), predicted frequency \(0.12\), observed lift \(0.3333/0.3000=1.1111\), and O/P \(1/0.36=2.7778\). The bottom band has observed frequency \(2/3=0.6667\), predicted frequency \(0.20\), observed lift \(0.6667/0.3000=2.2222\), and O/P \(2/0.60=3.3333\).
The check is that the band claim counts sum to \(3\), the band exposures sum to \(10\), and the predicted counts sum to \(1.28\). Common mistake: removing the top band because it has no claims would discard four exposure units and change the validation population. Zero claims are a valid observed outcome, not missing data. O/P in the top band is zero; predicted divided by observed is undefined there.
Worked Example: Equal Record Counts Can Mean Unequal Exposure
A hypothetical loss-cost model produces the following four records, already sorted from highest to lowest prediction. An analyst forms two equal-record groups, with the top two records in one group. Calculate the exposure-weighted rates and lifts, and explain what the group sizes imply.
| Record | Exposure | Observed loss | Predicted loss cost |
|---|---|---|---|
| A | 5 | $2,500 | $400 |
| B | 4 | $800 | $300 |
| C | 1 | $0 | $200 |
| D | 1 | $100 | $100 |
Method. The top group has two records but exposure \(5+4=9\). Its observed loss is \(\$2{,}500+\$800=\$3{,}300\), so its observed cost is \(3{,}300/9=\$366.6667\). Its predicted total is \(5(400)+4(300)=2{,}000+1{,}200=\$3{,}200\), so its predicted cost is \(3{,}200/9=\$355.5556\).
The bottom group also has two records, but exposure \(1+1=2\). Its observed cost is \((0+100)/2=\$50\). Its predicted total is \(1(200)+1(100)=\$300\), so predicted cost is \(300/2=\$150\).
Portfolio exposure is \(11\), observed loss is \(\$3{,}400\), and predicted loss is \(\$3{,}500\). Portfolio costs are \(3{,}400/11=\$309.0909\) observed and \(3{,}500/11=\$318.1818\) predicted. The top group’s observed and predicted lifts are \(366.6667/309.0909=1.186\) and \(355.5556/318.1818=1.117\). The bottom group’s lifts are \(50/309.0909=0.162\) and \(150/318.1818=0.471\).
The top group contains \(9/11=81.82\%\) of portfolio exposure, despite containing half the records. The bottom group represents only \(2/11=18.18\%\). A table with equal record counts therefore does not have equal exposure represented in each row. The total check is \(9(366.6667)+2(50)=\$3{,}400\) observed, subject to rounding, and \(9(355.5556)+2(150)=\$3{,}500\) predicted. Common mistake: describing these rows as equally sized without specifying that “size” means record count; for an exposure-weighted analysis, their exposure sizes are very different.
Common Traps and Practical Checks
- Averaging rates without weights. If observations have unequal exposure, use exposure shares as weights or aggregate the numerator and denominator directly.
- Confusing group lift with O/P. Lift compares a group rate with a portfolio rate; O/P compares the group’s observed total with its predicted total. They answer different questions.
- Dropping zero-outcome groups. Retain their exposure and observed outcome. Use a defined ratio such as observed divided by positive predicted total; do not reverse the ratio when the observed total is zero.
- Ignoring group construction. State whether groups are based on record counts or exposure, and report actual exposures. Ties and approximate boundaries can affect interpretation.
- Comparing incompatible quantities. A prediction per policy term cannot be compared directly with an observed amount per exposure unit unless both are first put on the same basis.
Before interpreting a lift table, verify that group exposures add to portfolio exposure, group observed totals add to the portfolio observed total, and group predicted totals add to the portfolio predicted total. Recalculate rates from those totals rather than averaging displayed group rates to recover a portfolio result. The table is a descriptive validation summary aligned with the CAS Exam 8 (Advanced Ratemaking) syllabus; it does not by itself establish why a model ranks risks as it does or whether the differences will persist.
Check Your Understanding
For each problem, calculate using aggregate outcomes and exposure rather than an unweighted mean of record-level rates.
- A two-record group has exposures 1 and 3, observed losses of $100 and $600, and predicted loss costs of $80 and $150. Find the exposure-weighted observed cost, predicted cost, and O/P. (Answer: observed cost $175.00; predicted cost $132.50; O/P 1.3208)
- A frequency band has exposure 5, zero observed claims, and predicted frequency 0.06. Find its observed frequency and O/P. (Answer: observed frequency 0.0000; O/P 0.0000, with exposure retained)
- A portfolio has exposure 8, observed loss $2,000, and predicted loss $1,600. A group within it has exposure 2, observed loss $800, and predicted loss $500. Find the group observed lift, predicted lift, and O/P. (Answer: observed lift 1.600; predicted lift 1.250; O/P 1.6000)
- Two groups have equal record counts. Group A has exposure 10 and observed loss $1,500; Group B has exposure 2 and observed loss $500. Find each group’s observed cost and explain why the record counts do not make their exposure sizes equal. (Answer: Group A $150.00 per exposure unit; Group B $250.00 per exposure unit; their exposures are 10 and 2, respectively)
- A validation table has two bands: band H has exposure 4, observed loss $1,200, and predicted loss $1,000; band L has exposure 6, observed loss $600, and predicted loss $900. Find portfolio observed and predicted costs, then band H’s observed lift and O/P. (Answer: portfolio observed cost $180.00; predicted cost $190.00; band H observed lift 1.6667; O/P 1.2000)