Backtesting Increased Limits Factors

Why Backtest Increased Limits Factors?

An increased limits factor (ILF) converts an expected loss at one limit into an expected loss at a higher limit. Estimating the factor is only part of the task: a backtest asks whether the factor would have predicted losses adequately for claims not used to construct it. The comparison should be made at several limits when possible, since a factor can fit one point on the curve while missing another.

A useful backtest separates two questions. First, does the estimated curve describe the relative change in limited losses as the limit rises? Second, is the base-limit loss level itself accurate? If the validation data are used as the base anchor, the backtest assesses relative curve shape, not the accuracy of the base estimate. If the base estimate comes from an independent source, the comparison also reflects any error in that base level.

Definition: Let $X_j$ be the ground-up loss for validation record $j$, $w_j$ its nonnegative weight, and $u$ a policy limit. With no deductible or other payment modification, the limited loss is $\min(X_j,u)$. The weighted observed limited loss at limit $u$ is $$ O(u)=\sum_j w_j\min(X_j,u). $$ For a base limit $b$, the observed ILF is $O(u)/O(b)$, provided $O(b)>0$. A calibration ratio is observed limited loss divided by predicted limited loss; a ratio above 1 indicates underprediction, and a ratio below 1 indicates overprediction.

The weights may represent claim multiplicities or other appropriate exposure weights. They must be applied consistently at every limit. The losses must also be on a consistent basis: for example, the validation losses and the estimated factors should agree on coverage, valuation, development, trend, and the treatment of expenses. Earlier material on exposure-curve adjustments addresses how to establish that basis. A change in deductible, coinsurance, or payment definition requires the corresponding payment function; the simple $\min(X_j,u)$ calculation does not apply unchanged.

Set Up a Like-for-Like Comparison

For a factor estimated from base limit $b$ to limit $u$, denote it by $\widehat{\operatorname{ILF}}(b,u)$. If the selected base-loss anchor is $B$, the predicted limited loss is

P(u)=B\,\widehat{\operatorname{ILF}}(b,u).

When the validation cohort itself supplies the anchor, set $B=O(b)$. When a separate pricing or exposure estimate supplies the base loss, use that estimate and state its source. The observed validation loss remains $O(u)$ in either case. Then calculate

C(u)=\frac{O(u)}{P(u)}.

A backtest is most informative when the factor is applied to later or held-out experience, not to the same observations used to fit or select the curve. Reusing the fitting data can make performance look better than it will be on new experience. Also report the base limit and all tested limits: a calibration ratio has no useful interpretation if its denominator or comparison limit is unclear.

Define the validation basis.
Identify the held-out claims, weights, loss basis, base limit, and limits to be tested.

Cap each loss at each limit.
For every record and limit, calculate the applicable limited payment before applying its weight.

Build predictions.
Multiply the stated base-loss anchor by the estimated ILF for each limit.

Compare and interpret.
Calculate observed-to-predicted ratios at each limit and look for a systematic pattern across the curve.

Worked Backtests

Worked Example: Correcting the Limited-Loss Calculation

A hypothetical held-out validation set contains ground-up losses of $80 and $180, with weights 2 and 1. The base limit is $100, the higher limit is $300, and the estimated ILF from $100 to $300 is 1.25. Use the validation losses at the base limit as the prediction anchor. Calculate the observed limited losses, the predicted loss at $300, and the calibration ratio.

Method. Cap each claim at each limit, multiply each capped amount by its weight, and sum. The weights apply to limited payments, not to the uncapped loss before applying the limit.

At the $100 base limit, the two limited payments are $\min(80,100)=80$ and $\min(180,100)=100$. Therefore,

$$ O(100)=2(80)+1(100)=160+100=260. $$

At the $300 limit, the payments are $\min(80,300)=80$ and $\min(180,300)=180$. Thus,

O(300)=2(80)+1(180)=160+180=340, \qquad \frac{O(300)}{O(100)}=\frac{340}{260}=1.3077.

The base-anchored prediction is

$$ P(300)=260(1.25)=325. $$

The calibration ratio is

C(300)=\frac{340}{325}=1.0462.

Check and interpretation. The observed loss at $300 is greater than the predicted $325, so the factor underpredicts this validation total by about 4.62% relative to the prediction. The observed ILF of 1.3077 is also greater than the estimated 1.25.

Common mistake avoided. Counting the weighted ground-up amounts as though neither loss were capped at $100 gives an incorrect base total. The base payments are $80 and $100, not $80 and $180; using the latter would produce an incorrect $O(100)$ and distort both the observed ILF and the calibration ratio.

Worked Example: Applying Weights at Two Limits

A hypothetical validation set has ground-up losses of $50, $150, and $350, with weights 1, 2, and 1. The base limit is $100 and the tested limit is $250. The estimated ILF is 1.65. Use the observed base-limit loss as the anchor. Find the weighted observed losses, predicted loss, and calibration ratio.

At $100, the capped payments are $50, $100, and $100. Applying the weights gives

$$ O(100)=1(50)+2(100)+1(100)=50+200+100=350. $$

At $250, the capped payments are $50, $150, and $250. The weighted sum is

$$ O(250)=1(50)+2(150)+1(250)=50+300+250=600. $$

The observed ILF is $600/350=1.7143$. The estimated-factor prediction is

$$ P(250)=350(1.65)=577.50. $$

Therefore,

C(250)=\frac{600}{577.50}=1.0390.

Check and interpretation. The factor underpredicts the observed limited loss by 3.90% relative to the prediction. As a direct check, $577.50(1.0390)$ is approximately $600, allowing for rounding.

Common mistake avoided. The weighted payment for the $350 loss at the $250 limit is $250, not $350. The correct contribution at that limit is $1(250)$, so the total is $600, not $700.

Worked Example: Reading Calibration Across the Curve

A hypothetical held-out set contains ground-up losses of $60, $140, $320, and $700, with weights 1, 2, 1, and 1. The base limit is $100. Estimated ILFs are 1.55 at $200 and 2.70 at $500. Anchor both predictions to the observed base-limit loss. Calculate observed and predicted losses and interpret the pattern.

At $100, the capped payments are $60, $100, $100, and $100. Thus,

$$ O(100)=1(60)+2(100)+1(100)+1(100)=60+200+100+100=460. $$

At $200, the payments are $60, $140, $200, and $200. Hence,

$$ O(200)=1(60)+2(140)+1(200)+1(200)=60+280+200+200=740. $$

At $500, the payments are $60, $140, $320, and $500, giving

$$ O(500)=1(60)+2(140)+1(320)+1(500)=60+280+320+500=1160. $$

The predicted losses are

P(200)=460(1.55)=713, \qquad P(500)=460(2.70)=1242.

The observed ILFs and calibration ratios are

\frac{O(200)}{O(100)}=\frac{740}{460}=1.6087, \qquad C(200)=\frac{740}{713}=1.0379, $$ $$ \frac{O(500)}{O(100)}=\frac{1160}{460}=2.5217, \qquad C(500)=\frac{1160}{1242}=0.9340.

Check and interpretation. At $200, the estimated factor is below the observed ILF, producing an underprediction. At $500, the estimated factor is above the observed ILF, producing an overprediction. This pattern suggests a difference in curve calibration across the tested range: the estimate is relatively low at the middle limit and relatively high at the higher limit. It does not, by itself, identify the cause of that pattern.

Common mistake avoided. Looking only at the $500 result would hide the underprediction at $200. A backtest across several limits can reveal where the estimated curve diverges from validation experience.

Why the Observed Ratio Is a Useful Check

The ILF is a ratio of expected limited losses. For a common population and consistent loss basis, the expected loss at limit $u$ is the expected loss at base limit $b$ multiplied by the true relative factor. The empirical counterpart is the ratio of the two weighted limited-loss totals. The justification is simply the way the prediction is constructed: if the base expected loss is $B$, an estimated factor $q$ predicts $Bq$ at the higher limit. Comparing that prediction with observed $O(u)$ gives $O(u)/(Bq)$. If $B=O(b)$, then the comparison can be written as

C(u)=\frac{O(u)}{O(b)\widehat{\operatorname{ILF}}(b,u)} =\frac{O(u)/O(b)}{\widehat{\operatorname{ILF}}(b,u)}.

Thus, with an observed base anchor, the calibration ratio is the observed ILF divided by the estimated ILF. This identity explains why a ratio above 1 means that the observed relative increase exceeds the estimated one. It also shows the limitation: at the base limit itself, the ratio is 1 by construction, so this comparison says nothing about whether the base loss level is adequate.

If instead the base prediction is an independent estimate $B$, then $C(u)=O(u)/(B\widehat{\operatorname{ILF}}(b,u))$. A non-unit ratio can reflect both a base-level difference and a curve-shape difference. To isolate the curve, also compare the observed relative factor $O(u)/O(b)$ with the estimated factor. To evaluate overall price level, retain the independent base anchor and interpret the total ratio accordingly.

Common Traps and Practical Interpretation

Applying the cap in the wrong order. For each claim, calculate $\min(X_j,u)$ first, then multiply by $w_j$. In general, capping an aggregate or weighting an uncapped claim before treating the limit does not give the same result.
Changing the loss basis between limits. The claims, weights, and adjustment basis must be consistent. Otherwise a ratio can reflect changes in the data rather than the ILF curve.
Confusing a calibration ratio with a percentage error. A ratio of 1.04 means observed is 1.04 times predicted, or 4% above predicted. It is not a 104% error.
Forgetting the anchor. An observed-base anchor makes the base-point calibration exact by construction. An independent base estimate does not, and its error affects the ratios.
Overinterpreting a small validation set. A few large claims can materially affect limited losses at high limits. A point estimate alone does not establish that a difference is statistically meaningful; consider validation volume and uncertainty before drawing a firm conclusion.
Reading one limit in isolation. A factor may underpredict at one limit and overpredict at another. Compare the full set of tested limits before describing calibration.

Backtesting is a diagnostic, not a substitute for defining the exposure curve correctly. A well-designed comparison uses an independent validation cohort where available, applies consistent weights and limits, records the base anchor, and reports observed and predicted losses together with their ratios. These practices support a technically sound review aligned with the CAS Exam 8 (Advanced Ratemaking) syllabus.

Key takeaway: At each tested limit, calculate observed limited losses claim by claim, build the prediction from a clearly stated base anchor and estimated ILF, and compare observed with predicted. The pattern across limits distinguishes relative curve calibration from base-level performance.

Check Your Understanding

For each problem, apply limits to individual losses before weighting and state whether the factor underpredicts or overpredicts when a calibration ratio is requested.

A validation set has losses $40 and $120 with weights 1 and 2. The base limit is $80 and the higher limit is $200. An estimated ILF is 1.50, anchored to observed base loss. Find $O(80)$, $O(200)$, the prediction, and the calibration ratio. (Answer: $O(80)=200$, $O(200)=280$, prediction $=300$, ratio $=0.9333$, so the factor overpredicts.)
Losses are $90, $160, and $300 with weights 2, 1, and 1. At base limit $100 and higher limit $250, the estimated ILF is 1.60 and is anchored to observed base loss. Find the observed losses and calibration ratio. (Answer: $O(100)=380$, $O(250)=590$, prediction $=608$, ratio $=0.9704$, so the factor overpredicts.)
A held-out set has losses $50, $180, and $420 with weights 1, 1, and 2. The base limit is $100; estimated ILFs are 1.70 at $200 and 2.80 at $400. Use observed base loss as the anchor. Calculate both calibration ratios and interpret them. (Answer: $O(100)=350$, $O(200)=630$, $O(400)=1030$; predictions are $595 and $980; ratios are $1.0588$ and $1.0510$, so both factors underpredict.)
A validation set has losses $70, $130, and $260 with weights 1, 2, and 1. At base limit $100 and higher limit $250, an independent base-loss estimate is $400 and the estimated ILF is 1.50. Calculate the validation limited losses and calibration ratio. (Answer: $O(100)=370$, $O(250)=580$, prediction $=600$, ratio $=0.9667$, so the prediction overstates observed loss.)
At base limit $100, observed validation loss is $500. At limits $150 and $300, observed losses are $575 and $1050. Estimated ILFs are 1.10 and 2.20, respectively, anchored to the observed base loss. Calculate the two predictions and ratios, then describe the relative pattern. (Answer: predictions are $550 and $1100; ratios are $575/550=1.0455$ and $1050/1100=0.9545$. The factor underpredicts at $150 and overpredicts at $300.)

What You'll Learn