Why Validation Bases Must Match
A validation result is meaningful only when the observed outcome and the model prediction refer to the same thing. A prediction may be an annual expected loss per policy-year at a specified rate level, while the available validation data may contain partial-year exposures, losses that are not fully developed, or premiums earned under earlier rates. Comparing those figures without adjustment can make a sound prediction appear inaccurate—or conceal a real difference.
The essential task is to define a target basis before comparing predictions with outcomes. Specify the coverage and exposure unit, the period to which losses are brought, the claim maturity represented by the losses, and—when premium is used—the rate level of that premium. These adjustments do not improve a model; they make the observed data commensurate with the quantity the model predicts.
An earned exposure measures the portion of covered risk actually provided during a period. For example, six months of coverage for one vehicle is \(0.5\) vehicle-years. A policy count is not automatically an exposure measure: policies can begin or end at different dates, and policy terms may differ. Use the same exposure definition in the validation outcome and the model prediction.
Claim maturity is a separate issue. An incurred loss total at a valuation date may omit later reported or paid development. A loss development factor (LDF) converts losses at a stated maturity to an estimated ultimate amount, subject to the appropriateness of that factor for the claims, coverage, and valuation age. Trend then moves the loss level from its applicable historical period to the selected target period. The LDF and trend must be selected for their respective bases; neither factor repairs a mismatch in exposure.
Putting Losses and Exposure on a Common Basis
Suppose a segment has incurred losses \(L_{\mathrm{inc}}\), an applicable LDF \(D\), earned exposure \(E\), and an annual effective loss trend rate \(r\). If trend is to be applied for \(t\) years from the loss period represented by the developed amount to the target period, the target-level loss and loss cost per exposure unit are
Here \(t\) is measured in years and may be fractional. The formula assumes the stated LDF is appropriate and that the annual effective trend rate applies over the specified interval with annual compounding. If trend is supplied on a different basis, such as monthly compounding, use that basis rather than silently treating it as an effective annual rate. If losses are already ultimate or already at the target trend level, do not apply the corresponding adjustment again.
Worked Example: Developing and Trending a Validation Loss Total
A hypothetical validation segment has $200,000 of incurred loss, an LDF of 1.25 to ultimate, and 500 earned exposure units. Assume a 4% effective annual trend rate applies for one-half year from the loss period represented by the developed amount to the target period. Find target-level losses and target-level loss cost per exposure unit.
Method. Develop the losses, trend the developed amount for \(0.5\) years, and divide by earned exposure. The assumed trend multiplier is \((1.04)^{0.5}=\sqrt{1.04}=1.0198039027\).
Development. The ultimate estimate before trend is \(200{,}000 \times 1.25 = 250{,}000\) dollars.
Trend. The target-level total is \(250{,}000 \times 1.0198039027 = 254{,}950.9757\) dollars, or $254,950.98 to the nearest cent.
Exposure normalization. Divide the unrounded target total by the 500 earned exposure units: \(254{,}950.9757 \div 500 = 509.9019514\), or $509.90 per exposure unit to the nearest cent.
Check and trap. The adjusted total exceeds the $250,000 developed amount, as a positive trend over a positive interval requires. A common error is to report $254,950.98 as the per-unit cost, forgetting to divide by 500. Another is to use a full-year factor of 1.04: the stated interval is one-half year. The result is therefore $254,950.98 in target-level losses and $509.90 per exposure unit.
Aligning Earned Premium and Exposure
Earned premium and earned exposure answer different questions. Exposure measures the amount of covered risk; earned premium measures the premium attributable to coverage provided. If premium was earned under an earlier rate level, it may need an on-level adjustment before it is used alongside losses at a target rate level. An on-level factor adjusts premium for rate changes; it does not change the number of exposure units or develop claims.
A useful sequence is to establish exposure first, then adjust losses for maturity and trend, and adjust premium for rate level if a premium-based comparison is required. Confirm that the premium covers the same policies, coverage, and earned period as the exposure and losses. A written premium total for policies bound during the period is not necessarily the earned premium for coverage provided during that period.
Worked Example: Reconciling Partial-Term Exposure and Earned Premium
A hypothetical validation cohort consists of 160 policies, each providing an average of one-half year of coverage during the observation interval. The cohort has $58,000 of incurred loss, an LDF of 1.25 to ultimate, and an effective annual loss trend rate of 3% for one year to the target period. Its earned premium is $64,000 at an earlier rate level; an approved on-level factor of 1.05 applies to bring that premium to the target rate level. Calculate earned exposure, target-level loss cost, on-level earned premium, and the resulting loss ratio.
Method. Convert the partial policy terms to earned exposure, adjust losses for development and trend, and adjust premium separately for rate level.
Earned exposure. Each policy contributes \(0.5\) policy-year, so \(160 \times 0.5 = 80\) earned policy-years.
Develop and trend losses. Development gives \(58{,}000 \times 1.25 = 72{,}500\) dollars. One year of 3% effective annual trend gives \(72{,}500 \times 1.03 = 74{,}675\) dollars of target-level losses.
Loss cost per exposure unit. \(74{,}675 \div 80 = 933.4375\), or $933.44 per earned policy-year to the nearest cent.
On-level premium and loss ratio. The target-level earned premium is \(64{,}000 \times 1.05 = 67{,}200\) dollars. The ratio of target-level losses to on-level earned premium is \(74{,}675 \div 67{,}200 = 1.1112351\), or 111.1% to one decimal place.
Check and trap. The 160 policies produce 80, not 160, earned policy-years because each contributes only one-half year. Using 160 as the denominator would halve the calculated loss cost. Also, the on-level factor applies to premium, not losses or exposure. The result uses 80 earned policy-years and $67,200 of on-level earned premium.
Validation Periods and Loss Maturity
For out-of-time or rolling-origin validation, an observation window and a valuation date are not interchangeable. The observation window identifies the risks and loss period being evaluated. The valuation date determines how much claim development is visible. A cohort observed for the same accident period can have different maturity if one data extract is valued later than another. Conversely, two cohorts with the same valuation date can represent different loss periods and therefore require different trend intervals.
Before comparing a model prediction with validation outcomes, write down the relevant dates and bases explicitly:
State the period to which the prediction and observed loss level are intended to refer.
Use earned units over the same coverage and observation interval as the outcome being summarized.
Identify the valuation age of the incurred losses and use an LDF appropriate to that maturity and loss definition.
Move the developed loss level from its represented period to the target period using the stated trend basis.
If premium enters the comparison, use earned premium for the same coverage and period, adjusted to the applicable rate level.
A practical check is to express the outcome in the same units as the prediction before examining any difference. If the prediction is an annual loss cost per vehicle-year, the observed quantity should also be a loss amount per earned vehicle-year at the same loss maturity and target period. If a total is compared instead, multiply the predicted per-unit amount by the matching earned exposure. This avoids mistaking a larger cohort or a longer coverage period for poorer prediction.
Worked Example: Matching a Midyear Validation Cohort to an Annual Prediction
A hypothetical model predicts an annual loss cost of $720 per policy-year at a target period. A validation cohort contains 240 earned policy-months and $14,000 of incurred losses. The losses are valued at a maturity for which the applicable LDF to ultimate is 1.15. Assume a 5% effective annual trend rate applies for one-half year from the loss period represented by the developed amount to the target period. Find the aligned observed loss cost and the predicted total loss for the cohort.
Method. Convert policy-months to policy-years, develop and trend the losses, and put the prediction on the same total-exposure basis.
Exposure conversion. There are 12 policy-months per policy-year, so \(240 \div 12 = 20\) earned policy-years.
Development. \(14{,}000 \times 1.15 = 16{,}100\) dollars of estimated ultimate loss before trend.
Trend. The half-year trend factor is \((1.05)^{0.5}=\sqrt{1.05}=1.0246950766\). Therefore the target-level losses are \(16{,}100 \times 1.0246950766 = 16{,}497.5907\) dollars, or $16,497.59 to the nearest cent.
Observed loss cost. \(16{,}497.5907 \div 20 = 824.879535\), or $824.88 per policy-year to the nearest cent.
Predicted total. The prediction is $720 per policy-year, so for 20 earned policy-years it corresponds to \(720 \times 20 = 14{,}400\) dollars.
Check and trap. Both observed and predicted quantities are now expressed per policy-year or for the same 20 policy-years. Comparing $16,497.59 directly with $720 would mix a cohort total with a per-year prediction. Treating 240 policy-months as 240 policy-years would also create a twelvefold exposure error. The aligned observed loss cost is $824.88, and the prediction corresponds to $14,400 for this cohort.
Common Basis Errors
The arithmetic of validation adjustments is usually straightforward; the more consequential errors are choosing the wrong base or applying a valid factor to the wrong quantity. In particular, development and trend must correspond to distinct, stated intervals, while exposure and premium must correspond to the risks and coverage being evaluated.
- Using written policies as earned exposure. Policy counts ignore partial terms and midperiod cancellations. Convert coverage duration into the exposure unit used by the prediction.
- Applying trend for the wrong interval. The exponent is the time from the period represented by the loss level to the target period, not automatically the time between data extracts.
- Using an LDF that does not match maturity. An LDF for a different valuation age or loss definition may not develop the validation losses appropriately.
- Trending premium as though it were loss. Loss trend and premium on-level adjustments represent different changes. Apply each to the relevant quantity and only when needed.
- Adjusting an already adjusted amount twice. Confirm whether losses are already ultimate or target-trended and whether premium is already on level before applying factors.
Check Your Understanding
For each problem, identify the stated bases and keep exposure conversion, loss development, trend, and premium adjustment separate.
- A hypothetical cohort has $96,000 of incurred losses, an LDF of 1.25, a 4% effective annual trend rate for one-half year, and 240 earned exposure units. Find target-level loss cost per exposure unit. (Answer: $509.90 per exposure unit.)
- A model predicts $640 per policy-year. A validation cohort has 360 earned policy-years and $180,000 of incurred loss; the applicable LDF is 1.10 and the 2% effective annual trend applies for one year. Find the adjusted observed loss cost and the predicted total loss for the cohort. (Answer: $561.00 per policy-year; $230,400 predicted total.)
- A cohort records 96 earned policy-months, $6,400 of incurred loss, an LDF of 1.25, and a 4% effective annual trend rate for one-half year. Find earned policy-years and target-level loss cost per policy-year. (Answer: 8 earned policy-years; $1,019.80 per policy-year.)
- A segment has 120 earned exposure units, $72,000 of incurred losses, an LDF of 1.20, and no additional trend to the target period. Earned premium is $100,000 at an earlier rate level, with an on-level factor of 1.06. Find target-level loss cost and the loss ratio using on-level premium. (Answer: $720.00 per exposure unit; 81.51%.)
- Two hypothetical validation groups are to be brought to the same target period. Group A has 100 exposure units, $40,000 incurred loss, and LDF 1.10; its trend interval is one year. Group B has 50 exposure units, $30,000 incurred loss, and LDF 1.30; its trend interval is one-half year. Both use a 3% effective annual trend rate. Find the combined target-level loss cost. (Answer: $566.00 per exposure unit.)