Why Backtest a Prior Reserve Estimate?
A reserve estimate is a prediction made with the information and assumptions available at a particular valuation date. Backtesting evaluates that prediction after time has passed: What was paid, what liability remains according to a later estimate, and how does that combined amount compare with the earlier selected liability? The exercise helps an actuary assess the performance of a prior estimate, but it does not turn a later estimate into a known ultimate result.
A useful backtest requires a like-for-like comparison. The prior liability and later amounts should refer to the same claims or accident-period cohort, the same coverage and expense scope, and a consistent valuation basis. For the examples below, all amounts are undiscounted dollars, and payments are cumulative from the prior valuation date through the later valuation date.
The word estimate matters. Payments are observed cash flows, but \(R_1\) is still an estimate of future payments. Thus \(P+R_1\) is a later view of total cost for the cohort, not necessarily its final ultimate cost. As claims continue to develop, the later liability may itself change.
Constructing a Valid Comparison
Before calculating a difference, confirm the basis of the values being compared. For example, a prior reserve might be net of reinsurance while later payments are gross, or the later estimate might include claim expenses excluded from the prior selection. Such a comparison mixes unlike quantities. Likewise, if the cohort includes only claims incurred by the prior valuation date, payments on later accidents must not be added to it.
If estimates are discounted, the amounts must be put on a consistent time-value basis. A discounted prior liability cannot be compared directly with undiscounted future payments and a nominal later liability. Either use an undiscounted basis throughout or bring cash flows and estimates to a common valuation date using the appropriate discounting convention. The calculations here use no discounting.
Specify which claims are included and whether the comparison is gross or net, and whether it includes allocated or other claim expenses.
Measure payments made since the prior valuation and obtain a later estimate of the remaining liability for that same cohort.
Compute \(P+R_1-R_0\), state the sign convention, and investigate the sources of the result rather than treating the sign as a complete diagnosis.
Separating Old-Basis Experience from Changed Assumptions
The total difference alone does not explain why the backtest changed. In particular, the change \(R_1-R_0\) is not, by itself, an assumption effect: it compares liabilities at different dates, after some claims have been paid and new information has emerged. A more informative bridge uses a later estimate calculated on the prior valuation's assumptions and a later estimate calculated on the revised assumptions.
This identity is an accounting bridge: adding the two terms cancels \(R_1^{(0)}\), leaving \(P+R_1^{(1)}-R_0\). The first term measures the combined result of payments and the remaining liability under the old basis. It should not automatically be called pure random error. It can reflect claim experience, information revealed over time, data corrections, and the way the old assumptions apply to the later claim information. The second term isolates the effect of the revisions defined in the bridge, provided the comparison holds the other elements constant.
When several assumptions change, the measured effect of each change can depend on the order in which they are applied. An actuary should describe the bridge method and identify any interaction or residual rather than implying that every component is uniquely determined. A change prompted by new claim facts may also be a change in circumstances rather than a general change in an actuarial parameter; labeling it clearly helps the reader interpret the result.
Worked Example: Reconciling a Prior Reserve to a Revised Liability
A hypothetical insurer selected an undiscounted reserve of $2.40 million for a fixed claim cohort at the prior valuation date. By the later valuation date, it had paid $0.50 million on the cohort. Applying the prior assumptions to the later claim information gives a remaining liability estimate of $1.90 million. Revised assumptions increase that later estimate by $0.12 million. Calculate and interpret the backtest difference, then reconcile it.
Method. Add cumulative payments to the later remaining liability, then compare that total with the prior selected reserve.
Step 1: Calculate the revised later liability. $$ R_1^{(1)}=\$1.90\text{ million}+\$0.12\text{ million} =\$2.02\text{ million}. $$
Step 2: Calculate the backtested amount. $$ P+R_1^{(1)} =\$0.50\text{ million}+\$2.02\text{ million} =\$2.52\text{ million}. $$
Step 3: Compare with the prior reserve. $$ D=\$2.52\text{ million}-\$2.40\text{ million} =+\$0.12\text{ million}. $$ The backtest difference is adverse by $0.12 million: the payments plus revised remaining estimate exceed the prior selected reserve by that amount.
Step 4: Reconcile the difference. Under the prior assumptions, payments plus remaining liability are $$ \$0.50\text{ million}+\$1.90\text{ million} =\$2.40\text{ million}. $$ So the old-basis component is \(\$2.40-\$2.40=\$0.00\) million. The revision component is \(\$2.02-\$1.90=+\$0.12\) million. Their sum is \(\$0.00+\$0.12=+\$0.12\) million, agreeing with the direct calculation. The result is explained by the stated revision in this bridge; it does not establish that the revised estimate is the eventual final cost.
Common mistake avoided. Do not report a total that fails to equal the listed components. Here the direct difference and the bridge both give +$0.12 million.
Reading a Backtest Without Overclaiming
A small total difference may arise because the prior estimate performed well, but it can also result from offsetting effects. For example, payments and remaining liability under the old basis could be below the prior estimate while revised assumptions increase the later liability. In that case, a near-zero total does not mean every part of the earlier estimate was accurate.
Similarly, a large adverse difference is a signal to investigate, not a verdict on one assumption. Useful questions include whether the cohort was held constant, whether payments were complete and correctly classified, whether the prior estimate included the same expenses, and whether the later revision reflects newly observed claim facts or a changed parameter. The backtest should distinguish what is observed from what remains estimated.
| Backtest item | What it represents | Interpretive caution |
|---|---|---|
| Prior selected liability, \(R_0\) | The unpaid estimate selected at the earlier valuation | Confirm its scope and valuation basis before comparing |
| Subsequent payments, \(P\) | Observed payments on the fixed cohort after the earlier valuation | Do not add payments for claims outside the cohort |
| Later remaining liability, \(R_1\) | A later estimate of unpaid obligations for the cohort | It is still an estimate, not an observed final cost |
| Backtest difference, \(D\) | Payments plus later liability less the prior liability | A total can conceal offsetting components |
Worked Example: Separating an Old-Basis Result from a Revision
For a hypothetical cohort, the prior selected liability is $860,000. Subsequent payments are $310,000. At the later date, the remaining liability is estimated at $565,000 under the prior assumptions; a separately identified change in circumstances adds $35,000. Calculate the total difference and its two bridge components.
Step 1: Calculate the revised remaining liability. $$ R_1^{(1)}=\$565{,}000+\$35{,}000=\$600{,}000. $$
Step 2: Calculate the total backtest difference. $$ D=\$310{,}000+\$600{,}000-\$860{,}000 =\$50{,}000. $$ The total is adverse by $50,000.
Step 3: Calculate the old-basis component. $$ P+R_1^{(0)}-R_0 =\$310{,}000+\$565{,}000-\$860{,}000 =\$15{,}000. $$
Step 4: Calculate the circumstance-change component and check. $$ R_1^{(1)}-R_1^{(0)} =\$600{,}000-\$565{,}000 =\$35{,}000, $$ and \(\$15{,}000+\$35{,}000=\$50{,}000\), matching the direct difference. The $15,000 component is the old-basis combined result, not necessarily pure process error. The $35,000 is the effect of the stated circumstance change within this bridge.
Common mistake avoided. Calling the entire $50,000 an assumption change would ignore the $15,000 old-basis component. Conversely, calling the entire amount prediction error would ignore the separately measured $35,000 revision.
Worked Example: A Small Net Difference Can Hide Larger Components
A hypothetical prior reserve for a cohort is $1.20 million. Subsequent payments total $260,000. At the later date, the old-basis estimate of remaining liability is $890,000. A revised assumption increases this amount by $50,000. Calculate the net difference and explain what the bridge says about the apparent result.
Step 1: Find the revised remaining liability. $$ R_1^{(1)}=\$890{,}000+\$50{,}000=\$940{,}000. $$
Step 2: Calculate the total backtest difference. $$ D=\$260{,}000+\$940{,}000-\$1{,}200{,}000 =\$0. $$ The later combined amount equals the prior reserve.
Step 3: Calculate each bridge component. $$ \text{Old-basis component} =\$260{,}000+\$890{,}000-\$1{,}200{,}000 =-\$50{,}000, $$ $$ \text{Revision component} =\$940{,}000-\$890{,}000 =+\$50{,}000. $$ Their sum is \(-\$50{,}000+\$50{,}000=\$0\), as required.
The net result is neutral, but the components are not. The old-basis combined amount was $50,000 below the prior selection, while the revision added $50,000. It would be misleading to describe this as an uncomplicated confirmation that the prior estimate was exactly right.
Common mistake avoided. Do not interpret a zero net difference as proof that no errors or changes occurred; check the components before drawing that conclusion.
Common Backtesting Traps
- Comparing unlike scopes. Gross payments and a net reserve, or indemnity-only and indemnity-plus-expense estimates, do not form a valid bridge without adjustment.
- Mixing valuation bases. Do not compare discounted and undiscounted values without converting them to a consistent basis.
- Treating the later estimate as final. The remaining liability is uncertain and may change again.
- Calling every residual “prediction error.” The old-basis component includes the effects of payments and later claim information under the chosen method; it need not isolate a single cause.
- Ignoring the sign convention. With \(D=P+R_1-R_0\), positive is adverse and negative is favorable. State the convention when presenting results.
Check Your Understanding
For each problem, use the sign convention \(D=P+R_1-R_0\). Amounts are undiscounted and in the same scope unless stated otherwise.
- A prior reserve is $750,000. Subsequent payments are $220,000, and the later remaining liability is $545,000. Calculate and interpret the difference. (Answer: \(D=+\$15{,}000\), adverse.)
- A prior reserve is $600,000, subsequent payments are $180,000, and the old-basis later remaining estimate is $410,000. A revised assumption increases the remaining estimate by $25,000. Find the direct difference and both bridge components. (Answer: \(D=+\$15{,}000\); old-basis component \(-\$10{,}000\), revision component \(+\$25{,}000\).)
- A prior reserve is $2.00 million. Payments are $0.60 million. The old-basis later estimate is $1.32 million; a change reduces it by $0.04 million and another change increases it by $0.09 million. Find the net revised estimate and difference. (Answer: revised estimate \(=\$1.37\) million; \(D=-\$0.03\) million, favorable.)
- A prior liability of $900,000 includes a fixed cohort and claim expenses. Later payments of $250,000 are for that cohort, but the later $610,000 estimate excludes claim expenses. Can \(250{,}000+610{,}000-900{,}000\) be interpreted directly as a valid backtest difference? State the needed correction. (Answer: No; the later estimate must include the same claim-expense scope, or the prior liability and other amounts must be adjusted to a consistent scope before interpreting the difference.)
- A prior reserve is $1.50 million. Payments are $420,000. The old-basis later estimate is $1.08 million. Three sequential revisions change that estimate by +$60,000, +$40,000, and \(-\$20,000\). Calculate the revised liability, total difference, and revision component. (Answer: revised liability \(=\$1.16\) million; \(D=+\$80{,}000\); revision component \(=+\$80{,}000\), with old-basis component \(\$0\).)