Counterfactual Evaluation of Plan Changes

What a Counterfactual Plan Comparison Measures

When an insurer considers changing an experience-rating plan, the immediate question is often how the new formula would have treated the risks already observed. A counterfactual comparison answers that question by applying both the current and proposed formulas to the same risks, exposure measures, and experience data. The resulting difference is a modeled effect of the formula change on that fixed portfolio, not evidence that the change caused different claim outcomes.

This distinction matters because experience-rating plans can affect more than one quantity. A formula can change a risk’s plan-implied loss estimate, its modification, and its premium. These amounts need not move by the same dollar amount: premiums may use base premiums that differ from expected losses, and a separate rate-level adjustment may change charged premiums without changing the plan’s loss predictions.

Definition: A counterfactual evaluation of plan changes compares the outputs of two or more experience-rating formulas after holding the portfolio, exposure, and experience inputs fixed. It estimates how those formulas would rate the same observed risks under the stated assumptions.

Specify the Plan and the Comparison Inputs

For a transparent comparison, define the quantities to be held fixed and the formula being changed. For risk $i$, let $E_i$ be expected losses for the evaluation period, $L_i$ be the corresponding experience-period losses adjusted to the same basis, and $B_i$ be the base premium before the experience modification. In the examples, losses and premiums are in dollars. Assume $E_i>0$; the ratio-based formula below is not defined for a risk with zero expected losses.

Consider a plan with credibility weight $Z$, where $0\leq Z\leq1$. Before any floor or cap, its credibility-weighted loss prediction is a blend of expected and observed losses. The corresponding raw modification is that prediction divided by expected losses:

\widehat{L}_i(Z)=(1-Z)E_i+ZL_i

m_i^{\mathrm{raw}}(Z)=\frac{\widehat{L}_i(Z)}{E_i} =1+Z\left(\frac{L_i}{E_i}-1\right)

A plan may also impose a minimum modification $f$ and maximum modification $c$. Its applied modification and the two principal dollar outputs are:

m_i=\min\!\left(c,\max(f,m_i^{\mathrm{raw}})\right),\qquad \widehat{L}_i^{\,\mathrm{plan}}=E_i m_i,\qquad P_i=B_i m_i

Here $\widehat{L}_i^{\,\mathrm{plan}}$ is the plan-implied loss prediction after applying the cap or floor, and $P_i$ is the premium before any other adjustments. This loss prediction is an output of the specified rating formula; it is not a guarantee of future losses. If a plan includes additional components, such as different loss weights or a different experience period, those must also be stated and held fixed or deliberately changed in the comparison.

Why the Credibility Formula Has This Form

The formula is a direct consequence of blending two loss estimates. The expected loss $E_i$ is the starting estimate, while $L_i$ is the observed experience. Credibility $Z$ assigns weight $Z$ to observed experience and $1-Z$ to the expected value. Dividing that blend by $E_i$ produces a relative modification:

\frac{(1-Z)E_i+ZL_i}{E_i} =(1-Z)+Z\frac{L_i}{E_i} =1+Z\left(\frac{L_i}{E_i}-1\right)

Thus, if observed losses equal expected losses, $L_i/E_i=1$, the raw modification is 1 for every value of $Z$. If losses exceed expectations, increasing $Z$ increases the raw modification; if losses are below expectations, increasing $Z$ decreases it. A floor or cap can restrict those movements, so the final applied modification need not respond linearly to a change in $Z$.

A Reliable Counterfactual Workflow

Freeze the comparison basis.
Use the same risks, exposure period, expected losses, experience losses, and base premiums under every plan. State any adjustments already made to the data.

Apply each plan separately.
Calculate its raw modification, then apply that plan’s floor, cap, credibility weight, and other specified rules.

Calculate risk-level outputs.
Multiply each applied modification by expected losses to obtain the plan-implied loss prediction, and by base premium to obtain premium before other adjustments.

Aggregate and interpret.
Sum dollar amounts across risks, compare the totals and risk-level changes, and state whether any rate-level adjustment has been applied.

The portfolio impact is a dollar-weighted result, not generally the simple average of risk-level modifications. A 10% change on a large base premium has a larger premium effect than a 10% change on a small one. Report both absolute changes and percentage changes with a clearly specified denominator.

Worked Example: Comparing Two Credibility Weights

A hypothetical insurer considers replacing Plan A with Plan B. Both use expected losses as base premiums, so $B_i=E_i$. Plan A has $Z=0.25$, a floor of 0.80, and a cap of 1.20. Plan B has $Z=0.50$, a floor of 0.70, and a cap of 1.30. For Risk 1, $E_1=\$100{,}000$ and $L_1=\$160{,}000$. For Risk 2, $E_2=\$200{,}000$ and $L_2=\$80{,}000$. Apply both formulas to the same data.

Risk 1 under Plan A. The loss ratio is $160{,}000/100{,}000=1.60$. The raw modification is $1+0.25(1.60-1)=1.15$, within the 0.80 to 1.20 range. The applied modification is 1.15. Its predicted losses and premium are each $\$100{,}000(1.15)=\$115{,}000$.

Risk 1 under Plan B. The raw modification is $1+0.50(1.60-1)=1.30$, exactly the cap. The applied modification is 1.30. Predicted losses and premium are each $\$100{,}000(1.30)=\$130{,}000$.

Risk 2 under Plan A. The loss ratio is $80{,}000/200{,}000=0.40$. The raw modification is $1+0.25(0.40-1)=0.85$, within the plan limits. Predicted losses and premium are each $\$200{,}000(0.85)=\$170{,}000$.

Risk 2 under Plan B. The raw modification is $1+0.50(0.40-1)=0.70$, exactly the floor. Predicted losses and premium are each $\$200{,}000(0.70)=\$140{,}000$.

Aggregate and check. Expected losses and base premium total $\$300{,}000$. Plan A produces total predicted losses and premium of $\$115{,}000+\$170{,}000=\$285{,}000$. Plan B produces $\$130{,}000+\$140{,}000=\$270{,}000$. Thus Plan B is $\$15{,}000$ lower than Plan A on this fixed portfolio, a change of $-\$15{,}000/\$285{,}000=-5.263\%$ relative to Plan A. Because $B_i=E_i$, the premium and predicted-loss totals agree, which checks the calculation. The method avoids comparing the plans on different risks or silently ignoring a binding floor or cap.

Premiums and Loss Predictions Are Different Outputs

When base premiums are not equal to expected losses, the same modification produces different dollar changes in premium and predicted losses. The next example uses two risks to show why those quantities should be calculated separately.

Worked Example: Separating Premium Impact from Prediction Impact

A hypothetical plan comparison uses the same Plan A and Plan B rules as in the previous example. Risk 1 has $E_1=\$80{,}000$, $L_1=\$160{,}000$, and $B_1=\$100{,}000$. Risk 2 has $E_2=\$120{,}000$, $L_2=\$60{,}000$, and $B_2=\$150{,}000$. Calculate each plan’s portfolio premium and plan-implied loss prediction.

Risk 1. Its loss ratio is $160{,}000/80{,}000=2.00$. Plan A’s raw modification is $1+0.25(2.00-1)=1.25$, so its 1.20 cap applies. Plan B’s raw modification is $1+0.50(2.00-1)=1.50$, so its 1.30 cap applies. Plan A therefore predicts $\$80{,}000(1.20)=\$96{,}000$ and charges $\$100{,}000(1.20)=\$120{,}000$. Plan B predicts $\$80{,}000(1.30)=\$104{,}000$ and charges $\$100{,}000(1.30)=\$130{,}000$.

Risk 2. Its loss ratio is $60{,}000/120{,}000=0.50$. Plan A’s raw modification is $1+0.25(0.50-1)=0.875$, and Plan B’s is $1+0.50(0.50-1)=0.75$. Both are within their respective limits. Plan A predicts $\$120{,}000(0.875)=\$105{,}000$ and charges $\$150{,}000(0.875)=\$131{,}250$. Plan B predicts $\$120{,}000(0.75)=\$90{,}000$ and charges $\$150{,}000(0.75)=\$112{,}500$.

Aggregate and check. Plan A’s predicted losses total $\$96{,}000+\$105{,}000=\$201{,}000$, and its premiums total $\$120{,}000+\$131{,}250=\$251{,}250$. Plan B’s predicted losses total $\$104{,}000+\$90{,}000=\$194{,}000$, and its premiums total $\$130{,}000+\$112{,}500=\$242{,}500$. Relative to Plan A, Plan B reduces premiums by $\$8{,}750$, or $8{,}750/251{,}250=3.483\%$, and reduces predicted losses by $\$7{,}000$, or $7{,}000/201{,}000=3.483\%$ to three decimal places. The dollar reductions differ because the base premiums are not the same as expected losses. The method avoids treating a premium change as though it were automatically the same amount as a change in predicted losses.

Separate Plan Design from Rate-Level Adjustment

A proposed plan can change the distribution of premiums across risks and also change the portfolio’s total premium. Those are distinct effects. If the intended comparison holds total premium fixed, a separate normalization can be applied to the plan’s premium indications. For a fixed portfolio with baseline total premium $T_0=\sum_i B_i$, and raw plan premium indications $Q_i=B_i m_i$, a revenue-neutral factor is:

c_{\mathrm{RN}}=\frac{T_0}{\sum_i Q_i},\qquad P_i^{\mathrm{RN}}=c_{\mathrm{RN}}Q_i

This adjustment makes total charged premium equal to $T_0$ for the portfolio being evaluated. It is not part of every experience-rating plan, and it depends on the chosen fixed portfolio. Unless the comparison explicitly includes it, do not assume that premium neutrality has been imposed. Nor should the factor be applied to loss predictions: it changes the premium level, not the credibility-weighted loss estimate.

Worked Example: Revenue-Neutral Premiums with Different Predictions

Two hypothetical risks each have a base premium and expected loss of $\$100{,}000$. Risk 1 has observed losses of $\$150{,}000$; Risk 2 has observed losses of $\$75{,}000$. Plan A uses $Z=0.40$, and Plan B uses $Z=0.60$. Neither plan has a binding floor or cap. Compare their raw predictions and then calculate revenue-neutral premiums using the original total premium of $\$200{,}000$.

Plan A predictions. Risk 1 has raw modification $1+0.40(1.50-1)=1.20$, giving predicted losses and raw premium of $\$120{,}000$. Risk 2 has raw modification $1+0.40(0.75-1)=0.90$, giving predicted losses and raw premium of $\$90{,}000$. Plan A’s raw premium total is $\$210{,}000$, so $c_{\mathrm{RN}}=\$200{,}000/\$210{,}000=20/21$. Its revenue-neutral premiums are $\$120{,}000(20/21)=\$114{,}285.71$ and $\$90{,}000(20/21)=\$85{,}714.29$, rounded to cents.

Plan B predictions. Risk 1 has raw modification $1+0.60(1.50-1)=1.30$, giving predicted losses and raw premium of $\$130{,}000$. Risk 2 has raw modification $1+0.60(0.75-1)=0.85$, giving predicted losses and raw premium of $\$85{,}000$. Plan B’s raw premium total is $\$215{,}000$, so $c_{\mathrm{RN}}=\$200{,}000/\$215{,}000=40/43$. Its revenue-neutral premiums are $\$130{,}000(40/43)=\$120{,}930.23$ and $\$85{,}000(40/43)=\$79{,}069.77$, rounded to cents.

Interpretation and check. Both adjusted premium totals equal $\$200{,}000$. Plan B shifts $\$6{,}644.52$ more premium to Risk 1 and $\$6{,}644.52$ less to Risk 2 than Plan A, after rounding to cents. The plan-implied loss predictions remain $\$210{,}000$ under Plan A and $\$215{,}000$ under Plan B; they are not scaled by the revenue-neutral factors. This method avoids confusing a fixed-total-premium comparison with a comparison of the plans’ loss predictions.

Common Exam Trap and Practical Limits

A frequent error is to calculate a new plan’s modification using a different loss or exposure basis from the old plan. That does not isolate the plan change. For example, in the first worked example, Plan A predicts $\$285{,}000$. A mistaken comparison might use Plan B’s modification for Risk 1 but retain Plan A’s modification for Risk 2, obtaining $\$130{,}000+\$170{,}000=\$300{,}000$. That is not Plan B’s portfolio result. The correct calculation applies Plan B to both risks and gives $\$130{,}000+\$140{,}000=\$270{,}000$, a $-\$15{,}000$ difference from Plan A.

Other checks help keep the comparison interpretable:

Use matching bases. Expected and observed losses must refer to comparable periods and loss definitions. If losses have been adjusted, apply the same stated adjustment basis to both plans.
Apply limits to each risk. Calculate the raw modification first, then apply that plan’s floor and cap before aggregating.
Report dollar-weighted totals. Do not multiply an unweighted average modification by total premium unless the weights make that operation valid.
State the denominator for a percentage change. A change relative to Plan A is not the same as a change relative to Plan B or the unmodified total.
Do not infer behavioral or causal effects. A fixed-data comparison estimates how the formulas rate the same observed risks. It does not establish how future claims, retention, or risk behavior would change after implementation.

For an exam response aligned with the CAS Exam 8 (Advanced Ratemaking) syllabus, identify the fixed inputs, show each plan’s calculation, aggregate premiums and plan-implied predictions separately, and explain any rate-level adjustment. The result is a transparent estimate of the mechanical effect of changing formulas on the specified portfolio, subject to the assumptions and data used.

Check Your Understanding

For each problem, apply the stated plan rules to the fixed data and distinguish premiums from plan-implied loss predictions when they differ.

A risk has $E=\$50{,}000$, $L=\$70{,}000$, and $B=\$60{,}000$. A plan uses $Z=0.30$ with no binding floor or cap. Calculate its modification, premium, and predicted losses. (Answer: $m=1.12$; premium $=\$67{,}200$; predicted losses $=\$56{,}000$.)
A risk has $E=\$120{,}000$, $L=\$90{,}000$, and $B=\$150{,}000$. Plan A uses $Z=0.25$; Plan B uses $Z=0.50$. Neither has a binding floor or cap. Give each plan’s premium and predicted losses. (Answer: Plan A $m=0.9375$, premium $\$140{,}625$, predicted losses $\$112{,}500$; Plan B $m=0.875$, premium $\$131{,}250$, predicted losses $\$105{,}000$.)
Two risks each have $E=B=\$80{,}000$. Their observed losses are $\$100{,}000$ and $\$60{,}000$. A plan uses $Z=0.40$ with no binding limits. Find both modifications and the portfolio premium and predicted-loss total. (Answer: Modifications $1.10$ and $0.90$; premium total $\$160{,}000$; predicted-loss total $\$160{,}000$.)
Risk 1 has $E=\$40{,}000$, $L=\$80{,}000$, $B=\$40{,}000$; Risk 2 has $E=\$60{,}000$, $L=\$30{,}000$, $B=\$60{,}000$. A plan uses $Z=0.50$, a floor of 0.75, and a cap of 1.25. Calculate total premium and total predicted losses. (Answer: Applied modifications $1.25$ and $0.75$; premium total $\$95{,}000$; predicted-loss total $\$95{,}000$.)
Two risks each have $B=E=\$100{,}000$; observed losses are $\$150{,}000$ and $\$75{,}000$. Plan A uses $Z=0.40$; Plan B uses $Z=0.60$, with no binding limits. If each plan’s premiums are normalized to the original $\$200{,}000$ total, what are Plan B’s normalized risk premiums, and what are its unscaled predicted losses? (Answer: Normalized premiums are $\$120{,}930.23$ and $\$79{,}069.77$; unscaled predicted losses are $\$130{,}000$ and $\$85{,}000$, totaling $\$215{,}000$.)

What You'll Learn