Sensitivity Analysis for Model Risk

Why Sensitivity Analysis Matters

A rate indication depends on choices: which data are included, how future costs are projected, and which model specification is used. Even a technically correct calculation can appear precise while relying on uncertain choices. Sensitivity analysis makes the effect of those choices visible by recalculating an output under alternative, clearly stated conditions.

For ratemaking, the output might be an indicated average premium, a class rate, or a loss cost. The central question is not simply whether the number changes. It is how much it changes, which choice caused the change, whether impacts differ across segments, and whether the difference is meaningful for the decision at hand. A sensitivity result is conditional: it shows what the model produces under an alternative scenario. It is not, by itself, a probability-weighted forecast or evidence that either scenario is correct.

Definition: A sensitivity scenario is a specified alternative value, assumption, model specification, or data treatment used to recalculate an output while documenting what has changed. A useful comparison identifies the baseline, the alternative, the output measure, and any other inputs held constant.

Two basic summaries make comparisons easier to interpret. If $R_0$ is a baseline indication and $R_1$ is an alternative indication, the absolute change is $R_1-R_0$. The percentage change uses the baseline as its denominator. That denominator matters: the percentage answers how large the change is relative to the baseline, not relative to the alternative.

\text{Absolute change}=R_1-R_0, \qquad \text{Percentage change}= \frac{R_1-R_0}{R_0}\times 100\%.

For a fair comparison, define a baseline and change one feature at a time when the goal is to attribute an effect to that feature. Multiway scenarios can also be useful when assumptions plausibly vary together, but their combined result should not be described as the effect of any one assumption. In either case, the alternative must be plausible and clearly explained; an arbitrary extreme is a stress test, not automatically a reasonable central estimate.

Comparing a Projection Assumption

A common sensitivity question is how a rate indication responds to a change in a projection assumption. The following simplified formula assumes a current loss cost is trended for $n$ years at annual rate $t$, fixed expenses per exposure are $F$, and the variable expense ratio is $v$. It assumes the same exposure basis and expense assumptions in both scenarios, and that the variable expense ratio is applied to the indicated premium.

R(t)=\frac{L(1+t)^n+F}{1-v}, \qquad 0\leq v<1,

Here $L$ is the current loss cost per exposure and $R(t)$ is the indicated premium per exposure. The formula is a simplified illustration, not a universal ratemaking formula: additional components, different expense treatments, or a different projection structure require a correspondingly different calculation.

Worked Example: Changing the Annual Trend Assumption

A hypothetical book has a current loss cost of $500 per exposure, a two-year projection period, fixed expenses of $60 per exposure, and a variable expense ratio of 20%. Compare a 4% annual trend baseline with a 6% annual trend alternative. Calculate both indications and the alternative percentage change. Assume annual compounding and hold all other inputs fixed.

Method. Apply the trend for two years, add fixed expenses, and divide by the portion of premium remaining after variable expenses. This comparison isolates the trend assumption.

Baseline at 4%. The two-year trend factor is $1.04^2=1.0816$. The trended loss cost is $500(1.0816)=\$540.80$. Adding fixed expenses gives $\$540.80+\$60=\$600.80$. Therefore,

R(0.04)=\frac{600.80}{1-0.20} =\frac{600.80}{0.80} =\$751.00.

Alternative at 6%. The two-year trend factor is $1.06^2=1.1236$. The trended loss cost is $500(1.1236)=\$561.80$. Adding fixed expenses gives $\$561.80+\$60=\$621.80$. Thus,

R(0.06)=\frac{621.80}{0.80}=\$777.25.

The absolute increase is $\$777.25-\$751.00=\$26.25$ per exposure. The percentage increase, rounded to two decimal places, is $(26.25/751.00)\times100\%=3.4953\%$, or 3.50%. Check: the alternative indication exceeds the baseline, as expected when a positive trend assumption increases and all else is held constant.

Common mistake avoided: applying the annual trend only once over a two-year period would use $1.06$, not $1.06^2$, and understate the alternative loss cost. The stated projection period requires two annual compounding steps.

Why a Small Change in an Assumption Can Matter

A finite scenario comparison gives the actual model-output difference between two specified assumptions. A derivative can provide a local explanation of how an output responds near a baseline. For the simplified indication above, differentiate with respect to $t$, treating $L$, $F$, $n$, and $v$ as constants:

R(t)=\frac{L(1+t)^n+F}{1-v} \quad\Longrightarrow\quad \frac{dR}{dt} = \frac{1}{1-v}\frac{d}{dt}\left[L(1+t)^n+F\right] = \frac{nL(1+t)^{n-1}}{1-v}.

The derivative is positive when $L>0$, $n>0$, and $v<1$, so increasing the trend increases the indication under these assumptions. It measures dollars of rate change per unit change in the decimal trend rate. A change of one percentage point is a change of $0.01$, not a change of 1.

For the baseline in the worked example, the derivative at $t=0.04$ is $2(500)(1.04)/0.80=\$1{,}300$ per unit change in $t$. A one-percentage-point increase therefore corresponds to a local change of approximately $\$1{,}300(0.01)=\$13.00$. This is an approximation near 4%, not a replacement for recalculating a finite change. For example, moving from 4% to 5% gives an exact indication of $[500(1.05^2)+60]/0.80=\$764.0625$, or $764.06 to the nearest cent, which is $13.06 above $751.00.

This derivation helps explain both direction and scale. It does not establish that a trend assumption is appropriate, nor does it measure uncertainty in the underlying data. A sensitivity calculation answers a conditional question about the model; the basis for choosing assumptions must be considered separately.

Testing Alternative Data Treatments

Sensitivity analysis can also reveal how a data treatment affects an indication. Examples include alternative treatments of a questionable record or a defined large loss. The scenarios must reflect defensible interpretations of the data, not convenient adjustments chosen to produce a desired rate. If an alternative changes both the loss amount and the exposure denominator, report both changes so the comparison can be understood.

Worked Example: Capping a Large Loss in a Sensitivity Scenario

A hypothetical portfolio has 1,000 exposures and $720,000 of losses, including one $40,000 loss. A simplified indication divides loss cost by a target loss ratio of 60%. Compare the baseline with an alternative sensitivity treatment that caps that one loss at $20,000. Assume the exposure count and target loss ratio do not change.

Method. Calculate total losses, loss cost per exposure, and the indicated premium under each treatment. Then calculate the absolute and percentage changes against the uncapped baseline.

Baseline. Loss cost is $\$720{,}000/1{,}000=\$720$ per exposure. The indicated premium is $\$720/0.60=\$1{,}200.00$.

Alternative treatment. The cap reduces the included loss by $\$40{,}000-\$20{,}000=\$20{,}000$. Alternative total losses are $\$720{,}000-\$20{,}000=\$700{,}000$. Loss cost is $\$700{,}000/1{,}000=\$700$, giving an indicated premium of $\$700/0.60=\$1{,}166.67$, rounded to the nearest cent.

The absolute change, using the rounded indications, is $\$1{,}166.67-\$1{,}200.00=-\$33.33$. Using the unrounded alternative of $\$1{,}166.666\ldots$, the percentage change is $(-\$33.333\ldots/\$1{,}200)\times100\%=-2.7777\ldots\%$, or -2.78% to two decimal places. The result is a comparison of two treatments, not a conclusion that the capped treatment is preferable.

Common mistake avoided: subtracting the full $40,000 loss from the baseline would treat the alternative as if the loss were removed altogether. A $20,000 cap reduces the included amount by only $20,000.

Specification Changes and Segment-Level Effects

An alternative model specification can change a total indication and its distribution across rating segments. To make a controlled comparison, use the same exposure mix and the same definition of the output in both scenarios. An aggregate change alone can hide offsetting class-level movements, while separate class changes should not be added without weighting them by exposure.

Worked Example: Comparing Two Model Specifications

A hypothetical portfolio has 600 exposures in Class A and 400 in Class B. Under a baseline specification, the indicated premiums are $900 for Class A and $1,200 for Class B. An alternative specification gives $930 and $1,140, respectively. Calculate the portfolio average under each specification, the class-level percentage changes, and the aggregate percentage change. Assume the exposure counts and all other rating inputs are unchanged.

Baseline portfolio average. The exposure-weighted premium total is $600(\$900)+400(\$1{,}200)=\$540{,}000+\$480{,}000=\$1{,}020{,}000$. Dividing by $600+400=1{,}000$ exposures gives a baseline average of $\$1{,}020.00$.

Alternative portfolio average. The weighted total is $600(\$930)+400(\$1{,}140)=\$558{,}000+\$456{,}000=\$1{,}014{,}000$. The alternative average is $\$1{,}014{,}000/1{,}000=\$1{,}014.00$.

Class A changes by $(\$930-\$900)/\$900=3.3333\ldots\%$, or +3.33% to two decimal places. Class B changes by $(\$1{,}140-\$1{,}200)/\$1{,}200=-5.00\%$. The aggregate change is $\$1{,}014-\$1{,}020=-\$6.00$, and the aggregate percentage change is $(-\$6/\$1{,}020)\times100\%=-0.5882\ldots\%$, or -0.59% to two decimal places.

The check is the weighted total: the $18,000 increase for Class A is partly offset by the $24,000 decrease for Class B, leaving a $6,000 decrease across 1,000 exposures. The small aggregate movement therefore does not mean the specification has little effect on every class.

Common mistake avoided: taking an unweighted average of the two class premiums would give each class equal weight despite their different exposure counts. The portfolio average must reflect the stated exposure mix.

Interpreting and Communicating Results

A sensitivity table is often more informative than a single comparison. It can show the baseline and alternative values, absolute and percentage changes, and segment-level results. State the units and rounding convention, and make clear which inputs changed. When multiple scenarios are shown, identify whether they are one-at-a-time sensitivities or combinations of assumptions.

Review question	What to examine
Is the comparison controlled?	Confirm the baseline, the changed feature, and the inputs held constant.
How large is the effect?	Report both the absolute change and the percentage change against the baseline.
Is the aggregate masking differences?	Review relevant segment-level impacts as well as the total.
Is the alternative interpretable?	State its rationale, limitations, and whether it is a plausible scenario or a stress test.

Materiality is contextual. A small average change may matter if it is concentrated in a particular segment, while a larger change may be less consequential for a decision if it falls within an understood range and does not alter the conclusion. There is no universal percentage threshold that makes a sensitivity material in every setting. The interpretation should consider the size of the effect, its distribution, the purpose of the analysis, and the uncertainty around the inputs.

Sensitivity analysis complements, but does not replace, validation and review. A model can be stable across the scenarios tested and still be misspecified; it can also be sensitive to an assumption for a legitimate reason. Explain what was tested and what was not. Do not present a scenario as a probability, a confidence interval, or a forecast unless a separate method supports that interpretation.

Key takeaway: A useful sensitivity analysis changes a clearly identified input or specification, recalculates the same output on the same basis, and reports both the size and distribution of the effect. Interpret the result as conditional evidence about model dependence, not as proof that an alternative is correct.

Check Your Understanding

For each problem, compare against the stated baseline and round money to the nearest cent and percentage changes to two decimal places unless noted otherwise.

A hypothetical indication uses a $420 current loss cost, a three-year period, a 3% annual trend, $45 fixed expenses per exposure, and a 25% variable expense ratio. Using $R=[L(1+t)^n+F]/(1-v)$, calculate the indicated premium. (Answer: $671.93.)
A portfolio has $540,000 of losses and 900 exposures, with an indication equal to loss cost divided by a 60% target loss ratio. In an alternative treatment, $18,000 of losses are excluded and exposures and target loss ratio remain unchanged. Find both indications and the percentage change. (Answer: Baseline $1,000.00; alternative $966.67; change -3.33%.)
Two classes have 250 and 750 exposures. Baseline premiums are $800 and $1,000; alternative premiums are $840 and $960. Find the two exposure-weighted portfolio averages and the aggregate percentage change. (Answer: Baseline $950.00; alternative $930.00; change -2.11%.)
An indication rises from $1,045 to $1,086.80 under an alternative assumption. Calculate the absolute and percentage changes relative to the baseline. (Answer: +$41.80; +4.00%.)
For $R(t)=[L(1+t)^n+F]/(1-v)$, use $L=\$600$, $n=2$, $t=0.05$, and $v=0.20$ to calculate $dR/dt$. Then state the approximate change for a one-percentage-point increase in $t$. (Answer: $1,575 per unit change in $t$; approximately +$15.75 for a one-percentage-point increase.)

What You'll Learn