Sensitivity and Scenario Testing for Catastrophe Models

Why Sensitivity Testing Matters

A catastrophe-model result is conditional on assumptions about the hazard, the damage caused by that hazard, the exposure being insured, and the policy terms applied to loss. A single modeled average annual loss or tail estimate can conceal how strongly the result depends on any one of those assumptions. Sensitivity testing makes that dependence visible: it changes a specified input while holding the others fixed, then compares the outputs.

A sensitivity test is not, by itself, a forecast that the changed assumption will occur. A scenario is a defined alternative set of assumptions, which may be chosen to explore a plausible stress even if it has no assigned probability. For a useful comparison, state what changes, what remains fixed, which outputs are compared, and whether each result is conditional on a modeled annual distribution or on a particular stress.

Definition: A sensitivity test measures how a model output changes when one input assumption is varied while other inputs are held fixed. A scenario test evaluates output under a specified alternative set of assumptions, potentially changing several inputs together. Neither term implies that the changed assumptions are predictions or have an assigned probability.

Choose the Input and Output Deliberately

The main input groups should be tested distinctly where possible. Exposure assumptions include the number, location, value, and characteristics of insured risks. Hazard assumptions describe the occurrence or intensity of the peril. Vulnerability assumptions describe how an exposed risk is damaged at a given intensity. Financial assumptions include deductibles, limits, and other policy terms that determine how ground-up loss becomes insured loss. The earlier material on hazard and vulnerability functions, policy terms, and event loss tables provides the model components; the task here is to test their sensitivity coherently.

Useful output measures include average annual loss (AAL), an exceedance probability at a stated loss amount, and a loss quantile at a stated probability. AAL is the expected annual loss. For a discrete annual loss distribution with outcomes $L_j$ and probabilities $p_j$,

\operatorname{AAL}=E[L]=\sum_j p_jL_j,\qquad \sum_jp_j=1.

For a discrete loss distribution, use the quantile convention

Q_q=\inf\{x:P(L\leq x)\geq q\}.

This definition matters when a cumulative probability lands exactly on $q$, or when the distribution has a jump over $q$. Always state the probability level and the convention used. A change in AAL does not determine the change in a tail quantile: the two measures summarize different features of the distribution.

Key principle: A controlled comparison changes only the selected assumption and keeps the exposure definition, policy terms, other model settings, and output definitions consistent. If several assumptions change together, report the result as a combined scenario rather than attributing the full change to one input.

A Practical Testing Sequence

A well-designed test starts with a documented baseline and then changes assumptions in a way that can be interpreted. When a model is stochastic, use the same simulation setup where feasible so that differences are less obscured by random variation. A change in event probabilities or event severities should be distinguished from simulation noise; the model’s stochastic features and the method used to compare runs should be recorded.

Specify the baseline.
Record the exposure set, hazard and vulnerability assumptions, policy terms, model settings, and the baseline outputs to be compared.

Define the test.
Name the input being changed and its new value or range. For a multi-input stress, list each changed assumption explicitly.

Hold other assumptions fixed where appropriate.
This makes a one-factor sensitivity interpretable. If inputs cannot reasonably be separated, explain why and treat the result as a scenario.

Recalculate on the same basis.
Apply the same annual-loss definition, financial terms, quantile convention, and reporting units in the baseline and test.

Interpret the difference.
Report both the absolute and, when useful, percentage change; explain which assumption caused the change and what the test does not establish.

Why the Calculation Works

For a finite set of mutually exclusive annual outcomes, the expected loss is the probability-weighted average of the loss in each outcome. If the outcome probabilities change but the losses remain fixed, then

\Delta\operatorname{AAL} =\sum_j p_j^{\text{test}}L_j-\sum_jp_j^{\text{base}}L_j =\sum_j\left(p_j^{\text{test}}-p_j^{\text{base}}\right)L_j.

The second equality follows by subtracting the two finite sums term by term. It shows why a hazard-frequency test can change AAL even when event severities are unchanged: probability shifted toward higher-loss outcomes raises the weighted average, while a shift toward lower-loss outcomes reduces it. The result does not imply that every tail measure changes by the same amount, because a quantile depends on where cumulative probability crosses its specified level.

A vulnerability change requires additional care. If ground-up loss $X$ is multiplied by a severity factor $c$, insured payment need not also be multiplied by $c$. For a deductible $d$ and per-loss limit $u$, the payment is $Y^L=\min((X-d)_+,u)$, where $(z)_+=\max(z,0)$. In general, $\min((cX-d)_+,u)$ is not equal to $c\min((X-d)_+,u)$. The deductible and limit can make the response nonlinear.

Worked Example: A Hazard-Probability Sensitivity

A hypothetical portfolio has three mutually exclusive annual loss outcomes. In the baseline, the losses are $0, $200,000, and $700,000, with probabilities 0.85, 0.10, and 0.05. A hazard sensitivity changes only the probabilities to 0.80, 0.13, and 0.07, respectively. Calculate baseline and test AAL, then calculate the 95th-percentile annual loss in each case using $Q_q=\inf\{x:P(L\leq x)\geq q\}$.

Method and assumptions. Use the probability-weighted average for AAL. The outcomes are annual losses in dollars, the probabilities in each distribution sum to 1, and loss amounts are unchanged in the hazard test.

Baseline AAL. The three weighted losses are $0.85(0)=0$, $0.10(200{,}000)=20{,}000$, and $0.05(700{,}000)=35{,}000$. Therefore,

\operatorname{AAL}_{\text{base}}=0+20{,}000+35{,}000=\$55{,}000.

The baseline probabilities sum to $0.85+0.10+0.05=1.00$. For the 95th percentile, cumulative probability is 0.85 at $0 and 0.95 at $200,000. Thus $Q_{0.95}=\$200{,}000$.

Test AAL. The test probabilities sum to $0.80+0.13+0.07=1.00$. The weighted losses are $0.80(0)=0$, $0.13(200{,}000)=26{,}000$, and $0.07(700{,}000)=49{,}000$. Hence,

\operatorname{AAL}_{\text{test}}=0+26{,}000+49{,}000=\$75{,}000.

For the test distribution, cumulative probability is 0.80 at $0, then $0.80+0.13=0.93$ at $200,000, and 1.00 at $700,000. The first loss with cumulative probability at least 0.95 is $700,000, so $Q_{0.95}=\$700{,}000$.

Check and interpretation. The AAL increases by $20,000, from $55,000 to $75,000; the 95th percentile increases by $500,000. The calculation checks because both probability sets sum to 1 and both outputs are computed from their own distribution. The common mistake this method avoids is changing event probabilities without stating the new probabilities, which would leave both test results undetermined.

Worked Example: Vulnerability Severity With a Deductible

In a hypothetical annual model, a single insured loss is $200,000 with probability 0.10, $600,000 with probability 0.02, and $0 otherwise. The policy has a $100,000 deductible and no payment limit. A vulnerability sensitivity increases each nonzero ground-up loss by 15%, without changing the probabilities or deductible. Calculate baseline and test AAL and the 95th-percentile insured annual loss.

Method and assumptions. Apply the deductible to each ground-up loss first: payment is $Y^L=(X-d)_+$, with $d=\$100{,}000$. The event outcomes are mutually exclusive. The probability of no loss is $1-0.10-0.02=0.88$.

Baseline payments and AAL. A $200,000 ground-up loss produces payment $200{,}000-100{,}000=\$100{,}000$. A $600,000 loss produces payment $600{,}000-100{,}000=\$500{,}000$. The expected annual payment is

\operatorname{AAL}_{\text{base}} =0.88(0)+0.10(100{,}000)+0.02(500{,}000) =10{,}000+10{,}000 =\$20{,}000.

The cumulative insured-loss probability is 0.88 at $0 and 0.98 at $100,000. Therefore, the 95th-percentile insured annual loss is $100,000.

Test payments and AAL. The increased ground-up losses are $200{,}000(1.15)=\$230{,}000$ and $600{,}000(1.15)=\$690{,}000$. After the unchanged deductible, the payments are $230{,}000-100{,}000=\$130{,}000$ and $690{,}000-100{,}000=\$590{,}000$. Thus,

\operatorname{AAL}_{\text{test}} =0.88(0)+0.10(130{,}000)+0.02(590{,}000) =13{,}000+11{,}800 =\$24{,}800.

The cumulative probability at $130,000 is $0.88+0.10=0.98$, so the test 95th percentile is $130,000. Check: the AAL rises by $4,800, and the test quantile is the first loss level at which cumulative probability reaches 0.95. The common mistake is to multiply the baseline insured AAL by 1.15; the deductible means that a 15% ground-up severity increase does not generally produce a 15% insured-loss increase.

Exposure and Financial-Term Scenarios

Exposure and policy terms should also be varied separately when the objective is to understand their individual effects. An exposure change may alter the number or value of risks represented in an event loss calculation. A financial-term change modifies the payment for a given ground-up loss. If both change in a single run, the result is still useful as a combined scenario, but it cannot identify either input’s effect in isolation.

Worked Example: Separating Exposure, Deductible, and Severity Tests

A hypothetical portfolio contains 50 identical insured locations. In an annual model, no catastrophe occurs with probability 0.96; with probability 0.04, one event causes a $20,000 ground-up loss at each location. Each location has a $5,000 deductible and a $30,000 per-loss limit. Calculate baseline AAL and the 99th-percentile annual insured loss. Then test (a) 10% more locations, (b) an $8,000 deductible with the original exposure count, and (c) a 20% increase in event severity with the original exposure count and terms.

Method and assumptions. Each location’s payment is $\min((X-d)_+,u)$, where $X$ is ground-up loss, $d$ is the deductible, and $u$ is the limit. The two annual outcomes are zero and the total payment for the stated event. Treat each specified exposure count and event outcome as given; no additional dependence assumption is needed.

Baseline. Per location, payment in the event is $\min(20{,}000-5{,}000,30{,}000)=\$15{,}000$. The event-year portfolio loss is $50(15{,}000)=\$750{,}000$. Therefore,

\operatorname{AAL}_{\text{base}}=0.96(0)+0.04(750{,}000)=\$30{,}000.

The cumulative probability is 0.96 at $0 and 1.00 at $750,000. Since 0.96 is below 0.99, the 99th-percentile annual loss is $750,000.

(a) Exposure count increases by 10%. The new count is $50(1.10)=55$. Per-location event payment remains $15,000, so the event-year loss is $55(15{,}000)=\$825{,}000$. The test AAL is $0.04(825{,}000)=\$33{,}000$, and the 99th percentile is $825,000. Both event loss and AAL are 10% higher because the modeled risks are identical and the terms and event probability are unchanged.

(b) Deductible increases to $8,000. With the original 50 locations, each event payment is $\min(20{,}000-8{,}000,30{,}000)=\$12{,}000$. Event-year loss is $50(12{,}000)=\$600{,}000$, and AAL is $0.04(600{,}000)=\$24{,}000$. The 99th percentile is $600,000.

(c) Event severity increases by 20%. Ground-up loss per location becomes $20{,}000(1.20)=\$24{,}000$. Payment is $\min(24{,}000-5{,}000,30{,}000)=\$19{,}000$, so event-year loss is $50(19{,}000)=\$950{,}000$. AAL is $0.04(950{,}000)=\$38{,}000$, and the 99th percentile is $950,000.

Check and interpretation. The event probability remains 0.04 in all tests, and every AAL equals 0.04 times its corresponding event-year loss. The 10% exposure increase raises AAL from $30,000 to $33,000, whereas the deductible increase reduces it to $24,000. The 20% severity increase raises it to $38,000, not $36,000: the deductible is fixed in dollars, so the payment per location changes from $15,000 to $19,000. The common mistake is to apply a percentage change to insured loss without recalculating the policy payment.

Interpreting and Communicating Results

A sensitivity table can make controlled comparisons easy to audit. Include the baseline value, test value, output under each, and the difference. For example, report “AAL increases by $4,800 (24% of baseline)” rather than reporting only the percentage; percentages can be misleading when the baseline is small. Keep dollar amounts, annual units, probability levels, and quantile conventions consistent.

Test type	Example changed input	Interpretation to preserve
Hazard	Event probabilities or intensity assumptions	Separate probability changes from changes in event loss amounts
Vulnerability	Damage response at a given hazard intensity	Reapply deductibles and limits to revised ground-up losses
Exposure	Number, location, or insured value of risks	State which exposure attributes changed and which were held fixed
Financial terms	Deductible or limit	Apply the revised term to the same ground-up loss distribution

A stress test identifies model response under its stated assumptions. It does not establish that the stress is likely, measure the uncertainty of the model, or show that a change in one modeled input is the only real-world cause of a changed outcome. A sound interpretation also checks that the model responds in a direction consistent with the tested mechanics. For instance, increasing a deductible should not increase payment for an unchanged loss under standard deductible application.

Common Exam Traps

Unspecified alternative assumptions. If a scenario changes probabilities, give the new probabilities. Baseline probabilities alone cannot determine scenario AAL or a scenario quantile.
Changing multiple inputs but claiming a single cause. Call this a combined scenario, or run separate controlled tests before attributing the change.
Scaling insured loss directly. Recalculate payment from ground-up loss and policy terms; deductibles and limits can make the response nonlinear.
Reporting a tail number without its definition. Identify the percentile or exceedance probability and the convention used, particularly for a discrete distribution.
Confusing a stress with a forecast. A scenario result is conditional on its assumptions; it does not assign a probability to those assumptions unless one is explicitly provided.

Key takeaway: A useful catastrophe sensitivity test is controlled, fully specified, and interpreted on a consistent output basis. Change the intended assumption, recalculate losses through the applicable model and policy mechanics, and distinguish the resulting conditional estimate from a prediction.

Check Your Understanding

For each problem, use the stated discrete quantile convention where a quantile is requested: $Q_q=\inf\{x:P(L\leq x)\geq q\}$.

A hypothetical annual loss distribution has losses $0, $100,000, and $500,000 with probabilities 0.90, 0.08, and 0.02. A hazard test changes the probabilities to 0.86, 0.10, and 0.04, leaving losses fixed. Find test AAL and the test 97th-percentile loss. (Answer: AAL = $30,000; 97th percentile = $500,000)
An annual model has ground-up losses $80,000 and $300,000 with probabilities 0.10 and 0.03, and zero otherwise. A $50,000 deductible applies, with no limit. Vulnerability increases both nonzero losses by 10%, with probabilities and deductible unchanged. Find the new AAL. (Answer: $12,200)
A hypothetical portfolio has 40 identical locations. An event occurs with annual probability 0.025 and causes $12,000 ground-up loss per location; each location has a $2,000 deductible and no limit. Find baseline AAL, then AAL if exposure increases to 44 identical locations with all else fixed. (Answer: baseline AAL = $10,000; test AAL = $11,000)
Annual ground-up loss is $0 with probability 0.90, $60,000 with probability 0.08, and $200,000 with probability 0.02. A $10,000 deductible and a $100,000 per-loss limit apply. Find AAL, then find AAL if only the limit increases to $150,000. (Answer: baseline AAL = $6,000; test AAL = $7,000)
A baseline annual distribution has losses $0, $250,000, and $1,000,000 with probabilities 0.92, 0.06, and 0.02. A scenario changes the probabilities to 0.90, 0.07, and 0.03, leaving losses fixed. Find scenario AAL and scenario 98th-percentile loss. (Answer: AAL = $47,500; 98th percentile = $1,000,000)

What You'll Learn