Why Catastrophe Model Changes Need Governance
A catastrophe model result depends on more than the vendor or software version. It also depends on exposure data, assumptions, settings, code, and the way results are transferred into internal reporting. A change to any of these can alter modeled loss, reinsurance estimates, capital measures, or decisions about risk appetite. The governance task is to establish what changed, assess its effect on decisions, and ensure that only an authorized, tested version is used.
Governance does not mean rejecting every change that produces a large movement. A material increase may be a valid reflection of revised hazard information or corrected exposure data. Conversely, a small portfolio-wide movement may conceal a large impact in a particular region, peril, or decision metric. Controls should therefore connect the change to its cause, quantify its effect on relevant outputs, and route it to the appropriate reviewers.
Changes commonly include a new vendor model release, a revised hazard or vulnerability assumption, corrected exposure records, a change in financial terms, or a software and data-processing update. These are different sources of change and should not be combined without a record of their separate effects. The intended use of a result also matters: a change affecting a pricing decision may require different review from one affecting a capital estimate or reinsurance purchase.
A Controlled Assessment and Approval Process
A sound process begins before implementation. The change owner records the reason for the change, the affected model components, the portfolios and outputs in scope, and the proposed effective date. The owner then compares the proposed configuration with the current approved baseline. That comparison should use consistent exposure scope, financial terms, output definitions, and reporting periods wherever possible. Otherwise, an apparent model effect may actually be caused by a difference in what was measured.
Record whether the source is a model release, assumption, exposure-data revision, financial-term update, or implementation change. Identify the accountable owner and affected uses.
Freeze the approved baseline, scope, and output definitions. Specify which portfolios, perils, and measures will be compared, and note any unavoidable differences in basis.
Calculate absolute and relative movements in relevant outputs. Review whether the movement is concentrated in a segment or changes a decision, and identify limitations in the comparison.
Require review by people with appropriate modeling, data, business, and risk responsibilities. Apply approval thresholds from the organization’s own policy, not a universal percentage.
Use an authorized release, preserve the prior approved version, test the production process, and retain a means to restore the prior version if implementation fails.
Record the decision, evidence, assumptions, limitations, implementation date, and affected outputs. Explain the effect and its cause to decision-makers in terms relevant to their decisions.
Impact should be reported in both absolute and relative terms where the baseline is nonzero. If \(L_0\) is a baseline measure and \(L_1\) is the measure after the change, the absolute change is \(L_1-L_0\), while the relative change is \((L_1-L_0)/L_0\). The denominator must be stated. If \(L_0=0\), the relative change is undefined, so report the absolute change and use a pre-established alternative materiality test.
A useful way to separate causes is to change one component at a time in a stated sequence. Let \(f(v,a,d)\) be an output produced using model version \(v\), assumptions \(a\), and data \(d\). For a change from \((v_0,a_0,d_0)\) to \((v_1,a_1,d_1)\), one exact decomposition is:
The identity holds because the intermediate terms cancel: the final positive term \(f(v_1,a_0,d_0)\) in the first bracket cancels the corresponding negative term in the second, and \(f(v_1,a_1,d_0)\) cancels between the second and third. The remaining terms are exactly the final output minus the baseline output. The components are attributed along the chosen sequence; if model, assumption, and data effects interact, another sequence can produce different component amounts even though the total change is the same. The sequence must therefore be recorded rather than presented as a uniquely determined attribution.
Worked Assessments
Worked Example: Assessing a Model-Version Update
A hypothetical insurer compares an approved catastrophe model with a proposed release. In a simplified annual event-count representation, the number of events \(N\) is Poisson: \(N\sim\operatorname{Poisson}(\lambda)\). Each event causes a fixed gross loss of $60 million, and annual aggregate loss is \(X=60N\) million dollars. Assume the baseline has \(\lambda=0.08\), while the proposed version has \(\lambda=0.10\); all other inputs and the portfolio scope are held fixed. The insurer’s hypothetical internal policy requires escalation if expected annual loss increases by more than 15%. Should the change be escalated?
Method. Compare expected annual loss and the probability of at least one loss under the two Poisson models. For a Poisson count, \(E[N]=\lambda\), so \(E[X]=60E[N]=60\lambda\) million dollars.
Baseline. \(E[X_0]=60(0.08)=4.80\) million dollars. The probability of at least one event is \(P(N\ge 1)=1-P(N=0)=1-e^{-0.08}=0.0769\), to four decimal places.
Proposed version. \(E[X_1]=60(0.10)=6.00\) million dollars. The probability of at least one event is \(1-e^{-0.10}=0.0952\), to four decimal places.
Impact and decision. The expected-loss increase is \(6.00-4.80=1.20\) million dollars. Relative to the baseline, this is \(1.20/4.80=0.25\), or 25%. The probability of at least one event rises by \(0.0952-0.0769=0.0183\), or 1.83 percentage points using the unrounded probabilities. Since the expected-loss increase exceeds the stated 15% escalation threshold, the change requires escalation under this hypothetical policy.
Check and common mistake. The result is consistent with the event-rate ratio \(0.10/0.08=1.25\): with fixed event severity, expected loss rises by 25%. A common error is to report the absolute increase of $1.20 million as though it were the relative change. The correct relative change divides by the baseline expected loss of $4.80 million.
Worked Example: Governing a Corrected Exposure Record
A hypothetical property portfolio has insured value of $80 million in a region. A data correction changes the value to $92 million. In the simplified model, one annual event occurs with probability \(p=0.04\); conditional on an event, gross loss is 25% of insured value; otherwise loss is zero. A hypothetical reinsurance contract pays the amount of event loss above a $5 million attachment, capped at a $20 million limit. The internal policy requires review for an expected annual loss increase of 10% or more. Quantify the impact and identify the appropriate governance response.
Method and model assumptions. The annual loss has a two-outcome distribution: it is zero with probability 0.96 and equals 25% of insured value with probability 0.04. Expected annual loss is therefore \(0.04\) times the event loss. For event loss \(x\), the recovery is \(\min(\max(x-5,0),20)\), with all dollar amounts in millions.
Before correction. Event loss is \(0.25(80)=20\) million dollars. Expected annual loss is \(0.04(20)=0.80\) million dollars. Event recovery is \(\min(\max(20-5,0),20)=15\) million dollars, so expected annual recovery is \(0.04(15)=0.60\) million dollars.
After correction. Event loss is \(0.25(92)=23\) million dollars. Expected annual loss is \(0.04(23)=0.92\) million dollars. Event recovery is \(\min(\max(23-5,0),20)=18\) million dollars, so expected annual recovery is \(0.04(18)=0.72\) million dollars.
Impact and decision. Expected annual loss increases by \(0.92-0.80=0.12\) million dollars, or \(0.12/0.80=15\%\). Expected annual recovery increases by \(0.72-0.60=0.12\) million dollars. The expected-loss impact meets the 10% review threshold. The correction should be supported by the responsible data owner, assessed against the source record, and reviewed for its effect on both gross and reinsurance outputs before the corrected data is approved for production use.
Check and common mistake. The recovery is calculated from event loss, not directly from insured value. Applying the attachment to the $92 million insured value would be wrong: the modeled event loss is $23 million, and the layer recovery is $18 million. Keeping the event probability and loss ratio fixed also makes clear which inputs changed.
Worked Example: Blocking an Implementation Error
A hypothetical implementation update changes the way event losses are imported. For the annual aggregate, two independent event indicators are used: event A occurs with probability 0.10 and causes a $12 million loss; event B occurs with probability 0.05 and causes a $20 million loss. The proposed import mistakenly reads event B’s $20 million value as $2 million. Determine the effect on expected annual loss and on the probability that aggregate loss is at least $20 million. Should the proposed implementation be approved?
Method and assumptions. Each indicator is Bernoulli with the stated probability, the indicators are independent, and annual aggregate loss is the sum of the two event losses. For an event with loss \(c\) and occurrence probability \(p\), its expected loss is \(pc\).
Approved implementation. Expected annual aggregate loss is \(0.10(12)+0.05(20)=1.20+1.00=2.20\) million dollars. Aggregate loss is at least $20 million whenever event B occurs: if B occurs, its $20 million loss alone meets the threshold, whether or not A occurs. Thus the probability is 0.05.
Proposed implementation. With the erroneous $2 million event B loss, expected annual aggregate loss is \(0.10(12)+0.05(2)=1.20+0.10=1.30\) million dollars. The change is \(1.30-2.20=-0.90\) million dollars, or \(-0.90/2.20=-40.91\%\), rounded to two decimals. The possible aggregate losses are $0 million, $2 million, $12 million, and $14 million, so the probability of aggregate loss of at least $20 million is zero.
Decision and check. The proposed implementation must not be approved. The independent calculation shows a 40.91% expected-loss decrease and removes all modeled probability of reaching the stated threshold, solely because of a unit-conversion error. A control that reconciles imported event-loss values to the approved source would identify that $20 million was incorrectly converted to $2 million. The calculation also checks: after the error, even both events together produce only \(12+2=14\) million dollars.
Approvals, Records, and Decision Communication
Approval should be proportionate to the source and potential consequence of a change. A routine correction with a clear source record may need a defined data-owner sign-off and model-owner review. A major version release, material assumption change, or implementation affecting production calculations may call for independent review, business-owner acceptance, and approval at a higher risk or governance level. Internal policies should state who can approve which classes of change, what impact thresholds require escalation, and what evidence is required. Thresholds support consistent routing; they do not replace judgment about concentration, uncertainty, or a decision that could change.
The change record should allow a reviewer to reconstruct what was approved. At a minimum, it should identify:
- the old and new model versions, data snapshots, assumptions, and relevant settings;
- the owner, rationale, scope, effective date, and affected business uses;
- the comparison method, output movements, limitations, and materiality decision;
- the reviewers, approvals, test evidence, and any conditions attached to approval;
- the production implementation, access or release controls, and rollback arrangement; and
- the people notified, the decision implications communicated, and any follow-up actions.
Communication should distinguish what changed from what the change means. Decision-makers need to know the direction and size of movement, which parts of the portfolio are affected, whether the change reflects new information or a process correction, and what uncertainty or limitation remains. A single portfolio-wide percentage may not be sufficient when the change alters a reinsurance decision, a capital plan, or a risk limit in a concentrated segment. The message should be concise enough to support action but traceable to the approved comparison and record.
A frequent governance trap is to treat vendor publication, model-owner acceptance, and production approval as the same event. They are separate: a release can be available without being validated for a particular use, and an approved change can still be implemented incorrectly. A second trap is comparing results that differ in both model version and exposure scope, then attributing the entire difference to the model release. Preserve a controlled baseline and isolate causes where feasible; if isolation is not feasible, state the combined nature of the impact rather than claiming a precise attribution.
Check Your Understanding
For each original scenario, calculate the requested impact and identify the relevant control decision.
- A proposed model version changes an annual event count from Poisson with \(\lambda=0.06\) to Poisson with \(\lambda=0.09\). Each event has fixed severity of $40 million. Calculate baseline and proposed expected annual loss, relative change, and probability of at least one event under each version. (Answer: $2.40 million and $3.60 million; 50% increase; probabilities 0.0582 and 0.0861, respectively.)
- A data correction changes insured value from $50 million to $56 million. In a hypothetical model, an annual event has probability 0.05 and causes 20% of insured value as loss. Find the expected annual loss change and relative change. (Answer: $0.50 million to $0.56 million, an increase of $0.06 million or 12%.)
- Two independent annual event indicators have probabilities 0.08 and 0.03 and fixed losses of $10 million and $25 million, respectively. Calculate expected aggregate loss and the probability aggregate loss is at least $25 million. (Answer: $1.55 million and 0.0300.)
- A proposed change increases an approved expected-loss measure from $4.00 million to $4.60 million. Internal policy requires escalation for increases greater than 10%. Calculate the change and state whether escalation is required. (Answer: $0.60 million, or 15%; escalation is required.)
- Along a stated attribution path, a model-version change moves expected loss from $3.00 million to $3.30 million, and a subsequent assumption change moves it from $3.30 million to $3.96 million. Calculate the path contributions and total relative change from baseline. (Answer: version contribution $0.30 million; assumption contribution $0.66 million; total increase $0.96 million, or 32% of baseline.)