Materiality Testing for Rating-Model Updates

Why Test Materiality?

A rating-model update can change indicated premiums substantially for one segment while barely moving the portfolio total. The reverse can also occur: modest changes across several large segments can accumulate into a meaningful portfolio impact. A materiality test makes these effects visible before a change proceeds through review and approval.

Materiality is not a universal percentage embedded in a formula. It is a documented governance rule applied to comparable measurements. The reviewer needs to know which indications are compared, which exposure base is held fixed, what thresholds apply, and whether a threshold is triggered by any one measure or by a combination of measures. The approach below is aligned with the CAS Exam 8 (Advanced Ratemaking) syllabus.

Definition: A materiality test for a rating-model update compares pre-change and post-change indicated premiums or relativities on a defined, comparable basis, then applies documented criteria to determine whether the measured impact requires additional review or approval. The test is a governance screen; it does not, by itself, establish that a model change is actuarially sound or unsound.

Set Up a Comparable Comparison

Let $P_s^0$ be the pre-change indicated premium for segment $s$, and let $P_s^1$ be its post-change indicated premium. These amounts should be evaluated on the same exposure basis and under consistent assumptions about the portfolio, premium period, and other inputs. The segment’s dollar change is $P_s^1-P_s^0$. Its percentage change, when $P_s^0>0$, is measured relative to the pre-change amount.

\Delta_s=P_s^1-P_s^0, \qquad g_s=\frac{P_s^1-P_s^0}{P_s^0}.

A positive $g_s$ indicates an increase; a negative value indicates a decrease. Percentage change is undefined when the pre-change premium is zero, so in that case use an appropriate absolute measure or a separately documented method rather than dividing by zero.

For a portfolio with total pre-change premium $P^0=\sum_s P_s^0>0$, define the total percentage change using aggregate premiums. This is not generally the unweighted average of segment percentage changes.

g_{\mathrm{portfolio}} =\frac{\sum_s P_s^1-\sum_s P_s^0}{\sum_s P_s^0} =\sum_s w_s g_s, \qquad w_s=\frac{P_s^0}{\sum_j P_j^0}.

Why the Portfolio Change Is Premium-Weighted

The weighting formula follows directly from summing the segment dollar changes. Since $\Delta_s=P_s^0g_s$, the total change divided by the old portfolio premium is:

\frac{\sum_s \Delta_s}{\sum_s P_s^0} = \frac{\sum_s P_s^0g_s}{\sum_s P_s^0} = \sum_s \left(\frac{P_s^0}{\sum_j P_j^0}\right)g_s.

Thus, each segment’s percentage change is weighted by its share of pre-change premium. A small segment with a large percentage change may have limited influence on the portfolio result, while a modest change in a large segment may have substantial influence. This calculation assumes the segment premiums being aggregated are comparable and collectively represent the portfolio being tested.

Worked Example: Apply Segment and Portfolio Thresholds

A hypothetical insurer compares rating indications for the same exposure set. Pre-change premiums are $1,200,000 for Segment A, $800,000 for Segment B, and $500,000 for Segment C. Post-change indications are $1,236,000, $776,000, and $550,000, respectively. The documented screening rule calls for review if the portfolio change exceeds 2.0% in absolute value, if any segment change exceeds 5.0% in absolute value, or if the absolute portfolio dollar change exceeds $75,000. Any one trigger is sufficient.

Step 1: Calculate segment dollar changes. Segment A changes by $1,236,000 − $1,200,000 = $36,000. Segment B changes by $776,000 − $800,000 = −$24,000. Segment C changes by $550,000 − $500,000 = $50,000.

Step 2: Calculate segment percentage changes. Segment A changes by $36,000 ÷ $1,200,000 = 0.030, or 3.0%. Segment B changes by −$24,000 ÷ $800,000 = −0.030, or −3.0%. Segment C changes by $50,000 ÷ $500,000 = 0.100, or 10.0%.

Step 3: Aggregate the premiums. Pre-change portfolio premium is $1,200,000 + $800,000 + $500,000 = $2,500,000. Post-change premium is $1,236,000 + $776,000 + $550,000 = $2,562,000. The portfolio dollar change is $2,562,000 − $2,500,000 = $62,000, and the percentage change is $62,000 ÷ $2,500,000 = 0.0248, or 2.48%.

Step 4: Apply the rule. The portfolio percentage change of 2.48% exceeds 2.0%, so that threshold is triggered. Segment C’s 10.0% change exceeds 5.0%, so the segment threshold is also triggered. The $62,000 portfolio dollar change does not exceed $75,000. Because the rule requires review when any threshold is triggered, the update requires review.

Check. The pre-change premium weights are 0.48, 0.32, and 0.20. Their weighted percentage change is $(0.48)(3.0\%)+(0.32)(-3.0\%)+(0.20)(10.0\%)=2.48\%$, matching the aggregate calculation. The method avoids treating a non-triggered dollar threshold as if it overrode the other triggered criteria.

Test Relativities as Well as Total Premium

A rating-model update can redistribute indicated premium among segments without changing the portfolio total. This can happen when segment relativities are normalized to a common overall level. Comparing only total premium would miss the redistribution; comparing only unnormalized factors could confuse a change in overall level with a change in relative segmentation.

A relativity comparison should state its reference, normalization basis, and exposure weights. For example, if raw factors are normalized so their exposure-weighted mean is one, the normalized factor for segment $s$ is its raw factor divided by the exposure-weighted mean raw factor. Compare pre- and post-change normalized factors using the same exposure weights. For positive factors, the percentage relativity change is the post-change normalized factor divided by the pre-change normalized factor, minus one.

Worked Example: Detect a Relativity Redistribution

A hypothetical program has 6,000 exposures in Segment A, 3,000 in Segment B, and 1,000 in Segment C. The old raw factors are 1.00, 1.50, and 2.50; the updated raw factors are 1.00, 1.65, and 2.25. The factors are normalized to an exposure-weighted mean of one, and the overall base premium is held at $100 per exposure.

Step 1: Calculate the old and new weighted mean raw factors. The old weighted mean is $(6,000\times1.00+3,000\times1.50+1,000\times2.50)/10,000=13,000/10,000=1.30$. The new weighted mean is $(6,000\times1.00+3,000\times1.65+1,000\times2.25)/10,000=13,200/10,000=1.32$.

Step 2: Normalize and compare the factors. Segment A’s old and new normalized factors are $1.00/1.30=0.769231$ and $1.00/1.32=0.757576$, so its change is $0.757576/0.769231-1=-1.5152\%$. Segment B’s normalized factors are $1.50/1.30=1.153846$ and $1.65/1.32=1.250000$, giving $1.250000/1.153846-1=8.3333\%$. Segment C’s normalized factors are $2.50/1.30=1.923077$ and $2.25/1.32=1.704545$, giving $1.704545/1.923077-1=-11.3636\%$.

Step 3: Check the premium redistribution. Old premiums are $100 multiplied by exposure and normalized factor: Segment A is $461,538.46, Segment B is $346,153.85, and Segment C is $192,307.69. Their total is $1,000,000. New premiums are $454,545.45, $375,000.00, and $170,454.55, also totaling $1,000,000. The changes are −$6,993.01, +$28,846.15, and −$21,853.14, which sum to $0.00 after rounding to cents.

Check. The total stays fixed because both sets of factors are normalized to an exposure-weighted mean of one and the base premium and exposures are unchanged. The substantial segment changes are still relevant to a materiality screen. The method avoids the mistake of concluding that no change matters merely because the portfolio total is unchanged.

Separate Model Effects From Exposure-Mix Effects

A comparison can change because the rating model changed, because the distribution of exposures changed, or because both changed. To isolate the model update, apply old and new indications to the same fixed exposure base. If the exposure mix differs between the two calculations, the resulting portfolio comparison combines model and mix effects and may not answer the intended governance question.

A separate calculation using a different exposure mix can be useful as a sensitivity, but it should be labelled as such. The reference portfolio should be chosen and documented before the comparison; it might be a defined in-force cohort or another agreed exposure set. The example below shows why that choice can affect the aggregate result even when segment-level rate changes remain the same.

Worked Example: Hold the Portfolio Basis Constant

A hypothetical update changes Segment A’s indicated rate from $200 to $220 and Segment B’s rate from $400 to $360. The documented screen requires review when the absolute portfolio percentage change exceeds 2.0% or any segment change exceeds 5.0%. First, use a reference portfolio of 800 A exposures and 200 B exposures.

Step 1: Calculate segment changes. Segment A’s rate change is $220 − $200 = $20, or $20 ÷ $200 = 10.0%. Segment B’s change is $360 − $400 = −$40, or −$40 ÷ $400 = −10.0%.

Step 2: Compare premiums on the reference exposure base. Old premium is $800\times\$200+200\times\$400=\$160,000+\$80,000=\$240,000$. New premium is $800\times\$220+200\times\$360=\$176,000+\$72,000=\$248,000$. The dollar change is $8,000, and the percentage change is $8,000 ÷ $240,000 = 3.3333%.

Step 3: Apply the thresholds. The portfolio change exceeds 2.0%, and both segment changes exceed 5.0% in absolute value. The update requires review.

Step 4: Show the effect of a different mix. With 400 A exposures and 600 B exposures, old premium would be $400\times\$200+600\times\$400=\$320,000$, while new premium would be $400\times\$220+600\times\$360=\$304,000$. That is a $16,000 decrease, or −5.0% relative to $320,000. This is a valid result for the different portfolio, but it is not the model-only impact on the original reference portfolio.

Check. Segment changes remain +10.0% and −10.0% in both calculations, while the portfolio result changes from +3.3333% to −5.0% because the exposure mix differs. Holding exposures fixed avoids attributing a mix change to the rating-model update.

Document the Threshold Rule and Its Limits

Thresholds should come from the organization’s documented governance process, not from an assumed industry-wide standard. A policy might set separate limits for aggregate premium impact, segment-level changes, dollar impact, or the number of segments exceeding a limit. The documentation should specify:

the pre- and post-change versions and the date or data basis used;
the exposure set, premium period, and any normalization applied;
the measures being tested and the exact thresholds;
whether a trigger is based on “any” criterion or a specified combination;
the action associated with a trigger, such as additional review or a higher approval level; and
the calculation, rounding convention, exceptions, and reviewer’s conclusion.

A threshold result is a prompt for the stated governance action, not a substitute for professional judgment. A change below a numeric threshold may still need attention if there is a relevant qualitative concern, while a change above a threshold does not prove that the update is inappropriate. Materiality testing is also distinct from model validation: it measures the size and distribution of the proposed impact, not whether the underlying model is sufficiently supported.

Common Mistakes

A frequent error is to take an unweighted average of segment percentage changes. In the first worked example, that would give $(3.0\%-3.0\%+10.0\%)/3=3.3333\%$, not the correct portfolio impact of 2.48%. The correction is to aggregate premiums or weight the segment changes by their pre-change premium shares.

Other mistakes include comparing premiums calculated on different exposure mixes without acknowledging the mix effect; overlooking large changes in smaller segments because the portfolio total is stable; comparing raw relativities when the overall level has also changed; and treating a threshold as a universal actuarial standard. State the comparison basis and the documented decision rule before interpreting the result.

Key takeaway: A defensible materiality test uses a defined, consistent comparison basis; reports both portfolio and segment impacts; distinguishes premium-level changes from relativity redistribution; and applies documented thresholds exactly as written.

Check Your Understanding

For each problem, compare the figures on the stated exposure basis and apply only the stated rule.

A two-segment portfolio has pre-change premiums of $400,000 and $600,000. The changes are +4.0% and −1.0%, respectively. What is the portfolio percentage change? (Answer: +1.0%)
A fixed portfolio has 500 exposures in A and 100 in B. Old rates are $300 and $900; new rates are $315 and $810. Find the portfolio percentage change and the two segment percentage changes. (Answer: Portfolio −0.625%; A +5.0%; B −10.0%)
There are 4,000 exposures in A and 1,000 in B. Old raw factors are 1.00 and 3.00; new raw factors are 1.10 and 2.60. Both sets are normalized to an exposure-weighted mean of one. Find the normalized relativity changes. (Answer: A +10.0%; B −13.3333%)
A documented rule requires review if portfolio impact exceeds 1.5% in absolute value or any segment impact exceeds 6.0% in absolute value. A fixed-basis comparison gives a portfolio impact of +1.2% and segment impacts of +2.0%, −6.5%, and +0.4%. Is review required? (Answer: Yes; the −6.5% segment change exceeds the segment threshold.)
Old portfolio premium is $2,000,000. On the same exposure basis, post-change indications total $1,960,000. A rule triggers at an absolute portfolio change greater than 2.0%, or an absolute dollar change greater than $30,000. Does either portfolio criterion trigger? (Answer: Yes; the change is −2.0%, which does not exceed 2.0%, but the dollar decrease is $40,000, which exceeds $30,000.)

What You'll Learn