Workshop: Reviewing a Rating Variable for Fairness

Reviewing a Rating Variable as a Whole

A rating variable can have a plausible name and still be a poor choice for a rate plan. The underlying data may be incomplete, the measured relationship with losses may be weak or unstable, or the variable may create material differences across groups without enough actuarial justification. A defensible review therefore asks more than whether a model assigns the variable a statistically significant coefficient.

The review should connect five questions: Where did the value come from? Does it measure what it claims to measure? Is its relationship with expected insurance cost credible? How does its use affect relevant groups? Is there a better, practicable alternative? The answers support a recommendation to use the variable, use it with restrictions or adjustments, collect more evidence, or not use it.

Definition: A rating variable review is an evaluation of a candidate variable’s source and measurement, relationship to expected insured losses, associations with protected traits or relevant groups, effects on indicated prices, and available alternatives. A finding that a variable predicts losses does not, by itself, establish that every proposed use is appropriate.

For a basic cost comparison, let $Z$ denote the candidate rating variable and let $C$ denote expected loss cost per exposure. If claim frequency and average claim severity are defined on the same exposure basis, then expected loss cost is their product:

E[C\mid Z=z] = E[N\mid Z=z]\,E[M\mid Z=z]

Here $N$ is claim count per exposure and $M$ is claim severity. This product is appropriate when the severity term is the average cost per claim, with frequency and severity measured consistently for the same population and period. It is not a formula for total premium: expenses, profit and contingencies, coverage terms, and other rate components may also matter.

A Practical Review Sequence

The sequence below keeps evidence about measurement distinct from evidence about rates and impacts. A variable can pass one step and fail another. For example, a source may be reliable but measure something that has little support as a rating characteristic; a cost relationship may be real while a proposed factor is larger than that relationship supports.

Define the proposed use.
State what the variable measures, its permitted values, the policy population, and how it would change the rate. A definition such as “risk score” is not precise enough to review.

Trace and assess the data source.
Check who or what supplies the value, when it is observed, how missing values are treated, whether values can be verified, and whether the data represent the intended insured population.

Test the cost rationale.
Compare credible expected losses across values of the variable, considering exposure, coverage, time period, and relevant confounding factors. Examine stability and uncertainty, not just a point estimate.

Assess association and impact.
Check how values are distributed across relevant groups and calculate the implied subgroup rate effects. Interpret group comparisons in context; they are not proof of causation or unfair treatment on their own.

Compare alternatives and decide.
Consider a more direct measure, a simpler factor, a limited use, or no use. Record what the evidence supports, what remains uncertain, and what monitoring or controls a conditional recommendation requires.

From Cost Evidence to a Rating Relativity

Suppose the candidate variable has categories $z$. A simple indicated relativity compares the expected cost in category $z$ with a chosen reference category $z_0$:

r_z=\frac{E[C\mid Z=z]}{E[C\mid Z=z_0]}

This is a cost comparison, not automatic permission to apply the resulting factor. The estimate should be credible for the intended population and should account for the rating plan’s coverage and exposure basis. Where a multivariable model is used, the conditional relationship may differ from the unadjusted comparison. In either case, predictive association is not proof that changing $Z$ would change losses: the variable may be associated with other characteristics that explain some or all of the observed difference.

Derivation: If a policy in category $z$ has expected loss cost $c_z$, and a policy in reference category $z_0$ has expected loss cost $c_0>0$, a factor $r_z=c_z/c_0$ gives $r_zc_0=c_z$. Thus multiplying the reference loss-cost level by this factor reproduces the category’s indicated expected loss cost. This justification assumes the cost estimates are comparable and that the factor is being applied to the same loss-cost component; it does not justify applying it to unrelated premium components or ignoring uncertainty and other rate requirements.

Worked Example: Reviewing an Odometer-Based Mileage Band

A hypothetical personal auto insurer is considering annual mileage, estimated from odometer readings, with a low band as the reference and a high band as the candidate factor. In a credible, exposure-adjusted study, expected annual loss costs are $500 per exposure in the low band and $600 in the high band. In the proposed book, Group A has 40% of its exposures in the high band; Group B has 20%. Assume, only for this illustration, that the same base loss-cost level of $500 applies before the mileage factor.

Step 1: Calculate the cost relativity. The low band is the reference, so its factor is $500/500=1.00$. The high-band factor is $600/500=1.20$. The measured cost difference is $100, or 20% of the low-band cost.

Step 2: Calculate average indicated factors. Group A’s average factor is $0.60(1.00)+0.40(1.20)=0.60+0.48=1.08$. Group B’s is $0.80(1.00)+0.20(1.20)=0.80+0.24=1.04$.

Step 3: Interpret the subgroup comparison. The ratio of average indicated factors is $1.08/1.04=1.03846$, so Group A’s average indicated amount is about 3.85% higher than Group B’s under these assumptions. At the $500 reference level, the respective averages are $500(1.08)=\$540$ and $500(1.04)=\$520$.

Step 4: Review source, limits, and alternatives. Odometer readings are relatively direct evidence of distance traveled, but the reviewer should still examine reading dates, missing or implausible values, vehicle changes, and how annual mileage is estimated between readings. A continuous mileage measure or verified periodic readings may be preferable to a coarse band if they are reliable and produce a stable relationship.

Recommendation. The example supports investigating mileage as a cost-related variable and quantifies its distributional effect; it does not settle regulatory acceptability or establish that every subgroup difference is justified. The reviewer should validate the source and cost relationship, examine the applicable requirements, and document why the chosen measurement and factor are appropriate. The common mistake avoided here is treating a 20% high-versus-low cost difference as a 20% difference in every group’s average rate: group averages depend on each group’s distribution across bands.

Association, Subgroup Effects, and Alternatives

A review should distinguish at least three findings. First, the variable may be associated with a protected trait or another group characteristic. Second, the variable’s use may produce different average indicated prices across groups because the groups have different distributions of its values. Third, the variable may or may not have a credible relationship with expected losses. These findings are related, but none substitutes for the others.

For a group $g$, let $p_{gz}$ be the proportion of that group in category $z$, and let $r_z$ be the factor applied to that category. The group’s average factor is the exposure-weighted average:

\bar r_g=\sum_z p_{gz}r_z, \qquad \sum_z p_{gz}=1

This identity explains why a variable can create different average indications even if the same category factors apply to everyone: the groups may have different category shares. It is a descriptive calculation under the specified factors, not a causal estimate and not a complete fairness test. Reviewers should define the comparison population and metric carefully, since different fairness measures can answer different questions.

The cost analysis should also consider whether an apparent relationship is robust to reasonable adjustments and choices of time period, and whether the variable adds useful information beyond existing factors. If an external vendor supplies the value, review the underlying source, population coverage, update process, and how the vendor handles missing or uncertain records. A score with opaque construction or weak representation may be difficult to validate even when it is easy to add to a model.

Worked Example: Evaluating a Telematics Night-Driving Measure

A hypothetical insurer proposes a factor for the share of recorded trips occurring at night. In an illustrative study, the low-night category has claim frequency $0.100$ per exposure and average severity of $4,000. The high-night category has frequency $0.115$ and the same average severity. Group A has 50% of exposures in the high-night category; Group B has 25%. For this calculation, assume the data are comparable by coverage and period, and use the low-night loss cost as the reference.

Step 1: Calculate expected costs. Low-night cost is $0.100\times\$4{,}000=\$400$ per exposure. High-night cost is $0.115\times\$4{,}000=\$460$ per exposure.

Step 2: Calculate the indicated factor. The high-night relativity is $\$460/\$400=1.15$. The proposed factor should therefore be assessed against a 15% indicated cost difference, not against frequency alone without checking severity.

Step 3: Calculate average indicated amounts by group. Using $400 as the base, Group A’s average is $0.50(\$400)+0.50(\$460)=\$200+\$230=\$430$. Group B’s is $0.75(\$400)+0.25(\$460)=\$300+\$115=\$415$. The ratio is $430/415=1.03614$, or about 3.61% higher for Group A under the assumptions.

Step 4: Review whether the evidence supports the use. The reviewer should check how “night” and a “trip” are defined, whether device participation and recording completeness vary across policyholders, and whether the study population represents the proposed book. An observed association does not show that night driving itself caused the cost difference; other driving patterns or selection into telematics participation may contribute.

Recommendation. Treat the 1.15 as an indicated association requiring validation, not a final approved factor. Consider whether a more complete measure of exposure or a narrower, better-supported use is available, and document relevant regulatory and practical constraints. The common mistake avoided is multiplying the 15% high-category relativity by the share in that category and calling the result the subgroup difference; each group’s full weighted average must be calculated.

When the Data Relationship Is Too Weak

A variable should not be retained simply because a model can assign it a coefficient. The size and stability of the cost difference, source limitations, the variable’s practical meaning, and its group impacts all matter. If uncertainty is substantial, the point estimate may not support a fine-grained factor. If the value is difficult to explain or correct, a policyholder may also be exposed to an error they cannot reasonably identify.

Worked Example: Reviewing a Vendor Payment-Regularity Score

A hypothetical vendor offers an external payment-regularity score with low and high categories. In a study supplied to an insurer, observed expected loss costs are $400 for the low-score category and $410 for the high-score category. The proposed rate plan nevertheless assigns a factor of 1.15 to the high-score category. Group A has 70% of exposures in the high category, while Group B has 30%. Assume the $400 low-category cost is the base for this illustration.

Step 1: Compare the cost evidence with the proposed factor. The observed cost relativity is $410/400=1.025$, or 1.025. The proposed factor of 1.15 is 15% above the base, while the point estimate supports a 2.5% difference. The gap is $1.15-1.025=0.125$ relativity points. It requires explanation and evidence; it is not justified by the supplied cost comparison alone.

Step 2: Calculate the group averages under the proposed factor. Group A’s average is $0.70(1.15)+0.30(1.00)=0.805+0.300=1.105$. Group B’s is $0.30(1.15)+0.70(1.00)=0.345+0.700=1.045$. Their ratio is $1.105/1.045=1.05742$, or about 5.74% higher for Group A.

Step 3: Separate impact from justification. The calculated 5.74% is the impact of the proposed factor given the group distributions. It does not show why the groups have different score distributions, whether the score is a proxy for a protected trait, or whether the difference is permissible. The reviewer needs evidence on the vendor’s source, coverage, construction, and the score’s incremental cost value.

Recommendation. Do not adopt the 1.15 factor on these facts alone. Seek credible support for the size of the factor and for the data’s relevance to the insured risk; compare a direct, verifiable alternative; and assess applicable requirements before any use. If the vendor cannot provide adequate evidence, rejection is more defensible than treating an unexplained score as validated. This avoids the common error of assuming that an external score is fair or accurate merely because it is numerical and available.

Turning Findings into a Defensible Recommendation

A useful conclusion is specific and conditional where the evidence is incomplete. It should state the intended use, summarize the source and cost evidence, quantify material subgroup effects, identify limitations and alternatives, and explain what action follows. Avoid a bare conclusion such as “the variable is fair” or “the variable is predictive”: neither communicates the basis or limits of the decision.

Possible recommendations include the following:

Use as proposed when the data source is fit for purpose, the cost relationship is credible and stable, subgroup impacts have been reviewed, and the proposed application meets relevant requirements.
Use with changes or controls when evidence supports only a narrower definition, a less granular factor, a verified source, or a specified monitoring and review process.
Defer pending evidence when missing validation, uncertainty, or source limitations prevent a reliable decision.
Do not use when the relationship is unsupported, the data are not fit for purpose, or the identified concerns cannot be addressed adequately.

A common exam trap is to use a single disparity statistic as a complete fairness verdict. Another is to say “the factor is actuarially justified” because the variable is correlated with losses, without checking measurement quality, the proposed factor’s magnitude, group distributions, alternatives, and jurisdiction-specific constraints. A sound review states what is known, what is not established, and how those limits affect the recommendation. This approach is aligned with the CAS Exam 8 (Advanced Ratemaking) syllabus.

Key takeaway: Review a rating variable as a chain of evidence: source and measurement, credible relationship to expected cost, group association and rate impact, and comparison with alternatives. Keep each conclusion within what the evidence actually supports.

Check Your Understanding

For each hypothetical review, identify the calculation or decision that follows from the stated assumptions.

A candidate factor distinguishes two categories. The reference category has frequency $0.08$ and average severity $6,000; the other has frequency $0.10$ and the same severity. What is the other category’s indicated cost relativity? (Answer: $0.10(6{,}000)/[0.08(6{,}000)]=1.25$.)
A proposed high-category factor is 1.30, with the low category at 1.00. Group A has 30% high-category exposure and Group B has 10%. Calculate each group’s average factor and their ratio. (Answer: A $=1.09$, B $=1.03$, ratio $1.09/1.03=1.0583$, so A is about 5.83% higher.)
A verified measure has expected costs of $360 in its reference category and $450 in the other category. What is the indicated relativity, and what does it not establish by itself? (Answer: $450/360=1.25$; it does not by itself establish that a proposed factor is permissible, fair in every relevant sense, or causally related to losses.)
A vendor provides a score that is associated with losses, but cannot explain its source, missing-value treatment, or population coverage. Should a positive model association alone support adoption? (Answer: No; source fitness and representativeness remain unestablished, so seek evidence or an auditable alternative before use.)
A variable has a credible cost relationship, but the proposed factor is materially larger than the indicated relativity and one group is much more likely to receive the high factor. What should the review do next? (Answer: Quantify and explain the group impact, investigate the factor-size discrepancy and alternatives, assess applicable constraints, and defer or revise use unless the evidence supports the proposal.)

What You'll Learn