Why Test a Rating Factor for Proxy Effects?
A rating factor may not explicitly use a protected trait, yet its values may be associated with that trait. That association matters when assessing whether a factor could act as a proxy. It does not, by itself, show that the factor is prohibited, unfair, or unjustified: a factor may also have a legitimate relationship with expected insurance costs. The review therefore asks two different questions: how much protected-trait information does the factor carry, and what evidence supports its stated actuarial purpose?
This tutorial uses three complementary checks: association between the factor and a protected trait, comparison of outcome patterns within protected-trait subgroups, and analysis of what changes when the factor is removed from a rating model. Each answers a different question. Results should be interpreted in context, with attention to data quality, uncertainty, applicable restrictions, and the intended use of the model.
Association Between a Factor and a Protected Trait
For a binary protected-trait indicator \(G\) and binary rating factor \(X\), a basic diagnostic is the difference in factor prevalence between groups. Define \(p_1=P(X=1\mid G=1)\) and \(p_0=P(X=1\mid G=0)\). The difference \(p_1-p_0\) has a direct interpretation in percentage points. The Pearson correlation of the two binary indicators, also called the phi coefficient, provides a standardized association measure.
To see why prevalence differences and correlation are related, let \(g=P(G=1)\). By conditioning on \(G\), \(P(X=1)=gp_1+(1-g)p_0\). Therefore, \(P(G=1,X=1)-P(G=1)P(X=1)=g(1-g)(p_1-p_0)\). Substitution gives \(\phi=\sqrt{g(1-g)}(p_1-p_0)/\sqrt{P(X=1)(1-P(X=1))}\). Thus, for nondegenerate binary variables, the sign of the correlation matches the sign of the prevalence difference. Its magnitude also depends on how common each category is.
For a factor with several categories, inspect category prevalences by group and consider an appropriate categorical association measure, such as Cramér’s V. A single summary can hide which categories drive the result. For continuous factors, examine distributions and appropriate association measures rather than assuming a linear relationship. In every case, sample size and uncertainty matter: a small observed difference may be imprecise, while a precisely estimated association is still not proof of a harmful effect.
Worked Example: Measuring Association With a Binary Factor
In a hypothetical portfolio of 1,000 policies, an audit compares protected-trait group \(G=1\) with \(G=0\) and a binary rating factor \(X\). The counts are: 180 policies have both \(G=1\) and \(X=1\); 220 have \(G=1,X=0\); 120 have \(G=0,X=1\); and 480 have \(G=0,X=0\). Calculate the group prevalence difference and phi coefficient. The method is a two-by-two association analysis; it avoids treating a raw count as a rate.
Step 1: Find group sizes and factor prevalence. The \(G=1\) group has \(180+220=400\) policies, so \(P(X=1\mid G=1)=180/400=0.45\). The \(G=0\) group has \(120+480=600\), so \(P(X=1\mid G=0)=120/600=0.20\). The difference is \(0.45-0.20=0.25\), or 25 percentage points.
Step 2: Calculate phi. The marginal proportions are \(P(G=1)=400/1000=0.40\), \(P(X=1)=300/1000=0.30\), and \(P(G=1,X=1)=180/1000=0.18\). Hence the numerator is \(0.18-(0.40)(0.30)=0.06\). The denominator is \(\sqrt{(0.40)(0.60)(0.30)(0.70)}=\sqrt{0.0504}\approx0.2245\). Thus \(\phi=0.06/0.2245\approx0.2673\).
Check and interpretation. The positive correlation agrees with the higher prevalence of \(X=1\) in group \(G=1\). The result identifies an association to investigate; it does not establish why the factor differs by group or whether its use is justified. The common mistake this method avoids is comparing 180 with 120 without accounting for the different group sizes.
Compare Outcomes Within Protected-Trait Groups
Association with a protected trait does not answer whether a factor differentiates expected losses within the groups. For that question, compare a relevant outcome—such as claim frequency or loss cost—across factor levels separately within each group. A comparison within groups can reveal whether the factor has a similar, different, or absent relationship with the outcome across groups. It should not be treated as causal proof: exposure, other rating variables, data credibility, and claim-development choices may affect observed differences.
If group composition differs across factor levels, also calculate standardized comparisons using the same group weights for both factor levels. This removes differences in group mix from that particular comparison, but it does not control for every possible confounder.
Worked Example: Within-Group Claim Frequency and Standardization
A hypothetical insurer evaluates a binary vehicle-use factor \(X\) using earned exposure units and reported claims. For group \(G=1\), \(X=1\) has 100 exposure units and 12 claims, while \(X=0\) has 300 units and 24 claims. For group \(G=0\), \(X=1\) has 300 units and 36 claims, while \(X=0\) has 100 units and 10 claims. Calculate subgroup frequencies and standardized frequencies using equal group weights. The method is a within-group comparison followed by direct standardization; it avoids interpreting pooled frequencies as if group mix were identical.
Step 1: Calculate subgroup claim frequencies. For \(G=1\), the frequencies are \(12/100=0.12\) for \(X=1\) and \(24/300=0.08\) for \(X=0\), a difference of 0.04. For \(G=0\), they are \(36/300=0.12\) and \(10/100=0.10\), a difference of 0.02.
Step 2: Standardize to equal group weights. For \(X=1\), the standardized frequency is \((0.5)(0.12)+(0.5)(0.12)=0.12\). For \(X=0\), it is \((0.5)(0.08)+(0.5)(0.10)=0.09\). The standardized difference is \(0.12-0.09=0.03\), and the standardized ratio is \(0.12/0.09=1.333\).
Step 3: Check the pooled comparison. Pooled \(X=1\) frequency is \((12+36)/(100+300)=48/400=0.12\). Pooled \(X=0\) frequency is \((24+10)/(300+100)=34/400=0.085\). The pooled difference is 0.035, not the standardized difference of 0.03, because the group weights differ between factor levels.
Check and interpretation. The factor is associated with higher observed frequency in both subgroups here, though the differences are not equal. Standardization compares the levels under a common 50/50 mix. This calculation does not establish predictive significance or explain the group association; uncertainty and other relevant variables still require review. The common mistake it avoids is assuming that a pooled comparison has controlled for group composition.
Variable-Removal Tests in a Rating Model
A variable-removal test asks how a model or its rates change when the candidate factor is excluded and the model is refitted. The comparison should use the same data, target, exposure definition, and other modeling choices wherever feasible. For example, in a Poisson frequency model with a log link and exposure offset, expected claim count \(N_j\) for observation \(j\) may be modeled as \(\log E[N_j]=\log(e_j)+\beta_0+\beta_X X_j+\boldsymbol{\beta}^{T}\mathbf{Z}_j\), where \(e_j\) is exposure and \(\mathbf{Z}_j\) denotes other included predictors. This specification assumes the conditional mean follows the stated log-linear form; the Poisson variance assumption is also used for standard Poisson inference and likelihood-based comparisons. A different response distribution or model requires its corresponding formulation.
Removing \(X\) and refitting can show its contribution to fit and how the model’s indicated rates change. Fairness review can then compare group-level rates under the full and reduced models using a clearly stated population and weighting. Separately, one can test whether \(X\) adds information about the protected trait after specified legitimate variables are controlled for, for example by comparing nested models for the trait with and without \(X\). These are different tests: one concerns the rating outcome and the other concerns protected-trait predictability. Neither alone determines whether use is appropriate.
Worked Example: Rate Changes When a Factor Is Removed
Consider a hypothetical simplified pure-premium model with two equally exposed protected-trait groups. The full model indicates a base rate of $100 per exposure unit and multiplies that rate by 1.5 when factor \(X=1\). In group \(G=1\), 80% of exposure has \(X=1\); in group \(G=0\), 20% does. Calculate the average full-model rate by group, then refit an intercept-only model that preserves the portfolio’s indicated total. This variable-removal comparison avoids the mistake of comparing group rates without holding the total rate level consistently.
Step 1: Calculate group averages under the full model. For \(G=1\), the average is \((0.20)(\$100)+(0.80)(\$150)=\$20+\$120=\$140\). For \(G=0\), it is \((0.80)(\$100)+(0.20)(\$150)=\$80+\$30=\$110\).
Step 2: Calculate the portfolio average. The groups have equal exposure, so the indicated portfolio average is \((\$140+\$110)/2=\$125\) per exposure unit. Removing \(X\) and refitting an intercept-only model to preserve this total gives a rate of $125 for each group.
Step 3: Compare group differentials. Under the full model, the group rate difference is \(\$140-\$110=\$30\), and the rate ratio is \(140/110\approx1.273\). Under the reduced model, the difference is $0 and the ratio is \(125/125=1.000\).
Check and interpretation. In this constructed example, the factor materially differentiates group-average indicated rates, and its removal eliminates that difference while preserving the overall average. The result does not show that the factor lacks predictive value or that its use is impermissible. Predictive performance should be assessed separately, preferably on appropriate validation data. The common mistake this test avoids is holding the base rate fixed after removing a factor, which can confuse a change in total rate level with a change in group differentials.
How to Combine the Evidence
A defensible review treats these diagnostics as complementary. Correlation identifies whether factor values are distributed differently across groups. Subgroup outcome comparisons examine whether the factor has an observed relationship with the actuarial outcome within those groups. Removal tests quantify the impact of excluding the factor from a specified model. A strong association with a protected trait together with a small or unstable predictive contribution may warrant different questions than an association accompanied by a well-supported outcome relationship. Neither pattern automatically resolves the applicable fairness or regulatory judgment.
Specify the protected-trait indicator, candidate factor, relevant outcome, population, time period, and exposure or other denominator. Check missingness, coding, and data representativeness.
Report group-specific factor distributions and a suitable association summary. Include uncertainty where it can be estimated reliably, and inspect which categories or values drive the result.
Calculate outcome rates with consistent definitions, then consider standardized comparisons. Identify sparse cells and avoid treating noisy subgroup estimates as stable conclusions.
Compare the full and reduced models on fit, predictive performance, and group-level indicated rates, stating the weighting and rate-level treatment used.
Separate observed association, predictive contribution, and normative or regulatory conclusions. Record unresolved questions and the assumptions behind each comparison.
A common trap is to call any correlation a proxy effect and immediately recommend removing the factor. Correlation does not identify a causal pathway, and a rating factor’s predictive contribution cannot be inferred from correlation alone. The reverse error is equally important: a factor’s predictive value does not, by itself, settle whether its use is acceptable under relevant restrictions or fairness objectives. A careful analysis reports what each test measures and what it cannot establish. This disciplined evidence-gathering approach is aligned with the CAS Exam 8 (Advanced Ratemaking) syllabus.
Check Your Understanding
For each problem, identify what the diagnostic establishes and what conclusion would require additional evidence.
- A hypothetical portfolio has 500 policies in each of two protected-trait groups. A binary factor applies to 150 policies in the first group and 75 in the second. Calculate the prevalence difference and state its direction. (Answer: 30% versus 15%; difference \(=0.15\), higher in the first group.)
- A rating factor has claim frequencies of 0.10 versus 0.06 in one group, and 0.08 versus 0.07 in another, comparing factor level 1 with level 0. Calculate the within-group differences. (Answer: 0.04 and 0.01, respectively; the factor has a positive observed difference in both groups, larger in the first.)
- Two protected-trait groups have weights 0.25 and 0.75. For factor level 1, subgroup frequencies are 0.08 and 0.12; for level 0, they are 0.04 and 0.10. Calculate standardized frequencies and their difference using the stated common weights. (Answer: level 1 \(=0.25(0.08)+0.75(0.12)=0.11\); level 0 \(=0.25(0.04)+0.75(0.10)=0.085\); difference \(=0.025\).)
- A full model indicates group-average rates of $96 and $84 with equal exposure. After a factor is removed and the intercept is refitted to preserve the portfolio average, what is the common reduced rate, and what happens to the group difference? (Answer: common rate \(=(96+84)/2=\$90\); the group difference falls from $12 to $0.)
- A factor is strongly associated with a protected trait, but within-group outcome comparisons are noisy and variable removal changes validation performance only slightly. What can be concluded from these facts alone? (Answer: the factor carries observed protected-trait information, while evidence of a stable within-group outcome relationship or material predictive contribution is inconclusive; the facts alone do not establish causation or an acceptability determination.)