Why Test a Classification Variable?
A variable can divide policyholders into groups and still be a poor rating variable. The groups may have little difference in expected cost, the recorded classifications may be unreliable, or the observed differences may reflect a different mix of risks rather than the proposed variable itself. A variable can also be difficult to explain or raise fairness and regulatory concerns.
Testing a proposed classification variable therefore means assessing several distinct questions. Does it separate expected costs after allowing for exposure and relevant risk mix? Are its values accurate and reproducible? Does the separation persist across time? Can policyholders and administrators understand the classification? Could it function as an inappropriate proxy, or conflict with applicable law or regulation? No single statistical test answers all of these questions.
The assessment should match the intended use. For a claim-frequency measure, compare claim counts per exposure. For a claim-cost measure, compare losses per exposure on a consistent basis, such as the same valuation date and coverage definition. Raw totals are generally not comparable when class exposures differ. As in earlier material on one-way relativities and overlapping classification effects, observed class differences can also reflect portfolio mix or other variables.
Evidence to Assess
| Assessment | Questions to ask | Evidence to examine |
|---|---|---|
| Predictive separation | Do expected costs differ by class on a comparable basis? Is the difference material and sufficiently precise? | Rates per exposure, uncertainty, consistency within relevant risk segments, and a plausible connection to insured cost. |
| Data reliability | Are class values accurate, complete, and consistently defined? | Source documentation, missing-value rates, coding checks, audit results, and the effect of classification errors. |
| Temporal stability | Does the relationship persist across experience periods? | Class rates and relativities by period, considering changes in definitions, coverage, and underlying conditions. |
| Interpretability | Can the classification be explained and applied consistently? | Clear definitions, reproducible assignments, data availability at the time of rating, and an understandable rationale. |
| Fairness and regulation | Could use of the variable create unjustified effects or conflict with applicable requirements? | Legal and regulatory review, possible proxy relationships, and the distribution of the variable across relevant groups. |
Statistical evidence needs context. A small rate difference may be estimated very precisely in a large portfolio but have little practical rating value. A larger apparent difference in a small or poorly coded class may be too uncertain to support a conclusion. The analyst should report the estimates and their limitations rather than treat a significance threshold as an automatic approval rule.
Comparing Class Experience
For a simple comparison, let \(N_j\) be the claim count and \(E_j\) the exposure in class \(j\). The observed frequency is \(N_j/E_j\). If the intended measure is claim cost, replace the count with the consistently measured loss amount. This calculation describes experience; it does not, by itself, adjust for other rating variables or establish that a proposed variable causes the difference.
The standard error follows from the approximate variance of a Poisson count: \(\operatorname{Var}(N_j)=E_j\lambda_j\). For sufficiently large counts, the variance of \(\log(N_j/E_j)\) is approximately \(1/N_j\). For independent classes, the variance of the difference of their log frequencies is the sum of those variances. Taking the square root gives the stated standard error. This supports an approximate interval for the rate ratio; it does not account for confounding by other variables.
Worked Example: Testing Frequency Separation
A hypothetical insurer is considering annual distance band as a classification variable. During one experience period, the lower-distance group has 120 claims over 6,000 car-years; the higher-distance group has 90 claims over 3,000 car-years. Assume independent Poisson claim counts with constant frequency within each group. Calculate the observed frequency ratio and an approximate 95% interval, then interpret the evidence.
Method and point estimate. The lower-distance frequency is \(120/6{,}000=0.0200\) claims per car-year. The higher-distance frequency is \(90/3{,}000=0.0300\). The higher-to-lower ratio is \(0.0300/0.0200=1.500\).
Uncertainty. The standard error of the log ratio is \(\sqrt{1/120+1/90}=\sqrt{0.008333+0.011111}=\sqrt{0.019444}=0.1394\). The log ratio is \(\log(1.500)=0.4055\). Using 1.96 standard errors, the interval on the log scale is \(0.4055\pm1.96(0.1394)=0.4055\pm0.2732\), or \((0.1323,0.6787)\). Exponentiating the endpoints gives an approximate ratio interval of \((1.141,1.971)\). The interval excludes 1 under the stated approximation.
Interpretation and check. The observed frequency is 50% higher in the higher-distance group, and this simple model indicates evidence of a difference. This does not establish that distance band adds useful information after other rating variables are considered, nor that it is appropriate to use. The ratio is consistent with the group rates: \(0.0200(1.500)=0.0300\).
Common mistake avoided. Dividing total claims by total exposure gives \(210/9{,}000=0.0233\), the portfolio frequency, not a class comparison. It cannot show how the two groups differ.
Adjusting for Risk Mix
A proposed variable may appear predictive because its classes contain different proportions of risks that are already known to have different costs. One diagnostic is direct standardization: calculate each candidate class's rate using the same weights for relevant strata. This makes the compared class rates reflect a common mix for those strata.
Standardization addresses only the factors included in the strata and chosen weights. It is not a universal correction for confounding. If a stratum has no observations for a class, its rate cannot be estimated directly from that class's data without an additional modeling assumption.
Worked Example: Separating a Class Effect from Portfolio Mix
A hypothetical insurer evaluates whether a usage category is informative. The table gives claim counts and exposures for two existing risk strata. Assume claim counts are measured consistently. Use equal weights for the two strata to compare the candidate classes.
| Candidate class | Lower-risk stratum | Higher-risk stratum |
|---|---|---|
| A | 9 claims / 900 exposures | 10 claims / 100 exposures |
| B | 2 claims / 100 exposures | 90 claims / 900 exposures |
Crude rates. Class A has \(19\) claims over \(1{,}000\) exposures, so its crude rate is \(19/1{,}000=0.019\). Class B has \(92\) claims over \(1{,}000\) exposures, so its crude rate is \(92/1{,}000=0.092\). The crude B-to-A ratio is \(0.092/0.019=4.842\), which looks very large.
Rates within strata. In the lower-risk stratum, A's rate is \(9/900=0.010\), while B's is \(2/100=0.020\). In the higher-risk stratum, A's rate is \(10/100=0.100\), while B's is \(90/900=0.100\).
Standardized comparison. With weights \(0.5\) for each stratum, A's standardized rate is \(0.5(0.010)+0.5(0.100)=0.055\). B's is \(0.5(0.020)+0.5(0.100)=0.060\). The standardized B-to-A ratio is \(0.060/0.055=1.091\), much closer to 1 than the crude ratio.
Interpretation and check. Class B contains far more exposure in the higher-risk stratum, accounting for much of its crude rate difference. After equal-mix standardization, the observed difference is about 9.1%, not the crude 384.2% increase. The standardized rates reproduce from the stated stratum rates and weights. This small example is only a diagnostic: uncertainty, other risk factors, and the suitability of the weights still require consideration.
Common mistake avoided. Treating the crude ratio as the effect of the candidate variable ignores the marked difference in stratum composition.
Reliability and Stability
A classification is useful only if it can be assigned correctly. Check how the value is obtained, whether it is missing or stale, whether different systems use the same definition, and whether an audit confirms recorded values. Overall agreement can conceal a serious problem if errors are concentrated in one class, so inspect errors by class and examine whether misclassification could materially change estimated rates.
Temporal stability is a separate question. Compare class rates and relativities across periods, using consistent definitions and appropriately comparable experience. A changing relativity can reflect random variation, a changing portfolio, changed data collection, or a genuine change in the relationship. A stable point estimate from a small number of claims is not necessarily strong evidence; uncertainty and exposure volumes matter.
Worked Example: Auditing a Classification and Comparing Periods
A hypothetical insurer evaluates an annual-mileage classification. In a verification audit, 228 of 240 sampled records agree with the source documents. The insurer also observes the following claim counts and exposures in two periods. Assume the class definitions are intended to be the same in both periods.
| Class | Period 1 | Period 2 |
|---|---|---|
| Lower mileage | 80 claims / 4,000 exposures | 105 claims / 5,000 exposures |
| Higher mileage | 60 claims / 2,000 exposures | 72 claims / 2,500 exposures |
Audit result. Agreement is \(228/240=0.950\), or 95.0%; disagreement is \(12/240=5.0\%\). This is a sample result, not proof that exactly 95% of all records are correct. The next checks should identify which classes contain the mismatches and whether the source data and classification rule are consistently applied.
Period 1 comparison. Lower-mileage frequency is \(80/4{,}000=0.0200\). Higher-mileage frequency is \(60/2{,}000=0.0300\). The higher-to-lower relativity is \(0.0300/0.0200=1.500\).
Period 2 comparison. Lower-mileage frequency is \(105/5{,}000=0.0210\). Higher-mileage frequency is \(72/2{,}500=0.0288\). The relativity is \(0.0288/0.0210=1.3714\), or 1.37 to two decimals.
Interpretation and check. The estimated relativity declined by \(1.3714/1.500-1=-0.0857\), approximately 8.57%. The direction of separation remains the same, but the estimated size changed. The audit also indicates a nonzero classification-error concern. Neither result alone proves that the variable is unsuitable: assess statistical uncertainty, error patterns, definition changes, and operational causes before drawing a conclusion.
Common mistake avoided. Comparing claim totals alone would suggest that the higher-mileage group had fewer claims in Period 2 than the lower-mileage group. That comparison ignores their different exposure volumes; the relevant frequencies are \(0.0288\) and \(0.0210\).
Interpretability, Fairness, and Regulatory Review
A variable should have a clear definition that can be applied consistently and explained in terms appropriate to its intended use. Ask whether the information is available when the policy is rated, whether it can be verified, and whether the same facts lead to the same classification. Complex or indirectly derived variables can be difficult to explain, maintain, or correct when a policyholder disputes the underlying data.
Also examine whether a variable may serve as a proxy for a protected characteristic or produce effects that warrant further review. A statistical relationship with claim cost does not settle legal, regulatory, or ethical questions. Requirements differ by jurisdiction and product; actuarial analysis should identify concerns and support review by the appropriate legal and compliance specialists, not substitute for that review.
Worked Example: A Predictive Variable with a Potential Proxy Concern
A hypothetical insurer considers a postal-zone category for a product. In its study data, the higher-cost zone has an average loss cost of $1,400 per exposure, compared with $1,000 in the lower-cost zone. A separate review finds that 240 of 800 policies in Group X and 240 of 400 policies in Group Y are assigned to the higher-cost zone. Assume these are descriptive figures and do not imply that group membership causes claim costs.
Predictive difference. The observed loss-cost ratio is \($1{,}400/$1{,}000=1.40\). In this hypothetical data, the higher-cost zone has a 40% higher average loss cost.
Distribution difference. The higher-cost-zone share is \(240/800=0.30\), or 30%, for Group X. It is \(240/400=0.60\), or 60%, for Group Y. The difference is \(0.60-0.30=0.30\), or 30 percentage points; the Group Y share is twice the Group X share.
Assessment. The observed cost separation is evidence to investigate, while the differing group distributions flag a potential proxy or disparate-impact concern. Before deciding whether the variable is appropriate, examine the source and meaning of the postal-zone measure, the credibility and consistency of its loss costs, available alternatives, and applicable requirements. These calculations do not establish that the variable is lawful or unlawful.
Common mistake avoided. A 1.40 loss-cost ratio is not, by itself, a complete justification for use. Conversely, the distribution difference alone does not prove that the variable is impermissible; it identifies an issue requiring contextual and regulatory review.
A Practical Decision Sequence
Specify how each risk is classified and whether the outcome is frequency, severity, or loss cost.
Use rates per exposure and consistent claim, coverage, and valuation definitions; inspect relevant mix differences.
Report class estimates, their differences or ratios, and limitations of the model and data.
Review audits, missing values, definition changes, and experience across periods.
Assess whether the classification is understandable and operationally reproducible, and refer potential fairness or regulatory issues for appropriate review.
The result is a reasoned assessment, not a mechanical pass-or-fail score. A variable with weak separation, unreliable assignment, or unstable experience may need revision or more evidence. A variable with strong separation still requires consideration of risk mix, implementation, explainability, and applicable requirements. These tests are aligned with the CAS Exam 5 (Basic Ratemaking and Estimated Claim Liabilities) syllabus.
Check Your Understanding
For each hypothetical situation, calculate the requested quantity and consider what the result does or does not establish.
- A candidate class has 48 claims over 2,400 exposures; the reference class has 36 claims over 1,200 exposures. Find both frequencies and the candidate-to-reference frequency ratio. (Answer: 0.0200 and 0.0300 claims per exposure; ratio 0.667.)
- Two candidate classes have rates in a lower-risk stratum of 0.010 and 0.020, respectively, and in a higher-risk stratum of 0.050 for both. Using common weights of 0.60 and 0.40, calculate the standardized rates and their ratio, second class to first. (Answer: 0.026 and 0.032; ratio 1.231.)
- An audit verifies 171 of 180 recorded classifications against source documents. Calculate the observed agreement and explain why it does not establish the error rate for every class. (Answer: 95.0%; the overall sample agreement does not show whether errors are concentrated in particular classes or establish population-wide accuracy.)
- A class relativity is 1.32 in one period and 1.20 in the next. Calculate the proportional change in the relativity from the first period to the second. (Answer: \(1.20/1.32-1=-0.0909\), a 9.09% decrease.)
- A candidate category appears for 25% of one policyholder group and 50% of another. Calculate the percentage-point difference and the ratio of the two shares, then state what additional conclusion is needed before rating use. (Answer: 25 percentage points; ratio 2.0. The distribution difference flags a concern to investigate but does not by itself establish a legal or actuarial conclusion.)