From Claim Occurrence to a Classification Tree
An insurer may want to estimate whether a policy will have at least one claim during the coming year. That outcome has two classes, so a classification tree can divide policies into groups with different observed claim frequencies. Its rules are easy to follow, but a tree's prediction is not automatically certain: a terminal node can contain both claim and no-claim policies.
This tutorial develops a small hypothetical example from split selection through individual predictions and error counts. It focuses on reading the rules and understanding what the terminal-node summaries do—and do not—say. The worked analysis is aligned with the SOA Exam PA (Predictive Analytics) syllabus.
In the calculations below, use the Gini impurity to compare splits. For a node with \(n\) observations, let \(p\) be the proportion with a claim. The node's Gini impurity is \(2p(1-p)\): it is 0 when all observations have the same class and is largest at \(p=0.5\). A split is assessed by the size-weighted average impurity of its child nodes. Lower weighted impurity means the proposed split separates the classes better according to this criterion.
Worked Example: Selecting Splits for Claim Occurrence
Worked Example: Selecting Splits for Claim Occurrence
Scenario. Consider a hypothetical training sample of 24 policies. The outcome is whether each policy had at least one claim during the year. There are 8 claim policies and 16 no-claim policies. A candidate root split is whether the policyholder had a prior incident. The “yes” branch contains 8 policies (6 claims and 2 no claims); the “no” branch contains 16 policies (2 claims and 14 no claims). A candidate split by vehicle age instead produces two groups of 12: one with 3 claims and 9 no claims, the other with 5 claims and 7 no claims. Calculate and compare the weighted Gini impurity of these root splits.
Method. Calculate the parent impurity, then calculate each candidate's child-node impurities and weight them by child size. The split with lower weighted impurity is preferred among these candidates.
At the root, \(p=8/24=1/3\), so the impurity before splitting is
For the prior-incident split, the “yes” child has claim proportion \(6/8=3/4\), giving impurity \(2(3/4)(1/4)=3/8=0.375\). The “no” child has claim proportion \(2/16=1/8\), giving impurity \(2(1/8)(7/8)=7/32=0.21875\). Weighting by the child sizes gives
For the vehicle-age split, the first child has claim proportion \(3/12=1/4\), so its impurity is \(2(1/4)(3/4)=3/8=0.375\). The second has claim proportion \(5/12\), so its impurity is \(2(5/12)(7/12)=70/144=35/72\approx 0.4861\). Both children contain 12 policies, so the weighted impurity is \((3/8+35/72)/2=31/72\approx 0.4306\).
The prior-incident split has the lower weighted impurity: \(0.2708<0.4306\). Its impurity reduction from the root is \(4/9-13/48=25/144\approx 0.1736\). By comparison, the vehicle-age split reduces impurity by \(4/9-31/72=1/72\approx 0.0139\). Thus, of these two candidates, prior incident is the better root split under the Gini criterion.
Now consider a candidate split of the prior-incident “yes” child by annual mileage, at 12,000 miles. The high-mileage child contains 4 claims and 0 no claims; the lower-mileage child contains 2 claims and 2 no claims. The parent node had 6 claims and 2 no claims, so these child counts sum correctly to its totals. Their weighted impurity is
This reduces the prior-incident “yes” node's impurity from \(0.375\) to \(0.25\). Suppose the fitted tree stops at this point because its chosen growth controls do not allow further splitting. Its rules and training summaries are then: prior incident = yes and mileage at least 12,000 leads to 4 claims out of 4; prior incident = yes and mileage below 12,000 leads to 2 claims out of 4; prior incident = no leads to 2 claims out of 16. The next sections interpret these terminal nodes.
Why the Weighted Impurity Is the Right Comparison
A split creates child nodes of potentially different sizes. The weighted impurity reflects the expected impurity of the child containing a randomly selected observation from the parent: the chance of selecting a particular child is its share of the parent observations. That is why a small child should not count as much as a large child in the overall split score.
The Gini formula also has a probability interpretation. Imagine drawing two class labels independently from the class proportions in one node. The chance that both labels are 1 is \(p^2\), and the chance that both are 0 is \((1-p)^2\). Therefore, the chance that the labels differ is
Thus, Gini impurity is the probability of different labels under this calculation. A pure node has no chance of different labels; a node with equal class proportions has the greatest chance. The weighted child impurity compares how much this measure changes after partitioning the observations.
Reading Terminal Nodes and Counting Training Errors
A terminal node summarizes the training observations that satisfy every rule along its path. Its estimated claim probability is the node's claim count divided by its total count. A common classification rule predicts a claim when that probability is greater than 0.5 and no claim when it is below 0.5. A value exactly equal to 0.5 is a tie, so the tie-breaking convention must be stated; in the examples here, ties are assigned to no claim.
Worked Example: Tracing Policies Through the Rules
Scenario. Apply the fitted tree from the previous example to three policies. Policy A had a prior incident and annual mileage of 15,000; its actual outcome was a claim. Policy B had a prior incident and annual mileage of 8,000; its actual outcome was a claim. Policy C had no prior incident and annual mileage of 20,000; its actual outcome was also a claim. Trace each policy, report the terminal-node claim probability and class prediction, and identify errors.
Method. Follow the root rule first. Only a policy in the prior-incident “yes” branch is tested against the mileage rule. Compare the actual outcome with the class predicted using the stated 0.5 threshold and no-claim tie-break.
Policy A. Prior incident = yes, so follow that branch. Mileage is 15,000, which is at least 12,000, so A reaches the high-mileage terminal node. The training claim proportion there is \(4/4=1\). The tree predicts claim, matching A's actual claim outcome.
Policy B. Prior incident = yes, but mileage is 8,000, below 12,000. B reaches the lower-mileage terminal node, whose training claim proportion is \(2/4=0.5\). Under the stated tie-break, the prediction is no claim. B actually had a claim, so this is an error.
Policy C. Prior incident = no, so C goes directly to that terminal node; the mileage rule is not applied on this branch. The estimated claim probability is \(2/16=0.125\), and the prediction is no claim. C's actual claim outcome makes this an error.
The tree's training-node summaries also show the number of training errors under this class rule. The high-mileage node predicts claim and has 0 errors among 4 policies. The lower-mileage node predicts no claim and misclassifies its 2 claims. The no-prior-incident node predicts no claim and misclassifies its 2 claims. The total is \(0+2+2=4\) errors among 24 training policies, or \(4/24=1/6\approx 0.1667\). This is a training error rate, not an estimate of performance on new policies.
The estimated probability and the class prediction answer different questions. For example, the lower-mileage node has an estimated claim probability of 0.5; its class label alone, “no claim,” hides that substantial uncertainty. Likewise, the no-prior-incident node predicts no claim but still contains 2 claims among 16 training policies. A class label should not be described as a guarantee that a policy will or will not have a claim.
Test-Set Errors and the Effect of a Threshold
Errors counted from the training observations describe how the fitted tree classifies the data used to form its terminal-node summaries. To examine how it classifies separate observations, trace a test set through the same fixed rules and compare its predictions with its actual outcomes. The test observations do not change the training claim proportions or the split rules in this calculation.
Worked Example: Assessing Errors on Held-Out Policies
Scenario. The tree above is applied to a separate hypothetical test set of 12 policies. Among the 2 test policies reaching the high-mileage node, 1 had a claim. Among the 3 reaching the lower-mileage node, 1 had a claim. Among the 7 reaching the no-prior-incident node, 2 had a claim. Use the original class rule, including its no-claim tie-break, to construct a confusion matrix and calculate accuracy, sensitivity, and specificity.
Method. The first two test policies are predicted as claims because the high-mileage leaf has training probability 1. The other two leaves predict no claim: the lower-mileage probability is tied at 0.5 and the no-prior-incident probability is 0.125. Count actual and predicted classes separately in each leaf.
The high-mileage leaf contributes 1 true positive and 1 false positive. The lower-mileage leaf predicts no claim for all 3 policies: its 1 claim is a false negative and its 2 no-claim policies are true negatives. The no-prior-incident leaf also predicts no claim: its 2 claims are false negatives and its 5 no-claim policies are true negatives. The totals are \(TP=1\), \(FP=1\), \(FN=3\), and \(TN=7\), which sum to 12.
| Actual outcome | Predicted claim | Predicted no claim | Total |
|---|---|---|---|
| Claim | 1 (TP) | 3 (FN) | 4 |
| No claim | 1 (FP) | 7 (TN) | 8 |
| Total | 2 | 10 | 12 |
Accuracy is the proportion classified correctly: \((TP+TN)/12=(1+7)/12=8/12=0.6667\). Sensitivity is the proportion of actual claims predicted as claims: \(TP/(TP+FN)=1/(1+3)=0.25\). Specificity is the proportion of actual no claims predicted as no claims: \(TN/(TN+FP)=7/(7+1)=0.875\). These results describe this hypothetical test set; they do not establish how the tree will perform in every future group.
A threshold changes the conversion of estimated probabilities into class labels; it does not change the fitted split rules or the leaf probabilities. For example, if the classification threshold were 0.4 instead of 0.5, the lower-mileage leaf's estimated probability of 0.5 would lead to a claim prediction. In the test set, that change would make its 1 claim a true positive and its 2 no claims false positives. The new totals would be \(TP=2\), \(FP=3\), \(FN=2\), and \(TN=5\). The labels and error counts change, while the underlying tree and its terminal-node probabilities stay the same.
Common Trap: Ignoring the Sizes of the Child Nodes
A frequent calculation error is to average child impurities equally even when the child nodes have different numbers of observations. In the root-split example, an unweighted average of the prior-incident child impurities would be
That is not the split's weighted impurity because the two children contain 8 and 16 policies, not the same number. The correct calculation is
The correction matters to split comparisons: the unweighted calculation gives a smaller child the same influence as a child twice its size. Always weight each child impurity by the number of observations it contains divided by the parent-node size.
- Do not apply a later rule on the wrong branch. A policy with no prior incident reaches the corresponding terminal node immediately; it does not also get classified by the mileage split.
- Do not equate a predicted class with certainty. A node predicted as no claim can still contain claim observations.
- Do not mix training and test errors. Training leaf counts come from the data used to fit the tree; test errors come from separate observations traced through fixed rules.
- State the threshold and tie rule. A leaf probability of exactly 0.5 does not determine a unique class without a convention.
Check Your Understanding
For each hypothetical problem, show the split or prediction steps before checking the answer.
- A node contains 30 policies: 10 with a claim and 20 without. A proposed split creates a child with 6 claims and 4 no claims, and another with 4 claims and 16 no claims. Calculate the parent Gini, the weighted child Gini, and the impurity reduction. (Answer: Parent Gini \(=4/9\approx0.4444\); weighted child Gini \(=0.3733\); reduction \(=0.0711\).)
- A terminal node contains 15 policies, 9 with claims and 6 without. Using a 0.5 threshold, determine its estimated claim probability, predicted class, and number of training classification errors. (Answer: Claim probability \(=9/15=0.6\); predict claim; 6 errors.)
- A tree first splits on prior incident. The “yes” branch then splits at 10,000 annual miles; its high-mileage leaf contains 3 claims and 1 no claim, and its low-mileage leaf contains 2 claims and 4 no claims. A policy has a prior incident, 8,000 miles, and an actual claim. With a 0.5 threshold and ties assigned to no claim, identify its leaf, estimated probability, predicted class, and whether it is misclassified. (Answer: Low-mileage leaf; probability \(=2/6=0.3333\); predict no claim; misclassified.)
- A test set has 20 policies. The tree predicts 5 claims and 15 no claims. Of the predicted claims, 3 actually had claims; among the 15 predicted no claims, 2 actually had claims. Calculate \(TP\), \(FP\), \(FN\), \(TN\), accuracy, sensitivity, and specificity. (Answer: \(TP=3\), \(FP=2\), \(FN=2\), \(TN=13\); accuracy \(=16/20=0.8000\); sensitivity \(=3/5=0.6000\); specificity \(=13/15=0.8667\).)
- A terminal node's estimated claim probability is 0.35. A test set sends 8 policies to the node, of which 3 have claims and 5 do not. Compare the number of test-set errors if the class threshold is 0.5 versus 0.3, and state whether the tree rules or leaf probability changed. (Answer: At 0.5 predict no claim, giving 3 errors; at 0.3 predict claim, giving 5 errors. The rules and leaf probability are unchanged.)