Why Tree Rules Need Careful Interpretation
A tree turns predictor values into a sequence of rules. Its compact form can make the result look more definitive than it is: a split may be mistaken for a causal finding, a leaf proportion may be read as a guaranteed outcome, or a rule supported by only a few observations may be presented as broadly reliable. Tree construction also involves choices, such as how to measure node impurity, that should be applied consistently.
A useful interpretation separates three things: how the tree was constructed, what the fitted tree predicts, and how much evidence supports that prediction. The examples below use hypothetical data. The goal is to report what the tree supports without claiming more.
Check the Split Calculation
For a classification tree, one common measure of impurity is the Gini impurity. Suppose a node contains \(K\) classes, and let \(p_k\) be the proportion of observations in class \(k\). The Gini impurity is the probability that two independent class labels drawn according to these proportions differ. It is zero when all observations are in one class and is larger when the classes are more evenly represented.
The weighting matters because a child containing many observations should contribute more to the split’s overall impurity than a child containing very few. This follows directly from the probability interpretation: an observation routed from the parent enters child \(j\) with probability \(n_j/n\), where \(n\) is the parent size and \(n_j\) is the child size. Therefore the expected child impurity is \(\sum_j (n_j/n)G_j\), not the unweighted average of the child impurities.
Worked Example: Comparing Two Candidate Splits
A hypothetical portfolio’s classification-tree node contains 40 policies: 14 with a claim and 26 without. Two candidate splits both divide this same node. Split A produces a child of 10 policies with 8 claims and a child of 30 with 6 claims. Split B produces two children of 20 policies, with 10 and 4 claims. Compare the splits using weighted Gini impurity.
Method. Calculate the parent impurity, then each candidate’s size-weighted child impurity. The split with the smaller weighted child impurity has the larger Gini decrease.
The parent claim proportion is \(14/40=0.35\), so its Gini impurity is \(1-(14/40)^2-(26/40)^2=1-196/1600-676/1600=728/1600=0.455\).
For Split A, the first child has claim proportion \(8/10=0.8\), giving impurity \(1-0.8^2-0.2^2=0.32\). The second has claim proportion \(6/30=0.2\), also giving impurity \(1-0.2^2-0.8^2=0.32\). Its weighted child impurity is \((10/40)(0.32)+(30/40)(0.32)=0.32\). The decrease is \(0.455-0.32=0.135\).
For Split B, the first child has claim proportion \(10/20=0.5\), giving impurity \(1-0.5^2-0.5^2=0.50\). The second has proportion \(4/20=0.2\), giving impurity \(1-0.2^2-0.8^2=0.32\). Its weighted child impurity is \((20/40)(0.50)+(20/40)(0.32)=0.41\). The decrease is \(0.455-0.41=0.045\).
Split A has the larger Gini decrease, \(0.135\) rather than \(0.045\), so it is preferred by this criterion. Check: both splits preserve the parent total of 14 claims, and each set of child sizes sums to 40. A common mistake is to compare unweighted averages when child sizes differ; that would give the wrong criterion for Split A.
This calculation explains what a split criterion says: it identifies a division that improves class separation according to a chosen measure. It does not establish that the predictor causes the outcome, nor that the selected split is uniquely correct under every tree-building choice.
A Split Is Predictive, Not Necessarily Causal
A tree split describes a pattern in the data used to fit the tree. For example, if a tree separates policies by whether a device is installed, the split indicates that the observed outcome proportions differ across the resulting groups. It does not show what would happen to the same policyholders if the device were installed. The groups may differ in other ways, and the data may reflect how installation decisions were made.
Worked Example: Interpreting a Split on Device Installation
In a hypothetical set of 80 policies, a fitted tree splits on whether a monitoring device is installed. Among 20 policies with a device, 4 have a claim; among 60 without a device, 18 have a claim. Calculate the observed claim proportions and state what the split does and does not support.
Method. Divide the claim count by the number of policies in each group. Interpret these as observed proportions within the groups, not as effects of an intervention.
For policies with a device, the proportion is \(4/20=0.20\). For policies without a device, it is \(18/60=0.30\). Across all policies, there are \(4+18=22\) claims out of 80, giving \(22/80=0.275\).
The tree identifies a predictive association in this hypothetical data: the observed claim proportion is 0.10 lower in the device group. It does not show that installing a device reduces an individual policyholder’s claim probability by 0.10. For instance, the groups might differ in driver behavior or in how policies were selected for installation, and the split itself does not control for every such difference.
Check: the group sizes sum to 80 and the claim counts sum to 22, consistent with the overall proportion of 0.275. The common mistake is to turn “the groups have different observed rates” into “the device caused the difference.” The tree alone supports the first statement, not the second.
A Leaf Proportion Is an Estimate, Not a Promise
A classification tree may report the fraction of training observations in a leaf that belong to a class. This is an empirical proportion: it summarizes the observed sample in that leaf. It is not a guarantee that the next observation assigned to the leaf will have that outcome. Even when the leaf’s estimated proportion is high, outcomes can vary.
A simple way to see why leaf size matters is to consider a binomial sampling model. Suppose, for illustration, that observations in a leaf are independent and have the same underlying event probability \(p\). If \(n\) observations produce \(\hat p\), the usual estimated standard error of the sample proportion is \(\sqrt{\hat p(1-\hat p)/n}\). This plug-in estimate describes sampling variability under the stated model; it is not a guarantee that the model’s assumptions hold. In particular, a normal approximation based on this standard error can be unreliable for very small leaves.
Worked Example: Comparing Leaf Sizes
Two leaves of a hypothetical claim-occurrence tree have different sizes. Leaf A contains 8 policies, of which 6 have claims. Leaf B contains 60 policies, of which 36 have claims. Calculate each observed claim proportion and its plug-in standard error, then explain why the proportions should not be reported with equal confidence.
Method. Calculate \(\hat p=x/n\), where \(x\) is the number of claims, followed by the plug-in standard error \(\sqrt{\hat p(1-\hat p)/n}\). The interpretation relies on independent observations with a common event probability within each leaf; the standard error is a rough diagnostic, especially for a small sample.
For Leaf A, \(\hat p_A=6/8=0.75\). Its estimated standard error is \(\sqrt{(0.75)(0.25)/8}=\sqrt{0.1875/8}=\sqrt{0.0234375}\approx0.1531\).
For Leaf B, \(\hat p_B=36/60=0.60\). Its estimated standard error is \(\sqrt{(0.60)(0.40)/60}=\sqrt{0.24/60}=\sqrt{0.004}\approx0.0632\). The proportions are based on different sample sizes: 6 of 8 versus 36 of 60. Leaf A’s estimated standard error is much larger, so its 0.75 proportion is less precisely supported by these observations.
These calculations do not make 0.60 a certain probability or prove that Leaf B’s underlying event rate is lower than Leaf A’s. They show why the leaf size should accompany the proportion. Check: the event and non-event counts are 6 and 2 in Leaf A, and 36 and 24 in Leaf B. A common mistake is to report “75% risk” as if it were a certain outcome for each policy, without mentioning that the estimate comes from only eight observations.
A small leaf may also result from a sequence of several splits. The final rule can sound highly specific, but specificity is not the same as strong evidence. When reporting a leaf proportion, include the leaf size and distinguish a model estimate from an individual outcome. If the sample is small, avoid presenting a normal-approximation standard error as a precise uncertainty interval.
Rules Can Change When the Data Change
A tree-building procedure chooses splits from the available data. When alternative splits have similar performance, a small change in the training sample can lead to a different split or a different rule. Earlier material on tree stability and pruning motivates checking whether a rule persists under reasonable changes to the data or model settings. A single fitted tree should not automatically be presented as the only defensible description.
Worked Example: Reporting a Rule That Moves
A hypothetical insurer fits two classification trees after making a small change to the training sample. In the first tree, the split is at 5 years of driving experience: the lower-experience leaf has 8 claims among 20 policies, and the other leaf has 6 among 30. In the second tree, the split is at 6 years: the lower-experience leaf has 9 claims among 23 policies, and the other leaf has 5 among 27. Compare the rates and decide how to describe the result.
Method. Calculate the within-leaf proportions for each tree, then distinguish the pattern that recurs from the exact rule that changes. Because the trees were fitted after a change to the training sample, these counts need not refer to identical groups.
For the first tree, the two proportions are \(8/20=0.40\) and \(6/30=0.20\). For the second, they are \(9/23\approx0.3913\) and \(5/27\approx0.1852\). In both fits, the lower-experience leaf has the larger observed claim proportion. However, the dividing experience value changes from 5 to 6 years, and the group sizes and rates also change.
A measured report would say that both fitted trees show a higher observed claim proportion in the lower-experience group, while the exact threshold is sensitive to the training sample. It would not present “5 years” or “6 years” as a definitive boundary without further support. Check: the first fit totals 50 policies and 14 claims; the second also totals 50 policies and 14 claims. The common mistake is to treat the precise split from one fit as a stable, universal rule just because the tree displays it clearly.
Practical Checks Before Presenting a Tree
A careful review asks whether the numerical calculation is correct, whether the interpretation matches the design, and whether the evidence is adequate for the strength of the wording. These checks do not require discarding the tree. They make its predictive information more useful by keeping claims proportional to the data.
Check class counts, child sizes, and any weighted impurity calculations. Confirm that the children account for the parent observations under the stated handling of missing values.
Say which groups the rule separates and how their observed outcomes differ. Do not describe a predictive split as a causal effect unless the study design and analysis support that claim.
Report the number of observations along with a leaf proportion. Small counts make a proportion more sensitive to a few observations and less persuasive as a precise risk estimate.
If a small data or fitting change moves the split, report the recurring pattern and disclose that the exact rule is unstable.
Check Your Understanding
For each hypothetical scenario, calculate what is requested and give an interpretation that does not claim more than the tree supports.
- A parent node has 30 policies, 9 with claims. A candidate split gives a child of 10 policies with 6 claims and a child of 20 with 3 claims. Calculate the parent Gini impurity, the weighted child impurity, and the Gini decrease. (Answer: Parent \(G=0.42\); weighted child impurity \(=0.33\); decrease \(=0.09\).)
- A leaf contains 12 observations, 9 of which had the event. Calculate the leaf proportion and its plug-in standard error under an independent, common-probability binomial model. How many observations in the leaf did not have the event? (Answer: \(\hat p=0.75\); estimated standard error \(=0.1250\); 3 observations did not have the event.)
- One leaf has 4 events among 5 observations; another has 18 among 30. Calculate both proportions and plug-in standard errors. Which estimate is more sensitive to its small sample size? (Answer: Proportions \(0.80\) and \(0.60\); standard errors approximately \(0.1789\) and \(0.0894\); the first estimate is more sensitive to its small sample size.)
- A tree splits on device ownership. There are 20 device owners with 4 events and 80 non-owners with 20 events. Calculate both observed proportions and the overall proportion. State whether the tree establishes a causal device effect. (Answer: \(0.20\) for owners, \(0.25\) for non-owners, and \(24/100=0.24\) overall; the split describes an association and does not by itself establish causation.)