Why Tree Interpretation Requires More Than Reading the Predicted Class
A classification tree may display class counts, estimated probabilities, a predicted label, or a measure of impurity at each node. These quantities are related, but they answer different questions. Confusing them can lead to incorrect claims about a split’s quality, a case’s predicted class, or how well a tree will perform on new observations.
This tutorial focuses on four distinctions: impurity versus misclassification, probabilities versus labels, training performance versus generalization, and a rule that stops growth versus cost-complexity pruning. The earlier material on classification thresholds and tree splitting provides the context for interpreting these quantities.
The formula for the count follows directly from the prediction rule: the majority-class label correctly classifies \(\max_k n_k\) observations in that node. All other observations are counted as misclassified, so the count is \(n-\max_k n_k\). In a binary node with counts 8 and 2, this means 8 correct predictions and 2 errors—not 8 errors. With three or more classes, the errors are all observations outside the largest class, not merely the second-largest class.
Impurity Is Not the Same as Misclassification
Classification trees often use an impurity measure to evaluate candidate splits. Two standard measures are the Gini impurity and entropy. Both summarize how mixed the class proportions are at a node. They are not the same quantity as the proportion of observations that would be misclassified by assigning the node its majority-class label.
For a split into child nodes \(j=1,\ldots,J\), the weighted Gini impurity is \(\sum_j(n_j/n)G_j\), where \(n_j\) is the number of observations in child \(j\). The weighted entropy is defined in the same way using \(H_j\). Weighting matters: a small child should not count as much as a large child simply because both are children. A decrease in weighted impurity describes a change in class mixture; a decrease in misclassification describes a change in majority-label errors. The two decreases need not be equal.
Worked Example: Separating Gini Gain From Classification Error
A hypothetical insurance team splits 20 policies into two groups. In the first group, 8 policies have outcome A and 2 have outcome B. In the second group, 4 have outcome A and 6 have outcome B. Calculate the majority-label training errors before and after the split, and compare them with the weighted Gini impurity.
Method. Calculate class proportions and the majority-class error in each node. Separately calculate Gini impurity. The common mistake to avoid is treating \(n-\min(n_A,n_B)\) as the number of errors in a binary node; that expression gives the number correctly classified by the majority label.
Before the split. The total class counts are \(8+4=12\) for A and \(2+6=8\) for B. The root predicts A. It correctly classifies 12 policies and misclassifies \(20-12=8\), so its training error rate is \(8/20=0.40\). Its class proportions are \(12/20=0.60\) and \(8/20=0.40\), giving root Gini impurity \(1-(0.60^2+0.40^2)=1-(0.36+0.16)=0.48\).
First child. The child predicts A. It correctly classifies 8 policies and misclassifies \(10-8=2\), so its error rate is \(2/10=0.20\). Its Gini impurity is \(1-(0.80^2+0.20^2)=1-(0.64+0.04)=0.32\).
Second child. The child predicts B. It correctly classifies 6 policies and misclassifies \(10-6=4\), so its error rate is \(4/10=0.40\). Its Gini impurity is \(1-(0.40^2+0.60^2)=1-(0.16+0.36)=0.48\).
After the split. There are \(2+4=6\) errors among 20 policies, for an error rate of \(6/20=0.30\). The children are equal in size, so weighted Gini is \((10/20)(0.32)+(10/20)(0.48)=0.16+0.24=0.40\). The Gini reduction is \(0.48-0.40=0.08\), while the misclassification-rate reduction is \(0.40-0.30=0.10\).
Check. The child correct classifications total \(8+6=14\), so the total errors are \(20-14=6\), matching the direct error count. The two impurity reductions are not the same: \(0.08\ne0.10\).
A Probability at a Leaf Is Not a Class Label
A leaf can summarize the training observations that reached it by reporting class proportions. These are empirical estimates of class probabilities for observations assigned to that leaf. A classification rule then turns those probabilities into a label. Under the usual majority-class rule, the label is the class with the largest estimated probability. A different threshold or decision rule can produce a different label without changing the underlying proportions.
Worked Example: Reading a Leaf’s Probability and Label
A hypothetical claim-screening tree sends a new claim to a leaf containing 12 training claims with no recorded event and 8 with the event. Calculate the leaf’s empirical event probability and identify the predicted label under two specified rules: majority class, and an event threshold of 0.30.
Method. Divide the event count by the total count to obtain the empirical event proportion. Then apply each classification rule to that same probability. The common mistake to avoid is reporting the probability itself as though it were the predicted label.
Step 1: Estimate the event probability. The leaf contains \(12+8=20\) claims. The empirical event probability is \(8/20=0.40\); the empirical probability of no event is \(12/20=0.60\).
Step 2: Apply the majority-class rule. Since \(0.60>0.40\), the majority-class label is “no event.” The tree’s probability estimate for the event remains 0.40; its label is “no event.”
Step 3: Apply the stated event threshold. With an event threshold of 0.30, predict “event” when the estimated event probability is at least 0.30. Since \(0.40\ge0.30\), this rule predicts “event.”
Check. The leaf counts and probabilities did not change between the two rules. The label changed because the decision rule changed, not because the estimated event probability became different.
When reading a tree output, check whether a displayed number is a class proportion, a class label, or an impurity measure. A probability is not a guarantee that an individual observation belongs to a class. It is an estimate associated with the leaf, and a label is the result of applying a classification rule to that estimate.
Training Accuracy Does Not Establish Generalization
Training error is calculated on the observations used to fit the tree. It describes how the fitted tree classifies those observations; it does not, by itself, establish how well the tree will classify new observations. A sufficiently deep tree can fit irregularities in its training data, which can reduce training error while worsening performance on data not used to fit it.
Validation data can help compare candidate trees during model selection. Because that comparison uses the validation results, the selected tree’s validation error is not a completely independent final assessment. A separate test set, held aside until the selection decisions are complete, can provide that assessment. Results should also be interpreted in light of sample size and how representative the evaluation data are.
Worked Example: Choosing Between Training and Validation Results
A hypothetical analyst compares a small tree and a deeper tree. On 100 training observations, the small tree makes 18 errors and the deeper tree makes 5. On a separate validation set of 40 observations, the small tree makes 9 errors and the deeper tree makes 12. Calculate both error rates and state what the results support.
Method. Divide each error count by the size of its corresponding data set. Use validation results to compare the two candidates here; do not interpret the training rate as a direct estimate of future error. The common mistake to avoid is choosing the deeper tree solely because its training error is lower.
Small tree. Training error rate is \(18/100=0.18\). Validation error rate is \(9/40=0.225\).
Deeper tree. Training error rate is \(5/100=0.05\). Validation error rate is \(12/40=0.30\).
Interpretation. The deeper tree has lower training error, but higher validation error: \(0.30>0.225\). For these candidate trees, the validation results favor the small tree. They do not prove that the small tree will always perform better on every future population; they provide evidence from this validation set.
Check. The validation errors are \(9+12=21\) across the two candidate evaluations, but each candidate’s rate uses its own 40 validation observations. Thus the rates are \(9/40\) and \(12/40\), not \(9/100\) and \(12/100\).
A Growth Stopping Rule Is Not Cost-Complexity Pruning
A stopping rule restricts tree growth. For example, an algorithm may disallow a split if a child would contain too few observations or if a specified minimum impurity decrease is not met. Such a rule acts during construction and can prevent a branch from being created.
Cost-complexity pruning instead compares a fitted tree with smaller subtrees. In one common formulation, \(R(T)\) is the training misclassification rate for tree \(T\), \(|T|\) is its number of terminal nodes, and \(\alpha\ge0\) is a complexity penalty. The objective is \(R(T)+\alpha|T|\). For a fixed \(\alpha\), a smaller tree can be preferred even when its training error is higher, because it has fewer leaves. The scale of \(R(T)\) and \(\alpha\) must be consistent.
Worked Example: Comparing a Stopping Decision With Pruning
A hypothetical classification tree is fitted to 40 observations. The larger tree has four leaves and 6 training errors. A candidate pruned subtree has two leaves and 9 training errors. Using \(R(T)\) as the training error rate and the objective \(R(T)+\alpha|T|\), find the value of \(\alpha\) at which the two trees tie, and compare them at \(\alpha=0.02\) and \(\alpha=0.05\).
Method. Compute each error rate, form each penalized objective, and compare. The common mistake to avoid is treating the \(\alpha\) comparison as though it were a local split-stopping condition.
Step 1: Calculate the error rates. The larger tree has \(R(T)=6/40=0.15\). The pruned subtree has \(R(T)=9/40=0.225\).
Step 2: Write the objectives. For the larger tree, the objective is \(0.15+4\alpha\). For the subtree, it is \(0.225+2\alpha\).
Step 3: Find the tie. Set the objectives equal: \(0.15+4\alpha=0.225+2\alpha\). Subtract \(0.15+2\alpha\) from both sides to obtain \(2\alpha=0.075\), so \(\alpha=0.0375\).
Step 4: Compare at \(\alpha=0.02\). The larger-tree objective is \(0.15+4(0.02)=0.23\). The subtree objective is \(0.225+2(0.02)=0.265\). The larger tree has the lower objective.
Step 5: Compare at \(\alpha=0.05\). The larger-tree objective is \(0.15+4(0.05)=0.35\). The subtree objective is \(0.225+2(0.05)=0.325\). The subtree has the lower objective.
Check. Below the tie value, the larger tree’s lower training error outweighs its extra leaves; above it, the penalty favors the smaller subtree. This is a comparison of tree objectives, not a claim that a particular split failed a growth stopping rule.
Interpretation Checks to Use Consistently
Before drawing a conclusion from a classification-tree result, identify which quantity is being reported and what data it uses. In particular:
- For a leaf’s errors, use the total count minus the largest class count. In a binary node with counts 8 and 2, the majority label correctly classifies 8 and misclassifies 2.
- For impurity, calculate the stated measure from the class proportions. Do not call a Gini value or entropy value a misclassification rate.
- For a predicted label, identify the rule that converts probabilities into a class. A changed threshold can change the label while leaving the probabilities unchanged.
- For performance, state whether the rate is training, validation, or test error, and divide by the size of that corresponding data set.
- For tree size, distinguish a constraint that prevents growth from a pruning comparison that penalizes the size of an already fitted tree.
These checks help keep the interpretation aligned with the calculation. They are especially useful when a tree display places counts, probabilities, labels, impurity values, and error summaries close together.
Check Your Understanding
For each problem, identify the quantity being calculated before interpreting the result.
- A binary leaf contains 7 observations in class A and 3 in class B. Under the majority-class rule, how many are correctly classified, how many are misclassified, and what is the misclassification rate? (Answer: 7 correct, 3 errors, and \(3/10=0.30\).)
- A three-class node has counts 6, 3, and 1. What is its majority-class training misclassification rate? Explain why using only the smallest class count would be wrong. (Answer: \(4/10=0.40\); all observations outside the largest class are errors, so the 3 and 1 counts both contribute.)
- A leaf contains 15 event observations and 35 non-event observations. What are the empirical event probability and majority-class label? What label results under an event threshold of 0.20? (Answer: Event probability \(15/50=0.30\); majority label is non-event; at threshold 0.20 the predicted label is event.)
- A tree makes 14 errors on 80 training observations and 6 errors on 30 validation observations. Calculate both rates and state which is the validation rate. (Answer: Training \(14/80=0.175\); validation \(6/30=0.20\), so 0.20 is the validation rate.)
- Tree A has 3 leaves and error rate 0.12; Tree B has 2 leaves and error rate 0.16. Under \(R(T)+\alpha|T|\), at what \(\alpha\) do they tie, and which tree has the lower objective when \(\alpha=0.06\)? (Answer: \(0.12+3\alpha=0.16+2\alpha\), so the tie is at \(\alpha=0.04\); at 0.06 the objectives are 0.30 for A and 0.28 for B, so Tree B is lower.)