Interpreting Variable Importance In A Single Tree

What a Tree’s Importance Summary Measures

A fitted tree can be read as a set of rules, but its printed rules do not by themselves show how much each predictor contributed to the tree’s split decisions. A variable-importance summary provides one view of that contribution: it aggregates the improvements associated with splits that use each predictor. Reading the summary well requires knowing what counts as an improvement and what the aggregation does—and does not—mean.

The values depend on the fitted tree and its split criterion. They are not direct measurements of a predictor’s causal effect, nor are they a universal score of how useful that predictor would be in every model. In particular, two predictors that carry similar information can compete for splits, so their individual scores may depend on which one the tree selected.

Definition: For a fitted tree, a split improvement is the reduction in the tree’s chosen measure of node risk when a node is divided into child nodes. A predictor’s split-based importance is an aggregate of the improvement contributions assigned to that predictor across the tree’s splits. The measure of risk, contribution rules, and any normalization must be identified before interpreting the resulting values.

Split Improvement and Aggregation

Let $t$ denote an internal node, meaning a node that is split, and let $R(t)$ be its total node risk under the chosen criterion. If $t$ is split into children $t_L$ and $t_R$, the improvement is

$$ G_t=R(t)-R(t_L)-R(t_R). $$

A useful split reduces risk, so $G_t$ is positive. For a regression tree, risk is commonly based on squared prediction errors within nodes. For a classification tree, a split criterion may use an impurity measure such as the Gini index or information. The risk calculation must use the same criterion on the parent and its children. A decrease in one criterion should not be described as a decrease in another.

In practice, software may store a split-improvement quantity on a particular scale, and case weights or the split criterion can affect that quantity. For interpreting a fitted tree, use the improvement values consistently as supplied by the method; do not silently multiply them by node size if node size is already reflected in the recorded improvement.

If $v(t)$ is the predictor used for the primary split at node $t$, a simple primary-split summary for predictor $j$ is

I_j=\sum_{t:\,v(t)=j}G_t.

This sum is not a count of how many times the predictor appears. Each split contributes according to its improvement, so a predictor used once for a large reduction can receive more importance than one used several times for small reductions. Some tree implementations, including the usual variable-importance summary for an rpart fit, can also credit surrogate splits. A surrogate is an alternative rule used to approximate a primary split when the primary predictor is unavailable. Its contribution is distinct from a primary split’s contribution, and in rpart the surrogate credit is adjusted for the proportion of cases missing the primary predictor. Details of the method and settings matter when interpreting a particular output.

Worked Example: Add Improvements Across a Regression Tree

Suppose a hypothetical tree predicts repair cost using squared-error reduction as its split criterion. Its recorded split improvements are: at the root, predictor $A$ gives 2,400; at one child, predictor $B$ gives 900; at another node, predictor $A$ gives 300; and at a final internal node, predictor $C$ gives 800. Calculate primary-split importance and relative importance. Assume the improvements are on the same scale and that this calculation uses primary splits only.

Method. Sum the improvements for each predictor, then divide each sum by the total importance to express it as a percentage.

Predictor $A$ appears at two primary splits, so $I_A=2{,}400+300=2{,}700$. Predictor $B$ appears once, so $I_B=900$. Predictor $C$ also appears once, so $I_C=800$. The total is $2{,}700+900+800=4{,}400$.

The relative percentages are $100(2{,}700/4{,}400)=61.3636\ldots\%$ for $A$, $100(900/4{,}400)=20.4545\ldots\%$ for $B$, and $100(800/4{,}400)=18.1818\ldots\%$ for $C$. Rounded to two decimal places, the values are $61.36\%$, $20.45\%$, and $18.18\%$. They sum to $99.99\%$ because of rounding; before rounding, the percentages sum to $100\%$.

The result says that $A$ accounts for the largest share of these recorded primary-split improvements in this tree. It does not say that $A$ is responsible for 61.36% of repair cost, or that changing $A$ would change cost by that percentage. The common mistake avoided is treating relative split importance as a causal or outcome percentage.

Why the Improvements Add Up

The additive summary has a direct justification when each node’s risk is the total risk for the cases in that node. Assume a fixed tree, a consistent risk criterion, and child nodes that partition their parent’s cases. For every internal node, write the improvement as parent risk minus the risks of its children. Adding these expressions across the tree cancels the risk of every internal child once as a child and once as a parent. Only the root risk and terminal-node risks remain.

\sum_{t\text{ internal}}G_t = R(\text{root})-\sum_{\ell\text{ terminal}}R(\ell).

This telescoping identity explains why the sum of primary split improvements represents a reduction from the root’s initial risk to the total risk remaining in the terminal nodes. It also explains why a split higher in the tree can have a large contribution: it acts on a larger group of cases. If an implementation uses a different scale or adds surrogate credit, that reported importance need not equal this primary-split risk reduction exactly. The identity justifies the sum of the defined primary improvements, not every possible software-specific extension to the summary.

Worked Example: Compute Classification Improvement Before Importance

A hypothetical classification tree uses total weighted Gini impurity as node risk. At its root, 100 observations have Gini impurity $0.48$. The root is split on predictor $B$, producing child nodes with 60 and 40 observations and Gini impurities $0.30$ and $0.375$. The 60-observation child is split on $A$, producing two 30-observation nodes with impurities $0.20$ and $0.30$. The 40-observation child is split on $C$, producing two 20-observation nodes with impurities $0.20$ and $0.30$. Find the three primary-split improvements and their relative shares.

Method. Multiply each node’s Gini impurity by its number of observations to obtain total node risk, then subtract the children’s total risk from the parent’s total risk.

At the root, total risk is $100(0.48)=48$. Its children have total risk $60(0.30)+40(0.375)=18+15=33$. Thus the improvement for $B$ is $48-33=15$.

For the 60-observation node, parent risk is $60(0.30)=18$. Its children have total risk $30(0.20)+30(0.30)=6+9=15$. Thus the improvement for $A$ is $18-15=3$.

For the 40-observation node, parent risk is $40(0.375)=15$. Its children have total risk $20(0.20)+20(0.30)=4+6=10$. Thus the improvement for $C$ is $15-10=5$.

The total importance is $15+3+5=23$. The relative shares are $100(15/23)=65.2173\ldots\%$ for $B$, $100(3/23)=13.0434\ldots\%$ for $A$, and $100(5/23)=21.7391\ldots\%$ for $C$. Rounded to two decimal places, these are $65.22\%$, $13.04\%$, and $21.74\%$; the rounded values sum to $100.00\%$. The calculation checks against the telescoping result: root risk $48$ minus terminal risk $6+9+4+6=25$ equals $23$, the sum of the improvements.

The common mistake avoided is comparing unweighted impurity values alone. For example, the root’s child impurities are not simply averaged equally: the child sizes differ, so total risk uses $60$ and $40$ as weights.

Surrogates, Normalization, and Reading R Output

A primary split is the rule selected to divide a node. A surrogate split is an alternative rule that agrees with the primary rule on cases where both predictors are observed. Surrogates can be useful when a case lacks the primary predictor, but a surrogate’s existence does not make it the primary driver of the tree’s partition when all values are available.

When a software summary assigns surrogate importance, that contribution may be reduced according to how often the primary predictor is missing. For example, if 20 of 80 cases at a node are missing the primary predictor, the missing fraction is $20/80=0.25$. A surrogate contribution of 8 before this missingness adjustment would contribute $8(0.25)=2$ after adjustment under that stated rule. This is an illustration of the adjustment, not a claim that every package or setting defines surrogate credit identically.

Worked Example: Interpret a Surrogate Contribution

At one node in a hypothetical rpart classification tree, the primary predictor $X$ has a split contribution of 12. Predictor $Z$ is a surrogate with pre-adjustment contribution 8, and 20 of the 80 cases at this node are missing $X$. Elsewhere, predictor $Y$ has a primary-split contribution of 3. Using the missingness adjustment described above, calculate the resulting contributions and explain the interpretation.

Method. Multiply the surrogate contribution by the fraction missing the primary predictor, then aggregate contributions by predictor.

The missing fraction is $20/80=0.25$. Predictor $Z$’s adjusted surrogate contribution is $8(0.25)=2$. The resulting contributions are $I_X=12$, $I_Z=2$, and $I_Y=3$, for a total of $12+2+3=17$. Their relative shares are $100(12/17)=70.5882\ldots\%$, $100(2/17)=11.7647\ldots\%$, and $100(3/17)=17.6470\ldots\%$. Rounded to two decimal places, these are $70.59\%$, $11.76\%$, and $17.65\%$, which sum to $100.00\%$ after rounding.

The score for $Z$ reflects surrogate credit under the stated rule; it is not a primary-split contribution of 2, and it does not mean that $Z$ has an independent causal effect. The common mistake avoided is treating surrogate status as if the predictor had been selected for the same primary split.

In R, an rpart fit may store an importance summary in fit$variable.importance. Its values are generally presented as relative contributions on a common scale; if a percentage is needed, check whether the displayed values have already been normalized. For a manual comparison, use the same scale for all variables and calculate $100I_j/\sum_k I_k$ only when the total is positive. A zero total provides no basis for relative percentages.

Importance values can also change when the fitted tree changes. The data, the split criterion, stopping rules, pruning, and small changes in the available candidate splits can all alter which predictor is selected and where. Even if the predictive task is unchanged, a different fitted tree can distribute improvement differently. Importance is therefore a summary of a particular fitted tree, not an immutable property of the variables.

What Importance Does Not Establish

A high score means that a predictor received substantial split-improvement credit in this fitted tree under its particular rules. It does not establish that the predictor causes the outcome. A tree is a predictive construction: it chooses splits to improve a prediction criterion in the observed data. Confounding, measurement choices, and the way data were collected are not resolved by the importance calculation.

Nor is the score a unique measure of a predictor’s value. If two predictors are correlated or encode similar information, one may be selected for a split while the other is not. The selected predictor can receive the improvement credit even though the unselected predictor might have supported a similar split. In another fitted tree, a small data change could lead to a different allocation. A low score therefore does not always mean that a predictor contains no useful information.

Relative importance also has a denominator: the scores of the other predictors in that tree. A predictor’s share can change because another predictor gained or lost contribution, even if its own raw contribution stayed the same. When reporting results, distinguish the raw score from a normalized percentage and state that the result describes split-based contributions in the fitted tree.

Key takeaway: Split-based variable importance aggregates the improvement credited to predictors in a particular fitted tree. Interpret the split criterion, aggregation scale, and surrogate treatment; do not interpret the result as a causal effect or a unique, model-independent ranking of predictive value.

Check Your Understanding

Work through these original problems, keeping the split criterion and the meaning of each contribution explicit.

A regression tree has primary split improvements of 18 and 6 for predictor $M$, 9 for $N$, and 3 for $P$. Calculate each predictor’s raw importance and relative percentage, rounded to two decimal places. (Answer: $M=24$, $N=9$, and $P=3$; total $36$; relative importance $66.67\%$, $25.00\%$, and $8.33\%$, respectively.)
A classification node has 50 cases and Gini impurity $0.40$. A proposed split produces children of 20 and 30 cases with impurities $0.25$ and $0.20$. Calculate the total-risk improvement. (Answer: parent risk $50(0.40)=20$; child risk $20(0.25)+30(0.20)=5+6=11$; improvement $20-11=9$.)
In a tree, the primary-split contributions are 10 for $Q$ and 5 for $R$. A surrogate for $Q$ has pre-adjustment contribution 6, and 15 of 60 cases at that node are missing $Q$. Under the stated missingness adjustment, calculate the adjusted surrogate contribution and total importance. (Answer: missing fraction $15/60=0.25$; adjusted contribution for the surrogate $6(0.25)=1.5$; total $10+5+1.5=16.5$.)
A tree’s primary split on $D$ reduces total node risk by 14. Two lower splits on $E$ reduce risk by 4 and 3. What are the primary-only relative importance percentages for $D$ and $E$? (Answer: $I_D=14$, $I_E=4+3=7$, total $21$; $D=66.67\%$ and $E=33.33\%$, rounded to two decimal places.)
A predictor has zero primary-split importance in one fitted tree, while a strongly related predictor has a large score. State one valid interpretation and one conclusion that is not justified. (Answer: The tree allocated its recorded primary-split improvement to the selected predictor; the zero score does not establish that the other predictor has no predictive information, and neither score establishes a causal effect.)

What You'll Learn