Why Successive Splits Can Represent an Interaction
A regression tree predicts an outcome by sending each observation down a sequence of predictor-based splits to a terminal node. With two predictors, that sequence can let the effect of one predictor vary across regions defined by the other. For example, a tree might divide policies first by coverage amount, then divide only the lower-coverage group by policy duration. In that fitted tree, duration affects predictions in the lower-coverage region but not in the higher-coverage region.
This is one way a tree represents a predictor interaction. It does not fit a product term or estimate a single interaction coefficient. Instead, its interaction is expressed through different terminal-node predictions across regions. The relevant question is whether the tree’s predicted change associated with one predictor depends on the value or region of another predictor.
Reading a Branch-Dependent Split
Each split divides the observations in its current node, not the entire data set. Thus, if a tree first splits on \(X_1\), a later split on \(X_2\) within the left branch applies only when the \(X_1\) condition for that branch is satisfied. The same predictor can be split again in a different branch, possibly at a different threshold. Each terminal node corresponds to the conjunction of the conditions along its path.
Record which observations go into each child node, such as \(X_1 \leq 5\) and \(X_1 > 5\).
A split on \(X_2\) applies only within the branch where it appears. If the threshold differs by branch, retain both branch-specific conditions.
Combine the conditions on the path to that node. Its prediction is used only when every condition on that path holds.
To assess a fitted interaction, compare how predicted values change with one predictor across regions defined by the other.
Worked Example: Different Split Thresholds by Branch
A hypothetical insurer fits a regression tree to predict an annual claim-cost index \(Y\) from coverage band \(X_1\) and policy duration \(X_2\), measured in years. The root split is \(X_1 \leq 5\). In the left branch, the tree splits at \(X_2 \leq 2\); in the right branch, it splits at \(X_2 \leq 4\). The observed responses in the four terminal nodes are:
- \(X_1 \leq 5,\ X_2 \leq 2:\ 10,12\)
- \(X_1 \leq 5,\ X_2 > 2:\ 18,20\)
- \(X_1 > 5,\ X_2 \leq 4:\ 25,27\)
- \(X_1 > 5,\ X_2 > 4:\ 40,42\)
Write the prediction rule for each terminal node, compute its prediction, and determine the predicted values for \((X_1,X_2)=(4,3)\) and \((7,5)\).
Method. Translate each root-to-terminal path into a pair of conditions, then take the mean response in that terminal node. A regression tree’s terminal-node prediction is the sample mean of the responses in that node.
For \(X_1 \leq 5,\ X_2 \leq 2\), the mean is \((10+12)/2=22/2=11\). For \(X_1 \leq 5,\ X_2 > 2\), it is \((18+20)/2=38/2=19\). For \(X_1 > 5,\ X_2 \leq 4\), it is \((25+27)/2=52/2=26\). For \(X_1 > 5,\ X_2 > 4\), it is \((40+42)/2=82/2=41\).
Therefore the four rules are: predict 11 if \(X_1 \leq 5\) and \(X_2 \leq 2\); predict 19 if \(X_1 \leq 5\) and \(X_2 > 2\); predict 26 if \(X_1 > 5\) and \(X_2 \leq 4\); and predict 41 if \(X_1 > 5\) and \(X_2 > 4\).
For \((4,3)\), \(4 \leq 5\) sends the case left, and \(3>2\) sends it to that branch’s second terminal node, so the prediction is 19. For \((7,5)\), \(7>5\) sends it right, and \(5>4\) selects the final node, so the prediction is 41.
Check. Each prediction uses the threshold for the branch actually reached: \(X_2=3\) is compared with 2 when \(X_1=4\), but \(X_2=5\) is compared with 4 when \(X_1=7\). The mistake to avoid is applying one branch’s \(X_2\) threshold to the whole predictor space.
Comparing Predictor Effects Across Regions
A useful way to describe an interaction is to compare prediction differences. If a predictor’s change in predicted response is different in two regions defined by another predictor, the fitted tree’s prediction pattern is nonadditive across those regions. For a two-by-two set of regions, this comparison is often written as a difference of differences.
Here \(\hat f_{ab}\) denotes a tree prediction in the region with \(X_1\) in category or interval \(a\) and \(X_2\) in category or interval \(b\). The first parenthesis is the predicted change across the two \(X_2\) regions when \(X_1\) is in its first region; the second is that change when \(X_1\) is in its other region. A nonzero difference means these two predicted changes are unequal.
The calculation describes the fitted predictions over the regions being compared. With continuous predictors, it is a finite comparison between intervals, not a derivative or a claim that the effect is constant at every possible value inside an interval. Also, if a branch has no split on \(X_2\), the tree’s prediction is constant in \(X_2\) within that branch, even if the observed responses there vary with \(X_2\).
Worked Example: A Split on One Branch Only
A hypothetical claim-cost tree first splits on \(X_1 \leq 5\). It then splits the left branch on \(X_2 \leq 2\), but it makes no further split in the right branch. The responses in its terminal nodes are:
- Left branch, \(X_2 \leq 2:\ 8,10\)
- Left branch, \(X_2 > 2:\ 16,18\)
- Right branch, \(X_1 > 5:\ 28,30,32\)
Find all three terminal-node predictions. Then compare the tree-predicted change between the two \(X_2\) regions on the left with the corresponding change on the right.
Method. Compute each terminal-node mean, then compare the change across \(X_2\) categories separately in the two \(X_1\) branches. The right branch has no \(X_2\) split, so all its observations receive the same prediction.
The left, low-\(X_2\) mean is \((8+10)/2=18/2=9\). The left, high-\(X_2\) mean is \((16+18)/2=34/2=17\). The right-branch mean is \((28+30+32)/3=90/3=30\).
On the left, moving from the low-\(X_2\) terminal node to the high-\(X_2\) terminal node changes the prediction by \(17-9=8\). On the right, there is just one terminal node, so both \(X_2\) cases receive prediction 30; the tree-predicted change is \(30-30=0\). The difference of these changes is \(0-8=-8\), so the tree’s fitted change associated with \(X_2\) differs across the \(X_1\) branches.
For a new case with \(X_1=4,\ X_2=3\), the path is left and then high \(X_2\), giving prediction 17. For \(X_1=7,\ X_2=3\), the case goes right, where the only terminal-node prediction is 30.
Check. The tree imposes no change in prediction with \(X_2\) on the right because it has no split there. That does not prove that the underlying relationship is truly flat in that branch: the observed responses 28, 30, and 32 differ, but the fitted tree assigns them their common branch mean. The mistake to avoid is treating a flat fitted prediction as evidence of no real association.
Difference of Differences in a Two-by-Two Tree
When both predictors divide the predictor space into the same two regions on each branch, all four terminal-node predictions can be compared directly. This makes the difference-of-differences calculation especially clear. It also shows why a tree that successively splits on \(X_1\) and \(X_2\) can represent an interaction even though it contains no explicit interaction term.
Worked Example: Testing the Additive Pattern of Four Leaves
A hypothetical insurer uses \(X_1 \leq 5\) and \(X_2 \leq 2\) to define four terminal nodes. The response observations are:
- \(X_1 \leq 5,\ X_2 \leq 2:\ 10,14\)
- \(X_1 \leq 5,\ X_2 > 2:\ 15,19\)
- \(X_1 > 5,\ X_2 \leq 2:\ 18,24\)
- \(X_1 > 5,\ X_2 > 2:\ 32,38\)
Compute the four leaf predictions, the predicted change across \(X_2\) in each \(X_1\) region, and the difference of differences. State whether these predictions have an additive two-region pattern.
Method. Take the mean in each cell. Then subtract low-\(X_2\) prediction from high-\(X_2\) prediction within each \(X_1\) branch, and subtract the resulting changes.
The four means are \((10+14)/2=24/2=12\), \((15+19)/2=34/2=17\), \((18+24)/2=42/2=21\), and \((32+38)/2=70/2=35\). Thus the predictions are 12, 17, 21, and 35 in the order listed.
For \(X_1 \leq 5\), the change across \(X_2\) regions is \(17-12=5\). For \(X_1>5\), it is \(35-21=14\). The difference of differences, using the higher-\(X_1\) change minus the lower-\(X_1\) change, is \(14-5=9\). It is nonzero, so the four predictions do not have an additive two-region pattern.
Check. The direct contrast is also \(35-21-17+12=14-17+12=9\). The two calculations agree. An additive pattern would require equal \(X_2\) changes in both \(X_1\) regions; here they are 5 and 14. The common mistake is to compare only one pair of leaves and call that comparison an interaction. Interaction requires comparing the predictor change across the other predictor’s regions.
What an Interaction in a Tree Does—and Does Not—Mean
A tree’s interaction is local to its partition and prediction rules. It can appear because one branch is split while another is not, because the same predictor is split at different thresholds in different branches, or because different terminal-node means produce different changes across corresponding regions. These are all descriptions of how the fitted tree maps predictor values to predictions.
The interaction need not imply that both predictors appear in every path. For example, if the tree splits on \(X_1\) and then only one \(X_1\) branch is split on \(X_2\), predictions depend on \(X_2\) in that branch but not the other. Conversely, a later split on \(X_2\) in both branches may still yield an interaction if the prediction changes differ. The presence of two predictors in the tree alone is not enough to establish a nonadditive pattern; compare the predicted changes.
In actuarial applications, this distinction matters when interpreting a fitted relationship between policy attributes and a response such as claim cost. A tree may assign different cost predictions to duration groups for different coverage bands. That can be useful for describing the fitted model, but it should not be reported as proof that changing a policy attribute would cause the predicted difference. The tree’s regions and predictions summarize the model fit, not a causal experiment.
Common Interpretation Traps
- Applying a split everywhere. A condition in one branch does not apply to observations in the other branch. Follow the path before using a threshold.
- Assuming a common threshold. The same predictor can be split at different values in separate branches. Read each node’s condition independently.
- Calling any two-predictor tree an interaction. Compare predicted changes across regions. Two predictors can appear in paths without the changes forming a nonadditive pattern.
- Interpreting an unsplit branch as a scientific null. A tree’s prediction is constant within a terminal node, but the observed response may still vary with a predictor that was not used for another split.
- Confusing prediction with causation. A branch-dependent prediction is a property of the fitted tree. It does not, on its own, establish what would happen under an intervention.
Check Your Understanding
For each problem, follow the branch conditions carefully and compare predictions only after identifying the relevant terminal nodes.
- A tree splits first at \(X_1 \leq 8\). In the left branch it predicts 6 for \(X_2 \leq 3\) and 13 for \(X_2>3\); the right branch is unsplit and has prediction 20. Find the predictions for \((6,4)\) and \((10,2)\), then compare the \(X_2\) prediction change across the two \(X_1\) branches. (Answer: 13 and 20; the changes are 7 on the left and 0 on the right, so their difference, right minus left, is \(-7\).)
- A four-leaf tree has predictions 9 and 15 for low and high \(X_2\) when \(X_1\) is low, and 20 and 31 for low and high \(X_2\) when \(X_1\) is high. Compute both \(X_2\) changes and the difference of differences, high-\(X_1\) change minus low-\(X_1\) change. (Answer: the changes are 6 and 11; the difference of differences is 5.)
- The root condition is \(X_1 \leq 4\). The left branch splits at \(X_2 \leq 1\); the right branch splits at \(X_2 \leq 5\). What prediction rule applies to \((3,2)\), and what rule applies to \((6,2)\)? (Answer: \((3,2)\) follows \(X_1\leq4,\ X_2>1\); \((6,2)\) follows \(X_1>4,\ X_2\leq5\).)
- Four terminal-node means, ordered as low \(X_1\)/low \(X_2\), low \(X_1\)/high \(X_2\), high \(X_1\)/low \(X_2\), and high \(X_1\)/high \(X_2\), are 11, 16, 24, and 37. Calculate the two \(X_2\) changes and their difference. Can the four predictions be represented by an additive two-region pattern? (Answer: the changes are 5 and 13; their difference, high-\(X_1\) minus low-\(X_1\), is 8, so they are not additive.)
- A tree predicts 14 and 22 across the two \(X_2\) regions in the low-\(X_1\) branch, and has a single prediction of 30 in the high-\(X_1\) branch. What interaction contrast results if the high-\(X_1\) branch’s \(X_2\) change is defined as zero? What does that result establish? (Answer: \(0-(22-14)=-8\); it establishes that the fitted tree’s \(X_2\) prediction change differs by branch, not that the underlying relationship is causally different.)