Representing Interactions With Successive Splits

Why Successive Splits Can Represent Interactions

A predictor can matter differently for different kinds of observations. For example, the association between a service-use measure and expected cost might differ between new and renewing policyholders. A single split on service use imposes the same division wherever it is applied. A tree can instead split first by policyholder type, then make a separate service-use split within each branch. The service-use comparison is then conditional on the earlier split.

This is one way a tree represents an interaction: the effect associated with one predictor depends on the value or group defined by another predictor. Unlike a regression model with an explicit product term, a tree represents this dependency through its sequence of splits and the predictions in the resulting terminal nodes.

Definition: For two predictors $A$ and $B$, a predictive interaction is present in a set of predictions when the difference associated with changing $B$ is not the same across values or groups of $A$. In a tree, this can occur when a split on $B$, or its resulting predictions, differs across branches defined by $A$. This describes a pattern in predictions; it does not by itself establish a causal effect.

A tree applies one split at a time. After an observation follows a branch, a later split can refine that branch further. The groups created by the earlier split therefore determine which later comparisons are made. The tree need not use the same next predictor, or the same cut point, in every branch. This conditional structure is the key to representing interactions with successive splits.

Conditional Effects and a Difference-in-Differences Contrast

For a simple numeric illustration, suppose there are two binary predictors, $A$ and $B$, and a regression tree gives a prediction for each of the four combinations. Write $\hat{y}_{ab}$ for the prediction when $A=a$ and $B=b$. The predicted difference for changing $B$ from 0 to 1 when $A=0$ is $\hat{y}_{01}-\hat{y}_{00}$. When $A=1$, it is $\hat{y}_{11}-\hat{y}_{10}$. Their difference is the interaction contrast:

(\hat{y}_{11}-\hat{y}_{10})-(\hat{y}_{01}-\hat{y}_{00})

If this contrast is not zero, the predicted difference associated with $B$ changes with $A$. In a regression model containing only separate additive terms for $A$ and $B$, the difference associated with $B$ would be the same at either value of $A$, so this contrast would be zero. A tree can produce a nonzero contrast because its terminal-node predictions are allowed to differ across the branches.

Key result: A path that first separates observations by $A$ and then makes a $B$-based comparison within each $A$ branch can represent different $B$ effects for different $A$ groups. The interaction is implicit in the branch-specific predictions; no product column such as $A \times B$ is required.

This result follows directly from how the tree makes predictions. Each terminal node has its own prediction, based on the training observations that reach it. If the tree splits on $A$, and then on $B$ in each $A$ branch, the four combinations can reach different terminal nodes. Because those nodes have separate predictions, the difference between the two $B$ predictions in one branch need not equal the difference in the other branch.

For a regression tree whose terminal-node prediction is the mean response among observations in that node, the result is immediate: each of the four combinations has its own node mean when all four paths end separately. Substituting those means into the contrast gives the difference between the two branch-specific $B$ differences. The contrast is zero only if those differences happen to be equal; the tree structure does not require them to be equal.

Worked Examples

Worked Example: Different Usage Effects by Policyholder Group

A hypothetical insurer fits a regression tree to predict annual claim cost, in dollars. The root split separates new from renewing policyholders. In each branch, the tree splits on whether the recorded service-use count is low or high. The terminal-node predictions are: new/low, $180; new/high, $300; renewing/low, $220; renewing/high, $240. Calculate the high-versus-low prediction difference in each group and the interaction contrast. State what the result means.

Method. Calculate the usage difference separately within each policyholder group, then subtract the renewing-group difference from the new-group difference.

For new policyholders, the high-versus-low difference is $300-180=120$ dollars. For renewing policyholders, it is $240-220=20$ dollars.

The interaction contrast is $120-20=100$ dollars. Equivalently, using the four predictions directly:

$$ (240-220)-(300-180)=20-120=-100 $$

The displayed expression uses the opposite ordering of groups from the first calculation, so it is the negative of that contrast. Under the convention $(\text{new difference})-(\text{renewing difference})$, the contrast is $120-20=100$ dollars.

Thus, the tree’s predicted high-versus-low difference is $100 larger for new policyholders than for renewing policyholders. This is a conditional predictive pattern, not evidence that increasing service use causes costs to rise.

Check. The branch-specific differences are $120 and $20, and their difference is $100. The common mistake is to report one overall usage difference and imply that it applies to both groups; it would erase the pattern the branch-specific splits capture.

Worked Example: Branch-Specific Predictors for a Classification Outcome

A hypothetical lender uses a classification tree to estimate the probability of a missed payment. The root split separates standard from flexible repayment plans. Among standard-plan accounts, the tree splits on whether the balance exceeds $8,000. Among flexible-plan accounts, it instead splits on whether a recent payment was late. The terminal nodes have these counts:

Standard plan, balance at most $8,000: 4 missed payments among 40 accounts.
Standard plan, balance above $8,000: 12 missed payments among 40 accounts.
Flexible plan, no recent late payment: 3 missed payments among 30 accounts.
Flexible plan, recent late payment: 15 missed payments among 30 accounts.

Calculate each terminal-node event probability and compare the two within-plan differences. What interaction pattern does the tree represent?

Method. For each terminal node, divide the number of missed payments by the number of accounts. Then compare the resulting probabilities within each repayment-plan branch.

For the standard plan, the probabilities are $4/40=0.1000$ and $12/40=0.3000$. The high-balance versus lower-balance difference is $0.3000-0.1000=0.2000$.

For the flexible plan, the probabilities are $3/30=0.1000$ and $15/30=0.5000$. The recent-late-payment versus no-recent-late-payment difference is $0.5000-0.1000=0.4000$.

The tree represents a branch-dependent pattern: within standard plans it uses balance to distinguish predicted risk, while within flexible plans it uses recent payment history. The two differences are not a controlled comparison of the same predictor: the downstream predictors differ. Therefore, it would be incorrect to call $0.4000-0.2000=0.2000$ a formal interaction contrast for one common predictor. What can be concluded is that the tree uses different conditional splits in the two branches and assigns the displayed probabilities to their resulting terminal nodes.

Check. The four probabilities are $0.1000$, $0.3000$, $0.1000$, and $0.5000$, respectively. The trap is to compare branch-specific differences as if they measured the effect of the same predictor. Here, the tree’s interaction-like structure is represented by its different downstream tests, not by a numerical contrast for one shared predictor.

Worked Example: A Classification Pattern With No Marginal Signal

A hypothetical monitoring system classifies whether a machine requires inspection. Two binary indicators are recorded: $A$, whether vibration is elevated, and $B$, whether temperature is elevated. In a training sample, there are 20 machines in each predictor combination. The numbers requiring inspection are:

$A=0, B=0$: 0 of 20.
$A=0, B=1$: 20 of 20.
$A=1, B=0$: 20 of 20.
$A=1, B=1$: 0 of 20.

Describe the terminal-node probabilities for a tree that first splits on $A$ and then splits on $B$ within each branch. Explain why looking at the marginal event rate by either predictor alone would be misleading.

Method. Within each of the four combinations, divide the event count by 20. Then aggregate across the other predictor to calculate each marginal event rate.

The four combination probabilities are $0/20=0$, $20/20=1$, $20/20=1$, and $0/20=0$. A tree that splits first on $A$ and then on $B$ in both branches can assign these four probabilities to separate terminal nodes.

For $A=0$, the event probability across both $B$ groups is $(0+20)/(20+20)=0.5$. For $A=1$, it is $(20+0)/(20+20)=0.5$. Thus, the marginal probability is 0.5 at either value of $A$. Similarly, for $B=0$, it is $(0+20)/40=0.5$, and for $B=1$, it is $(20+0)/40=0.5$.

Each predictor alone has no marginal difference in this constructed sample, but their combination perfectly separates the outcomes. The tree captures this pattern through successive splits. A tree would need enough observations and a growth process that permits these splits; the pattern does not guarantee that every fitted tree will find them.

Check. The marginal event rate is $0.5000$ for each level of either predictor, while the four joint-group rates are 0, 1, 1, and 0. The mistake to avoid is concluding that neither predictor is useful because its marginal rates are equal.

How to Read the Interaction Structure

When examining a tree for conditional predictor effects, follow the branches in order. Ask what group an earlier split creates, then determine whether a later split makes a further distinction within that group. A later split on the same predictor with different cut points can also give different conditional comparisons across branches. The relevant evidence is the full path and the resulting terminal-node predictions, not merely the fact that a predictor appears somewhere in the tree.

Identify the first condition.
Record which predictor creates the initial groups and what values or categories define each branch.

Inspect what happens within each group.
Check whether later splits use the same predictor, different predictors, or different cut points in separate branches.

Compare predictions conditionally.
For regression, compare terminal-node response predictions. For classification, compare the event probabilities assigned to the relevant nodes.

State the conclusion narrowly.
Describe how the fitted predictions vary across the observed conditions. Do not turn a predictive pattern into a causal claim without an appropriate causal design and assumptions.

A tree’s conditional structure is useful because it can express patterns that a model with only separate additive predictor effects would miss. It also comes with limitations. A split is selected from the available data, and small changes in the data can alter which split is chosen. A branch may contain few observations, so its prediction can be uncertain. Further, the tree represents interactions through its chosen partitions; it does not provide a single coefficient that summarizes an interaction across all observations.

Common Mistake: Treating a Branch Effect as Universal

Suppose a tree’s predictions show a large high-versus-low difference in one branch and a small difference in another. It is incorrect to describe the larger difference as the universal effect of the predictor. The comparison must retain the condition defining each branch. Likewise, if branches use different downstream predictors, their differences are not directly comparable as if they were estimates of a common predictor’s effect.

Another trap is to interpret an interaction as necessarily causal. The tree describes how its predictions change among groups defined by recorded predictors. Differences may reflect confounding, selection, measurement choices, or sample variation. The tree’s structure alone does not establish why the pattern occurs.

Key takeaway: Successive splits let a tree make later comparisons conditional on earlier ones. When branch-specific predictions differ, the tree can represent predictor interactions without an explicit product term. Interpret the full path, calculate comparisons within the relevant branches, and describe the result as predictive unless causal evidence supports more.

Check Your Understanding

For each problem, calculate the requested quantities and keep the branch conditions in your interpretation.

A regression tree predicts repair cost of $90 and $150 for low and high temperature sensors among older units, and $110 and $130 among newer units. Find the high-versus-low difference in each unit group and the older-minus-newer interaction contrast. (Answer: Older difference = $60; newer difference = $20; contrast = $40.)
A classification tree has 5 events among 25 accounts in one terminal node and 9 events among 30 in another node within the same branch. Find both predicted event probabilities and their difference. (Answer: $0.2000$, $0.3000$, and $0.1000$.)
A tree splits first on region. In the east branch it next splits on age; in the west branch it next splits on deductible choice. Can the difference between the downstream split effects be called an interaction contrast for one common predictor? (Answer: No. The downstream predictors differ, so describe the branch-specific structure rather than treating the two differences as effects of the same predictor.)
For two binary predictors, four regression-tree predictions are $\hat{y}_{00}=12$, $\hat{y}_{01}=18$, $\hat{y}_{10}=20$, and $\hat{y}_{11}=35$. Calculate the $B$ difference at each $A$ value and the interaction contrast $(\hat{y}_{11}-\hat{y}_{10})-(\hat{y}_{01}-\hat{y}_{00})$. (Answer: At $A=0$, 6; at $A=1$, 15; interaction contrast = 9.)
A classification sample has 10 observations in each of four binary-predictor groups. Event counts for $(A,B)=(0,0),(0,1),(1,0),(1,1)$ are 2, 8, 8, and 2. Calculate all four group probabilities and the marginal event probability at each value of $A$ and $B$. (Answer: Group probabilities are $0.2000, 0.8000, 0.8000, 0.2000$; each marginal probability is $0.5000$ at either level of either predictor.)

What You'll Learn