From Split Rules to Regions in Predictor Space
A decision tree is often drawn as a branching diagram, but its predictions can also be understood geometrically. Each predictor value identifies a point in predictor space. A split on one numeric predictor divides that space into two parts; successive splits divide parts again. The resulting terminal-node regions are rectangular, and a regression tree assigns one constant prediction throughout each region.
This geometric view is useful when interpreting what a fitted tree says about a risk or outcome. It makes the model’s boundaries visible, clarifies how predictors combine, and highlights that a tree does not generally interpolate smoothly between observations. The focus here is the geometry and prediction surface created by numeric splits, aligned with the SOA Exam PA (Predictive Analytics) syllabus.
The conditions on a path are combined by intersection: an observation must satisfy all conditions on that path to belong to the region. A split on one predictor places a threshold boundary perpendicular to that predictor’s axis. For two predictors, the resulting pieces are rectangles or rectangular strips; in more dimensions, they are rectangular boxes, possibly unbounded on some sides.
Boundary conventions matter. For example, a split stated as \(x_1 \le 5\) versus \(x_1 > 5\) assigns a value exactly equal to 5 to the first branch, and the two branches do not overlap. When describing a tree’s regions, preserve the inequalities used by its splits. If a split is instead written \(x_1 < 5\) versus \(x_1 \ge 5\), the equality case belongs to the second branch.
Why the Prediction Surface Is Piecewise Constant
Assume a regression tree has been fitted and each terminal node \(m\) has a fixed prediction \(c_m\), such as the mean response among the training observations in that node. The prediction for a new predictor vector is the prediction of the single terminal node whose region contains that vector. Therefore, once a point is inside a region, changing its predictor values without crossing one of that region’s boundaries does not change its prediction.
This gives a stepwise-constant surface: flat within each region, with possible jumps at split boundaries. The tree’s split rules determine the geometry; the terminal-node estimates determine the heights of the flat pieces. The same geometric idea applies to classification trees, although the output may be a class or a class probability rather than a numeric response.
Here \(R_1,\ldots,R_M\) are the terminal-node regions, \(c_m\) is the prediction attached to region \(R_m\), and the indicator is 1 when its condition is true and 0 otherwise. The regions partition predictor space: every point follows one branch at each split and arrives in exactly one terminal node. Thus, exactly one indicator in the sum is 1 for a given point, so the displayed formula returns that region’s prediction.
Worked Example: Mapping a Two-Predictor Tree into Rectangles
A hypothetical insurer uses annual mileage \(x_1\), measured in thousands of miles, and vehicle age \(x_2\), measured in years, to predict an annual claim-cost index. Its tree first splits at \(x_1 \le 6\). In the left branch it splits at \(x_2 \le 4\), assigning predictions 70 and 110 to the two resulting leaves, respectively. In the right branch it splits at \(x_2 \le 8\), assigning predictions 130 and 190. State all regions and determine the prediction for a vehicle with \(x_1=6\) and \(x_2=8\).
Method. For each leaf, combine the root condition with the condition at the next split. Conditions along a path must all hold.
The first left leaf has \(x_1 \le 6\) and \(x_2 \le 4\). Its region is \(R_1=\{x_1 \le 6,\ x_2 \le 4\}\), with prediction 70. The other left-branch leaf has \(x_1 \le 6\) and \(x_2 > 4\). Its region is \(R_2=\{x_1 \le 6,\ x_2 > 4\}\), with prediction 110.
The first right leaf has \(x_1 > 6\) and \(x_2 \le 8\). Its region is \(R_3=\{x_1 > 6,\ x_2 \le 8\}\), with prediction 130. The final leaf has \(x_1 > 6\) and \(x_2 > 8\). Its region is \(R_4=\{x_1 > 6,\ x_2 > 8\}\), with prediction 190.
For the specified vehicle, \(x_1=6\) satisfies \(x_1 \le 6\), so it follows the left branch. Then \(x_2=8\) satisfies \(x_2>4\), so it reaches \(R_2\). The prediction is therefore 110. The value \(x_2=8\) is not compared with the right-branch threshold of 8 because this vehicle did not enter that branch.
Check. The four regions are disjoint and cover all pairs of values: every \(x_1\) is either at most 6 or greater than 6, and the applicable \(x_2\) split sends the observation to exactly one of two leaves. A common mistake is to apply every threshold in the tree to every observation; the path determines which later split is reached.
How Successive Splits Create Interactions
A tree can make the effect of one predictor depend on another without fitting a conventional product term. It does so by making a later split conditional on an earlier one. Geometrically, a predictor can be split at different thresholds in different parts of the predictor space. The boundaries remain axis-aligned, but their locations depend on the path.
The following justification explains why the region for a path is rectangular. Start with the full predictor space. A split on \(x_j\) intersects the current region with one of the half-spaces \(x_j \le c\) or \(x_j > c\). That operation adds a lower or upper bound on one coordinate and leaves the other coordinates’ bounds unchanged. Repeating it along a path adds coordinate-wise constraints. The final region is the intersection of those constraints, so it is a product of intervals, that is, an axis-aligned rectangle or box.
In this expression, \(P_m\) is the sequence of splits on the path to leaf \(m\), and \(j_s\) is the predictor used at split \(s\). The intersection is essential: replacing it with a union would allow a point that satisfies only one path condition to enter the leaf. If a predictor is split more than once along a path, its conditions combine into a narrower interval. For example, \(x_1>2\) followed by \(x_1 \le 7\) gives \(2<x_1 \le 7\).
Worked Example: Conditional Thresholds and a Stepwise Surface
A hypothetical maintenance team predicts a service-cost score using machine hours \(x_1\) and component age \(x_2\). A tree splits first at \(x_1 \le 10\). In that branch, it splits at \(x_2 \le 3\), with leaf predictions 40 and 65. For \(x_1>10\), it splits at \(x_2 \le 6\), with leaf predictions 90 and 150. Describe the regions and find predictions for \(A=(8,5)\), \(B=(12,5)\), and \(C=(12,7)\). Then explain how the tree represents a conditional effect.
For \(x_1 \le 10\), the first leaf has \(x_2 \le 3\), so \(R_1=\{x_1\le10,\ x_2\le3\}\) has prediction 40. The other leaf in that branch has \(x_2>3\), giving \(R_2=\{x_1\le10,\ x_2>3\}\) and prediction 65.
For \(x_1>10\), the first leaf has \(x_2\le6\), giving \(R_3=\{x_1>10,\ x_2\le6\}\) and prediction 90. The final leaf has \(x_2>6\), giving \(R_4=\{x_1>10,\ x_2>6\}\) and prediction 150.
For \(A=(8,5)\), the first coordinate satisfies \(8\le10\), and the second satisfies \(5>3\). Thus \(A\in R_2\) and its prediction is 65. For \(B=(12,5)\), \(12>10\) and \(5\le6\), so \(B\in R_3\) and its prediction is 90. For \(C=(12,7)\), \(12>10\) and \(7>6\), so \(C\in R_4\) and its prediction is 150.
The threshold for component age depends on the machine-hours branch: the age boundary is 3 when \(x_1\le10\), but 6 when \(x_1>10\). This is a conditional effect represented by successive splits. The tree does not create a smooth slope; it assigns flat predictions within each of the four regions. A common mistake is to claim there is one global age threshold. There are two different thresholds because the second split is conditional on the first branch.
Reading and Checking a Tree’s Geometry
To interpret a tree reliably, reconstruct one terminal-node region at a time. Begin with the root condition, then append only the conditions on the chosen branch. Keep the units and inequality signs attached to each predictor. After writing the region, test a point by evaluating its conditions in path order. This prevents a later split from being applied on a branch the point never reaches.
Write both branches as complementary inequalities, including which branch receives equality at the threshold.
For a particular leaf, add only the split condition encountered at each node along that path.
If the path splits a predictor more than once, intersect its conditions to obtain the final interval for that coordinate.
Label the resulting region with that leaf’s fitted prediction, class, or probability as appropriate.
Confirm that branch conditions partition the parent region and that a test point satisfies exactly one terminal-node path.
Worked Example: Combining Repeated Bounds in Three Predictors
A hypothetical equipment-screening tree uses vibration \(x_1\), operating temperature \(x_2\), and load \(x_3\). A path to one leaf requires \(x_1>2\), then \(x_2\le70\), then \(x_1\le9\), then \(x_3>4\). That leaf predicts a risk score of 0.32. Write its region and determine whether machines \(D=(5,68,6)\), \(E=(10,68,6)\), and \(F=(5,72,6)\) enter it.
Method. Intersect all conditions on the path. The two conditions on \(x_1\) combine to \(2<x_1\le9\). The complete region is \(R=\{2<x_1\le9,\ x_2\le70,\ x_3>4\}\), a rectangular box, with prediction 0.32.
For \(D=(5,68,6)\), \(2<5\le9\), \(68\le70\), and \(6>4\). All conditions hold, so \(D\) enters the leaf and receives 0.32.
For \(E=(10,68,6)\), the temperature and load conditions hold, but \(10\le9\) is false. Therefore \(E\) does not enter the leaf. For \(F=(5,72,6)\), the vibration and load conditions hold, but \(72\le70\) is false. Thus \(F\) also does not enter the leaf.
Check. The second vibration condition excludes values above 9 even though they met the first condition \(x_1>2\). A common mistake is to retain only the last bound on a predictor or to overlook a bound earlier on the path. Intersecting all conditions gives the correct interval.
What the Geometry Does—and Does Not—Say
An axis-aligned tree offers interpretable rectangular rules, but its geometry imposes limitations. A boundary such as \(x_1+x_2=10\), which is diagonal in two dimensions, cannot be represented by a single split on one predictor. A tree may approximate such a boundary using several successive vertical and horizontal splits, creating a staircase pattern. More splits can make the approximation more detailed, but each added boundary also makes the rule structure more involved.
Likewise, stepwise predictions do not imply that the underlying outcome itself changes abruptly at a threshold. The jump is a property of the fitted model’s partition. Two observations very close to opposite sides of a split can receive different predictions, while two observations far apart within the same region receive the same prediction. Interpret predictions as outputs of the fitted tree, not as proof of a sharp real-world discontinuity.
A useful final check is to separate three questions: what conditions define the region, which observations fall inside it, and what value the tree assigns there. Geometry answers the first two; the terminal-node estimate answers the third. Keeping them distinct makes tree predictions easier to communicate and reduces boundary and path errors.
Check Your Understanding
For each problem, write the relevant path conditions before deciding the region or prediction.
- A tree splits \(x_1\le4\) first. The left branch splits at \(x_2\le2\), with predictions 12 and 18. State the region and prediction for the left-branch leaf where \(x_2>2\). (Answer: \(\{x_1\le4,\ x_2>2\}\), prediction 18.)
- A tree’s path requires \(x_1>3\), then \(x_1\le8\), and \(x_2>5\). Does \((7,5)\) enter the leaf? State the combined region. (Answer: No; the region is \(\{3<x_1\le8,\ x_2>5\}\), and \(x_2=5\) fails the strict inequality.)
- A tree splits at \(x_1\le6\). Its left branch splits at \(x_2\le4\), with predictions 20 and 30. Its right branch splits at \(x_2\le9\), with predictions 50 and 80. What is the prediction at \((6,7)\), and why does the threshold 9 not apply? (Answer: 30; \(x_1=6\) follows the left branch, where the applicable \(x_2\) threshold is 4.)
- A leaf path requires \(x_1>1\), \(x_2\le12\), \(x_1\le5\), and \(x_3>0\), and its prediction is 0.45. Write its region and determine whether \((4,10,0)\) receives this prediction. (Answer: \(\{1<x_1\le5,\ x_2\le12,\ x_3>0\}\); no, because \(x_3=0\) does not satisfy \(x_3>0\).)
- In two predictors, a fitted tree uses successive splits only on one predictor at a time. Can one split create the diagonal boundary \(x_1+x_2=8\)? What geometric form can successive splits use to approximate it? (Answer: No; a split is axis-aligned and changes one predictor at a time. Successive splits can approximate the diagonal with a staircase of rectangular regions.)