From Split Rules to Regions
A regression tree divides predictor space by applying threshold rules one at a time. Each rule sends an observation to one of two branches, such as \(X_1 \leq 4\) or \(X_1 > 4\). After several splits, an observation reaches a terminal region only if it satisfies every rule along that path. Translating those rules into a region is a matter of collecting the constraints and keeping their boundaries straight.
A split on one predictor draws a line in two dimensions, a plane in three dimensions, or the corresponding coordinate-aligned boundary in higher dimensions. Because the split is perpendicular to one predictor axis, the resulting regions have sides parallel to the coordinate axes. A path through the tree combines such constraints, producing an axis-aligned rectangle in two dimensions or, more generally, a rectangular region in predictor space.
The branch convention matters: equality belongs to the \( \leq \) branch, not the \(>\) branch. Once each split has a specified convention, path regions do not overlap. Any predictor value follows exactly one branch at each split and therefore belongs to exactly one terminal path, unless the path constraints themselves are contradictory and define an empty region.
Why a Path Makes a Rectangle
Each rule on a path restricts just one coordinate. For example, a path might require \(X_1 \leq 8\), \(X_2 > 3\), and \(X_1 > 2\). The two conditions on \(X_1\) combine to give \(2 < X_1 \leq 8\), while the condition on \(X_2\) gives \(X_2 > 3\). There is no restriction on any other predictor. These independent coordinate intervals form a rectangular region.
Here \(I_j\) is the interval left after combining all constraints on predictor \(X_j\) along that path. A lower bound may be open when it comes from a \(>\) rule; an upper bound may be closed when it comes from a \( \leq \) rule. If a path requires both \(X_1 \leq 2\) and \(X_1 > 5\), no value can satisfy it, so the region is empty.
This result follows directly from how the rules are formed. A single split retains either one side of a coordinate threshold, without restricting the other coordinates. Along a path, the observation must satisfy all the retained conditions. Grouping those conditions by coordinate gives one interval for each predictor, and satisfying all those intervals is exactly membership in their Cartesian product. This is why a sequence of threshold splits creates axis-aligned rectangular regions rather than diagonal or curved regions.
This description specifies the region geometrically; it does not identify which training observations fall in it. To assign observations, test each observation against the path conditions, or equivalently check whether each predictor value lies in its interval. Both methods should give the same membership list.
Worked Region Assignments
Worked Example: Combining Branches in Two Predictors
A tree first splits at \(X_1=4\). The \(X_1 \leq 4\) branch is a terminal region. The other branch splits at \(X_2=10\). Its \(X_2 \leq 10\) branch is terminal; its \(X_2>10\) branch splits again at \(X_1=7\), with both resulting branches terminal. Assign the observations below to terminal regions.
| Observation | \(X_1\) | \(X_2\) |
|---|---|---|
| A | 2 | 8 |
| B | 4 | 12 |
| C | 5 | 9 |
| D | 7 | 11 |
| E | 8 | 8 |
| F | 6 | 12 |
Method. Write each terminal path as a set of conditions, then check the observations against those conditions. The first region is \(X_1 \leq 4\). It contains A, since \(2 \leq 4\), and B, since \(4 \leq 4\). The equality for B belongs on this branch.
The second region requires \(X_1>4\) and \(X_2 \leq 10\). C satisfies both because \(5>4\) and \(9 \leq 10\). E also satisfies both because \(8>4\) and \(8 \leq 10\).
The third region requires \(X_1>4\), \(X_2>10\), and \(X_1 \leq 7\). D satisfies all three: \(7>4\), \(11>10\), and \(7 \leq 7\). F also satisfies all three: \(6>4\), \(12>10\), and \(6 \leq 7\).
The fourth region requires \(X_1>7\) and \(X_2>10\). No observation satisfies both: E has \(X_1>7\) but \(X_2 \leq 10\), while D and F have \(X_2>10\) but \(X_1 \leq 7\). The complete assignments are therefore: first region A, B; second region C, E; third region D, F; fourth region empty. As a check, every one of the six observations appears exactly once.
Common mistake avoided: Treating \(X_1=4\) as belonging to the \(X_1>4\) branch would misassign B. The stated \( \leq \) rule includes equality, so B enters the first terminal region and does not proceed to the later splits.
Worked Example: Translating a Three-Predictor Tree
A tree splits first at \(X_3=2.5\). In the \(X_3 \leq 2.5\) branch, it splits at \(X_2=0.4\). In the \(X_3>2.5\) branch, it splits at \(X_1=100\). Each of the four resulting branches is terminal. Determine the region and observations for each terminal branch.
| Observation | \(X_1\) | \(X_2\) | \(X_3\) |
|---|---|---|---|
| P | 90 | 0.2 | 2.0 |
| Q | 120 | 0.7 | 2.4 |
| R | 80 | 0.6 | 3.1 |
| S | 110 | 0.3 | 2.8 |
| T | 100 | 0.4 | 2.5 |
| U | 100 | 0.4 | 2.6 |
Step 1: Write the four paths. The first requires \(X_3 \leq 2.5\) and \(X_2 \leq 0.4\). The second requires \(X_3 \leq 2.5\) and \(X_2>0.4\). The third requires \(X_3>2.5\) and \(X_1 \leq 100\). The fourth requires \(X_3>2.5\) and \(X_1>100\).
Step 2: Express the regions. In the coordinate order \(X_1,X_2,X_3\), the first region is \((-\infty,\infty)\times(-\infty,0.4]\times(-\infty,2.5]\). The second is \((-\infty,\infty)\times(0.4,\infty)\times(-\infty,2.5]\). The third is \((-\infty,100]\times(-\infty,\infty)\times(2.5,\infty)\). The fourth is \((100,\infty)\times(-\infty,\infty)\times(2.5,\infty)\).
Step 3: Assign observations. P has \(X_3=2.0 \leq 2.5\) and \(X_2=0.2 \leq 0.4\), so it enters the first region. Q has \(X_3=2.4 \leq 2.5\) and \(X_2=0.7>0.4\), so it enters the second. T has \(X_3=2.5\), which is included in the \( \leq \) branch, and \(X_2=0.4\), also included in the \( \leq \) branch; it enters the first region.
R has \(X_3=3.1>2.5\) and \(X_1=80 \leq 100\), so it enters the third region. U has \(X_3=2.6>2.5\) and \(X_1=100 \leq 100\), so it also enters the third. S has \(X_3=2.8>2.5\) and \(X_1=110>100\), so it enters the fourth. Thus the membership lists are first: P, T; second: Q; third: R, U; fourth: S. Each observation appears once.
Common mistake avoided: A path that splits on \(X_2\) leaves \(X_1\) unrestricted, and a path that splits on \(X_1\) leaves \(X_2\) unrestricted. The unconstrained intervals are not omitted; they are \((-\infty,\infty)\).
Worked Example: Intersecting Repeated Bounds on One Predictor
A tree first splits at \(X_1=6\). In the \(X_1 \leq 6\) branch, it splits again at \(X_1=2\). In the \(X_1>6\) branch, it splits at \(X_2=5\). Give the four terminal regions and assign the observations.
| Observation | \(X_1\) | \(X_2\) |
|---|---|---|
| J | 1 | 8 |
| K | 2 | 3 |
| L | 4 | 7 |
| M | 6 | 5 |
| N | 7 | 4 |
| O | 8 | 6 |
The first path requires \(X_1 \leq 6\) and \(X_1 \leq 2\). Both conditions together reduce to \(X_1 \leq 2\), giving \((-\infty,2]\times(-\infty,\infty)\). It contains J and K; K is included because \(X_1=2\).
The second path requires \(X_1 \leq 6\) and \(X_1>2\), so its region is \((2,6]\times(-\infty,\infty)\). It contains L and M, including M because \(X_1=6\) satisfies the root rule.
The third path requires \(X_1>6\) and \(X_2 \leq 5\), giving \((6,\infty)\times(-\infty,5]\). It contains N. The fourth requires \(X_1>6\) and \(X_2>5\), giving \((6,\infty)\times(5,\infty)\); it contains O. The resulting groups are J, K; L, M; N; and O. These disjoint intervals also check that every observation has been assigned once.
Common mistake avoided: The second region is not simply \(X_1 \leq 6\). It must also satisfy the path condition \(X_1>2\), so its lower boundary is open at 2.
Reading Boundaries and Checking the Partition
The most reliable way to translate a tree is to trace one complete path at a time, recording both the predictor and the branch condition at every split. Then combine constraints only when they refer to the same predictor. A small table can make this process systematic:
Record every split encountered from the root to a terminal branch, including whether the path takes the \( \leq \) side or the \(>\) side.
Place all conditions on \(X_j\) together. Leave predictors with no conditions unrestricted.
Use the strongest lower and upper bounds, retaining open or closed endpoints as required by the split rules.
Test its values against all the path conditions and confirm that it is assigned to one terminal region only.
A useful final check is to compare membership across all terminal paths. At each split, the two branches divide the incoming set into \(X_j \leq c\) and \(X_j>c\). These sets are disjoint and their union is the incoming set. Applying that fact successively means the terminal regions of a complete tree are disjoint and cover the predictor space. Some terminal regions may contain no training observations, even though they contain possible predictor values; an empty observed membership list is not by itself a contradiction.
A frequent exam trap is to follow the right split threshold but forget an earlier condition. For example, a terminal path \(X_1 \leq 6\), then \(X_1>2\), then \(X_2 \leq 5\) cannot be described by \(X_1 \leq 6\) and \(X_2 \leq 5\) alone. The missing \(X_1>2\) changes the region and may incorrectly add observations with \(X_1 \leq 2\). Another common error is to put a threshold equality on both sides. Under the stated convention, equality is in the \( \leq \) branch only.
The geometry clarifies what a tree's rules do: they partition predictor space into coordinate-aligned regions, and the observations in any one region are precisely those meeting every condition on that path. This representation is aligned with the SOA Exam SRM (Statistics for Risk Modeling) syllabus, where careful translation of sequential rules into regions is a practical step in interpreting a tree.
Check Your Understanding
For each problem, write the path region and identify the observations satisfying it. Use the convention that equality belongs to the \( \leq \) branch.
- A terminal path requires \(X_1 \leq 9\), then \(X_2>1\). Which of \((8,1)\), \((9,2)\), and \((10,3)\) belong? (Answer: \((9,2)\) only; the region is \((-\infty,9]\times(1,\infty)\).)
- A path requires \(X_1>3\), then \(X_1 \leq 8\), and \(X_2 \leq 4\). State its region. (Answer: \((3,8]\times(-\infty,4]\).)
- A root split is \(X_2 \leq 0\) versus \(X_2>0\). The left branch splits at \(X_1=5\), and the path of interest takes \(X_1>5\). Give the path region and determine whether \((6,0)\) belongs. (Answer: \((5,\infty)\times(-\infty,0]\); yes, \((6,0)\) belongs.)
- A path requires \(X_1 \leq 2\), followed by \(X_1>4\). Is its region empty or nonempty? (Answer: Empty, because no \(X_1\) can satisfy both inequalities.)
- A tree first splits at \(X_3=1.5\). Its \(X_3>1.5\) branch then splits at \(X_2=7\), and the path takes \(X_2 \leq 7\). Which of A \((4,7,2)\), B \((3,8,2)\), and C \((9,1,1.5)\) enter this terminal region? (Answer: A only; the region is \((-\infty,\infty)\times(-\infty,7]\times(1.5,\infty)\).)