Why Predictor Scale and Split Choice Are Different
A regression tree splits observations according to whether a numeric predictor falls on one side or the other of a threshold. The threshold is written in the predictor’s units, but the split itself is determined by which observations go to each child node. This distinction explains why changing a predictor’s scale can change the threshold label without changing the available splits or the split scores.
The result in this tutorial concerns a strictly increasing transformation of one numeric predictor. It does not say that every transformation leaves a tree unchanged. In particular, a transformation that changes the order of observations can change the candidate partitions. The response values, observations in the node, and split criterion are held fixed throughout.
For an unweighted regression tree using squared-error loss, the error of a node \(A\) is the sum of squared deviations of its responses from their mean. A candidate split’s score is the sum of the two child-node errors. The split selected at that node is a candidate partition with the smallest score, subject to any stated admissibility rules, such as a minimum number of observations in each child.
Here \(A_L\) and \(A_R\) are the observations assigned to the left and right child by the candidate split. The score depends on the responses and on which observations are in each child; it does not depend directly on the numerical value or units of the predictor threshold.
Ordering, Candidate Partitions, and Thresholds
Suppose the distinct observed values of a predictor, in increasing order, are \(x_{(1)} < x_{(2)} < \cdots < x_{(r)}\). A threshold between \(x_{(j)}\) and \(x_{(j+1)}\) separates the first \(j\) ordered values from the remaining values. If the usual midpoint convention is used, the threshold is \(\{x_{(j)}+x_{(j+1)}\}/2\). The midpoint is a convenient label for the partition, not a different kind of split.
Apply a strictly increasing transformation \(z=g(x)\). The observed values retain their order: \(g(x_{(1)}) < g(x_{(2)}) < \cdots < g(x_{(r)})\). Thus there is a corresponding gap between \(g(x_{(j)})\) and \(g(x_{(j+1)})\), and a threshold in that gap separates exactly the same observations. If midpoint thresholds are used on both scales, the new threshold is the midpoint of the transformed values. It need not equal the transformed old midpoint.
This statement assumes that candidate splits are generated from gaps between distinct observed predictor values and that any split constraints depend on the observations assigned to the children. It applies directly to standard unweighted squared-error splits. It does not automatically cover procedures whose rules depend on threshold magnitudes, predictor-dependent observation weights, or constraints expressed in the original units.
Why the Result Holds
The argument has two parts. First, strict increase preserves order. Second, a split score based on child membership has no way to distinguish two threshold labels that produce the same children.
If \(x_i < x_k\), strict increase gives \(g(x_i) < g(x_k)\). Equal observed values also remain equal because a function assigns the same output to the same input.
A gap between adjacent distinct values \(x_{(j)}\) and \(x_{(j+1)}\) corresponds to a gap between \(g(x_{(j)})\) and \(g(x_{(j+1)})\). A threshold in the transformed gap sends the first \(j\) ordered values to one child and all later values to the other.
The two children contain the same observations as before. Their response means and sums of squared deviations are therefore identical, so the candidate split score is identical.
The same reasoning applies recursively within a node: order is preserved among the observations in that node, so its candidate partitions correspond as well. With the same rules for choosing among tied best partitions, the sequence of partitions can therefore be unchanged. This conclusion concerns the partitions and predictions attached to observations, not the numerical cutoffs printed in a tree diagram.
Worked Example: An Affine Rescaling
A hypothetical claims model has four observations with predictor values \(x=10,20,30,40\) and responses \(y=2,2,8,8\). The tree uses unweighted squared-error loss and considers thresholds midway between adjacent distinct predictor values. Compare the candidate scores with those obtained after replacing \(x\) by \(z=2x+5\).
Method. Order the predictor values, list each partition, and compute the child errors from the responses. The response values do not change.
On the original scale, the candidate thresholds are \(15,25,35\), giving partitions after observations 1, 2, and 3 in predictor order. The parent response mean is \((2+2+8+8)/4=5\), and its error is \( (2-5)^2+(2-5)^2+(8-5)^2+(8-5)^2=9+9+9+9=36\).
For the split after observation 1, the left child contains \(2\), so its error is \(0\). The right child responses are \(2,8,8\), with mean \(18/3=6\), and error \( (2-6)^2+(8-6)^2+(8-6)^2=16+4+4=24\). The total split score is \(0+24=24\).
For the split after observation 2, the children contain \(2,2\) and \(8,8\). Each child is constant, so both errors are \(0\), and the total score is \(0\).
For the split after observation 3, the left responses \(2,2,8\) have mean \(12/3=4\), with error \(4+4+16=24\). The right child contains \(8\), with error \(0\). The total score is \(24\).
The transformed predictor values are \(25,45,65,85\), in the same order. Their midpoint thresholds are \(35,55,75\), producing exactly the same three partitions. The scores remain \(24,0,24\), so the unique best partition is still the split after observation 2.
Check. The transformed midpoint thresholds are \(2(15)+5=35\), \(2(25)+5=55\), and \(2(35)+5=75\), since this particular transformation is affine. The response values and child membership are unchanged, and the best score remains \(0\). A common mistake is to attribute the unchanged scores to the numerical thresholds themselves; the reason is the unchanged partition.
Nonlinear Transformations Can Change Threshold Labels
For a nonlinear increasing transformation, even the midpoint thresholds do not generally transform into one another. The candidate partitions still correspond because order is preserved, but the midpoint rule is applied separately on each scale. The next example shows both facts.
Worked Example: A Square-Root Transformation
A hypothetical risk study has predictor values \(x=1,4,9,16\) and responses \(y=0,4,8,8\). Compare the candidate partitions and squared-error scores when the predictor is replaced by \(z=\sqrt{x}\). Use midpoint thresholds on each scale.
Method. Compute the score for each gap in sorted predictor order, then match the gaps after transformation. Since the square-root function is strictly increasing for positive \(x\), the order should be preserved.
The original midpoint thresholds are \((1+4)/2=2.5\), \((4+9)/2=6.5\), and \((9+16)/2=12.5\). The parent response mean is \((0+4+8+8)/4=5\), with error \(25+1+9+9=44\).
For the split after observation 1, the right-child responses \(4,8,8\) have mean \(20/3\). Their error is \( (4-20/3)^2+(8-20/3)^2+(8-20/3)^2 =64/9+16/9+16/9=96/9=32/3\). The total is \(0+32/3=32/3\).
For the split after observation 2, the left-child responses \(0,4\) have mean \(2\) and error \(4+4=8\). The right-child responses \(8,8\) have error \(0\). The total is \(8\).
For the split after observation 3, the left responses \(0,4,8\) have mean \(12/3=4\), with error \(16+0+16=32\). The right child contains \(8\), with error \(0\). The total is \(32\). Thus the split after observation 2 has the smallest score, \(8\).
After transformation, the predictor values are \(1,2,3,4\), whose midpoint thresholds are \(1.5,2.5,3.5\). They again give the partitions after observations 1, 2, and 3, respectively. The scores are still \(32/3,8,32\), and the best partition is still after observation 2.
Check. The original threshold \(6.5\) maps to \(\sqrt{6.5}\), approximately \(2.5495\), not to the transformed-scale midpoint \(2.5\). Both values lie between transformed observations \(2\) and \(3\), so both represent the same partition. The two threshold labels differ, but the chosen split and its score do not.
Tied Predictor Values and Candidate Splits
If several observations share a predictor value, a threshold cannot separate those tied observations from one another. Candidate gaps are considered only between distinct values. Strictly increasing transformations preserve equalities as well as order, so the tied groups remain intact and the same group boundaries are available.
Worked Example: Ties Survive a Strictly Increasing Transformation
A hypothetical portfolio has five observations with predictor values \(x=2,2,5,8,8\) and responses \(y=1,3,4,10,12\). Compare the available partitions and scores after transforming the predictor to \(z=x^2\). The transformation is strictly increasing on the observed positive values.
Method. Treat each distinct predictor value as an indivisible group, compute the squared-error score for each gap, and check whether the transformed values retain those groups in order.
The original candidate thresholds are midway between distinct values: \((2+5)/2=3.5\) and \((5+8)/2=6.5\). There is no candidate split between the two observations having value \(2\), or between the two observations having value \(8\).
For the split after the group with value \(2\), the left responses \(1,3\) have mean \(2\), with error \(1+1=2\). The right responses \(4,10,12\) have mean \(26/3\), with error \( (4-26/3)^2+(10-26/3)^2+(12-26/3)^2 =196/9+16/9+100/9=312/9=104/3\). The total score is \(2+104/3=110/3\).
For the split after the group with value \(5\), the left responses \(1,3,4\) have mean \(8/3\), with error \( (1-8/3)^2+(3-8/3)^2+(4-8/3)^2 =25/9+1/9+16/9=42/9=14/3\). The right responses \(10,12\) have mean \(11\), with error \(1+1=2\). The total is \(14/3+2=20/3\), so this is the better partition.
The transformed values are \(4,4,25,64,64\). Their midpoint thresholds are \(14.5\) and \(44.5\), so the same two group boundaries, and no additional split within a tie, are available. The scores remain \(110/3\) and \(20/3\).
Check. The best partition still separates the observations with values \(2,2,5\) from those with values \(8,8\). The transformed midpoint \(44.5\) changes the threshold label, while the response-based score and child membership remain unchanged.
Common Mistake: Treating a Threshold as the Split
A tree diagram displays a numerical rule such as \(x \leq 6.5\). It is tempting to think that preserving a split requires preserving that exact number after rescaling. It does not. What must be preserved is which observations satisfy the rule. A strictly increasing transformation maps values on either side of an actual cutoff to values on the corresponding sides of its transformed cutoff; reapplying a midpoint convention may then display a different cutoff within the same gap.
For the square-root example, the incorrect calculation is to say that the original midpoint \(6.5\) becomes the new midpoint \(2.5\), because \(\sqrt{6.5}\ne 2.5\). The correct calculation is \(\sqrt{6.5}\approx2.5495\). If the transformed-scale procedure instead chooses the midpoint of \(2\) and \(3\), it reports \(2.5\). Both \(2.5495\) and \(2.5\) lie strictly between those same adjacent observations, so both assign the observations identically. The numerical labels differ; the candidate partition does not.
Likewise, do not infer that a transformation leaves every aspect of a fitted tree unchanged. This result concerns predictor ordering and partition-based splitting with the response held fixed. A transformation that is not strictly increasing on the observed domain can reorder values and alter the candidate partitions. For example, squaring values that include both negative and positive numbers need not preserve their order. Changes to the response scale are a different question because they can change the split scores themselves.
Check Your Understanding
For each problem, focus on the distinction between the predictor’s threshold and the observations assigned to each child.
- A node has predictor values \(3,6,10,15\). Under \(z=x+4\), list the original and transformed midpoint thresholds and state the observation partitions they represent. (Answer: Original thresholds \(4.5,8,12.5\); transformed thresholds \(8.5,12,16.5\); both scales give splits after observations 1, 2, and 3 in order.)
- A predictor takes values \(-2,0,5,11\), and a proposed transformation is \(g(x)=x^2\). Is the transformation strictly increasing over these observed values, and does the theorem guarantee the same ordered candidate partitions? (Answer: No; the transformed values are \(4,0,25,121\), so the order changes and the theorem does not guarantee the same candidate partitions.)
- A node has \(x=1,2,5,10\) and responses \(y=2,2,6,10\). Under \(z=\sqrt{x}\), identify the best squared-error partition and its score. (Answer: The split after observation 2 is best; its child errors are \(0\) and \(8\), for a total score of \(8\), and the same partition is best after transformation.)
- A predictor has values \(1,1,3,7\), and \(z=3^x\). How many candidate partitions are available, and what are the transformed-scale midpoint thresholds? (Answer: Two candidate partitions, after the first two observations and after the first three observations; transformed values are \(3,3,27,2187\), giving midpoint thresholds \(15\) and \(1107\).)
- A node has ordered predictor values \(2,4,6\) and responses \(0,1,0\). Each of the two possible splits has the same squared-error score. If the predictor is transformed by \(z=x^3\), what happens to the tie between the partitions? (Answer: The two corresponding partitions still tie: each has score \(1/2\), because each split leaves one child with responses \(0,1\) and the other with \(0\); the transformed predictor values \(8,64,216\) preserve their order.)