Common Error: Choosing a Split by Child Error Alone

Why the Parent Error Matters

A regression tree predicts a constant value within each node, usually the mean response of the observations in that node. A proposed split replaces one parent node with two child nodes. The split is useful when those two predictions fit the responses better than the single parent prediction. The key comparison is therefore not just “How much error do the children have?” but “How much error did the split remove from the parent?”

For squared-error trees, the two questions produce closely related scores. Within one fixed parent node, the total child error and the reduction from the parent error rank candidate splits in exactly the same order. The common error arises when that fact is extended beyond its conditions—for example, when raw child totals from different parent nodes are compared as if they had the same baseline.

Definition: For a node containing $n$ observations with responses $y_1,\ldots,y_n$, its squared-error total is $$ E_{\mathrm{node}}=\sum_{i=1}^{n}(y_i-\bar{y})^2, \qquad \bar{y}=\frac{1}{n}\sum_{i=1}^{n}y_i. $$ If a split creates left and right children, its post-split error is $E_L+E_R$, where each child error is calculated around that child’s own response mean. The split’s error reduction, or improvement, is $$ \Delta E=E_{\mathrm{parent}}-(E_L+E_R). $$ These scores apply to squared-error regression trees using the same observations and the same error definition for the parent and children.

Child Error and Improvement Within One Parent

Suppose several candidate thresholds are being considered for the same parent node. Its error $E_{\mathrm{parent}}$ is fixed. For any candidate split $s$, $$ \Delta E_s=E_{\mathrm{parent}}-E_{\mathrm{children},s}. $$ Consequently, if candidate A has a smaller child-error total than candidate B, then candidate A has a larger reduction by exactly the same amount. For this particular comparison, minimizing the child total and maximizing the reduction are equivalent. The parent error is still important: it tells us how much was gained, and it is essential when scores from different parent nodes are compared.

Worked Example: Comparing Thresholds Within One Node

A node contains four observations with predictor values $x=1,2,3,4$ and responses $y=0,8,2,10$, respectively. Compare the squared-error scores for thresholds after $x=1$, after $x=2$, and after $x=3$. Which threshold gives the greatest improvement?

Method. Calculate the parent error once, then calculate the two child means and child errors for each threshold. The lowest child-error total will have the greatest reduction because all three splits share this parent.

Parent error. The parent response mean is $(0+8+2+10)/4=5$. Its squared-error total is $ (0-5)^2+(8-5)^2+(2-5)^2+(10-5)^2=25+9+9+25=68 $.

Threshold after $x=1$. The left child contains $0$, so its error is $0$. The right child has responses $8,2,10$, mean $20/3$, and error $ (8-20/3)^2+(2-20/3)^2+(10-20/3)^2=(16+196+100)/9=104/3 $. The child total is $104/3$, and the reduction is $68-104/3=100/3$.

Threshold after $x=2$. The left child has responses $0,8$, mean $4$, and error $16+16=32$. The right child has responses $2,10$, mean $6$, and error $16+16=32$. Thus the child total is $64$, and the reduction is $68-64=4$.

Threshold after $x=3$. The left child has responses $0,8,2$, mean $10/3$, and error $ (0-10/3)^2+(8-10/3)^2+(2-10/3)^2=(100+196+16)/9=104/3 $. The right child contains $10$, so its error is $0$. The child total is $104/3$, and the reduction is $100/3$.

The lowest child total is $104/3$, achieved by the thresholds after $x=1$ and $x=3$; each gives the greatest reduction, $100/3$. The threshold after $x=2$ has the largest child total, $64$, and the smallest reduction, $4$. The ranking agrees because the parent error is the same in every comparison. Common mistake avoided: treating a smaller child total as a smaller improvement. The reduction is parent error minus child total, so a smaller child total means a larger improvement.

Why the Reduction Formula Works

The relationship between parent error and child errors can be justified by decomposing the parent’s deviations into within-child and between-child parts. Let the left and right children have sizes $n_L,n_R$, means $\bar{y}_L,\bar{y}_R$, and combined parent mean $\bar{y}$. For an observation in the left child, $y_i-\bar{y}=(y_i-\bar{y}_L)+(\bar{y}_L-\bar{y})$. When the squared terms are summed within the left child, the cross-product vanishes because $\sum_{i\in L}(y_i-\bar{y}_L)=0$. The same reasoning applies to the right child.

Combining the two children gives the parent-error decomposition:

E_{\mathrm{parent}} = E_L+E_R+ n_L(\bar{y}_L-\bar{y})^2+ n_R(\bar{y}_R-\bar{y})^2 = E_L+E_R+ \frac{n_Ln_R}{n_L+n_R}(\bar{y}_L-\bar{y}_R)^2.

The final equality follows because $\bar{y}=(n_L\bar{y}_L+n_R\bar{y}_R)/(n_L+n_R)$. Thus the improvement is $\frac{n_Ln_R}{n_L+n_R}(\bar{y}_L-\bar{y}_R)^2$: it is large when the child means are far apart, with the sizes of both children also affecting the score. This identity also shows why, for a fixed parent, child error plus reduction always equals the same parent error.

When Raw Child Totals Mislead

The equivalence in the threshold example depends on comparing splits of the same parent. When two candidate splits come from different parent nodes, their baselines can differ. A raw child total of $10$ may represent a substantial improvement from a parent error of $100$, but little improvement from a parent error of $12$. If choosing which current leaf to split next, compare improvements (or another explicitly specified criterion), not child totals alone.

Worked Example: A Smaller Child Total but a Smaller Improvement

A tree has two eligible parent nodes, A and B. A proposed split at A has parent error $68$ and child errors $2$ and $2$. A proposed split at B has parent error $26$ and child errors $0.5$ and $0.5$. If the next split is selected by largest squared-error reduction, which proposal should be chosen?

Method. Add the two child errors for each proposal, then subtract each total from its own parent error. The parent baselines are not interchangeable.

For A, the child total is $2+2=4$, so the reduction is $68-4=64$.

For B, the child total is $0.5+0.5=1$, so the reduction is $26-1=25$.

B has the smaller raw child total, $1$ rather than $4$, but A removes more error: $64$ rather than $25$. Choose A under the stated largest-reduction rule. Common mistake avoided: selecting B just because its child total is smaller, without accounting for the fact that its parent already had much less error. This comparison is across different parent nodes; it does not contradict the same-parent equivalence.

Worked Example: Checking Improvement With Child Means

A hypothetical claim-severity node has four responses, in thousands of dollars: $0,0,0,6$. A proposed predictor threshold places the three zero responses in the left child and the response $6$ in the right child. Calculate the parent error, the child total, and the improvement. Verify the improvement using the between-child-means identity.

Parent calculation. The parent mean is $(0+0+0+6)/4=1.5$. The parent error is $3(0-1.5)^2+(6-1.5)^2=3(2.25)+20.25=27$, in squared thousands of dollars.

Child calculation. Each child is constant, so each child error is $0$. The child total is $0+0=0$, and the improvement is $27-0=27$.

Identity check. Here $n_L=3$, $n_R=1$, the left mean is $0$, and the right mean is $6$. Therefore the between-child term is $(3\cdot1)/(3+1)\times(0-6)^2=(3/4)\times36=27$. It matches the direct reduction. The factor involving the child sizes matters: using only the squared difference between means would give $36$, not the correct improvement of $27$. Common mistake avoided: ignoring the size factor when checking a reduction through the difference in child means.

A Reliable Split-Scoring Process

Identify the comparison.
Check whether all candidate splits belong to one parent or to different parents. This determines whether their parent errors are shared.

Calculate each child error.
For squared-error trees, calculate each child’s response mean and sum squared deviations from that mean. Add the two child totals.

Calculate the parent-relative reduction.
Subtract the child total from the error of the parent that produced those children.

Rank using the right comparison.
Within one parent, the smallest child total and largest reduction give the same ranking. Across parents, compare reductions rather than raw child totals when the stated rule is largest improvement.

A split score should always be interpreted together with its baseline and its selection rule. This matters when choosing which leaf to split, comparing candidate thresholds, or checking a reported reduction. It also prevents a subtle but important overstatement: the smallest child-error total is not inherently a bad score. Within a fixed parent it is exactly the score that maximizes squared-error reduction; the mistake is using it without a common parent baseline.

Key takeaway: For candidate splits of the same parent, minimizing $E_L+E_R$ is exactly equivalent to maximizing $E_{\mathrm{parent}}-(E_L+E_R)$. For splits from different parents, each reduction must use its own parent error; the smallest raw child total need not give the largest improvement.

Check Your Understanding

For each question, distinguish the post-split child total from the improvement relative to the correct parent.

A node has responses $2,2,8,8$. A split puts the two 2s in one child and the two 8s in the other. Calculate the parent error, child total, and reduction. (Answer: Parent error $36$; child total $0$; reduction $36$.)
A node has responses $1,3,5,7$ and is split into $\{1,3\}$ and $\{5,7\}$. Calculate the child total and reduction from the parent. (Answer: Child total $4$; parent error $20$; reduction $16$.)
Proposal A has parent error $68$ and child total $4$. Proposal B, at a different parent, has parent error $26$ and child total $1$. Which has the larger reduction, and by how much? (Answer: A; its reduction is $64$, compared with $25$ for B, so it is larger by $39$.)
Two splits of the same parent have child totals $12$ and $17$. The parent error is $30$. State both reductions and identify the better split under a largest-reduction rule. (Answer: Reductions $18$ and $13$; choose the split with child total $12$.)
A split has child sizes $3$ and $2$, child means $4$ and $10$, and child errors $6$ and $8$. Use the between-child-means term to find the reduction, then find the parent error. (Answer: Reduction $(3\cdot2/5)(10-4)^2=43.2$; parent error $6+8+43.2=57.2$.)

What You'll Learn