Residual Patterns In Regression Trees

Why Inspect Regression-Tree Residuals?

A regression tree predicts a constant response within each terminal node. The tree therefore simplifies the relationship between predictors and the response into a set of regions. Residual analysis asks what remains after that simplification: are the prediction errors scattered without an obvious pattern, or do they still vary systematically with a predictor, fitted value, or terminal node?

Residuals are diagnostic evidence, not an automatic instruction to make the tree larger. A pattern can suggest that the current partition misses structure, but it can also reflect random variation, a few unusual observations, or limited data. In particular, a pattern in training residuals should be checked against data not used to fit the tree before treating it as evidence that a more complex tree will predict better.

Definition: For observation $i$, the regression residual is $e_i=y_i-\hat{y}_i$, where $y_i$ is the observed response and $\hat{y}_i$ is the tree’s fitted prediction. A positive residual means the observed response is greater than the prediction (the tree underpredicted); a negative residual means the observed response is less than the prediction (the tree overpredicted).

A residual plot places residuals on one axis and a quantity such as fitted value or a predictor on the other. The horizontal reference line $e_i=0$ marks exact prediction. The question is whether the residuals show structure beyond a roughly patternless scatter around that line. For trees, “patternless” does not mean that predictions must vary smoothly: tree predictions are stepwise and take only a limited number of values.

What Different Residual Views Can Show

View	What to inspect	Possible signal
Residuals versus fitted values	Residual centers, spread, and unusual points at each fitted value	Systematic bias in some prediction ranges or a change in error spread
Residuals versus a predictor	A trend, curve, clusters, or changing spread as the predictor changes	The current splits may not capture a relationship within a region
Residuals by terminal node	Mean, spread, and notable errors within each leaf	Different error behavior between leaves, especially on held-out data

A residual-versus-fitted plot for a tree often forms vertical bands: all observations assigned to the same terminal node have the same fitted value. Bands by themselves are not a defect. Look within and across the bands for residuals predominantly on one side of zero, changing spread, or individual errors that warrant investigation.

A Useful Training-Data Result

For a standard regression tree that uses unweighted squared-error loss, each terminal node predicts the mean response of its training observations. This has an important diagnostic consequence: the residuals in each training node sum to zero. That consequence is a property of how the node prediction is calculated, not evidence that the node fits well.

Result: If terminal node $j$ contains $n_j$ training responses and predicts their arithmetic mean, then $\sum_{i\in j} e_i=0$. This statement applies to the training observations used to calculate the node mean under squared-error loss. It need not hold for validation observations, and it does not imply that residuals within the node lack a pattern.

The reason is that the node prediction is chosen to minimize the sum of squared residuals. Write the responses in one node as $y_1,\ldots,y_n$, and consider a candidate constant prediction $c$. The sum of squared errors is $Q(c)=\sum_{i=1}^{n}(y_i-c)^2$. Differentiating and setting the derivative to zero gives the minimizing prediction; the second derivative is positive, so this is a minimum.

\frac{dQ(c)}{dc}=-2\sum_{i=1}^{n}(y_i-c)=0 \quad\Longrightarrow\quad c=\bar{y}

Substituting $c=\bar{y}$ into the residual sum gives $\sum_{i=1}^{n}(y_i-\bar{y})=\sum_{i=1}^{n}y_i-n\bar{y}=0$, because $\bar{y}=(1/n)\sum_{i=1}^{n}y_i$. This derivation assumes the node prediction is the unweighted mean under squared-error loss. With observation weights, the corresponding result concerns the weighted residual sum when the fitted constant is the weighted mean. Do not apply the unweighted result automatically to a different loss or prediction rule.

Worked Examples

Worked Example: Reading Residuals at Two Fitted Values

A hypothetical service-cost tree has two terminal nodes. In node A, the three observed costs are $10, $12, and $14; in node B, they are $18, $20, and $22. The tree predicts the training mean in each node. Calculate the predictions and residuals, then interpret the residual-versus-fitted view.

Method. Calculate each node mean, subtract the fitted prediction from each observed cost, and compare the residuals in each fitted-value band.

For node A, the prediction is $(10+12+14)/3=36/3=\$12$. Its residuals, in the order given, are $10-12=-\$2$, $12-12=\$0$, and $14-12=\$2$. Their sum is $-2+0+2=0$, and their sum of squares is $4+0+4=8$.

For node B, the prediction is $(18+20+22)/3=60/3=\$20$. Its residuals are $18-20=-\$2$, $20-20=\$0$, and $22-20=\$2$. Their sum is also $-2+0+2=0$, and their sum of squares is $4+0+4=8$.

The residual-versus-fitted plot has a band at fitted cost $12 with residuals $-2,0,2$, and a band at fitted cost $20 with the same residuals. Each band is centered at zero, but that centering follows from using the training mean. The identical spread here gives no indication that one fitted-value band has larger errors than the other. The total training sum of squared errors is $8+8=16$.

Check. There are six observations and their residuals are $-2,0,2,-2,0,2$, which sum to zero; the total squared error is $4+0+4+4+0+4=16$. This example avoids the mistake of treating a zero mean training residual in a node as proof that the tree has captured every relationship.

Worked Example: A Predictor Trend Within One Leaf

A hypothetical tree assigns four accounts to the same terminal node. Their values of a continuous predictor $x$ are $1,2,3,4$, and their observed responses are $4,7,10,13$. The tree predicts the node mean for each account. Calculate the residuals and decide whether the residuals suggest remaining structure with $x$.

Method. Calculate the leaf mean, form $y_i-\hat{y}_i$, and examine whether the residuals change systematically as $x$ increases.

The leaf prediction is $(4+7+10+13)/4=34/4=8.5$. The residuals are $4-8.5=-4.5$, $7-8.5=-1.5$, $10-8.5=1.5$, and $13-8.5=4.5$. Their sum is $-4.5-1.5+1.5+4.5=0$, as expected for training responses fitted by their mean.

In order of increasing $x$, the residuals increase steadily from $-4.5$ to $4.5$. The first two responses are below the prediction and the last two are above it. Thus, a residual-versus-$x$ plot shows a clear increasing pattern within the leaf. The tree’s constant prediction has not represented the systematic increase in response across these $x$ values. A candidate refinement would need to be assessed carefully, including on data not used to fit the tree.

Check. The squared errors are $20.25, 2.25, 2.25,$ and $20.25$, for a total of $45$. The zero residual sum does not cancel the visible trend. This example avoids the mistake of relying on a node’s average residual while ignoring the order of residuals against a predictor.

Worked Example: Comparing Validation Residuals by Terminal Node

A hypothetical cost tree has two terminal nodes with fixed predictions of $10 and $20. Six new validation observations fall into those nodes: node A has observed costs $11, $12, and $13; node B has observed costs $17, $20, and $21. Calculate each node’s validation residual mean and RMSE, then summarize the overall validation errors. RMSE is the square root of mean squared error.

Method. Use each observation’s assigned node prediction, compute observed minus predicted cost, and summarize the validation errors without assuming their means must be zero.

In node A, the residuals are $11-10=1$, $12-10=2$, and $13-10=3$. Their mean is $(1+2+3)/3=6/3=2$ dollars. Their squared errors sum to $1+4+9=14$, so node A’s RMSE is $\sqrt{14/3}=\sqrt{4.6667}$, approximately $2.1602.

In node B, the residuals are $17-20=-3$, $20-20=0$, and $21-20=1$. Their mean is $(-3+0+1)/3=-2/3$, approximately $-\$0.6667$. Their squared errors sum to $9+0+1=10$, so node B’s RMSE is $\sqrt{10/3}=\sqrt{3.3333}$, approximately $1.8257.

Across all six observations, the residual sum is $1+2+3-3+0+1=4$, giving an overall mean residual of $4/6=\$0.6667$. The total squared error is $14+10=24$, so the overall RMSE is $\sqrt{24/6}=\sqrt{4}=\$2.00$. Node A’s positive mean residual indicates underprediction on average in this validation sample; node B’s mean residual is negative. These are sample diagnostics, not proof of persistent bias.

Check. The node squared-error totals add to the overall total, $14+10=24$, and $24/6=4$, whose square root is $2.00. This example avoids the mistake of expecting validation residuals to sum to zero merely because predictions are constant within terminal nodes.

A Practical Inspection Sequence

A useful review moves from an overall view to more focused views. It should include the response scale and the data split used to fit the tree. If the tree was fitted on training data, calculate training residuals as observed response minus fitted prediction. For a separate validation set, calculate residuals using predictions from the already fitted tree; do not refit the tree on those observations before using them as a check.

Confirm the residual definition.
Use $e_i=y_i-\hat{y}_i$, so positive values mean underprediction and negative values mean overprediction. Verify that the response and prediction use the same units.

Plot residuals against fitted values.
Expect vertical bands for a tree. Within and across bands, look for one-sided residuals, changing spread, and observations with especially large errors. Do not mistake the bands themselves for a defect.

Plot residuals against relevant predictors.
Within a leaf, a trend or curve can indicate response structure that the current splits do not represent. Consider whether the pattern is repeated and credible rather than driven by a single observation.

Compare terminal nodes.
Summarize residual means and error sizes by node, distinguishing training from validation results. Training means are zero under the stated squared-error conditions, so training error spread and predictor patterns are more informative than that mean alone.

Decide what the evidence supports.
A repeated pattern may motivate investigating a different partition or model. Check whether the proposed change improves performance on data not used to choose it; do not increase tree complexity solely to remove a training residual pattern.

In R, for a fitted regression tree named fit and a training data frame named dat with response column y, fitted values and residuals can be calculated explicitly:

\texttt{pred <- predict(fit, newdata = dat)} \qquad \texttt{e <- dat\$y - pred}

A basic diagnostic is plot(pred, e); abline(h = 0); a predictor view can be made with plot(dat$x, e); abline(h = 0). For training observations in an rpart fit, fit$where identifies the corresponding rows of fit$frame; the frame row names can be used as terminal-node labels when summarizing training errors by node. Interpret training node means with the squared-error result above in mind.

Common Mistakes and Exam Traps

Reversing the residual sign. If an observation is 16 and its prediction is 20, the residual under the definition here is $16-20=-4$, not $+4$. The negative sign means overprediction. Reversing the subtraction reverses the interpretation.

Using the training-node mean as a fit check. Under squared-error fitting, a node’s training residual mean is zero by construction. That fact does not rule out a predictor trend, large errors, or a poor prediction for new observations.

Calling every vertical band a failure. A regression tree assigns one fitted value per terminal node, so vertical bands in residuals versus fitted values are expected. Assess residual position and spread within those bands, and compare with predictor-based views.

Changing the tree in response to one point. A large residual can be a data error, an unusual but valid outcome, or evidence of a broader weakness. Check the observation and surrounding pattern before drawing a conclusion; do not assume that removing it or adding a split is justified.

Residual inspection is useful because it tests the adequacy of the tree’s representation, not just the accuracy of its overall predictions. A residual pattern can reveal where a constant leaf prediction is systematically missing information. A lack of an obvious pattern is reassuring but not conclusive, especially in a small sample.

Key takeaway: Define residuals consistently as observed minus predicted. In a squared-error regression tree, training residuals average to zero within each leaf because the leaf prediction is its training mean. Diagnose adequacy by also examining residual spread and patterns against fitted values, predictors, and terminal nodes—particularly on validation data.

Check Your Understanding

For each problem, calculate the requested quantities before interpreting the pattern. Use residual $e=y-\hat{y}$.

A single training leaf contains responses 4, 8, and 12 and predicts their mean. Find the prediction, the three residuals, and the sum of squared errors. (Answer: prediction 8; residuals $-4,0,4$; sum of squared errors 32.)
A tree predicts 15 for each of three training observations with predictor values $x=1,2,3$ and responses $12,15,18$. Find the residuals and describe their pattern against $x$. (Answer: residuals $-3,0,3$; they increase with $x$, indicating a within-leaf trend despite a zero residual mean.)
On validation data, a node predicts 30 for responses 27 and 33. Find its mean residual and RMSE. (Answer: residuals $-3,3$; mean residual 0; RMSE 3.)
Two validation observations have predictions 40 and observed responses 36 and 44, respectively. State each residual and what its sign means. (Answer: residuals $-4$ and $4$; the first is an overprediction and the second an underprediction.)
A validation node predicts 50 for responses 46, 48, and 52. Find its mean residual and RMSE, rounding the RMSE to four decimal places. (Answer: residuals $-4,-2,2$; mean residual $-4/3$, approximately $-1.3333$; RMSE $\sqrt{24/3}=\sqrt{8}\approx2.8284$.)

What You'll Learn