Why a Terminal Node Uses a Mean
A regression tree divides predictor space into regions and assigns a constant prediction to each terminal region. Once a region is fixed, the tree must choose one number to represent the training responses that fall inside it. Under squared-error loss, the number that gives the smallest total error is the sample mean of those responses.
This result connects a tree’s simple prediction rule to a precise optimization problem. For a new case that falls into a terminal node, the fitted value is the same as for every other case in that node. The training responses in the node determine that shared value; the new case’s response is not needed to make the prediction.
The responses are numerical, and the objective penalizes each case according to the square of the difference between its observed response and the node’s constant prediction. Each training case has equal weight in this definition. The result below requires no probability distribution for the responses: it is an optimization fact about a fixed set of observed values.
Deriving the Least-Squares Node Prediction
Write the sample mean of the responses in the node as \(\bar y=\frac{1}{n}\sum_{i=1}^{n}y_i\). To see why it minimizes the squared-error objective, express each difference \(y_i-c\) as the sum of its deviation from the sample mean and the difference between the sample mean and \(c\).
For every response, \(y_i-c=(y_i-\bar y)+(\bar y-c)\). Substitute this into the sum of squared errors.
The cross-product terms contain \(\sum_{i=1}^{n}(y_i-\bar y)\), which is zero by the definition of the sample mean.
The remaining term that depends on \(c\) is \(n(c-\bar y)^2\), which is nonnegative and is zero only when \(c=\bar y\).
The middle term vanishes because \(\sum_{i=1}^{n}(y_i-\bar y)=\sum_{i=1}^{n}y_i-n\bar y=0\). The first term does not depend on \(c\), while the final term is nonnegative. Since \(n>0\), it is minimized uniquely at \(c=\bar y\). Thus, for a fixed nonempty terminal node, the sample mean is the unique constant prediction minimizing the sum of squared response errors.
The zero-sum property follows directly: \(\sum_{i=1}^{n}e_i=\sum_{i=1}^{n}(y_i-\bar y)=\sum_{i=1}^{n}y_i-n\bar y=0\). It is a useful arithmetic check, but it does not mean that every residual is zero or that the prediction is exact for each case. A node with a positive residual must have negative residuals elsewhere in the node to balance the sum.
Worked Calculations
Worked Example: Finding a Node Prediction and Its Residuals
A hypothetical insurer groups four training policies into one terminal node using their recorded predictor values. Their annual claim-cost responses, in thousands of dollars per policy, are \(8, 10, 12,\) and \(14\). Find the node prediction and each training residual, then verify the squared-error objective. Interest and mortality assumptions are not relevant to this regression calculation. Treat the observed responses as fixed; no response distribution or distribution parameters are assumed.
Method. Under equal-weight squared-error loss, calculate the sample mean, assign it as the fitted value for each policy, and subtract each fitted value from its observed response.
1. Calculate the mean. The response sum is \(8+10+12+14=44\), and the number of policies is \(n=4\). Therefore, \(\bar y=44/4=11\) thousand dollars per policy. The terminal node predicts \(11\) thousand dollars per policy.
2. Calculate the residuals. Using \(e_i=y_i-\hat y_i\), the residuals in the listed order are \(8-11=-3\), \(10-11=-1\), \(12-11=1\), and \(14-11=3\) thousand dollars.
3. Calculate and check the squared errors. Their squares are \(9,1,1,\) and \(9\), so \(Q(11)=9+1+1+9=20\) in squared thousands of dollars. The residuals sum to \(-3-1+1+3=0\), as they must when the prediction is the sample mean. The minimizing prediction is therefore \(11\) thousand dollars, with squared-error total \(20\).
Common mistake avoided: Do not report the mean response as a residual. The prediction is \(11\); the four residuals are the observed responses minus \(11\).
Worked Example: Predicting for a Case in a Specified Region
In a hypothetical portfolio analysis, one rectangular predictor-space region is a terminal node. Four training policies in that region have annual claim-cost responses of \(2,4,4,\) and \(10\) thousand dollars. A fifth policy enters the same region when a prediction is required. Find its prediction and the training residuals. Interest and mortality assumptions do not apply; the observed responses are treated as fixed, with no distribution assumed.
Method. Use the mean of the four training responses. The fifth policy’s response is not included, because it is the response to be predicted rather than a training observation.
1. Calculate the node mean. The sum is \(2+4+4+10=20\), and \(n=4\). Thus, \(\bar y=20/4=5\) thousand dollars. The fifth policy’s predicted annual claim cost is \(5\) thousand dollars, provided its predictors place it in this node.
2. Calculate the training residuals. The fitted value for each of the four training policies is \(5\). The residuals are \(2-5=-3\), \(4-5=-1\), \(4-5=-1\), and \(10-5=5\) thousand dollars.
3. Verify the residual and squared-error calculations. The residual sum is \(-3-1-1+5=0\). The squared errors are \(9,1,1,\) and \(25\), giving \(Q(5)=9+1+1+25=36\) in squared thousands of dollars.
Check against a different constant. If the node instead predicted \(4\), the squared errors would be \((2-4)^2=4\), \((4-4)^2=0\), \((4-4)^2=0\), and \((10-4)^2=36\). Their sum is \(40\), greater than \(36\). This confirms that \(4\) is not as good as the mean under squared-error loss.
Common mistake avoided: The fifth policy’s unknown response must not be added to the mean. The node prediction is calculated from training responses, then applied to cases that enter the node.
Worked Example: A Noninteger Mean and a Competing Prediction
A hypothetical group of five commercial property records falls in one terminal node. Their annual loss responses, in thousands of dollars, are \(2,5,8,9,\) and \(16\). Find the fitted value and residuals, and compare the resulting squared-error total with the total from predicting \(9\) for every record. Interest and mortality assumptions are not involved; use the fixed observed responses without assuming a distribution.
Method. Compute the mean of the five responses to obtain the least-squares prediction, then evaluate both constants using the same five observations.
1. Calculate the mean. The sum is \(2+5+8+9+16=40\). Since \(n=5\), \(\bar y=40/5=8\) thousand dollars. The terminal-node fitted value for each record is \(8\) thousand dollars.
2. Calculate the residuals at the mean. In order, they are \(2-8=-6\), \(5-8=-3\), \(8-8=0\), \(9-8=1\), and \(16-8=8\) thousand dollars. Their sum is \(-6-3+0+1+8=0\).
3. Calculate squared error at the mean. The squared residuals are \(36,9,0,1,\) and \(64\), so \(Q(8)=36+9+0+1+64=110\) in squared thousands of dollars.
4. Evaluate the competing prediction. At \(c=9\), the residuals are \(-7,-4,-1,0,\) and \(7\). Their squares sum to \(49+16+1+0+49=115\). Since \(115>110\), the prediction \(9\) has a larger squared-error total than the mean prediction \(8\).
Check. The mean-based residuals sum to zero, and the direct comparison agrees with the minimizing result. The node prediction is \(8\) thousand dollars, with squared-error total \(110\).
Common mistake avoided: Do not round a noninteger mean prematurely in a case where the arithmetic does not divide evenly. Here the mean is exactly \(8\); in other data sets, retain sufficient precision until the final reported prediction.
What the Result Does—and Does Not—Say
The mean rule is conditional on both the chosen node and the loss function. It says that among all constant predictions for the same nonempty collection of training responses, the mean minimizes the sum of squared errors. It does not say that the mean is the response of a typical individual, that every response will be close to it, or that the prediction will be correct for a new case.
The rule also depends on using squared errors. If the objective changes, the minimizing constant can change too. For example, the median minimizes the sum of absolute deviations, not the sum of squared deviations. For squared-error prediction in a regression tree, using the median in place of the mean generally fails to minimize the stated objective.
When interpreting a fitted tree, distinguish three quantities: the node’s single predicted value, each training case’s residual, and the total squared error across the node’s training cases. A new case assigned to the node receives the same predicted value, but it has no known residual until its response is observed. The sample mean describes the training responses in that region; it does not supply a separate prediction for each case.
This distinction is useful in actuarial prediction tasks, where a tree may assign policies with different predictor values to the same terminal region. Under squared-error fitting, the shared prediction is the average training response in that region. Knowing why the mean is used makes it easier to calculate fitted values correctly and to recognize when an alternative prediction rule is inconsistent with squared-error loss. This treatment is aligned with the SOA Exam SRM (Statistics for Risk Modeling) syllabus.
Check Your Understanding
For each fixed terminal node, use the training responses only and report predictions and residual checks as requested.
- A node contains responses \(3,7,\) and \(8\). Find its mean prediction, the three residuals, and the sum of squared errors. (Answer: prediction \(6\); residuals \(-3,1,2\); squared-error sum \(14\).)
- A node contains responses \(5,5,9,\) and \(13\). Find its prediction and squared-error sum. (Answer: prediction \(8\); squared-error sum \(44\).)
- Five records in a node have responses \(12,15,17,20,\) and \(26\). Find the prediction, residuals, and squared-error sum. (Answer: prediction \(18\); residuals \(-6,-3,-1,2,8\); squared-error sum \(114\).)
- A node contains responses \(2,4,9,\) and \(11\). Compare the squared-error totals for predictions \(6.5\) and \(7\), and identify the minimizing one. (Answer: \(Q(6.5)=53\), \(Q(7)=54\); the minimizing prediction is \(6.5\).)
- Terminal node A has responses \(1,2,\) and \(6\); terminal node B has responses \(4,8,12,\) and \(16\). Find each node’s prediction and squared-error sum, then give their combined squared-error sum. (Answer: A predicts \(3\) with sum \(14\); B predicts \(10\) with sum \(80\); combined sum \(94\).)