From a Terminal Node to a Class Prediction
A classification tree sorts observations through a sequence of splits. Each observation that follows the same path reaches the same terminal node, or leaf. To predict a categorical outcome for a new observation, the tree uses the training observations that reached that leaf. Their class frequencies provide estimated probabilities, and the most frequent class gives the usual majority-class prediction.
These are related outputs, but they answer different questions. A class prediction is a single label, such as “renew” or “lapse.” A probability estimate gives a proportion for every possible class, such as 0.60 for renewal and 0.40 for lapse. A label does not mean the model is certain, and the probability vector does not by itself determine a label unless a decision rule is specified.
The proportion uses only observations that reached the node. Counts from other leaves do not enter its calculation. For an incoming observation, first determine its terminal node by following the tree’s splits; then use that node’s class proportions. Consequently, all new observations reaching the same leaf receive the same estimated class-probability vector, and normally the same majority-class label.
This basic calculation treats observations as equally weighted. It requires no assumption that the response follows a particular parametric distribution. The proportions are empirical estimates of the class mix among observations in the region represented by the leaf. Interpreting them as useful estimates for future observations relies on the training data being relevant to the future population and the fitted tree being applied consistently. In particular, a node proportion is not a guarantee about an individual outcome, nor does it automatically establish that the probabilities are calibrated.
Why the Proportions Are Probabilities
For each training observation \(i\) that reaches node \(m\), define an indicator \(I(Y_i=k)\), equal to 1 if its response is class \(k\) and 0 otherwise. Adding these indicators counts the observations in class \(k\). Dividing that count by the number of observations in the node gives the fraction in class \(k\):
Here \(R_m\) denotes the predictor region represented by node \(m\), and \(X_i\) denotes the predictors for observation \(i\). The average of the class indicator is the observed fraction of cases in that class. It is therefore the natural empirical estimate of the conditional class probability for observations in the node. Since every observation belongs to exactly one of the \(K\) classes, the counts add to the node size: \(\sum_{k=1}^{K}n_{mk}=n_m\). Dividing by \(n_m\) shows why the proportions add to one:
The majority-class rule selects an index \(k\) with the largest \(\hat{p}_{mk}\). In a binary problem, this is equivalent to predicting the first class when its estimated proportion is greater than 0.5, and the second class when it is less than 0.5. At exactly 0.5, the proportions do not select a unique label. For multiple classes, a tie can likewise occur. A software implementation may use a specified class order or another tie convention, but the convention should not be mistaken for evidence that one tied class is more probable.
Worked Example: Calculating Probabilities in a Multiclass Leaf
A hypothetical insurer uses a classification tree to assign a policy to one of three renewal outcomes: renew, revise, or nonrenew. In one terminal node, 50 training policies are counted: 22 renewed, 17 were revised, and 11 were nonrenewed. Find the estimated class probabilities and the ordinary majority-class prediction for a new policy routed to this node.
Method. Divide each class count by the total number of observations in the leaf. First check that the counts account for the whole node: \(22+17+11=50\).
For renew, the estimated proportion is \(22/50=0.44\). For revise, it is \(17/50=0.34\). For nonrenew, it is \(11/50=0.22\). To four decimal places, the vector in the stated class order is \((0.4400, 0.3400, 0.2200)\).
The proportions check: \(0.44+0.34+0.22=1.00\). The largest is 0.44, so the majority-class prediction is renew. The model has not predicted renewal with certainty; it estimates that 44% of the training policies in this leaf renewed.
Common mistake avoided: Reporting only “renew” loses the information in the other two estimated probabilities. Conversely, reporting 0.44 as certainty confuses the predicted label with its estimated probability.
The Predicted Label Is a Decision
The majority-class prediction is one rule for converting probabilities into a label. Other rules can be appropriate when the cost of different errors is unequal. For example, a business may choose to flag a case for review whenever the estimated probability of an adverse outcome reaches a specified threshold. That rule can produce a different label from the majority-class rule without changing the probability estimates themselves.
It is therefore useful to state both the probability vector and the rule used to select a label. If a problem asks for the usual classification-tree prediction and gives no alternative rule, use the largest class proportion. If it specifies a threshold or cost-based decision, apply that rule instead. Do not silently replace the stated rule with a threshold that seems more cautious.
Worked Example: A Threshold Changes the Label, Not the Probabilities
A hypothetical service team routes accounts to routine handling or review. In a terminal node, 15 training accounts are observed: 9 received routine handling and 6 required review. The estimated probability of review is calculated from the node proportion. The team’s separate operating policy says to label an account “review” whenever this estimated probability is at least 0.35. Find the probability estimates, the majority-class prediction, and the policy label.
Method. Calculate the class proportions first, then apply each label rule separately. The counts sum to \(9+6=15\).
The routine proportion is \(9/15=0.60\). The review proportion is \(6/15=0.40\). The probability vector in the order (routine, review) is \((0.6000, 0.4000)\), rounded to four decimal places; its entries sum to 1.0000.
Under the ordinary majority-class rule, routine is predicted because \(0.60>0.40\). Under the stated operating policy, the review probability is \(0.40\), and \(0.40\geq0.35\), so the policy label is review. The estimated probabilities have not changed; only the rule for turning those probabilities into a label has changed.
Common mistake avoided: It would be incorrect to call review the majority class. Review is the policy label here, not the class with the largest observed count. It would also be incorrect to use a threshold of 0.50 when the problem explicitly specifies 0.35.
Ties and the Limits of a Single Label
A terminal node does not always have a unique majority class. When two or more classes share the largest count, the empirical proportions support a tie. A program still may return one label, but that return value depends on its tie-handling convention. Always distinguish the observed probability estimates from the implementation’s choice of one label.
Small leaves can also produce coarse estimates: one observation changes a class proportion by \(1/n_m\). For example, in a leaf of 10 observations, changing a count by one changes its proportion by 0.10. This does not alter the calculation, but it is a reason to avoid interpreting a node proportion as a highly precise fact about the wider population.
Worked Example: Reporting a Tied Multiclass Node
A hypothetical claims triage tree has a terminal node containing 20 training claims. The recorded outcomes are standard handling for 8 claims, specialist handling for 8 claims, and escalation for 4 claims. The implementation resolves tied class predictions by selecting the first class in the declared order: standard handling, specialist handling, escalation. Give the proportions and the returned label, and explain what the label means.
Method. Calculate all three proportions, identify the maximum, and then use the specified tie convention only to choose the returned label. The count check is \(8+8+4=20\).
Standard handling has proportion \(8/20=0.40\); specialist handling also has proportion \(8/20=0.40\); escalation has proportion \(4/20=0.20\). To four decimal places, the vector is \((0.4000,0.4000,0.2000)\). The sum is \(0.4000+0.4000+0.2000=1.0000\).
The largest proportion is tied between standard and specialist handling. There is no unique majority class based on the proportions alone. Under the stated ordering convention, the returned label is standard handling, because it comes first among the tied classes. That convention does not mean standard handling is estimated to be more likely than specialist handling; both estimates are 0.4000.
Common mistake avoided: Do not report a unique majority class without checking for ties, and do not treat a software tie-break as a difference in estimated probability.
A Reliable Calculation Sequence
For a terminal-node question, use a short sequence that keeps the count calculation and the decision rule separate.
Use only the training observations that reached the specified terminal node, and keep the class order clear.
Add the class counts and confirm that they equal the stated node size \(n_m\).
For every class, divide its count \(n_{mk}\) by \(n_m\). Check that the proportions sum to one, allowing for rounding.
Choose a class with the largest proportion for the ordinary majority rule; use a supplied threshold or tie convention only when specified.
State the probability estimates and the predicted label, and identify any tie or alternative decision rule.
This procedure is aligned with the SOA Exam PA (Predictive Analytics) syllabus: a complete interpretation reports what the tree estimates and how those estimates lead to a class decision. A frequent exam trap is to give a label without its basis, or to confuse a probability such as 0.40 with a 40% chance that the model’s label itself is correct. The node proportion estimates the share of a class among observations in that leaf; it is not automatically the model’s accuracy.
Check Your Understanding
For each problem, show the count check, calculate the class proportions, and apply the requested prediction rule.
- A terminal node contains 24 hypothetical applications: 10 approved, 8 deferred, and 6 declined. Give the probability vector in that order and the majority-class prediction. (Answer: \((0.4167, 0.3333, 0.2500)\); approved.)
- A binary leaf contains 16 hypothetical devices, 5 of which fail and 11 of which pass. Give the estimated failure probability and the ordinary majority-class prediction. (Answer: Failure probability \(5/16=0.3125\); pass.)
- A leaf contains 30 hypothetical service cases: 12 resolved online, 12 resolved by phone, and 6 unresolved. Is there a unique majority class? Give the probability vector. (Answer: No; online and phone are tied. The vector is \((0.4000, 0.4000, 0.2000)\).)
- A binary node contains 20 hypothetical applications, 7 high-risk and 13 low-risk. The default rule is majority class; a separate review policy labels an application high-risk if its estimated high-risk probability is at least 0.30. Give the probability vector, the default prediction, and the policy label. (Answer: \((0.3500, 0.6500)\) in the order (high-risk, low-risk); low-risk by majority; high-risk under the policy.)
- A terminal node has counts 9 in class A, 9 in class B, and 7 in class C. A system resolves a tie by choosing the alphabetically first tied class. Give the proportions and the returned label, stating whether the proportions themselves distinguish the tied classes. (Answer: \((0.3600, 0.3600, 0.2800)\); class A is returned by the tie rule, but the proportions do not distinguish A from B.)