Estimating Class Probabilities in Terminal Nodes

From a Terminal Node to a Probability Estimate

A classification tree sends each case down a sequence of branches and assigns it to one terminal node. The cases in that node share the branch conditions used to reach it. For a new case that reaches the same node, the tree can report a probability estimate for each class, based on the class frequencies among the training cases already in that node.

These are local estimates: the denominator is the number of training cases in the particular terminal node, not the total training set and not the number of cases in some other node. This simple choice of denominator is central to interpreting the result.

Definition: Suppose a terminal node $t$ contains $n_t$ training cases and the classification problem has $K$ classes. Let $n_{tk}$ be the number of cases in node $t$ that belong to class $k$. The estimated probability of class $k$ for a case in node $t$ is the observed class proportion $$ \widehat{P}(Y=k\mid t)=\frac{n_{tk}}{n_t}, \qquad k=1,\ldots,K. $$ The estimates for all classes form the node's class-probability vector.

The training cases used in this calculation are the cases that actually reached the node while the tree was being fit. Each such case contributes one count to exactly one class. Unless a problem explicitly defines a different procedure, each training case has equal weight in the proportion. The formula describes the empirical proportion; by itself, it does not guarantee that the estimate is accurate for future cases.

Why the Proportion Is the Empirical Conditional Probability

The formula follows directly from conditioning on being in the node. Within the training data, treat each case as one equally weighted observation. There are $n_t$ observations in the node, and $n_{tk}$ of them have class $k$. Thus the fraction of node cases with class $k$ is $n_{tk}/n_t$. Equivalently, average an indicator that equals one when a node case has class $k$, and zero otherwise:

\widehat{P}(Y=k\mid t) =\frac{1}{n_t}\sum_{i:\,i\text{ is in }t} I(Y_i=k) =\frac{n_{tk}}{n_t}.

Here $I(Y_i=k)$ is one if case $i$ belongs to class $k$, and zero otherwise. The sum therefore counts exactly the node cases in class $k$. This is a conditional empirical proportion because the calculation first restricts attention to cases in $t$, then measures the fraction in the class of interest.

The estimates are valid proportions: they are between zero and one, and they add to one. Every case in the node belongs to exactly one of the $K$ classes, so $n_{t1}+\cdots+n_{tK}=n_t$. Dividing both sides by $n_t$, when $n_t>0$, gives:

\sum_{k=1}^{K}\widehat{P}(Y=k\mid t) =\sum_{k=1}^{K}\frac{n_{tk}}{n_t} =\frac{n_t}{n_t} =1.

Worked Example: A Late-Payment Estimate in One Node

A hypothetical insurer fits a classification tree to flag whether a scheduled premium payment will be late. A particular terminal node contains 24 training policies: 7 had a late payment and 17 did not. Calculate both class-probability estimates and interpret the late-payment estimate for a new policy routed to this node.

Step 1: Identify the node denominator. The node contains $n_t=24$ training policies. The denominator for each class proportion is 24.

Step 2: Count each class. There are $n_{t,\text{late}}=7$ late-payment cases and $n_{t,\text{not late}}=17$ cases without a late payment. The counts check because $7+17=24$.

Step 3: Calculate the proportions.

\widehat{P}(\text{late}\mid t)=\frac{7}{24}=0.2917,\qquad \widehat{P}(\text{not late}\mid t)=\frac{17}{24}=0.7083.

The two estimates add to $0.2917+0.7083=1.0000$. Rounded to four decimal places, the estimated probability of a late payment for a new policy routed to this node is 0.2917. This is an estimate based on the node's observed training cases, not a guarantee about any particular future policy.

Method and mistake avoided. Use the observed proportion within the terminal node. Dividing the 7 late cases by the entire training-set size, if that size differs from 24, would answer a different question: the overall training-set proportion rather than the estimate conditional on this node.

Calculating Probabilities for Several Classes

The same calculation applies when there are more than two classes. Calculate a proportion for each class using the same node denominator. Do not omit a class merely because it is not the most frequent one in the node: the probability vector reports an estimate for every class in the problem.

Worked Example: Three Claim-Outcome Classes

A hypothetical claims-screening tree has a terminal node containing 28 training claims. Their recorded outcomes are 12 routine, 9 requiring additional review, and 7 requiring specialist review. Find the estimated probability for each outcome. Then calculate the corresponding expected numbers if 140 comparable future claims were routed to this node and the estimated proportions held.

Step 1: Verify the total and denominator. The outcome counts sum to $12+9+7=28$, so $n_t=28$.

Step 2: Calculate the three node proportions.

\widehat{P}(\text{routine}\mid t)=\frac{12}{28}=0.4286,

\widehat{P}(\text{additional review}\mid t)=\frac{9}{28}=0.3214,

\widehat{P}(\text{specialist review}\mid t)=\frac{7}{28}=0.2500.

The probabilities sum to $0.4286+0.3214+0.2500=1.0000$, to four decimal places.

Step 3: Translate the proportions into expected counts. Multiplying each estimate by 140 gives:

140(0.4286)\approx60,\qquad 140(0.3214)\approx45,\qquad 140(0.2500)=35.

Using the unrounded fractions gives $140(12/28)=60$, $140(9/28)=45$, and $140(7/28)=35$. These add to 140. The counts are expected amounts under the stated assumption that the estimated proportions apply; they are not a promise that exactly those numbers will occur.

Method and mistake avoided. Calculate all class proportions using 28, then use those estimates consistently for the projection. Using different denominators for different classes would not produce a valid probability vector.

Node-Specific Estimates Versus Overall Proportions

A terminal node's estimate is conditional on reaching that node. It can therefore differ from the overall class proportion in the training set. The overall proportion describes the training cases before restricting attention to the node. A node estimate describes the subset selected by the tree's branch conditions.

Worked Example: Using the Child Node, Not the Whole Data Set

In a hypothetical equipment-maintenance study, a tree has a parent node containing 40 training machines: 22 required repair within the study period and 18 did not. One terminal node below it contains 15 of those machines: 10 required repair and 5 did not. Calculate the repair probability estimate for a new machine routed to this terminal node, compare it with the parent-node proportion, and estimate the expected number of repairs among 45 comparable machines routed to the terminal node.

Step 1: Calculate the terminal-node proportions. The terminal-node denominator is 15, so:

\widehat{P}(\text{repair}\mid t)=\frac{10}{15}=0.6667,\qquad \widehat{P}(\text{no repair}\mid t)=\frac{5}{15}=0.3333.

The estimates add to one. The estimate for a new machine reaching this terminal node is therefore 0.6667 for repair and 0.3333 for no repair.

Step 2: Calculate the parent proportion for comparison. At the parent node, the repair proportion is $22/40=0.5500$, and the no-repair proportion is $18/40=0.4500$. These are not the estimates for a case known to reach the smaller terminal node; they summarize the larger parent group.

Step 3: Calculate the expected number of repairs. Applying the terminal-node estimate to 45 comparable cases gives $45(10/15)=45(2/3)=30$ expected repairs. The complementary expected number without repair is $45(5/15)=15$. The two expected counts sum to 45.

Common mistake and correction. A wrong calculation uses the terminal-node repair count but the parent denominator, $10/40=0.2500$, or the parent repair count and denominator, $22/40=0.5500$, as the terminal-node estimate. Neither is correct for the stated conditional question. The relevant calculation is $10/15=0.6667$, because both the count and denominator must refer to the same terminal node.

Method. First identify which cases the question conditions on, then use the matching node's class counts and total. This prevents a denominator mismatch.

Interpreting the Estimates Carefully

A terminal-node probability is an empirical estimate, not a known probability for every case in the node. It summarizes how often each class occurred among the training cases assigned there. Interpreting it as an estimate for future cases requires the training cases to provide relevant information about those future cases. If the future population or the process generating outcomes changes, the observed proportions may not remain appropriate.

The size of the node also matters when interpreting the estimate. A proportion from a small number of cases can change substantially when one additional case is observed. For example, one case changes a class proportion by $1/5=0.2000$ in a node of five cases, but by $1/100=0.0100$ in a node of 100 cases. This arithmetic does not by itself quantify uncertainty, but it shows why the same-looking proportion can have different stability depending on its denominator.

A class with zero observed cases has estimated proportion zero under this calculation. That means no training case in that node belonged to the class; it does not establish that the class is impossible for future cases. Conversely, a class proportion of one means all observed training cases in the node belonged to that class, not that every future case must do so. The empirical formula does not add adjustments for small samples or unobserved classes.

Key takeaway: For a case routed to terminal node $t$, estimate each class probability by dividing that class's training-case count in $t$ by the total training-case count in $t$. Use the same local denominator for every class, check that the estimates sum to one, and interpret them as empirical estimates rather than certainties.

Common Calculation Traps

Using the whole training set as the denominator. That gives an overall class proportion, not the estimate conditional on being in a particular terminal node.
Mixing counts from different nodes. The numerator and denominator in a node proportion must both refer to the same node.
Forgetting a class. Include every class when forming the probability vector, including classes with a zero observed count.
Treating an observed proportion as a certainty. A zero or one estimate records what happened in the training node; it does not prove what must happen in the future.
Reporting counts as probabilities. A count such as 7 is not a probability. Divide by the node total and check that all class proportions sum to one.

Check Your Understanding

For each problem, calculate the requested estimates from the stated terminal-node training cases. Probabilities are rounded to four decimal places when needed.

A terminal node for a hypothetical appliance service tree contains 16 training cases: 5 required a technician visit and 11 did not. Find both class-probability estimates. (Answer: visit $=0.3125$; no visit $=0.6875$.)
A terminal node contains 16 training applications in three categories: 8 standard, 5 manual review, and 3 declined. Calculate the three probability estimates and check their total. (Answer: standard $=0.5000$, manual review $=0.3125$, declined $=0.1875$; total $=1.0000$.)
A leaf in a hypothetical crop-disease tree contains 27 training plants: 9 diseased and 18 not diseased. What is the estimated probability of disease for a new plant routed to that leaf? (Answer: $9/27=0.3333$.)
A parent node contains 48 training records, but the terminal node relevant to a new case contains 12 records, of which 5 are in the class of interest. Calculate the appropriate node estimate and the overall parent proportion if 14 parent cases are in that class. (Answer: terminal-node estimate $=5/12=0.4167$; parent proportion $=14/48=0.2917$.)
A terminal node has 25 training cases across three outcomes, with counts 14, 7, and 4. Calculate each class probability, then estimate the expected numbers in a group of 75 comparable cases routed to that node. (Answer: probabilities $=0.5600, 0.2800, 0.1600$; expected counts $=42, 21, 12$.)

What You'll Learn