Why Probabilities and Predicted Labels Are Different
A classification tree may place an observation in a terminal node and report a predicted class. That label is useful when a decision requires one category, but it does not show the full information in the leaf. The leaf’s class probabilities describe the estimated distribution across all possible classes; the predicted label reports only the class with the largest estimated probability.
For example, two leaves can both predict “routine” while estimating very different proportions of routine and review cases. One leaf might contain a close split, while another has a much larger share of routine cases. Reading only their labels hides that difference. A careful interpretation reports the probability vector when the relative proportions matter and identifies the label separately.
These probabilities are estimates based on the observations in the training data that reached the node. They are not guarantees about an individual observation, nor do they necessarily equal the true class probabilities in the population. Unless a model explicitly applies a different estimation method, the empirical proportions are the probabilities being interpreted here.
How a Leaf Probability Vector Is Estimated
The proportion \(n_k/n\) is a natural estimate when the observations in a leaf are treated as outcomes from a common class distribution. To justify this estimate, suppose the observations reaching a fixed leaf have independent class labels, each with probabilities \(p_1,\ldots,p_K\), where \(p_k\geq 0\) and \(\sum_{k=1}^K p_k=1\). The likelihood of the observed class counts, apart from a factor that does not depend on the probabilities, is proportional to \(\prod_{k=1}^K p_k^{n_k}\). Maximizing this likelihood subject to the probabilities summing to 1 gives the empirical proportions.
For positive counts, introduce a multiplier \(\lambda\) for the sum constraint. Differentiating the constrained expression with respect to \(p_k\) gives \(n_k/p_k-\lambda=0\). Thus \(p_k=n_k/\lambda\). Summing over \(k\) and using \(\sum_k p_k=1\) gives \(\lambda=\sum_k n_k=n\), so \(p_k=n_k/n\). If a class has count zero, its likelihood-maximizing empirical probability is zero as well; the same proportions result. This derivation explains the empirical estimate, but it does not establish that it will be accurate for every future sample or population.
The last expression is a rule for converting the vector into one label. It is a lossy conversion: the vector has one component for each class, while the label identifies only a maximizing class. If the largest component is shared by multiple classes, the label cannot be specified without the tree’s tie-breaking rule.
Worked Example: Same Predicted Class, Different Probabilities
Worked Example: Same Predicted Class, Different Probabilities
In a hypothetical classification tree for sorting service requests, the classes are Routine and Review. Leaf A contains 25 training requests: 15 Routine and 10 Review. Leaf B contains 50: 35 Routine and 15 Review. Find each probability vector and predicted label, then compare the two leaves. Use the class order (Routine, Review).
Method. Divide each class count by the total count in its leaf to estimate the vector. Then select the class with the largest component as the predicted label.
For Leaf A, \(n=15+10=25\). Its Routine probability estimate is \(15/25=0.6000\), and its Review estimate is \(10/25=0.4000\). The vector is therefore \((0.6000,0.4000)\). Since \(0.6000>0.4000\), the predicted label is Routine.
For Leaf B, \(n=35+15=50\). Its Routine estimate is \(35/50=0.7000\), and its Review estimate is \(15/50=0.3000\). The vector is \((0.7000,0.3000)\). Since \(0.7000>0.3000\), this leaf also predicts Routine.
Both leaves predict Routine, but their estimated Routine proportions differ: 0.6000 in Leaf A and 0.7000 in Leaf B. Their Review proportions differ as well. The common label does not mean that the leaves have identical class distributions.
Check. Each vector sums to 1: \(0.6000+0.4000=1.0000\) and \(0.7000+0.3000=1.0000\). In each vector, Routine is the largest component, matching the stated label.
Common mistake avoided: Treating the shared label as if it were a complete description of either leaf. The label is Routine in both, but the two probability vectors retain information the label discards.
Reading the Full Vector in a Multiclass Leaf
With more than two classes, a predicted label still identifies only the largest component. The remaining components matter because they show how the other classes are represented in that node. A probability vector should be read in the same class order whenever vectors are compared; changing the order without saying so can make the comparison misleading.
Worked Example: Interpreting Three Class Probabilities
A hypothetical tree classifies equipment inspections into Low, Moderate, or High priority. A terminal node contains 40 training inspections: 18 Low, 14 Moderate, and 8 High. Find the probability vector in the stated class order, give the predicted label, and interpret the other components.
Method. Estimate each class probability as its count divided by 40. Compare all three components to find the largest.
The Low estimate is \(18/40=0.4500\). The Moderate estimate is \(14/40=0.3500\). The High estimate is \(8/40=0.2000\). Thus, in the order (Low, Moderate, High), the vector is \((0.4500,0.3500,0.2000)\).
The largest component is 0.4500, so the predicted label is Low. The vector also says that Moderate is estimated at 0.3500 and High at 0.2000 within this leaf. Reporting only “Low” would omit those two estimates.
Check. The components sum to \(0.4500+0.3500+0.2000=1.0000\), and 0.4500 is larger than both 0.3500 and 0.2000. The vector and label are consistent.
Common mistake avoided: Describing the predicted class as if every observation in the leaf belongs to that class. The label is Low, but 22 of the 40 training inspections in this example are not Low.
Leaf Probabilities, Pooled Proportions, and Averaging
A pooled class proportion across several leaves uses all observations in those leaves. Because the leaves may have different sizes, the pooled proportion is a size-weighted average of their class proportions. An unweighted average gives each leaf the same contribution regardless of how many observations it contains. These quantities answer different questions and are generally not equal.
Here \(J\) is the number of leaves being combined, \(n_j\) is the size of leaf \(j\), and \(n_{jk}\) is its count for class \(k\). The weight on leaf \(j\) is its share of all observations, \(n_j/\sum_r n_r\). The formula follows by writing the total count of class \(k\) as \(\sum_j n_{jk}\), and the total sample size as \(\sum_j n_j\). Since \(n_{jk}/n_j\) is the leaf proportion, multiplying it by \(n_j/\sum_r n_r\) gives \(n_{jk}/\sum_r n_r\). Summing these terms gives the pooled proportion.
Worked Example: Pooling Leaves of Unequal Sizes
Two terminal leaves of a hypothetical service-request tree contain Routine and Review cases. Leaf 1 has 20 cases, with 12 Routine and 8 Review. Leaf 2 has 80 cases, with 56 Routine and 24 Review. Both leaves predict Routine. Find the pooled Routine proportion and the unweighted average of the two leaf Routine proportions. State what each calculation represents.
Method. For the pooled proportion, add the Routine counts and divide by the combined sample size. For the unweighted average, calculate each leaf proportion first and then average the two proportions with equal weights.
Leaf 1’s Routine proportion is \(12/20=0.6000\). Leaf 2’s Routine proportion is \(56/80=0.7000\). The total Routine count is \(12+56=68\), and the total number of cases is \(20+80=100\). Therefore, the pooled Routine proportion is \(68/100=0.6800\).
The unweighted average of the leaf proportions is \((0.6000+0.7000)/2=1.3000/2=0.6500\). It gives each leaf weight \(1/2\), even though Leaf 1 contains 20 cases and Leaf 2 contains 80.
As a check using size weights, the pooled proportion is \((20/100)(0.6000)+(80/100)(0.7000)=0.1200+0.5600=0.6800\). The unweighted result, 0.6500, is different because it answers a different question: the average of the two leaf-specific proportions when each leaf counts equally.
Common mistake avoided: Calling the unweighted average the pooled proportion. The correct pooled result is 0.6800, not 0.6500, because observations—not leaves—receive equal weight in the pooled calculation.
What the Label Does and Does Not Tell You
A predicted label is appropriate when the requested output is one class under the tree’s stated majority-class rule. A probability vector is appropriate when the question asks about class composition or requires a comparison of the estimated probabilities. If only the label is reported, two leaves with the same label cannot be distinguished by their class proportions from that output alone.
The estimated vector also should not be overinterpreted. It summarizes the training observations assigned to a node under the fitted tree. It does not by itself establish that the estimates are accurate for new observations, and a large component is not a guarantee that a particular future case belongs to that class. The distinction here is between two forms of tree output: a set of estimated proportions and the single label selected from them.
A useful reporting habit is to state the class order, give the full vector when relevant, and then state the predicted label. For example: “In the order (Routine, Review), the estimated vector is \((0.70,0.30)\); the predicted label is Routine.” This makes the probability estimates and the classification decision explicit rather than treating them as interchangeable.
Check Your Understanding
For each question, distinguish the full class-probability information from the single predicted label. Give probabilities to four decimal places where needed.
- A binary-class leaf contains 14 cases of Class A and 6 cases of Class B. Give its probability vector in the order (A, B) and its majority-class prediction. (Answer: \((0.7000,0.3000)\); Class A)
- A three-class leaf contains 6 cases of Red, 9 of Blue, and 5 of Green. Give the vector in the order (Red, Blue, Green) and the predicted label. (Answer: \((0.3000,0.4500,0.2500)\); Blue)
- Two leaves have vectors \((0.6500,0.3500)\) and \((0.7000,0.3000)\), in the order (Accept, Review). What do they share, and what differs? (Answer: Both predict Accept; their estimated Accept probabilities differ by 0.0500, and their Review probabilities differ by 0.0500.)
- Leaf A has 25 cases, of which 15 are Class X. Leaf B has 60 cases, of which 42 are Class X. Find the pooled Class X proportion and the unweighted average of the two leaf proportions. (Answer: pooled \(57/85=0.6706\); unweighted average \((0.6000+0.7000)/2=0.6500\).)
- A leaf has 8 cases of Class 1, 8 of Class 2, and 4 of Class 3. Give its probability vector in that order. If the specified tie rule selects Class 2 between the tied largest components, give the predicted label. (Answer: \((0.4000,0.4000,0.2000)\); Class 2.)