From Tree Predictions to a Forest Class
A classification tree assigns a class to a case by following its predictor values to a terminal node. A classification forest combines the predictions of many trees, but there is more than one way to perform that combination. Majority voting counts each tree’s assigned class once. Probability averaging combines the class probabilities estimated by the trees, then assigns the class with the largest average probability. These methods can produce different predictions for the same case.
This distinction matters when interpreting a forest’s output. A reported class is not enough to tell you whether the forest used votes or averaged probabilities. You need to know the aggregation rule, and you need the tree-level outputs to reproduce the result.
For \(K\) possible classes, define \(I(\hat{C}_b=k)\) to be 1 when tree \(b\) predicts class \(k\), and 0 otherwise. The vote count for class \(k\) is \(V_k=\sum_{b=1}^{B} I(\hat{C}_b=k)\). The vote proportion is \(V_k/B\). Both majority voting and probability averaging are commonly described as forest aggregation, but they aggregate different quantities: assigned classes in the first case and estimated probabilities in the second.
The symbol \(\operatorname*{arg\,max}\) means “the class or classes attaining the largest value.” If exactly one class has the largest value, the prediction is unique. If multiple classes tie, the aggregation rule alone does not choose between them; the implementation’s tie policy must be checked or the tie reported explicitly.
Why the Two Rules Are Different
A hard class prediction retains only which class a tree chose. It discards how strongly that tree preferred the class. By contrast, a class-probability vector retains the tree’s relative support for all classes. A tree that assigns a class probability of 0.51 and a tree that assigns 0.99 both cast one vote for that class, but they contribute different amounts to the averaged probability.
The connection between vote counts and vote proportions follows directly from the definition. Since every tree contributes either 0 or 1 to a given class count, summing those indicators counts exactly the trees that chose the class. Dividing by the same positive number \(B\) for every class preserves the ordering of the counts.
Thus, reporting vote proportions rather than counts cannot change a majority-vote prediction. Probability averaging is different: it averages \(\hat{p}_{bk}\), not the indicator \(I(\hat{C}_b=k)\). In general, the ordering of the sums of probabilities need not match the ordering of the sums of indicators.
Worked Example: Calculating a Majority-Vote Prediction
A hypothetical classification forest contains nine trees and predicts one of three outcomes for a new application: Accept, Review, or Decline. The trees predict, in order: Accept, Review, Accept, Decline, Accept, Review, Accept, Decline, Accept. Find the vote counts, vote proportions, and forest class under unweighted majority voting.
Method. Count the assigned class for each tree; do not infer probabilities that were not supplied. There are \(B=9\) trees.
Accept: Trees 1, 3, 5, 7, and 9 predict Accept, so \(V_{\mathrm{Accept}}=5\). Its vote proportion is \(5/9=0.5556\) to four decimal places.
Review: Trees 2 and 6 predict Review, so \(V_{\mathrm{Review}}=2\). Its vote proportion is \(2/9=0.2222\) to four decimal places.
Decline: Trees 4 and 8 predict Decline, so \(V_{\mathrm{Decline}}=2\). Its vote proportion is \(2/9=0.2222\) to four decimal places.
The counts sum to \(5+2+2=9\), and the proportions sum to \(0.5556+0.2222+0.2222=1.0000\) after rounding. Accept has the unique largest vote count, so the majority-vote prediction is Accept. The common mistake avoided here is treating vote proportions as class probabilities: they describe the fraction of trees choosing a class, not the trees’ estimated probabilities for that class.
Probability Averaging Can Change the Winner
A tree may estimate class probabilities from the class composition of its terminal node. The tree’s assigned class is typically the class with the largest estimated probability, subject to any specified tie rule. The forest can then average the probability for each class across trees. The following examples use explicitly supplied probability vectors, so both aggregation methods can be calculated and compared.
Worked Example: Majority Vote and Probability Averaging Disagree
A hypothetical binary classifier has five trees. Each tree reports its estimated probability of Event, with the probability of No Event equal to one minus that value. The Event probabilities are 0.49, 0.49, 0.49, 0.99, and 0.99. Each tree predicts the class with the larger probability. Find both forest predictions.
Step 1: Determine each tree’s class. The first three trees have Event probability 0.49 and No Event probability \(1-0.49=0.51\), so each predicts No Event. The last two have Event probability 0.99 and No Event probability \(1-0.99=0.01\), so each predicts Event.
Step 2: Apply majority voting. No Event receives 3 votes and Event receives 2 votes. The vote proportions are \(3/5=0.60\) for No Event and \(2/5=0.40\) for Event. Majority voting therefore predicts No Event.
Step 3: Average the Event probabilities. Their sum is \(0.49+0.49+0.49+0.99+0.99=3.45\). The average is \(3.45/5=0.69\).
Step 4: Average the No Event probabilities. Their sum is \(0.51+0.51+0.51+0.01+0.01=1.55\). The average is \(1.55/5=0.31\). The two averages sum to 1.00. Since \(0.69>0.31\), probability averaging predicts Event.
The methods disagree: majority voting predicts No Event, while probability averaging predicts Event. Each tree has one vote under the first rule, but the two Event-predicting trees contribute much larger Event probabilities under the second rule. The common mistake avoided is assuming that the class with the most tree votes must also have the largest average probability.
Worked Example: Comparing Aggregation Rules for Three Classes
A hypothetical forest has five trees and three classes, A, B, and C. The trees’ probability vectors, in A–B–C order, are \((0.40,0.35,0.25)\), \((0.40,0.35,0.25)\), \((0.40,0.35,0.25)\), \((0.05,0.90,0.05)\), and \((0.05,0.90,0.05)\). Find the tree votes and both forest predictions.
Step 1: Check the tree-level probabilities and predictions. Each vector sums to 1.00. The first three trees predict A because \(0.40\) is their largest class probability. The final two predict B because \(0.90\) is their largest class probability. No tree predicts C.
Step 2: Calculate hard votes. A receives 3 votes, B receives 2, and C receives 0. The counts sum to 5. The majority-vote prediction is therefore A.
Step 3: Sum the probabilities class by class. For A, the sum is \(3(0.40)+2(0.05)=1.20+0.10=1.30\). For B, it is \(3(0.35)+2(0.90)=1.05+1.80=2.85\). For C, it is \(3(0.25)+2(0.05)=0.75+0.10=0.85\).
Step 4: Divide each sum by five. The averaged probabilities are \(\bar{p}_{A}=1.30/5=0.26\), \(\bar{p}_{B}=2.85/5=0.57\), and \(\bar{p}_{C}=0.85/5=0.17\). They sum to \(0.26+0.57+0.17=1.00\). The largest average is for B, so probability averaging predicts B.
The calculations are consistent: three trees vote for A, while the average probability is largest for B. The common mistake avoided is averaging only the probabilities of the classes each tree selected; the rule requires averaging each class’s probability across all trees.
Ties, Interpretation, and Exam Traps
A tie can occur under either aggregation rule. With an even number of trees, two classes might receive the same largest vote count. Averaged probabilities can also tie, including in a multiclass problem. When a tie occurs, state which quantities tie and avoid claiming a unique prediction unless a tie-breaking rule is supplied. A fitted implementation may apply a rule, but the rule can depend on the software and settings.
The standard calculations above assume each tree has equal weight. If a problem explicitly specifies tree weights, use the stated weighted aggregation instead; do not silently replace an unweighted formula with a weighted one. Likewise, a class decision rule that uses a specified probability cutoff or a different loss-based threshold is not the same as choosing the largest class probability. Follow the rule given in the problem.
Use class labels to calculate hard votes. Use class-probability vectors to calculate averaged probabilities. If both are supplied, calculate each method separately.
For voting, count one assigned class per tree. For probability averaging, sum each class’s probability across all trees, then divide by the number of trees.
Vote counts should sum to the number of trees. Vote proportions should sum to 1. Averaged class probabilities should also sum to 1, subject to rounding.
Name the aggregation method and identify the winning class, or report that the maximum is tied if no tie-breaking rule is provided.
A frequent exam trap is to confuse a vote proportion with an average class probability. For example, if four of seven trees predict a class, \(4/7\) is its vote proportion. It is not necessarily the forest’s averaged probability for that class: the trees might report probabilities close to 0.51 or close to 1.00, and those values lead to different averages. Another trap is to calculate an average probability for only the trees that voted for one class. Probability averaging uses that class’s probability from every tree, including trees whose winning class was different.
Check Your Understanding
For each problem, state the requested forest prediction and show enough intermediate arithmetic to distinguish the aggregation rule.
- A seven-tree forest predicts Red, Blue, Red, Green, Red, Blue, and Red. Find the majority-vote class and the vote proportion for each class. (Answer: Red; Red \(4/7\), Blue \(2/7\), Green \(1/7\).)
- Three binary trees report Event probabilities 0.52, 0.52, and 0.02. Each tree predicts its larger-probability class. Find the majority-vote and probability-averaging predictions. (Answer: Majority vote predicts Event; the average Event probability is \(1.06/3=0.3533\), so probability averaging predicts No Event.)
- Four trees predict classes A, A, B, and B. What is the majority-vote result if no tie-breaking rule is specified? (Answer: A and B tie at two votes each; there is no unique majority-vote class.)
- Three trees have probability vectors in X–Y order: \((0.70,0.30)\), \((0.40,0.60)\), and \((0.40,0.60)\). Find the vote counts and the averaged probabilities. (Answer: X receives one vote and Y receives two, so majority voting predicts Y; average probabilities are X \(1.50/3=0.50\) and Y \(1.50/3=0.50\), so probability averaging is tied.)
- Five trees report probabilities for class Green of 0.20, 0.20, 0.80, 0.80, and 0.80. Each predicts Green when its Green probability exceeds 0.50 and Blue otherwise. Find the Green vote proportion and average Green probability. (Answer: Green receives \(3/5=0.60\) of the votes; its average probability is \(2.80/5=0.56\).)