Fitting A Classification Forest With Ranger

From a Categorical Outcome to a Classification Forest

A classification forest predicts a category, such as whether a hypothetical insurance application is referred for review. In R, a critical first step is to represent that outcome as a factor. If a binary outcome is stored as the numbers 0 and 1, software may treat it as a numeric response and fit a regression forest instead of a classification forest. The values might look similar, but the prediction task is different.

The R package ranger fits random forests efficiently. For classification, its usual predictions are class labels. With probability estimation requested, predictions are class probabilities instead. These outputs answer related but distinct questions: which class is selected, and how much probability the model assigns to each class?

Definition: A classification forest predicts a categorical response by combining predictions from trees. In ranger, a factor response identifies a classification task. With the default classification prediction, the forest returns a predicted class; with probability = TRUE, it returns a probability for each response class.

Fitting a Classification Forest in ranger

Suppose each row represents one hypothetical application. The response, review, has two possible values, “No” and “Yes.” The predictors are numeric summaries calculated before an application decision. A minimal fit is:

library(ranger)
applications$review <- factor(applications$review, levels = c("No", "Yes"))
fit_class <- ranger(review ~ ., data = applications, num.trees = 500, mtry = 2, seed = 41)

The formula review ~ . uses review as the response and the other columns in applications as predictors. The explicit factor levels set the intended class order. The arguments shown request 500 trees, use two candidate predictors at each split, and set a seed for reproducibility. Here, mtry is supplied as a model setting; selecting its value is a separate modelling decision.

A useful first check is that the response really is a factor and that the data contain the intended labels. For example, if a data frame has 12 rows, of which 9 have “No” and 3 have “Yes,” then its observed class proportions are $9/12=0.75$ and $3/12=0.25$. Those proportions describe the data; they do not tell us whether the fitted forest will predict every case correctly.

For a standard classification fit, ranger can return out-of-bag predictions and a classification error estimate. The exact results depend on the data and the fitted forest, so inspect the object rather than assuming that the fit succeeded as intended. In particular, check the response type, the available prediction fields, and whether the returned predictions have the form expected for a classification model.

Worked Example: Setting Up a Binary Response

A hypothetical team has 8 completed application records. Six are labelled “No” and two are labelled “Yes” for referral. The data frame contains three predictor columns: age_band_score, document_count, and prior_contact. Prepare the response and write a ranger fit using two candidate predictors per split and 300 trees.

Step 1: Check the response counts. There are 8 records in total, with 6 “No” outcomes and 2 “Yes” outcomes. The observed proportions are $6/8=0.75$ for “No” and $2/8=0.25$ for “Yes.”

Step 2: Set the response type and order. Use factor(applications$review, levels = c("No", "Yes")). The response is now categorical, and the declared order makes the intended labels explicit.

Step 3: Fit the forest. Use ranger(review ~ ., data = applications, num.trees = 300, mtry = 2, seed = 19). The dot includes the three predictor columns, while review remains the response. The specified value of mtry is a model input, not a claim that it is optimal.

Step 4: Check the fit. Confirm that applications$review is a factor with levels “No” and “Yes.” Then inspect fit_class$prediction.error and the returned out-of-bag predictions, if present. No numerical error can be deduced from the class counts alone: it depends on the predictions for the individual records.

Method and trap avoided. The method is to encode the response as a factor before calling ranger. A common mistake is to leave the response as 0 and 1 and assume that the software will necessarily interpret it as classification. The factor check prevents that ambiguity.

Predicted Classes and Class Probabilities

To predict for new records, pass a data frame containing the predictor columns used by the fitted model. Do not include the response column as a predictor. For a standard classification forest, use predict(fit_class, data = newdata)$predictions. The result contains one predicted class for each row of newdata.

To obtain probabilities, fit a probability forest by setting probability = TRUE, then predict with that fit. The predictions are arranged by record and response class. Check the returned column names rather than relying on an assumed column order. Each row should give a probability for each class, and the row probabilities should sum to 1, up to numerical precision.

A probability prediction is not itself a class label. If a decision requires a class, a rule must map probabilities to classes. A common binary rule selects “Yes” when its predicted probability is at least 0.5, but that threshold is a decision choice, not a universal property of classification forests. The threshold should match the decision context.

Worked Example: Reading Probability Predictions

A probability forest is used to score three hypothetical applications. Assume the returned probability matrix has columns “No” and “Yes,” in that order, and contains these rows: application A, (0.72, 0.28); application B, (0.43, 0.57); application C, (0.50, 0.50). Find the predicted class for each row using a rule that predicts “Yes” when its probability is at least 0.55, and check the row totals.

Step 1: Check the probability totals. For A, $0.72+0.28=1.00$. For B, $0.43+0.57=1.00$. For C, $0.50+0.50=1.00$. All three rows satisfy the probability-sum check.

Step 2: Apply the stated decision rule. A has $P(\text{Yes})=0.28$, which is below 0.55, so its predicted class is “No.” B has $P(\text{Yes})=0.57$, which is at least 0.55, so its predicted class is “Yes.” C has $P(\text{Yes})=0.50$, below 0.55, so its predicted class is “No.”

Step 3: Report the results. The predicted classes are A: “No,” B: “Yes,” and C: “No.” The probabilities remain useful alongside those labels: application C is tied at 0.50 for each class, even though the stated rule assigns it to “No.”

Method and trap avoided. The method is to inspect the “Yes” probability and compare it with the specified threshold. Do not silently replace the requested threshold with 0.5, or treat the first probability column as “Yes” without checking its name.

Out-of-Bag Error and Its Calculation

In a bootstrap forest, each tree is fitted using a bootstrap sample. A training record that was not selected for a particular tree is out of bag for that tree. The forest can use only the trees for which a record was out of bag to produce an out-of-bag (OOB) prediction for that record. This gives a training-data-based estimate of classification error without predicting each record using every tree that trained on it.

Definition: For $n$ records with observed classes $y_1,\ldots,y_n$ and OOB predicted classes $\hat{y}_1^{\mathrm{OOB}},\ldots,\hat{y}_n^{\mathrm{OOB}}$, the OOB misclassification error is the fraction of records whose OOB predicted class differs from the observed class:

\widehat{E}_{\mathrm{OOB}} = \frac{1}{n}\sum_{i=1}^{n} I\!\left(\hat{y}_i^{\mathrm{OOB}}\ne y_i\right).

Here, $I(\cdot)$ equals 1 when the condition inside it is true and 0 otherwise. The formula follows by assigning one error indicator to each record: a correctly classified record contributes 0, and an incorrectly classified record contributes 1. Summing the indicators counts the errors, and dividing by $n$ gives the error proportion. Thus, for an OOB error of 0.20, the OOB predictions misclassify 20% of the records.

For a standard classification fit, fit_class$prediction.error reports the classification prediction error, and the fit can also provide OOB predictions and a confusion matrix. Verify the object fields for the fitted model. For a probability forest, the error reported by ranger is a probability-based error measure rather than the same class-misclassification fraction. Do not interpret those two reported quantities as interchangeable.

Worked Example: Calculating OOB Misclassification Error

A standard classification forest is fitted to 10 hypothetical training records. The observed and OOB-predicted classes produce this confusion matrix, with actual classes in rows and predicted classes in columns:

Actual class	Predicted No	Predicted Yes
No	5	1
Yes	2	2

Calculate the OOB misclassification error and accuracy.

Step 1: Count the records. The “No” row contains $5+1=6$ records. The “Yes” row contains $2+2=4$ records. Therefore, $n=6+4=10$.

Step 2: Count errors. The off-diagonal entries are errors: 1 actual “No” record was predicted “Yes,” and 2 actual “Yes” records were predicted “No.” The total error count is $1+2=3$.

Step 3: Divide by the total. The OOB misclassification error is $3/10=0.30$, or 30%. The number correctly classified is the diagonal total, $5+2=7$, so accuracy is $7/10=0.70$, or 70%. The two proportions sum to 1.

Method and trap avoided. The method is to use the off-diagonal entries for error and divide by all records. A common mistake is to divide the three errors by the number of “Yes” records or by the number of correct predictions. Neither denominator gives the overall OOB error.

Using ranger Outputs Carefully

The following sequence keeps the response, prediction type, and error interpretation aligned.

Prepare the response.
Convert the categorical outcome to a factor and specify its levels explicitly.

Fit the intended model.
Use the factor response in the ranger formula and choose whether the fit is for class predictions or probability predictions.

Predict and inspect.
Supply only the required predictors, then inspect the prediction structure and class or probability labels.

Interpret the error on its own scale.
For a standard classification fit, interpret the reported OOB error as misclassification error; for a probability fit, do not assume it is that same quantity.

A useful cross-check for class predictions is to compare the number of mismatches between the observed classes and the OOB predicted classes with the reported error. If the mismatch count is $e$ among $n$ records, the error should be $e/n$, subject to any records that lack an OOB prediction. Confirm the number of usable predictions before making that comparison.

OOB error is informative because each record's OOB prediction is formed without using that record to fit the contributing trees. It is not a guarantee of performance on every future population. Differences in the data source, outcome definition, predictor availability, or class prevalence can affect how well the fitted forest generalizes. For consequential decisions, report the error's meaning and the evaluation setting, rather than presenting a single number without context.

Common Mistakes

Leaving the outcome numeric. A 0/1 response may be treated as numeric. Convert a categorical response to a factor when fitting a classification forest.
Confusing classes and probabilities. A class prediction is a label; a probability prediction is a set of class probabilities. Use the output that matches the question.
Assuming a threshold. A probability does not uniquely determine a class until a decision rule is specified. State the threshold being used.
Ignoring class-column names. Confirm which probability column corresponds to each factor level before interpreting or thresholding values.
Calling every reported error “misclassification error.” A probability forest reports a probability-based error measure, so distinguish it from the fraction of incorrect class labels.

Key takeaway: For a ranger classification forest, make the outcome a factor, choose class or probability predictions deliberately, and interpret the reported OOB error according to the type of fit.

Check Your Understanding

For each problem, identify the appropriate ranger output or calculation before checking the answer.

A data frame has 20 records, with 15 “Retain” outcomes and 5 “Close” outcomes stored as character strings. What response conversion and factor level order could be used to fit a classification forest? (Answer: Convert the response to a factor, for example with levels = c("Retain", "Close"); the observed proportions are 15/20 = 0.75 and 5/20 = 0.25.)
A two-class probability prediction has columns “Pass” and “Review,” with one row equal to (0.38, 0.62). Under a rule predicting “Review” when its probability is at least 0.60, what class is assigned, and what is the row total? (Answer: “Review”; the row total is 0.38 + 0.62 = 1.00.)
An OOB confusion matrix has 18 correct predictions and 6 incorrect predictions across all records. Find the OOB misclassification error and accuracy. (Answer: Error = 6/24 = 0.25; accuracy = 18/24 = 0.75.)
A probability forest reports class probabilities of 0.48 for “Accept” and 0.52 for “Refer.” A decision rule assigns “Refer” only when its probability is at least 0.55. What class does the rule assign, and why is that not determined by the largest probability alone? (Answer: “Accept”; the “Refer” probability of 0.52 is below the specified 0.55 threshold, even though it is the larger probability.)
A standard classification fit has 50 usable OOB predictions, of which 11 disagree with the observed classes. What error should the OOB predictions imply, and what should be checked if the fit reports a different value? (Answer: 11/50 = 0.22; check that the same 50 usable records and OOB predictions were used and that the reported quantity is classification misclassification error.)

What You'll Learn