Factor Predictors In rpart Models

Why Factor Preparation Matters

A tree can use a factor predictor such as payment channel, region, or repair type to divide observations into groups. The fitted split is about the category labels, not the numeric codes R may use internally to store them. That distinction matters both when fitting a model and when applying it to new records: a model trained with one set of factor levels needs a deliberate policy for scoring data with different levels.

In R, a factor records a categorical variable and its permitted levels. Before fitting an rpart model, check that the factor represents the intended categories, that spelling and capitalization are consistent, and that an ordered factor is used only when the order has genuine meaning. A numeric-looking category code should not be left numeric simply because it contains digits.

Definition: A factor predictor is a categorical predictor represented in R by a factor, with a defined set of levels. For an unordered factor, an rpart split assigns a subset of its levels to one child node and the remaining levels to the other. An ordered factor also carries a specified order, which can support splits at boundaries in that order. In either case, a split concerns category membership; the stored integer codes do not represent a quantitative scale.

A reproducible workflow defines the factor levels for the training data and applies the same level definition to prediction data. For example, if the permitted payment channels are “Agent,” “Online,” and “Partner,” use those labels consistently rather than relying on automatic conversion from whichever values happen to occur in a particular data sample.

A level that is permitted but absent from a training sample is not evidence about that category’s relationship with the response. Likewise, a genuinely new category at scoring time cannot be assigned a learned effect merely by giving it an integer code. Decide how to handle such cases as part of data preparation—for example, by mapping them to a justified “Other” category if that category was represented during training, or by flagging them for review. Do not silently treat an unknown label as an ordinary known level.

Preparing Factors for rpart

A reliable preparation step converts category columns explicitly and checks their values before fitting. In R, a factor can be created with a declared level set, for example: train$channel <- factor(train$channel, levels = c("Agent", "Online", "Partner")). Apply that same level set to the scoring data. A factor’s levels are metadata; two columns with the same visible values but different level definitions may not behave as intended in model fitting or prediction.

A factor level is distinct from a missing value. A missing value is not automatically a substantive category such as “Unknown.” If missingness is to be represented as a category, create and use an explicit label consistently; otherwise, handle missing values using a suitable data-preparation policy. Also check whitespace, capitalization, and spelling: “Online” and “online” are different labels.

With unordered factors, do not interpret level order as a ranking. For example, a split that groups “Agent” and “Partner” together does not mean those labels are adjacent on a meaningful scale. For an ordered factor such as severity levels “Low,” “Medium,” and “High,” define the order deliberately. Treating an ordinal scale as unordered discards that ordering; treating a nominal category as ordered invents an ordering that is not justified.

Worked Example: Aligning Training and Scoring Factor Levels

A hypothetical insurer fits a claim classifier using payment channel. The training data contain “Agent,” “Online,” and “Partner.” The fitted tree reports the rule “channel belongs to Agent or Partner” for one branch. A scoring extract contains the rows Online, Partner, and BranchOffice. Prepare the values for prediction and determine which rows satisfy the reported branch rule.

Method. Define the training factor levels explicitly, use the identical level definition for known scoring categories, and check for values outside the permitted set before prediction.

Step 1: State the permitted levels. The fitted model’s factor domain is Agent, Online, Partner. The order in this list is only a storage convention because the factor is unordered.

Step 2: Check each scoring value. Online is a known level, Partner is a known level, and BranchOffice is not a known level. The first two can be represented using the training model’s factor levels. BranchOffice must be flagged or mapped according to a policy established before fitting; it must not be assigned an arbitrary numeric code.

Step 3: Apply the tree rule. The branch condition is membership in the set {Agent, Partner}. Online does not satisfy it, so it follows the complementary branch. Partner satisfies it, so it follows the stated branch. BranchOffice has no valid branch assignment under the fitted rule until the unknown-category policy is applied.

For example, the preparation could use factor(score$channel, levels = c("Agent", "Online", "Partner")) only after checking that every value is in that set. Depending on the modeling interface and R version, an unrecognized level may produce an error or otherwise prevent valid prediction. The robust correction is to detect it explicitly, not to rely on a particular failure mode.

Common mistake avoided. Converting “Agent,” “Online,” and “Partner” to codes 1, 2, and 3 and treating those codes as ordered numbers would imply that Online lies between Agent and Partner. That interpretation is not warranted for an unordered payment-channel factor.

What a Categorical Split Means

For a factor with several levels, a binary tree split forms two groups: a selected subset of levels and its complement. Thus, a condition such as $X \in \{\text{A},\text{B}\}$ sends A and B to one child and every other represented level to the other child. It is not a comparison such as $X < 2$. The exact labels in the fitted rule are the important part to preserve when explaining or implementing the model.

For a classification tree, the split is assessed by how well it separates the response classes according to the tree’s splitting criterion. As a concrete measure, the Gini impurity of a node with $K$ response classes is $1-\sum_{k=1}^{K}p_k^2$, where $p_k$ is the proportion of node observations in class $k$. A candidate split’s weighted child impurity is the sum of each child’s impurity multiplied by its share of the parent observations. The reduction is the parent impurity minus that weighted value.

G = 1 - \sum_{k=1}^{K} p_k^2, \qquad \Delta G = G_{\text{parent}} - \left( \frac{n_L}{n}G_L + \frac{n_R}{n}G_R \right)

Here $n$ is the parent-node count, $n_L$ and $n_R$ are the child counts, and $G_L$ and $G_R$ are their Gini impurities. This calculation describes the Gini criterion; it is not a formula for every possible tree-fitting method or splitting criterion.

Justification: Each child impurity describes the class mixture among observations in that child. A randomly selected parent observation lands in the left child with proportion $n_L/n$ and in the right child with proportion $n_R/n$. Therefore, the average post-split impurity is the weighted sum of the child impurities. Subtracting that average from the parent impurity measures the reduction in impurity caused by the split.

Worked Example: Checking a Factor Split With Gini Impurity

At one node, a hypothetical classification tree considers a four-level factor with levels A, B, C, and D. The observed class counts are: A has 6 Yes and 2 No; B has 4 Yes and 2 No; C has 1 Yes and 5 No; D has 1 Yes and 3 No. The fitted split groups A and B against C and D. Calculate the Gini reduction and interpret the rule.

Method. Add class counts within the parent and each child, calculate each Gini impurity, and weight child impurities by their observation counts.

Step 1: Parent impurity. There are $6+4+1+1=12$ Yes observations and $2+2+5+3=12$ No observations, for $n=24$. The class proportions are $12/24=0.5$ and $12/24=0.5$, so $G_{\text{parent}}=1-(0.5^2+0.5^2)=0.5$.

Step 2: Left child, A or B. The child has $6+4=10$ Yes and $2+2=4$ No, for 14 observations. Its impurity is $1-(10/14)^2-(4/14)^2=1-116/196=80/196=20/49$, approximately 0.4082.

Step 3: Right child, C or D. The child has $1+1=2$ Yes and $5+3=8$ No, for 10 observations. Its impurity is $1-(2/10)^2-(8/10)^2=1-0.04-0.64=0.32$.

Step 4: Weighted impurity and reduction. The weighted child impurity is $(14/24)(20/49)+(10/24)(0.32)=0.238095+0.133333=0.371429$. The reduction is $0.5-0.371429=0.128571$, or $9/70$. Rounded to four decimal places, the reduction is 0.1286.

The rule sends A and B to one child and C and D to the other; the corresponding class counts show that the first child has a larger Yes proportion. The calculation checks the reported split’s impurity reduction. It does not, by itself, prove that this split is the best among all candidate splits. Common mistake avoided: averaging the two child impurities equally would ignore that the children contain 14 and 10 observations, rather than the same number.

Reading and Applying Factor Rules

A fitted tree may display factor splits using a group of labels or a compact internal representation. Interpret the split as the set of levels assigned to one child and the complementary set assigned to the other. Then follow the full path from the root to a terminal node: a prediction rule combines every condition encountered along that path. A category condition at one node does not determine the final prediction unless the rest of the path is also satisfied.

For each terminal node, the fitted model supplies a prediction according to the response type and fitting method. In classification, node class proportions can be used as estimated class probabilities, and the predicted class depends on the model’s classification rule. In regression, the terminal node prediction is a numerical value. The factor condition determines which observations reach the node; it does not change the interpretation of the node prediction itself.

Worked Example: Following Factor Conditions to a Class Probability

A hypothetical fitted classification tree uses an unordered factor, service route, with levels {Broker, Direct, Mobile}, followed on one branch by a tenure split. Its reported rules are: if service route is Broker or Mobile and tenure is less than 2 years, the terminal node contains 3 Yes and 7 No outcomes; if service route is Broker or Mobile and tenure is at least 2 years, the terminal node contains 7 Yes and 2 No outcomes. Use the rules to score a record with service route Mobile and tenure 3 years. Apply a 0.5 probability threshold for predicting Yes.

Method. Check the factor condition first, then the numeric condition, then calculate the terminal node’s Yes proportion and compare it with the stated threshold.

Step 1: Check service route. Mobile belongs to the set {Broker, Mobile}, so the record follows the branch with the tenure condition.

Step 2: Check tenure. The record has 3 years of tenure, and $3 \geq 2$, so it reaches the terminal node with 7 Yes and 2 No outcomes.

Step 3: Calculate the node probability. The node has $7+2=9$ observations. Its estimated Yes proportion is $7/9=0.7778$ to four decimal places.

Step 4: Apply the stated threshold. Since $0.7778 \geq 0.5$, the predicted class is Yes. The factor condition alone would not have been enough: tenure determines which of the two terminal nodes supplies the estimate.

Common mistake avoided. Replacing “Broker or Mobile” with “service route code less than some number” could send a record down the wrong branch if the factor’s internal codes change. Use the category labels in the model rule.

Practical Checks Before Fitting and Scoring

Factor preparation is part of model reliability, not a cosmetic R detail. A careful analyst checks the training and scoring data using the same intended category definition and verifies that transformations have not changed that definition unexpectedly. In particular, dropping unused levels from one data set while retaining them in another can create inconsistent factor metadata. Dropping levels may be appropriate for a specific preparation step, but it should be deliberate and consistent with the model workflow.

An ordered factor requires an additional check: does the declared order express a real progression? If the categories are severity bands, a progression may be meaningful; if they are product names, it usually is not. A numeric variable with values 1, 2, and 3 should be converted to a factor when those numbers are category labels, while a true count or measurement should remain numeric.

Before scoring, inspect for categories that were not represented in the fitted model’s factor levels. A sound process does not guess how an unknown category should behave. It applies a documented mapping if one is justified, routes the case for review, or otherwise follows the deployment policy. This check also helps distinguish an actual new category from a simple data-entry inconsistency.

Confirm that each categorical predictor is represented as a factor rather than an accidental numeric code.
Check spelling, capitalization, whitespace, and missing values before fitting.
Use the same defined levels for training and scoring data.
Interpret an unordered-factor split by its named level set and its complement.
Confirm that any declared order for an ordered factor is meaningful.
Check for unseen scoring categories before requesting predictions.

Key takeaway: An rpart factor split is a rule about category membership. Define factor levels consistently, preserve category meaning between fitting and scoring, and explain the fitted rule using its labels rather than R’s internal codes.

Check Your Understanding

For each problem, identify the factor-level issue or apply the stated rule. Answers are provided for checking.

A fitted model uses an unordered factor with levels {Web, Phone, Desk}. A rule sends {Phone, Desk} to the left child. A scoring record has channel Web. Which child does it enter? (Answer: The right child, because Web is not in {Phone, Desk}.)
A parent node has 8 Yes and 8 No observations. A proposed factor split produces a left child with 6 Yes and 2 No, and a right child with 2 Yes and 6 No. Calculate the parent impurity, each child impurity, and the Gini reduction. (Answer: Parent impurity 0.5000; each child impurity 0.3750; weighted child impurity 0.3750; reduction 0.1250.)
Training levels are {Standard, Express, Courier}. Scoring contains Standard, Courier, and Locker. Which values match known levels, and what should happen before predicting the Locker record? (Answer: Standard and Courier are known; Locker is unseen and should be flagged or handled by a documented policy before prediction.)
A tree rule sends {North, East} to a terminal node with 5 positive and 15 negative outcomes. A new record has region East. Calculate the node’s estimated positive proportion and classify using a 0.30 threshold for positive. (Answer: $5/(5+15)=0.2500$; classify as negative because $0.2500 < 0.30$.)
A field takes the labels Bronze, Silver, and Gold, but the labels are product names with no guaranteed ranking. Should it be encoded as an ordered factor, and why? (Answer: No; use an unordered factor because the labels do not define a meaningful order.)

What You'll Learn