Categorical Predictors In Gradient-Boosting Implementations

Why Categorical Predictors Need Special Attention

A predictor such as policy channel may have values like “web,” “agent,” and “branch.” These labels identify groups; they do not have numerical distances or a natural order. A tree-based boosting implementation must still decide how to represent those groups when it searches for a split. The choice is not merely a data-storage detail: it can change which splits the algorithm considers and, in turn, the fitted trees.

In R, the correct preparation depends on the implementation and its version. The formula interface to gbm commonly expands factor predictors into indicator columns. Recent versions of xgboost can support categorical features natively when factors are supplied in a compatible data frame and categorical support is enabled. Other workflows, including older versions or numeric-matrix inputs, require explicit indicator encoding. Check the installed package’s documentation and the exact input type rather than assuming all boosting implementations handle factors alike.

How Common R Workflows Represent Categories

With the formula interface, gbm can accept a factor predictor in the data frame. Its formula processing represents categorical predictors through contrasts, typically using treatment contrasts: for a factor with \(K\) observed levels, the design matrix has \(K-1\) indicator columns when an intercept is included. The intercept is not a useful tree predictor, so formula processing handles the model matrix for fitting. In contrast, lower-level fitting interfaces expect numeric predictor inputs; prepare those inputs explicitly.

A typical formula workflow is:

For xgboost, recent R interfaces support categorical predictors with factors in a data-frame input when the data matrix is created with categorical support enabled. A common pattern is to create an xgb.DMatrix using enable_categorical = TRUE and fit with a compatible tree method, such as hist. A sparse matrix or a plain numeric matrix generally does not preserve factor metadata; in those workflows, create indicator columns instead. Native support and accepted input forms are version-dependent, so verify them for the installed release.

When using indicators, one common choice is treatment encoding: retain \(K-1\) columns and choose a reference level. Full one-hot encoding uses \(K\) columns, one for every level. Both can be useful for trees, but neither should be confused with assigning arbitrary integer labels. The key question is what split structures the representation permits.

Why Encoding Changes the Splits a Tree Can Use

For a numeric predictor, a standard tree split compares the feature with a threshold. For an indicator column, the useful split is usually zero versus one. A tree can combine such splits across levels, but the required tree structure may differ from a category-aware split that directly groups several levels together.

For squared-error regression, the split criterion can be understood through residual sums of squares. At a particular boosting step, let \(r_i\) be the current residual for observation \(i\). For a candidate split, calculate the mean residual in each child node and the sum of squared deviations from those child means. The reduction in residual sum of squares is the parent-node sum minus the two child-node sums. A larger reduction is preferable, before any implementation-specific penalties or constraints are applied.

A native categorical split may group a subset of levels against the remaining levels, subject to the implementation’s rules and regularization. Indicator encoding makes individual levels easy to isolate, while grouping several levels can require more than one split. Integer coding is riskier: a threshold on codes can only separate adjacent values in the arbitrary code order. It can therefore exclude a useful grouping entirely.

Native Categorical Splits and Indicator Splits

Native categorical support can consider category groupings more directly than a single split on one indicator. The exact search is implementation-specific: some versions or settings use one-category-versus-rest splits at lower cardinality, while other cases can use category partitions. Controls such as category cardinality thresholds can influence the split strategy. Do not assume that every version evaluates every possible partition or will choose the same split as another implementation.

Indicator encoding remains a practical, widely applicable approach, particularly when the implementation requires numeric inputs. But the representation affects tree depth and the sequence of splits needed to express a category grouping. When interpreting fitted splits, report the encoding and reference level. Two models trained on the same rows can have different candidate splits simply because their categorical inputs were prepared differently.

R Preparation and Level Consistency

Whichever implementation is used, training and prediction data need a consistent categorical schema. In R, a factor stores its levels, and those levels can affect both the design matrix and the treatment reference. Establish the intended levels from the training design, then apply the same schema to validation and future data. Do not silently assign different integer codes or reference levels in separate data sets.

For example, converting a factor with as.numeric() returns its internal integer codes, not the displayed labels and not a set of indicators. If these codes are passed as a numeric predictor, the model may split on their arbitrary ordering. Similarly, an unseen prediction-time level should not be allowed to become an accidental missing value without a deliberate plan. A common solution is to combine rare or future categories into a training-defined “other” level before fitting, provided that this is substantively appropriate. Any such rule must be applied consistently.

These choices matter for actuarial prediction because categories often represent channels, product types, regions, or administrative groups. Encoding can affect which groupings a tree can express, how deep a tree must be to express them, and how predictions behave when data contain unfamiliar levels. A sound workflow records the package version, input type, factor levels, contrasts, and any category pooling used. This tutorial is aligned with the SOA Exam PA (Predictive Analytics) syllabus.