Supervised Binning Using Outcome Information

Why Use Outcome Information?

A cut point based only on a predictor’s distribution does not use the outcome that a model is meant to predict. That can be useful when the goal is to describe the predictor, but it may leave groups together even when their observed outcomes differ. Supervised binning uses both the predictor and the outcome to choose candidate boundaries that improve a specified measure of separation in the observed data.

For example, if a numeric predictor is a hypothetical policyholder’s building age and the outcome is whether a claim occurred, a candidate cut may be assessed by how differently the two resulting age groups behave. The cut is not automatically better just because it uses the outcome: a boundary can appear highly predictive by fitting random variation in the sample. The objective is to find useful structure while remaining cautious about the evidence supporting it.

Definition: Supervised binning selects one or more intervals of a predictor using the observed outcome values as well as the predictor values. A candidate cut is evaluated by a stated criterion, such as reduction in within-bin squared error for a numeric outcome or reduction in weighted Gini impurity for a binary outcome. The criterion measures fit in the data used to select the cut; it does not by itself establish predictive performance on new observations.

Candidate Cuts and Split Criteria

For a single cut, first order observations by the predictor \(x\). A candidate boundary belongs between two adjacent distinct predictor values, so each resulting group has a well-defined membership. The boundary may be recorded as an interval split—for instance, values at or below one side versus values above it—rather than as a value that could ambiguously belong to both bins.

The appropriate criterion depends on the outcome. For a numeric outcome \(y\), a common criterion is the total within-bin sum of squared errors (SSE). For a binary outcome, a common criterion is Gini impurity. In either case, a better split has a smaller weighted total criterion than the unsplit parent group.

Numeric-outcome criterion: For a group \(G\) of \(n_G\) observations, define its mean outcome as \(\bar y_G\) and its within-group SSE as \(\sum_{j\in G}(y_j-\bar y_G)^2\). For left and right groups \(L\) and \(R\), the split SSE is \(\operatorname{SSE}(L)+\operatorname{SSE}(R)\). The SSE gain is parent SSE minus split SSE; a positive gain means the two group means fit the observed outcomes better than the parent mean does.

Binary-outcome criterion: If the event proportion in a group is \(p\), its Gini impurity is \(2p(1-p)\). For a split, the weighted child impurity is \((n_L/n)G(L)+(n_R/n)G(R)\), where \(G(L)\) and \(G(R)\) are the child impurities and \(n=n_L+n_R\). Gini improvement is parent impurity minus weighted child impurity. A group containing only one outcome class has impurity zero.

The weights matter: a small child group should not count as much as a large one merely because its impurity is low. The formula weights each child by its fraction of the observations. The chosen criterion also matters. A cut that reduces SSE is judged on numeric response variation; a cut that reduces Gini impurity is judged on binary class composition. These are not interchangeable scores.

Why the SSE Gain Has This Form

The split gain can be understood by decomposing the parent’s variation into within-group variation and variation between the child means. Write \(\bar y\) for the parent mean, and let \(n=n_L+n_R\). For each child \(G\), add and subtract its mean:

\sum_{j\in G}(y_j-\bar y)^2 = \sum_{j\in G}(y_j-\bar y_G)^2+n_G(\bar y_G-\bar y)^2.

The cross-product term vanishes because \(\sum_{j\in G}(y_j-\bar y_G)=0\). Summing across the two children gives the parent SSE as the two child SSEs plus \(n_L(\bar y_L-\bar y)^2+n_R(\bar y_R-\bar y)^2\). Since \(\bar y=(n_L\bar y_L+n_R\bar y_R)/n\), substitution and simplification yield:

\operatorname{SSE}(\text{parent})-\operatorname{SSE}(\text{split}) = \frac{n_Ln_R}{n}(\bar y_L-\bar y_R)^2.

This identity explains why a split has greater SSE gain when the child means are farther apart, while also accounting for group sizes. It does not say that the cut will predict well beyond the observations used to calculate it.

Worked Example: Comparing Numeric-Outcome Cuts

A hypothetical analyst sorts six buildings by age and records the following numeric outcome, a one-year maintenance cost index: \(1,1,4,4,10,10\). Compare cuts after observation 2 and observation 4 using SSE. The method is to calculate the parent SSE, then subtract each split SSE. The trap to avoid is miscalculating squared deviations from the parent mean.

Parent group. Its mean is \((1+1+4+4+10+10)/6=30/6=5\). The deviations are \(-4,-4,-1,-1,5,5\), so:

\operatorname{SSE}(\text{parent}) =2(4^2)+2(1^2)+2(5^2) =32+2+50=84.

Cut after observation 2. The left outcomes are \(1,1\), with mean \(1\) and SSE \(0\). The right outcomes are \(4,4,10,10\), with mean \(28/4=7\). Their squared deviations sum to \(2(4-7)^2+2(10-7)^2=18+18=36\). Split SSE is \(0+36=36\), so the gain is \(84-36=48\).

Cut after observation 4. The left outcomes \(1,1,4,4\) have mean \(10/4=2.5\) and SSE \(2(1-2.5)^2+2(4-2.5)^2=4.5+4.5=9\). The right outcomes \(10,10\) have mean \(10\) and SSE \(0\). Split SSE is \(9\), so the gain is \(84-9=75\).

Among these two cuts, the cut after observation 4 gives the larger training SSE reduction, \(75\) rather than \(48\). The arithmetic checks: parent SSE \(84\) equals split SSE plus gain for either candidate (\(36+48=84\), \(9+75=84\)). A larger observed gain is evidence of a better fit to these outcomes, not proof of better future prediction.

Worked Example: Comparing Binary-Outcome Cuts With Gini Impurity

A hypothetical dataset has six ordered predictor values and binary outcomes \(0,0,0,1,1,1\), where \(1\) denotes an event. Compare a cut after observation 2 with a cut after observation 3. Use weighted Gini impurity. The method is to calculate each group’s event proportion, apply \(2p(1-p)\), and weight by group size. The trap is to use the parent impurity for a child without recalculating that child’s event proportion.

Parent. There are three events among six observations, so \(p=3/6=1/2\). Parent Gini impurity is \(2(1/2)(1/2)=0.5\).

Cut after observation 2. The left outcomes are \(0,0\), so their impurity is \(0\). The right outcomes are \(0,1,1,1\); their event proportion is \(3/4\), giving impurity \(2(3/4)(1/4)=6/16=0.375\). Weighted child impurity is \((2/6)(0)+(4/6)(0.375)=0.25\). Improvement is \(0.5-0.25=0.25\).

Cut after observation 3. The left outcomes are all \(0\), and the right outcomes are all \(1\). Both child impurities are \(0\), so weighted child impurity is \(0\) and improvement is \(0.5-0=0.5\).

The cut after observation 3 has the larger Gini improvement, and its children are pure in this sample. The calculation checks against the definition: pure groups have zero impurity, and the parent impurity is \(0.5\). Purity here describes only these six observations; with so little data, it is not strong evidence that the boundary is reliable.

When Supervised and Unsupervised Cuts Differ

An unsupervised cut is chosen without consulting the response, for example to create bins with similar numbers of predictor observations. A supervised cut can differ because it seeks a boundary aligned with differences in the observed response. The next example compares a two-bin equal-frequency cut with a Gini-guided cut. It also shows why an apparently perfect split should be interpreted cautiously.

Worked Example: Outcome-Guided Cut Versus Equal-Frequency Cut

Eight hypothetical properties have ordered predictor values \(10,20,30,40,50,60,70,80\) and binary outcomes \(0,0,0,1,1,1,1,1\). Compare a two-bin equal-frequency cut after observation 4 with candidate Gini cuts. The method is to calculate the parent impurity and each weighted child impurity. The mistake to avoid is treating a zero training impurity as a guarantee of zero future errors.

The parent has five events and three non-events, so \(p=5/8\) and its Gini impurity is \(2(5/8)(3/8)=30/64=0.46875\).

The equal-frequency cut after observation 4 creates two groups of four. The left group has one event and three non-events, so its impurity is \(2(1/4)(3/4)=0.375\). The right group is all events, so its impurity is \(0\). Weighted child impurity is \((4/8)(0.375)+(4/8)(0)=0.1875\), giving improvement \(0.46875-0.1875=0.28125\).

A supervised cut after observation 3 creates a left group of three non-events and a right group of five events. Both child impurities are \(0\), so weighted child impurity is \(0\) and improvement is \(0.46875\). This exceeds the equal-frequency cut’s improvement by \(0.46875-0.28125=0.1875\). The boundary lies between predictor values \(30\) and \(40\).

The outcome-guided cut separates the observed classes perfectly, but it is based on only eight observations. A new sample could place events on both sides of the boundary. The calculation supports the statement that this cut fits the observed labels better; it does not establish that its apparent separation will persist.

Overfitting, Boundaries, and Interpretation

When many candidate boundaries are examined, the best observed gain can be large partly because one candidate happens to align with random fluctuations. The more possible cuts considered, the more opportunities there are to find an impressive-looking split by chance. This is a form of overfitting: the selected boundary captures details of the observed outcomes that may not recur.

Boundaries can also be unstable. A small change in the observations may move the best cut to a different gap between predictor values, especially when several candidate cuts have similar gains or when few observations lie near the boundary. Such a change can alter group membership substantially even when the overall fit criterion changes little. Therefore, report the chosen boundary and its criterion in context, and distinguish a sample-based finding from a dependable risk distinction.

Outcome-guided binning can be useful when a simple grouped representation is desired and the predictor-outcome relationship has meaningful changes across its range. It is less compelling when the apparent benefit depends on a few observations, when candidate groups are very small, or when the selected cut changes materially under modest data perturbations. A reasonable assessment considers both the gain and how much evidence supports it, and checks predictive performance using observations not used to choose the cut.

Common exam trap: For two events and four non-events, the event proportion is \(2/6\), so parent Gini is \(2(2/6)(4/6)=16/36=0.4444\), not \(8/36=0.2222\). The factor of 2 in \(2p(1-p)\) must be included. For numeric outcomes, likewise calculate every squared deviation: for values \(1,1,3,3,7,7\), the mean is \(22/6=11/3\), and parent SSE is \(2(1-11/3)^2+2(3-11/3)^2+2(7-11/3)^2=112/3\), not \(100/3\). If the split after the fourth observation has SSE \(4\), the gain is \(112/3-4=100/3\), approximately \(33.3333\).

Key takeaway: Supervised binning chooses cuts to improve a stated outcome-based criterion in the observed sample. Calculate gains with the correct group weights and outcome proportions, then interpret them as sample fit—not as a guarantee that the same boundaries or predictive differences will hold elsewhere.

Check Your Understanding

For each problem, identify the criterion and check that the reported gain equals parent criterion minus split criterion.

Four observations are ordered by a predictor and have numeric outcomes \(2,2,5,5\). Calculate parent SSE and the SSE gain from cutting after observation 2. (Answer: Parent SSE \(=9\); split SSE \(=0\); gain \(=9\).)
Binary outcomes in predictor order are \(0,0,1,1\). Calculate parent Gini and the improvement from cutting after observation 2. (Answer: Parent Gini \(=0.5\); weighted child Gini \(=0\); improvement \(=0.5\).)
Binary outcomes are \(0,0,1,1,0,1\). Calculate parent Gini and weighted child Gini for a cut after observation 3. (Answer: Parent Gini \(=0.5\); each child Gini \(=4/9\), weighted child Gini \(=4/9\); improvement \(=1/18\), approximately \(0.0556\).)
For the eight outcomes \(0,0,0,1,1,1,1,1\), compare the equal-frequency cut after observation 4 with the cut after observation 3 using Gini improvement. (Answer: Parent Gini \(=0.46875\); gains are \(0.28125\) after observation 4 and \(0.46875\) after observation 3.)
Six ordered observations have numeric outcomes \(2,4,4,10,10,10\). Find the parent SSE and the SSE gain from cutting after observation 3. (Answer: Parent mean \(=20/3\); parent SSE \(=208/3\); split SSE \(=8/3\); gain \(=200/3\), approximately \(66.6667\).)

What You'll Learn