A Hand Calculation of Gini Split Gain

Why Hand-Calculate a Gini Split?

A classification tree considers candidate partitions of a node and measures how much each partition reduces class impurity. For a small data set, calculating the values directly is a useful way to check whether a proposed split is better than its alternatives. The key is to use the class counts in every child and to weight each child’s impurity by the fraction of observations it contains.

Suppose a hypothetical insurer is exploring whether a policy attribute can separate applications into two groups with different rates of a binary outcome, such as claim or no claim. The calculations below describe only these hypothetical data; they do not imply that a particular attribute predicts claims in practice.

Definition: For a node containing class proportions \(p_1,\ldots,p_K\), its Gini impurity is \(G=1-\sum_{k=1}^{K}p_k^2\). For a binary-labelled node with \(n_1\) observations in class 1 and \(n_0\) in class 0, \(p_1=n_1/(n_1+n_0)\), \(p_0=n_0/(n_1+n_0)\), and \(G=1-p_1^2-p_0^2=2p_1p_0\).

Gini impurity is zero when a node contains only one class and is largest for a binary node when the classes are equally represented. A candidate split sends observations from a parent node into child nodes. The post-split impurity is the size-weighted average of the child impurities, and the Gini gain is the parent impurity minus that weighted average.

Key result: If a parent has \(n\) observations and a candidate split creates children \(j=1,\ldots,J\), with \(n_j\) observations and Gini impurity \(G_j\), then

G_{\text{after}}=\sum_{j=1}^{J}\frac{n_j}{n}G_j, \qquad \text{Gini gain}=G_{\text{parent}}-G_{\text{after}}.

A larger gain means a greater reduction in training-set Gini impurity among the candidates being compared.

Why Child Impurities Must Be Weighted

The weight \(n_j/n\) is the fraction of the parent’s observations sent to child \(j\). To see why it belongs in the calculation, select one observation at random from the parent. The probability that it came from child \(j\) is \(n_j/n\). The average impurity experienced across the children must therefore average each child’s impurity using that probability, rather than giving each child the same influence regardless of size.

Equivalently, if a child’s impurity is written as one minus the sum of its squared class proportions, weighting that impurity by \(n_j/n\) makes the total post-split impurity an average over observations in the parent. This is why a child containing one observation cannot count as much as a child containing ten.

G_{\text{after}} =\sum_{j=1}^{J}\frac{n_j}{n} \left(1-\sum_{k=1}^{K}p_{jk}^{\,2}\right),

Here \(p_{jk}\) is the proportion of class \(k\) in child \(j\). For a binary outcome, each child’s impurity can be calculated from its two class proportions or from \(G_j=2p_{j1}p_{j0}\). These forms are equivalent because \(p_{j1}+p_{j0}=1\).

Worked Comparisons

Worked Example: Comparing Two Splits of Ten Applications

In a hypothetical training sample of 10 applications, 6 have the positive outcome and 4 have the negative outcome. Candidate split A produces children with counts \((4,1)\) and \((2,3)\), where each pair lists positive and negative observations. Candidate split B produces children with counts \((3,2)\) and \((3,2)\). Calculate the parent impurity, each split’s weighted post-split impurity and gain, then identify the better candidate by Gini gain.

Method: calculate from class counts, then weight by child size. The parent class proportions are \(6/10=0.6\) and \(4/10=0.4\), so

G_{\text{parent}} =1-(0.6)^2-(0.4)^2 =1-0.36-0.16 =0.48.

For split A, the first child has 5 observations. Its class proportions are \(4/5\) and \(1/5\), giving impurity \(1-(4/5)^2-(1/5)^2=1-16/25-1/25=8/25=0.32\). The second child also has 5 observations; its proportions are \(2/5\) and \(3/5\), giving impurity \(1-4/25-9/25=12/25=0.48\). Thus

G_{\text{after,A}} =\frac{5}{10}(0.32)+\frac{5}{10}(0.48) =0.16+0.24 =0.40, \qquad \text{gain}_A=0.48-0.40=0.08.

For split B, each child has proportions \(3/5\) and \(2/5\), so each has impurity \(1-9/25-4/25=12/25=0.48\). The weighted post-split impurity is \((5/10)(0.48)+(5/10)(0.48)=0.48\), and the gain is \(0.48-0.48=0\). Split A is preferred: it reduces training-set impurity by 0.08, while split B does not reduce it.

Check. In both candidates, the child counts add back to 6 positive and 4 negative observations. The weighted impurity for A, 0.40, is below the parent impurity of 0.48; for B it equals the parent impurity. The common mistake avoided here is assuming that any split creates a gain.

Worked Example: A More Uneven Pair of Candidate Splits

A hypothetical node contains 12 records: 5 with the positive label and 7 with the negative label. Candidate split A produces children \((2,2)\) and \((3,5)\). Candidate split B produces children \((4,2)\) and \((1,5)\). Calculate the gain of each candidate and choose the larger one.

Parent. The proportions are \(5/12\) and \(7/12\), so

G_{\text{parent}} =1-\left(\frac{5}{12}\right)^2-\left(\frac{7}{12}\right)^2 =\frac{144-25-49}{144} =\frac{70}{144} =\frac{35}{72} \approx 0.4861.

Candidate A. The first child has 4 observations, split evenly between the labels, so \(G_1=1-(1/2)^2-(1/2)^2=1/2\). The second child has 8 observations, with proportions \(3/8\) and \(5/8\), so \(G_2=1-9/64-25/64=30/64=15/32\). Weight by child sizes:

G_{\text{after,A}} =\frac{4}{12}\left(\frac{1}{2}\right) +\frac{8}{12}\left(\frac{15}{32}\right) =\frac{1}{6}+\frac{5}{16} =\frac{23}{48} \approx 0.4792.

Therefore, \(\text{gain}_A=35/72-23/48\). With denominator 144, this is \(70/144-69/144=1/144\approx 0.0069\).

Candidate B. The first child has 6 observations, with proportions \(4/6\) and \(2/6\). Its impurity is \(1-16/36-4/36=16/36=4/9\). The second child has proportions \(1/6\) and \(5/6\), so its impurity is \(1-1/36-25/36=10/36=5/18\). The children have equal sizes, giving

G_{\text{after,B}} =\frac{6}{12}\left(\frac{4}{9}\right) +\frac{6}{12}\left(\frac{5}{18}\right) =\frac{2}{9}+\frac{5}{36} =\frac{13}{36} \approx 0.3611.

Thus \(\text{gain}_B=35/72-13/36=35/72-26/72=9/72=1/8=0.1250\). Candidate B has the greater gain.

Check. Candidate A’s children contain \(2+3=5\) positive and \(2+5=7\) negative observations; candidate B’s children also contain \(4+1=5\) positive and \(2+5=7\) negative observations. Candidate B’s weighted impurity is lower than A’s, consistent with its larger gain. The method avoids judging a split by one child alone: both children contribute to the post-split value.

Worked Example: A Pure Small Child and a Larger Mixed Child

A hypothetical node has 15 records, with 9 positive and 6 negative labels. One candidate split creates a child with \((5,0)\) and another with \((4,6)\). A competing split creates children with \((6,1)\) and \((3,5)\). Calculate both gains and compare them.

Parent. The proportions are \(9/15=0.6\) and \(6/15=0.4\), giving \(G_{\text{parent}}=1-0.36-0.16=0.48\).

First candidate. The pure child has 5 observations and impurity 0. The other child has 10 observations, with proportions \(4/10\) and \(6/10\), so its impurity is \(1-0.16-0.36=0.48\). Hence \(G_{\text{after}}=(5/15)(0)+(10/15)(0.48)=0.32\), and the gain is \(0.48-0.32=0.16\).

Second candidate. The child with 7 observations has proportions \(6/7\) and \(1/7\), so its impurity is \(1-36/49-1/49=12/49\). The child with 8 observations has proportions \(3/8\) and \(5/8\), giving impurity \(1-9/64-25/64=30/64=15/32\). Therefore,

G_{\text{after}} =\frac{7}{15}\left(\frac{12}{49}\right) +\frac{8}{15}\left(\frac{15}{32}\right) =\frac{4}{35}+\frac{1}{4} =\frac{51}{140} \approx 0.3643.

The second candidate’s gain is \(0.48-51/140=12/25-51/140=81/700\approx 0.1157\). The first candidate’s gain, 0.16, is larger. Both partitions preserve the parent totals: each has 9 positive and 6 negative observations.

Check. The first candidate does not win merely because it makes a pure child. Its larger child remains mixed, so its impurity must still be included with weight \(10/15\). The calculation shows that, for these counts, the pure child and the other child together yield the larger reduction.

Common Calculation Traps

A frequent error is to take the ordinary, unweighted average of the child impurities. That calculation is correct only when the children have equal sizes. For example, in the 15-record example, the first candidate’s child impurities are 0 and 0.48. Averaging them equally gives 0.24, but the children contain 5 and 10 records, so the correct post-split impurity is \((5/15)(0)+(10/15)(0.48)=0.32\). The incorrect value would overstate the gain as \(0.48-0.24=0.24\), rather than the correct 0.16.

Another trap is to compute the impurity of each child from the parent’s proportions. A split changes the class proportions in each child, so each \(G_j\) must use that child’s own class counts. Also check that all child counts sum to the parent counts before comparing candidate gains. A mismatch indicates that an observation or class count has been omitted or assigned incorrectly.

Practical sequence: Record the parent’s two class counts; calculate its Gini impurity; for every candidate, record each child’s counts, size, and impurity; weight each impurity by \(n_j/n\); subtract the weighted total from the parent impurity; then compare gains. Keep unrounded fractions or sufficient precision until the final comparison.

This hand calculation is aligned with the SOA Exam SRM (Statistics for Risk Modeling) syllabus. In an exam-style comparison, state the parent impurity, show the child weights, and report the gain for each candidate. A larger gain means a greater in-sample reduction in Gini impurity; it does not, by itself, establish that a split will predict well on new observations.

Check Your Understanding

For each problem, calculate the weighted post-split impurity before finding the gain. Unless stated otherwise, report final decimal values to four decimal places.

A parent has 3 positive and 5 negative observations. A split creates children \((2,2)\) and \((1,3)\). Find the parent impurity, post-split impurity, and gain. (Answer: parent \(0.4688\); post-split \(0.4375\); gain \(0.0313\).)
A parent has 4 positive and 6 negative observations. A candidate creates children \((3,1)\) and \((1,5)\). Find the weighted post-split impurity and gain. (Answer: post-split \(19/60\approx0.3167\); gain \(49/300\approx0.1633\).)
A parent has 7 positive and 5 negative observations. Candidate A creates children \((4,2)\) and \((3,3)\); candidate B creates children \((5,1)\) and \((2,4)\). Which candidate has the greater Gini gain, and what is that gain? (Answer: candidate B; gain \(1/8=0.1250\).)
A parent has 6 positive and 3 negative observations. One split creates children \((3,0)\) and \((3,3)\). Calculate the correct gain and compare it with the gain obtained by incorrectly giving the two children equal weight. (Answer: correct post-split impurity \(1/3\), correct gain \(1/9\approx0.1111\); equal-weight impurity \(1/4\), incorrectly calculated gain \(7/36\approx0.1944\).)
A parent has 8 positive and 6 negative observations. A candidate split creates children \((4,1)\) and \((4,5)\). Find the parent impurity, weighted post-split impurity, and gain. (Answer: parent \(24/49\approx0.4898\); post-split \(136/315\approx0.4317\); gain \(128/2205\approx0.0580\).)

What You'll Learn