Deriving the Kruskal–Wallis Rank-Sum Formula

Why Derive the Rank-Sum Formula?

The Kruskal–Wallis procedure compares \(k\) independent groups by ranking all observations together, then examining how far each group’s mean rank lies from the pooled mean rank. Its familiar calculation is often written using group rank sums. Understanding how that formula arises helps prevent errors with unequal group sizes and clarifies what the statistic measures.

This tutorial derives the formula for observations with no ties. If ties occur, average ranks can be used, but the usual statistic also requires a tie adjustment; that correction is treated separately. The derivation below focuses on how dispersion among group mean ranks leads to the rank-sum expression.

Definition: Let \(k\) independent groups contain \(n_1,\ldots,n_k\) observations, with total sample size \(N=\sum_{j=1}^k n_j\). Pool the observations and assign ranks \(1,\ldots,N\) in increasing order. For group \(j\), let \(R_j\) be its rank sum and \(\bar R_j=R_j/n_j\) its mean rank. With no ties, the pooled mean rank is \(\bar R=(N+1)/2\).

The statistic is large when the group mean ranks are spread away from the pooled mean rank. Since a group mean is based on a different number of observations when group sizes differ, each squared deviation must be weighted by its group size. The centered form is

H=\frac{12}{N(N+1)}\sum_{j=1}^{k}n_j\left(\bar R_j-\frac{N+1}{2}\right)^2.

This expression is equivalent to the standard rank-sum form:

H=\frac{12}{N(N+1)}\sum_{j=1}^{k}\frac{R_j^2}{n_j}-3(N+1).

The two forms are useful for different checks. The centered form makes the comparison with the pooled mean rank visible; the rank-sum form is often quicker once the \(R_j\) values have been calculated.

From Mean-Rank Dispersion to Rank Sums

The conversion uses \(\bar R_j=R_j/n_j\) and \(\sum_j n_j=N\). Expanding the squared deviations gives

\begin{aligned} \sum_{j=1}^{k}n_j\left(\bar R_j-\frac{N+1}{2}\right)^2 &=\sum_{j=1}^{k}n_j\bar R_j^2 -(N+1)\sum_{j=1}^{k}n_j\bar R_j +\frac{(N+1)^2}{4}\sum_{j=1}^{k}n_j\\ &=\sum_{j=1}^{k}\frac{R_j^2}{n_j} -(N+1)\sum_{j=1}^{k}R_j +\frac{N(N+1)^2}{4}. \end{aligned}

Every pooled rank appears in exactly one group, so the group rank sums satisfy \(\sum_j R_j=1+2+\cdots+N=N(N+1)/2\). Substituting this identity into the expansion yields

\begin{aligned} \sum_{j=1}^{k}n_j\left(\bar R_j-\frac{N+1}{2}\right)^2 &=\sum_{j=1}^{k}\frac{R_j^2}{n_j} -\frac{N(N+1)^2}{2} +\frac{N(N+1)^2}{4}\\ &=\sum_{j=1}^{k}\frac{R_j^2}{n_j} -\frac{N(N+1)^2}{4}. \end{aligned}

Multiplying by \(12/[N(N+1)]\) converts the last term to \(3(N+1)\), which proves the rank-sum formula. In particular, each squared rank sum is divided by its own group size. That divisor is what makes the rank-sum expression equivalent to a size-weighted comparison of group mean ranks.

Why the Scaling Uses \(N(N+1)\)

The multiplier \(12/[N(N+1)]\) gives the statistic a useful scale under the null hypothesis that all group distributions are the same. Under this null, the ranks can be regarded as randomly assigned to group memberships, subject to the fixed group sizes. The pooled ranks have mean \((N+1)/2\), and their finite-population variance, using denominator \(N-1\), is \(N(N+1)/12\).

For a group of size \(n_j\), random assignment is sampling without replacement from those \(N\) rank values. The variance of its mean rank is therefore

\operatorname{Var}(\bar R_j) =\left(1-\frac{n_j}{N}\right)\frac{1}{n_j}\frac{N(N+1)}{12} =\frac{(N-n_j)(N+1)}{12n_j}.

Because the mean rank has null expectation \((N+1)/2\), its expected squared deviation from that value is its variance. Consequently,

\begin{aligned} E\left[\sum_{j=1}^{k} n_j\left(\bar R_j-\frac{N+1}{2}\right)^2\right] &=\sum_{j=1}^{k} n_j\operatorname{Var}(\bar R_j)\\ &=\frac{N+1}{12}\sum_{j=1}^{k}(N-n_j)\\ &=\frac{N(N+1)(k-1)}{12}. \end{aligned}

The last equality follows because \(\sum_j(N-n_j)=kN-N=N(k-1)\). Applying the multiplier in the centered formula gives \(E[H]=k-1\) under this random-assignment null model. This expectation helps explain the scaling; it does not by itself give a decision rule or a reference distribution.

Calculating the Statistic

Worked Example: Three Equal-Sized Groups

A hypothetical laboratory compares a measurement from three independent treatment groups, each containing three units. After pooling all nine observations, the groups receive ranks \(1,4,7\); \(2,5,8\); and \(3,6,9\), respectively. Calculate \(H\) in both forms and check that the results agree. Assume there are no ties.

Method. Calculate each group’s rank sum and mean rank, then use the rank-sum formula and verify with weighted deviations from the pooled mean rank.

The total size is \(N=3+3+3=9\). The rank sums are \(R_1=1+4+7=12\), \(R_2=2+5+8=15\), and \(R_3=3+6+9=18\). They check against the pooled rank total: \(12+15+18=45=9(10)/2\). The group mean ranks are \(4,5,6\), while the pooled mean rank is \((9+1)/2=5\).

Using the rank-sum form,

\begin{aligned} H&=\frac{12}{9(10)} \left(\frac{12^2}{3}+\frac{15^2}{3}+\frac{18^2}{3}\right)-3(10)\\ &=\frac{12}{90}(48+75+108)-30\\ &=\frac{12}{90}(231)-30\\ &=30.8-30=0.8. \end{aligned}

The centered check is \(12/90\) times \(3[(4-5)^2+(5-5)^2+(6-5)^2]\). The weighted squared deviations sum to \(3(1+0+1)=6\), so \(H=(12/90)(6)=0.8\), as required. The common mistake avoided here is treating the three mean ranks as unweighted observations rather than weighting each by its group size.

Worked Example: Unequal Group Sizes

Three independent groups in a hypothetical materials study contain \(2\), \(3\), and \(4\) specimens. Their pooled ranks are \(1,8\); \(2,4,9\); and \(3,5,6,7\), respectively. Assuming no ties, calculate \(H\) and check it using mean ranks.

Method. Use the rank-sum formula with a separate divisor for each group, then check the weighted mean-rank deviations.

Here \(N=2+3+4=9\). The rank sums are \(R_1=9\), \(R_2=15\), and \(R_3=21\). Their total is \(9+15+21=45=9(10)/2\), so the ranks have been accounted for correctly. The mean ranks are \(9/2=4.5\), \(15/3=5\), and \(21/4=5.25\); the pooled mean is \(5\).

\begin{aligned} H&=\frac{12}{9(10)} \left(\frac{9^2}{2}+\frac{15^2}{3}+\frac{21^2}{4}\right)-30\\ &=\frac{12}{90}(40.5+75+110.25)-30\\ &=\frac{12}{90}(225.75)-30\\ &=30.1-30=0.1. \end{aligned}

For the check, the weighted squared deviations are \(2(4.5-5)^2+3(5-5)^2+4(5.25-5)^2=2(0.25)+0+4(0.0625)=0.75\). Thus \(H=(12/90)(0.75)=0.1\). The small value is consistent with mean ranks close to the pooled mean. Unequal group sizes cause no difficulty provided the group-size weights or divisors are retained.

Worked Example: Groups at Opposite Ends of the Ranks

A hypothetical screening study has four independent groups of two observations each. In ascending pooled order, their group memberships correspond to ranks \(\{1,2\}\), \(\{3,4\}\), \(\{5,6\}\), and \(\{7,8\}\). Calculate \(H\), showing how the centered form makes the result easy to verify. Assume no ties.

Method. Find the four rank sums and mean ranks, then compute the weighted squared deviations from the pooled mean.

The total size is \(N=8\), and the rank sums are \(3,7,11,15\). They total \(36=8(9)/2\). Dividing each by its group size of \(2\) gives mean ranks \(1.5,3.5,5.5,7.5\). The pooled mean rank is \(9/2=4.5\).

\begin{aligned} H&=\frac{12}{8(9)} \left[2(1.5-4.5)^2+2(3.5-4.5)^2 +2(5.5-4.5)^2+2(7.5-4.5)^2\right]\\ &=\frac{12}{72}\left[2(9)+2(1)+2(1)+2(9)\right]\\ &=\frac{1}{6}(40)=\frac{20}{3}\approx 6.6667. \end{aligned}

As a rank-sum check, \(\sum_j R_j^2/n_j=(9+49+121+225)/2=202\), so \(H=(12/72)(202)-27=33.6667-27=6.6667\). Both calculations agree. The mistake avoided is using \(N+1\), rather than \((N+1)/2\), as the pooled mean rank.

Common Formula Errors and a Practical Check

A frequent error is to square each rank sum but omit division by its group size. For the unequal-size example above, that incorrect calculation would be

\frac{12}{90}(9^2+15^2+21^2)-30 =\frac{12}{90}(747)-30 =69.6,

rather than the correct \(0.1\). The incorrect result fails because it does not represent weighted dispersion of mean ranks; it also makes the answer depend improperly on the sizes of the groups through their raw totals. Use \(R_j^2/n_j\), not \(R_j^2\).

Before accepting a calculation, check both the pooled rank total and the equivalence of the two forms. With no ties, \(\sum_j R_j=N(N+1)/2\). Then calculate \(H\) once from rank sums and once from weighted deviations if the arithmetic is not extensive. A discrepancy usually indicates a missing group-size divisor, an incorrect mean rank, or a wrong pooled mean.

Key takeaway: The Kruskal–Wallis rank-sum formula is the centered, group-size-weighted dispersion of mean ranks rewritten algebraically. For no ties, calculate \(H=\frac{12}{N(N+1)}\sum_j R_j^2/n_j-3(N+1)\), keeping each group’s own size in its divisor.

Check Your Understanding

Assume all pooled observations in these problems are distinct. Calculate or verify the requested rank-sum statistic.

Two groups each have size \(2\), with ranks \(\{1,4\}\) and \(\{2,3\}\). Find \(H\) using the rank-sum formula. (Answer: \(H=0\))
Three groups each have size \(2\), with ranks \(\{1,2\}\), \(\{3,4\}\), and \(\{5,6\}\). Find \(H\) using weighted deviations from the pooled mean rank. (Answer: \(H=32/7\approx4.5714\))
Groups of sizes \(2,1,3\) receive ranks \(\{1,6\}\), \(\{2\}\), and \(\{3,4,5\}\). Calculate \(H\) and check that the group rank sums total \(N(N+1)/2\). (Answer: \(H=6/7\approx0.8571\); rank-sum total \(21\))
Three groups each have size \(3\), with ranks \(\{1,5,9\}\), \(\{2,6,7\}\), and \(\{3,4,8\}\). Calculate \(H\). (Answer: \(H=0\))
For groups of sizes \(2,3,4\) with rank sums \(9,15,21\), a candidate calculates \(12[9^2+15^2+21^2]/90-30\). Identify the error and give the correct \(H\). (Answer: The group-size divisors are missing; \(H=12[9^2/2+15^2/3+21^2/4]/90-30=0.1\))

What You'll Learn