Kruskal–Wallis Reference Distributions and Conditions

Why the Reference Distribution Matters

The Kruskal–Wallis statistic summarizes how far the groups’ pooled rank sums depart from what would be expected if the groups came from the same distribution. To decide whether that departure is unusual, the statistic must be compared with a reference distribution. For moderate or large samples, the usual reference is a chi-square distribution with $k-1$ degrees of freedom, where $k$ is the number of independent groups.

That comparison is an approximation, not an exact identity for every sample size and data pattern. Small groups, many tied observations, or a design that violates independence can make the usual chi-square tail probability unreliable. The practical task is therefore twofold: calculate the appropriate statistic and decide whether its reference distribution is credible for the data at hand.

Definition: Let $k$ independent groups have sample sizes $n_1,\ldots,n_k$, with total sample size $N=\sum_{j=1}^k n_j$. Rank all $N$ observations together, assigning average ranks to tied observations, and let $R_j$ be the rank sum for group $j$. The uncorrected Kruskal–Wallis statistic is $$ H_0=\frac{12}{N(N+1)}\sum_{j=1}^{k}\frac{R_j^2}{n_j}-3(N+1). $$ If there are tied values, the usual tie-corrected statistic is $H=H_0/C$, where $$ C=1-\frac{\sum_g(t_g^3-t_g)}{N^3-N}. $$ Here $t_g$ is the number of observations in the $g$th tied group of equal values. With no ties, $C=1$ and $H=H_0$.

Conditions for the Chi-Square Approximation

The standard asymptotic result concerns the null hypothesis that the independent groups have the same underlying distribution. The observations must be independent within and across groups, and the groups must be formed so that observations are not paired or repeatedly measured on the same units. Under the null, pooled ranks are exchangeable across group labels. These conditions support the rank-based comparison; they do not make every small-sample chi-square approximation accurate.

When the approximation is adequate, $H$ is compared with a chi-square distribution having $k-1$ degrees of freedom. The degrees of freedom depend on the number of groups, not on the total sample size or the balance of the group sizes. At significance level $\alpha$, reject the null for a sufficiently large $H$, using the upper-tail critical value $\chi^2_{k-1,\,1-\alpha}$, or report the corresponding upper-tail probability.

Theorem: Under the null of identical group distributions, independent sampling, and regularity conditions supporting the large-sample approximation, the tie-adjusted Kruskal–Wallis statistic has an approximate chi-square distribution with $k-1$ degrees of freedom. Ties are handled in the usual asymptotic calculation by dividing $H_0$ by $C$. If $C=0$, all observations are tied and the statistic contains no information for comparing groups.

The degrees of freedom have a simple rank-sum explanation. Under the null, each group’s rank sum fluctuates around the rank total expected for its size. Those $k$ deviations cannot vary independently: their sum is zero because the pooled ranks have a fixed total. Thus only $k-1$ independent group contrasts remain. In large samples, the standardized contrasts are approximately jointly normal under the null. Their appropriate squared-distance statistic is approximately a sum of $k-1$ squared standard normal components, which is a chi-square distribution with $k-1$ degrees of freedom. The statistic $H$, with the tie adjustment when needed, provides that large-sample comparison.

This reasoning also explains why one should not use $k$ degrees of freedom: the rank-total constraint removes one independent contrast. It does not, however, establish that the approximation is good when the samples are very small. The asymptotic result describes behavior as the sample sizes grow.

Sample Size, Ties, and Practical Judgment

There is no universal sample-size threshold that guarantees a dependable chi-square approximation in every Kruskal–Wallis problem. A common classroom rule of thumb is to be cautious when any group has fewer than about five observations. Having at least five observations per group makes the approximation more plausible than having only two or three, while larger group sizes generally provide stronger support. These are guidelines rather than a theorem: group balance, the rank pattern, and how much probability is concentrated on tied values also matter.

Ties do not automatically invalidate the test. Average ranks are assigned to equal observations, and the correction factor $C$ adjusts the statistic’s scale. But a correction is not a guarantee that the chi-square tail probability is accurate in a small sample with many ties. When samples are small, the exact conditional permutation distribution of the rank statistic can be used: hold the observed pooled values and their ranks fixed, reassign group labels in ways consistent with the group sizes, and compare the observed statistic with that permutation distribution. This preserves the tie pattern rather than relying on a large-sample approximation.

The correction factor is between zero and one when there are ties. Consequently, $H=H_0/C$ is at least as large as $H_0$ whenever $C>0$. Omitting this adjustment can therefore affect a borderline decision. If every observation has the same value, then the single tie group has size $N$, giving $C=0$; there is no variation in ranks to distinguish the groups, so a chi-square comparison is not meaningful.

Worked Example: Four Groups With No Ties

A hypothetical assessment compares completion times from four independent training methods. The group sizes are $8,9,7,$ and $10$; all 34 times are distinct. The pooled rank sums are $104,128,157,$ and $206$. Assess the chi-square reference and determine whether the result is significant at the 5% level.

Method and conditions. There are $k=4$ independent groups, each with at least five observations, and no ties. The chi-square approximation is reasonable as a large-sample working approximation, although the sample sizes are not so large that it should be mistaken for an exact result. Since there are no ties, $C=1$.

First check the rank-sum total. The pooled ranks sum to $34(35)/2=595$, and the supplied group sums give $104+128+157+206=595$, as required.

Calculate the uncorrected statistic, which is also the corrected statistic:

\begin{aligned} H_0 &=\frac{12}{34(35)} \left(\frac{104^2}{8}+\frac{128^2}{9}+\frac{157^2}{7}+\frac{206^2}{10}\right)-3(35)\\ &=\frac{12}{1190} \left(1352+1820.4444+3521.2857+4243.6\right)-105\\ &\approx 5.2924. \end{aligned}

The degrees of freedom are $k-1=3$. The 5% upper-tail critical value is approximately $7.815$. Since $5.2924<7.815$, do not reject the null at the 5% level. This conclusion is based on the chi-square approximation, not an exact test.

Check. The rank sums total 595, the degrees of freedom are one less than the four groups, and the statistic is below the stated critical value. The common mistake avoided here is using $k=4$ degrees of freedom instead of $k-1=3$.

Worked Example: A Borderline Result With Ties

A hypothetical study compares scores from three independent groups of six participants each. Among the 18 scores, one score occurs four times and all remaining scores are distinct. The rank sums are $71,69,$ and $31$. The pooled ranks total $171$, matching $18(19)/2$. Calculate the corrected statistic and use the 5% chi-square reference to assess the result.

Method and conditions. There are $k=3$ groups, with six observations in each. The sample sizes are modest, so the approximation needs caution. Apply the standard tie correction for the one tied group of size $t=4$.

The uncorrected statistic is

\begin{aligned} H_0 &=\frac{12}{18(19)} \left(\frac{71^2}{6}+\frac{69^2}{6}+\frac{31^2}{6}\right)-3(19)\\ &=\frac{12}{342}\left(\frac{5041+4761+961}{6}\right)-57\\ &=\frac{10763}{171}-57\\ &\approx 5.9415. \end{aligned}

The tie correction uses $N=18$:

C=1-\frac{4^3-4}{18^3-18} =1-\frac{60}{5814} =\frac{959}{969} \approx 0.9897.

Thus

H=\frac{5.9415}{0.9897}\approx 6.0035.

The reference distribution has $k-1=2$ degrees of freedom, with a 5% upper-tail critical value of approximately $5.991$. The corrected statistic is just above this value, so the approximate test rejects at 5%. Because the groups are small and the result is close to the cutoff, an exact or permutation calculation would be a prudent check if available.

Check. The corrected statistic is slightly larger than the uncorrected one because $C<1$. Using $H_0=5.9415$ without correction would put the statistic below the critical value and give a different decision. The correction matters here, but it does not remove the small-sample caution.

Worked Example: Very Small Groups

A hypothetical pilot comparison has three independent groups, each containing three observations, with no ties. The rank sums are $12,15,$ and $18$, totaling $45=9(10)/2$. Calculate the asymptotic statistic and explain whether its chi-square probability should be treated as dependable.

Method and conditions. The groups are independent and there are no ties, so $C=1$. However, each group has only three observations. The chi-square calculation can be shown, but the small sample sizes make an exact or permutation reference preferable for inference.

Using $N=9$, calculate

\begin{aligned} H_0 &=\frac{12}{9(10)} \left(\frac{12^2}{3}+\frac{15^2}{3}+\frac{18^2}{3}\right)-3(10)\\ &=\frac{12}{90}(48+75+108)-30\\ &=30.8-30\\ &=0.8. \end{aligned}

The asymptotic reference has $3-1=2$ degrees of freedom. For two degrees of freedom, the upper-tail probability is $e^{-H/2}$, so the asymptotic p-value is $e^{-0.4}\approx 0.6703$. This is the chi-square approximation’s result, not an exact p-value. With only three observations per group, the discreteness of the possible rank allocations can be substantial; use an exact or permutation reference when a reliable small-sample inference is needed.

Check. The statistic and approximate probability are internally consistent, but the sample-size condition is weak. The common mistake avoided is reporting the asymptotic p-value as if the chi-square distribution were exact for these groups.

Choosing and Reporting the Reference

A sound Kruskal–Wallis report makes the reference choice visible. State the number of groups, whether ties occurred, whether the tie correction was applied, the degrees of freedom, and whether the reported probability is asymptotic or based on an exact or permutation distribution. In particular, do not describe a chi-square p-value as exact simply because software prints it to several decimal places.

Before using the usual chi-square reference, check that the observations are independent, the group sizes are adequate for an approximation, and the tie pattern is not so concentrated that the asymptotic result is questionable. If the data are paired or repeatedly measured, the independent-groups model is not appropriate. If groups are tiny or have many tied observations, prefer an exact or permutation reference when feasible.

Key takeaway: For $k$ independent groups, compare the tie-corrected Kruskal–Wallis statistic $H$ with an approximate $\chi^2_{k-1}$ distribution only when the large-sample conditions are credible. Small samples and substantial ties call for caution; when the approximation is doubtful, use an exact or permutation reference rather than treating the chi-square result as definitive.

Check Your Understanding

For each problem, decide how the reference distribution should be handled and check the numerical conclusion.

Four independent groups have sizes $12,12,12,$ and $12$, with no ties, and the calculated statistic is $8.10$. At the 5% level, use the appropriate degrees of freedom and decide whether to reject. (Answer: $df=3$; the critical value is $7.815$, so reject using the chi-square approximation.)
Three independent groups have five observations each. One pooled value occurs three times and all others are distinct; the uncorrected statistic is $4.80$. Find the tie correction and corrected statistic, then compare with the 5% critical value. (Answer: $C=1-24/3360=0.9929$; $H\approx4.8345$, with $df=2$, so do not reject using the chi-square approximation.)
Three independent groups have only two observations each and no ties. Which reference is preferable for a test, and why? (Answer: Prefer an exact or permutation reference; two observations per group are too few to rely comfortably on the chi-square approximation.)
Five independent groups have sizes $8,9,10,8,$ and $11$, with no ties, and $H=10.0$. At the 5% level, state the degrees of freedom and decision. (Answer: $df=4$; the critical value is $9.488$, so reject using the chi-square approximation.)
All observations in three groups have exactly the same recorded value. What happens to the tie correction, and can the usual chi-square comparison provide an informative group test? (Answer: $C=0$; there is no rank variation to compare, so the usual corrected statistic and chi-square comparison are not informative.)

What You'll Learn