Common Kruskal–Wallis Errors and Assumption Checks

Why a Correct Kruskal–Wallis Calculation Can Still Be Wrong

A Kruskal–Wallis test compares independent groups using the ranks of all observations pooled together. Its arithmetic is compact, but a sound conclusion depends on more than substituting rank sums into a formula. The observational units must be independent, the ranks must be assigned across all groups together, tied observations must be handled correctly, and the conclusion must match what the test actually addresses.

The checks in this tutorial are useful both before and after calculating the statistic. They help distinguish a computational error from a design problem or an overstatement of the evidence. The procedures are aligned with the SOA VEE Mathematical Statistics syllabus.

Definition: For $k$ independent groups with sample sizes $n_1,\ldots,n_k$, let $N=\sum_{j=1}^k n_j$, and let $R_j$ be the sum of the pooled ranks in group $j$. The uncorrected Kruskal–Wallis statistic is $$ H=\frac{12}{N(N+1)}\sum_{j=1}^k \frac{R_j^2}{n_j}-3(N+1). $$ If the pooled observations contain tie groups of sizes $t_1,\ldots,t_g$, the tie correction is $$ C=1-\frac{\sum_{r=1}^g(t_r^3-t_r)}{N^3-N}, \qquad H_c=\frac{H}{C}. $$ Here $H_c$ is the corrected statistic. Under the usual large-sample conditions, it is compared with a chi-square distribution with $k-1$ degrees of freedom.

The test’s null hypothesis is that the groups have the same distribution. A rejection is evidence that the group distributions are not all the same; it does not by itself identify which groups differ or establish that their medians differ. If the distributions have the same shape and spread and differ only in location, a location or median interpretation may be reasonable. Without that additional condition, differences in spread or shape can also contribute to a significant result.

Check the Design Before Ranking

The independence requirement concerns the observational units, not merely whether the recorded values look different. Each unit should contribute an observation to only one group, and observations from different units should be independent under the study design. For example, measurements from separate randomly sampled policyholders may be independent, subject to the sampling design. Measurements of the same policyholders at several follow-up visits are repeated measurements, not independent groups.

A common error is to use Kruskal–Wallis for repeated measurements because the response is ordinal or non-normal. Ranking does not remove dependence. A within-subject design requires an analysis that accounts for the matching or repeated observations; a rank-based repeated-measures procedure such as the Friedman test is a standard alternative when its design and assumptions fit. If observations are clustered, such as several people sampled from each household, independence may also fail even when each person appears in only one nominal group.

Worked Example: Repeated Measurements Mistaken for Independent Groups

A hypothetical study records each of 8 workers’ fatigue scores on a 1-to-10 scale at the start, middle, and end of a shift. The analyst proposes a Kruskal–Wallis test with three groups of 8 scores. Identify the design issue and the appropriate correction to the analysis plan.

Step 1: Identify the units. There are 8 workers. Each worker contributes three scores, one at each time. Thus, the 24 scores do not represent 24 independent workers.

Step 2: Check the required independence. A worker’s three measurements are linked by the worker’s characteristics. Treating the start, middle, and end groups as independent ignores this within-worker dependence. The Kruskal–Wallis independence condition is therefore not met.

Step 3: Correct the plan. Retain the worker as the matching unit and use a method for repeated measurements, such as the Friedman test for a suitable rank-based analysis. Do not calculate a Kruskal–Wallis $H$ from these three sets and interpret its reference distribution as if the observations were independent.

Check. The issue remains even if the scores are all distinct and the pooled ranks are calculated perfectly: correct ranking cannot repair a dependent study design. No interest-rate or mortality assumption applies; this is a repeated-measures sampling design.

Audit the Pooled Ranks and Tie Correction

For Kruskal–Wallis, sort all $N$ observations across every group, then assign ranks from 1 through $N$. When equal observations occupy several rank positions, each receives the average of those positions. Do not rank each group separately: group-specific ranks discard the cross-group ordering that the test uses.

Ties do not change the pooled total rank $N(N+1)/2$, because assigning each tied observation the average of its occupied positions preserves their combined rank sum. Ties do change the rank variation used to calibrate the test. The factor $C$ adjusts for that reduction; when $C>0$, dividing $H$ by $C$ gives the tie-corrected statistic. With no ties, every $t_r=1$, so $C=1$ and the correction has no effect.

Worked Example: Midranks and the Tie Correction

Three independent groups have observations $A=(1,4,4)$, $B=(2,4,7)$, and $C=(3,6,8)$. Assume observations from different units are independent and that the groups are independent samples. Calculate the tie-corrected Kruskal–Wallis statistic and identify the error made by omitting the correction.

Step 1: Rank the pooled observations. In increasing order they are $1,2,3,4,4,4,6,7,8$. The three values equal to 4 occupy ranks 4, 5, and 6, so each receives midrank $(4+5+6)/3=5$. The full ranks are therefore $1,2,3,5,5,5,7,8,9$.

Step 2: Find each group’s rank sum. Group A has ranks $1,5,5$, so $R_A=11$. Group B has ranks $2,5,8$, so $R_B=15$. Group C has ranks $3,7,9$, so $R_C=19$. Each group has size 3, and $N=9$. As a rank check, $R_A+R_B+R_C=11+15+19=45=9(10)/2$.

Step 3: Calculate the uncorrected statistic. $$ H=\frac{12}{9(10)} \left(\frac{11^2}{3}+\frac{15^2}{3}+\frac{19^2}{3}\right)-3(10) =\frac{2}{15}\left(\frac{707}{3}\right)-30 =\frac{64}{45} \approx 1.4222. $$

Step 4: Correct for the tie of size 3. There is one tie group with $t_1=3$, so $$ C=1-\frac{3^3-3}{9^3-9} =1-\frac{24}{720} =\frac{29}{30}. $$ Hence $$ H_c=\frac{64/45}{29/30}=\frac{128}{87}\approx 1.4713. $$

Common mistake. Reporting $1.4222$ as the final statistic omits the tie correction. The corrected value is $1.4713$. As a check, $C<1$, so dividing the positive uncorrected statistic by $C$ increases it. At the 5% level, the large-sample reference has $k-1=2$ degrees of freedom and critical value approximately 5.991; $1.4713<5.991$, so this procedure does not reject the null at that level. With only three observations per group, however, the chi-square approximation may be rough; an appropriate exact or permutation method may be preferable. No interest-rate or mortality assumption applies; the stated model is independent sampling with tied ordinal or numeric outcomes.

The check above illustrates two separate questions: was the statistic calculated correctly, and is its reference distribution suitable? A correct tie correction does not guarantee that a small-sample chi-square approximation is accurate. There is no universal sample-size cutoff that makes the approximation exact. For small samples, sparse group sizes, or a design that makes the asymptotic approximation doubtful, consider an exact or permutation approach that preserves the independent-group design.

Keep the Conclusion Within the Evidence

A significant Kruskal–Wallis result is not automatically a test of equal medians. It tests whether the rank patterns are consistent with the groups having the same distribution. If one group is much more variable, for instance, its observations may occupy more extreme ranks even when its median is similar to those of other groups. A statement such as “the medians differ” needs additional justification, usually a defensible same-shape, same-spread location-shift assumption.

A second interpretive error is to treat an omnibus rejection as proof that every pair differs. The omnibus test says only that the evidence is inconsistent with all groups having the same distribution. Pairwise follow-up tests address particular pairs and create a multiple-testing problem if several are conducted. Plan the comparisons and an error-rate adjustment, or use a suitable post-hoc procedure. A correction cannot make an unsuitable comparison appropriate, but it can control the familywise error rate across a specified family of tests.

Worked Example: Avoiding a Median Claim and Controlling Pairwise Tests

A hypothetical analysis compares three independent samples of recovery times from different treatment groups. The omnibus Kruskal–Wallis test has $H_c=8.40$ and 2 degrees of freedom, giving an approximate $p$-value of 0.0150. The group distributions have visibly different spreads. A researcher reports that all three medians differ and then gives unadjusted pairwise $p$-values of 0.012, 0.028, and 0.041. Assess both claims using familywise significance level 0.05.

Step 1: Interpret the omnibus test. Since $0.0150<0.05$, reject the null that all three group distributions are the same, using the stated approximate reference distribution. The different spreads mean the result does not establish that the medians differ; the distributional differences need not be location shifts.

Step 2: Define the pairwise family. Three groups yield $\binom{3}{2}=3$ pairwise comparisons. Applying the Bonferroni method gives the per-comparison threshold $0.05/3=0.0167$, rounded to four decimal places.

Step 3: Compare each result with the adjusted threshold. The first value, $0.012<0.0167$, is significant. The second, $0.028>0.0167$, is not significant at the adjusted level. The third, $0.041>0.0167$, is also not significant. Equivalently, the Bonferroni-adjusted $p$-values are $3(0.012)=0.036$, $3(0.028)=0.084$, and $3(0.041)=0.123$, all capped at 1 if necessary; only the first is below 0.05.

Correction. The evidence supports an omnibus distributional difference and, under this specified adjustment, a difference for the first tested pair. It does not support the claim that all three medians differ. The pairwise conclusion also assumes the comparisons and their $p$-values are appropriate for the data; adjustment does not validate a violated independence assumption. No interest-rate or mortality assumption applies; the stated model is three independent samples with differing observed spreads.

A Practical Assumption and Error Checklist

Before reporting a result, work through the checks in order. This prevents a polished calculation from concealing a mismatch between the design, the method, and the conclusion.

Confirm the sampling units.
Identify what was sampled and whether each unit belongs to only one independent group. Flag repeated measurements, matching, and clustering rather than treating them as independent observations.

Inspect the response and data collection.
Confirm that responses can be meaningfully ordered and that the sampling or assignment supports the intended inference. A rank test does not correct biased sampling or dependence.

Rebuild the pooled ranks.
Rank observations across all groups, assign average ranks to ties, and check that all rank sums total $N(N+1)/2$.

Check the calculation and approximation.
Use the correct group sizes in $H$, include the tie correction when ties occur, and assess whether the chi-square approximation is adequate for the sample sizes.

Limit the interpretation.
State an omnibus distributional conclusion unless a location-shift interpretation is justified. For multiple pairwise tests, specify the comparison family and the adjustment.

Common traps can be summarized as follows:

“It is nonparametric, so dependence does not matter.” False. Rank procedures still require a design and reference distribution that account for dependence.
“The groups should be ranked separately.” False. Kruskal–Wallis uses ranks pooled across all groups.
“Tied values can be assigned whichever tied rank is convenient.” False. Use their average rank and apply the tie correction.
“A significant result proves different medians.” Not without conditions that support a location interpretation; the test concerns distributions.
“A significant omnibus result makes every pair significant.” False. Pairwise conclusions require their own evidence and a plan for multiplicity.

Key takeaway: Audit the design first, then the pooled ranks and tie correction, then the reference distribution, and finally the interpretation and follow-up comparisons. A valid Kruskal–Wallis conclusion must satisfy all four checks.

Check Your Understanding

For each problem, identify the main audit issue and determine the supported conclusion.

A lab records one measurement from each of 12 independently sampled specimens in each of three groups. Two observations are tied. What ranking and calculation checks are required before testing? (Answer: Rank all 36 observations together, give the tied pair their average occupied rank, verify the rank sums total $36(37)/2=666$, and apply the tie correction for a tie of size 2.)
The same 10 machines are assessed under four settings, but an analyst proposes Kruskal–Wallis on four groups of 10 readings. What is the design error? (Answer: The readings are repeated within machine and are dependent; use a repeated-measures method, such as Friedman’s test when appropriate, rather than an independent-groups Kruskal–Wallis test.)
A Kruskal–Wallis test rejects equality of distributions for three groups. The sample medians are 14, 15, and 15, but the spreads differ substantially. Is “the medians differ” established? (Answer: No. Rejection supports a difference among the group distributions, but differing spreads prevent attributing it to median or location differences alone.)
For four groups, six pairwise tests are planned at familywise level 0.05. Using Bonferroni, what is the per-comparison threshold? (Answer: $0.05/6=0.0083$ to four decimal places.)
An analysis with five independent groups reports $H_c=7.5$ and uses a chi-square reference with 4 degrees of freedom, but each group has only two observations. What should be checked before accepting the $p$-value? (Answer: The degrees of freedom are $5-1=4$, but the chi-square approximation may be poor with such small groups; consider a suitable exact or permutation method and ensure it preserves the independent-group design.)

What You'll Learn