What a Kruskal–Wallis Rejection Says
The Kruskal–Wallis test is often described as a test of whether several population medians are equal. That description needs a condition: the group distributions must have the same shape and spread, differing only by horizontal shifts. Without that condition, a rejection is evidence against equality of the group distributions, but it does not by itself establish that their medians differ.
This distinction matters in applications. For example, hypothetical treatment groups might have the same typical response but different variability. A rank-based test can respond to that difference in spread. Calling the result a difference in medians would then overstate what the test established.
The usual Kruskal–Wallis statistic is compared with a chi-square reference distribution with \(k-1\) degrees of freedom when the applicable approximation conditions hold. The p-value measures how unusual a statistic at least as large as the observed one would be under the null model. It does not state the probability that the null is true, and it does not describe the size or practical importance of a difference.
The statistic is built from group rank sums, or equivalently group mean ranks. Thus it is useful to distinguish three statements:
- “The test rejects equality of distributions.” This is the direct inferential conclusion, at the stated significance level.
- “At least one population median differs.” This may be a reasonable interpretation if the group distributions have a common shape and spread and differ only by location.
- “A particular group differs from another.” The omnibus result alone does not identify which groups differ.
The last statement requires additional analysis. A significant omnibus test is a signal to investigate further, not a result that names the groups responsible.
Why Rank Differences Reflect Distributional Differences
A pooled rank records how an observation compares with all observations in the combined sample. The following simple argument explains why equal distributions produce equal expected ranks across groups.
For a continuous observation \(X\) in group \(j\), its pooled rank is one plus the number of pooled observations smaller than \(X\).
Under \(H_0\), all \(N\) observations have the same continuous distribution. For any other observation, the probability it is smaller than \(X\) is \(1/2\), by symmetry between the two independent observations.
There are \(N-1\) other observations, so the expected rank for an observation from any group is \(1+(N-1)/2=(N+1)/2\). The groups therefore have the same expected mean rank under the null.
If distributions differ, the pairwise probabilities of one group’s observations exceeding another group’s observations need not be \(1/2\). That can produce differences in mean ranks, which the test detects. But it is possible for distributions to differ while their mean-rank behavior is similar or equal; the Kruskal–Wallis statistic is not a universal detector of every possible shape difference. Ties alter the rank calculations and are handled by a tie correction where appropriate.
When a Median Interpretation Is Reasonable
Suppose each group distribution is a horizontal shift of one common distribution. In symbols, for some common distribution function \(F_0\) and group-specific locations \(\theta_j\),
This model preserves the distribution’s shape and spread across groups: only its location changes. If the common distribution has median \(m_0\), then group \(j\) has median \(m_0+\theta_j\). Therefore, under this common-shape location-shift model, a distributional difference is a location difference and hence a median difference.
In practice, the common-shape condition is an assumption to assess, not something established by a significant Kruskal–Wallis result. Compare group distributions using plots and summaries of spread and shape, as well as medians. Similar interquartile ranges alone do not prove that shapes match; skewness, tail behavior, and clustering may also differ. Equal variances alone are not enough to guarantee a common shape.
| Observed pattern | Careful interpretation of a rejection |
|---|---|
| Similar shapes and spreads, with distributions shifted horizontally | Evidence consistent with a location difference; a median interpretation is reasonable under the shift model. |
| Similar medians but visibly different spreads or shapes | Evidence against identical distributions; do not describe it as evidence that medians differ. |
| No rejection | Insufficient evidence to reject identical distributions; this does not establish that the distributions are equal. |
Worked Examples
Worked Example: Rejection With Similar Shapes
A hypothetical study compares response times, in minutes, for three independently sampled service protocols. The Kruskal–Wallis calculation gives \(H=6.000\), with \(k=3\) groups. The observed sample medians are 18, 22, and 26 minutes; plots show broadly similar shapes and spreads. Interpret the result at \(\alpha=0.05\), using the chi-square approximation.
Step 1: Identify the reference distribution. There are \(k-1=3-1=2\) degrees of freedom. For a chi-square random variable with 2 degrees of freedom, the upper-tail probability at \(H\) is \(e^{-H/2}\).
Step 2: Calculate the p-value.
Step 3: Compare with the significance level. Since \(0.0498<0.05\), reject the null hypothesis that the three population distributions are identical.
Step 4: Interpret in context. The result is evidence that response-time distributions are not all identical. The similar observed shapes and spreads make a location-shift interpretation plausible; under the common-shape shift assumption, the result supports a difference in group medians. The test itself does not establish that assumption or identify which protocols differ.
Common mistake avoided: Saying “the p-value is below 0.05, so all three medians differ.” Rejection supports neither that all medians differ nor that each pair differs. It rejects the all-distributions-identical null.
Worked Example: Rejection With Different Spreads
In a hypothetical comparison of claim-processing times for three independent workflow designs, the Kruskal–Wallis statistic is \(H=7.600\). The sample medians are all 50 minutes, but the observed interquartile ranges are 4, 9, and 16 minutes, and the plots show visibly different spreads. Interpret the result at \(\alpha=0.05\), using the chi-square approximation.
Step 1: Set the degrees of freedom. With \(k=3\), the reference distribution has \(k-1=2\) degrees of freedom.
Step 2: Calculate the p-value.
Step 3: Make the test decision. Because \(0.0224<0.05\), reject the hypothesis that the three population distributions are identical.
Step 4: State what the summaries permit. The result is evidence against identical distributions, and the observed spreads differ substantially. The equal sample medians do not undo the rejection, because the test is not exclusively a test of medians. Nor does the result establish that the population medians are equal: sample medians are estimates, and the test’s conclusion concerns distributional rank behavior.
Common mistake avoided: “The test rejects, so the population medians cannot all be equal.” That conclusion is not justified when shape or spread may differ. Here, the observed spread differences specifically make a median-only explanation unsafe.
Worked Example: A Non-Significant Result
A hypothetical assessment compares scores from three independent instructional formats. The Kruskal–Wallis statistic is \(H=2.100\), and the group plots have considerable overlap. Interpret the result at \(\alpha=0.05\), using the chi-square approximation.
Step 1: Determine the degrees of freedom. There are three groups, so \(k-1=2\) degrees of freedom.
Step 2: Calculate the p-value.
Step 3: Decide whether to reject. Since \(0.3499>0.05\), do not reject the null hypothesis.
Step 4: Give the appropriately limited conclusion. The sample does not provide sufficient evidence, at the 5% level, that the three score distributions differ. This is not proof that they are identical. The result may reflect genuinely similar distributions, limited sample information, or differences the rank-based test does not detect well.
Common mistake avoided: Writing “the three formats have the same score distribution.” A non-rejection is not an affirmative demonstration of equality.
Common Interpretation Traps
- Equating ranks with medians. Ranks reflect comparisons throughout the distributions, not only their centers. A difference in spread or shape can affect ranks.
- Treating a significant result as a pairwise conclusion. The omnibus test says the distributions are not all identical; it does not identify a particular group pair.
- Assuming similar medians settle the question. Equal observed medians can coexist with different distributions, as when spreads or shapes differ.
- Turning non-rejection into proof of equality. The conclusion is insufficient evidence to reject, not evidence that the null must be true.
- Ignoring the model behind “median test.” A median-shift interpretation requires distributions with a common shape and spread that differ only by horizontal location shifts.
Check Your Understanding
For each problem, distinguish the direct distributional conclusion from any conclusion that depends on a common-shape shift assumption. Use the chi-square approximation where a statistic is given.
- A hypothetical study compares four independent groups. It reports \(H=8.000\), and the group plots show similar shapes and spreads with shifted centers. State the degrees of freedom, calculate the p-value to 4 decimals, and interpret at \(\alpha=0.05\). (Answer: \(3\) degrees of freedom; \(p\approx0.0460\); reject identical distributions, with a median-shift interpretation reasonable only under the common-shape assumption.)
- Three independent groups have sample medians 30, 30, and 31, but their observed spreads differ considerably. The Kruskal–Wallis test rejects at the 5% level. What is the safest conclusion? (Answer: Evidence against identical group distributions; the result does not by itself establish a difference in population medians.)
- For three independent groups, \(H=4.000\). Calculate the approximate p-value to 4 decimals and state the decision at \(\alpha=0.05\). (Answer: \(2\) degrees of freedom; \(p=e^{-2}=0.1353\); do not reject identical distributions, but do not claim they are proven equal.)
- A significant Kruskal–Wallis result is obtained for five independent groups, and all five sample medians are different. Does the omnibus test establish that every pair of population medians differs? Explain briefly. (Answer: No. It is evidence against all five distributions being identical; it neither establishes every pairwise difference nor proves the common-shape shift assumption.)
- Two population distributions have equal medians but different spreads. Explain why a Kruskal–Wallis rejection in a multi-group comparison could still be consistent with the observed summaries. (Answer: The test uses pooled rank behavior across the distributions, so spread differences can affect ranks even when medians agree; rejection is not exclusively evidence about medians.)