Why Compare These Two Tests?
When a study compares three or more independent groups, one-way ANOVA and the Kruskal–Wallis test are two common choices. Both can detect evidence that the groups differ, but they do not test the same feature of the data. One-way ANOVA compares population means under a model with particular distributional assumptions. Kruskal–Wallis compares the distributions of the groups using the pooled ranks of the observations.
This distinction matters in practice. A few extreme values can strongly affect means and ANOVA’s sums of squares, while ranking limits how much the magnitude of an extreme value can affect Kruskal–Wallis. But ranking does not make a test assumption-free, and a significant Kruskal–Wallis result does not automatically mean that the group medians differ. The choice should follow the study design, the scientific question, and the data’s characteristics.
Hypotheses and Assumptions
Let \(Y_{ij}\) be observation \(i\) from group \(j\), where \(j=1,\ldots,k\). Write \(\mu_j\) for the population mean in group \(j\). In one-way ANOVA, the hypotheses are
A standard one-way ANOVA model is \(Y_{ij}=\mu_j+\varepsilon_{ij}\), with independent errors having mean zero and a common variance \(\sigma^2\). For the usual exact F reference distribution, the errors are also normally distributed. Thus, the model assumes independent observations, a quantitative response, equal within-group variances, and normal errors. The test can be reasonably robust to modest departures from normality, particularly with balanced group sizes and no severe outliers, but that robustness is not a substitute for checking whether the model is plausible.
For Kruskal–Wallis, the null hypothesis is that the \(k\) population distributions are identical. The usual alternative is that at least one distribution differs. Observations must be independent across and within groups, and the response must be at least ordinal so that meaningful ranks can be assigned. The test does not require normally distributed observations or equal population variances. However, when group distributions differ in spread or shape, a rejection can reflect those differences as well as a location difference.
| Feature | One-way ANOVA | Kruskal–Wallis |
|---|---|---|
| Primary comparison | Population means | Population distributions, through pooled ranks |
| Response scale | Quantitative; differences in the response scale are meaningful | Ordinal or quantitative; order is meaningful |
| Key conditions | Independent observations, common variance, and normal errors for the exact F result | Independent observations; rankable outcomes |
| Effect of extreme magnitude | Can be substantial because means and squared deviations use the values themselves | Limited by ranks, though an extreme observation still receives an extreme rank |
| Interpretation of rejection | Evidence that not all population means are equal | Evidence that not all group distributions are identical |
How the Statistics Reflect the Difference
ANOVA compares variation between the group means with variation among observations within groups. If the group means are equal, the two sources of variation estimate the same error variance, so their ratio should usually be near one. A large ratio is evidence against equality of means.
where \(N\) is the total sample size, \(SS_{\text{between}}=\sum_{j=1}^{k} n_j(\bar{Y}_j-\bar{Y})^2\), and \(SS_{\text{within}}=\sum_{j=1}^{k}\sum_{i=1}^{n_j}(Y_{ij}-\bar{Y}_j)^2\). Here, \(\bar{Y}_j\) is the sample mean in group \(j\), and \(\bar{Y}\) is the grand mean.
The Kruskal–Wallis statistic instead summarizes how the ranks are distributed across groups. Pool all \(N\) observations and assign ranks, using average ranks for ties. If \(R_j\) is the sum of the ranks in group \(j\), the uncorrected statistic is
When there are no ties and the sample sizes are adequate, \(H\) is commonly compared with a chi-square distribution with \(k-1\) degrees of freedom. Ties require the standard tie correction; small samples may call for an exact or permutation reference distribution instead of relying on the chi-square approximation.
Worked Comparisons
Worked Example: Ordered Group Differences on a Common Scale
A hypothetical laboratory compares a quantitative assay result from five independent samples in each of three treatment groups. The measurements are: Group A, \(1,2,3,4,5\); Group B, \(7,8,9,10,11\); and Group C, \(13,14,15,16,17\). Assume the observations are independent and that the ANOVA model’s normal-error and common-variance conditions are reasonable. Calculate both statistics and compare their conclusions at the 5% level, using the chi-square approximation for Kruskal–Wallis.
Method: one-way ANOVA. The group means are \(3\), \(9\), and \(15\), and the grand mean is \((15+45+75)/15=9\). Within each group, the squared deviations from its mean sum to \(10\), so \(SS_{\text{within}}=30\). The between-group sum of squares is
The degrees of freedom are \(2\) between groups and \(12\) within groups. Therefore, \(MS_{\text{between}}=360/2=180\), \(MS_{\text{within}}=30/12=2.5\), and \(F=180/2.5=72\). The F-reference p-value is approximately \(0.0000\) to four decimals (\(0.0000002\) before rounding), so ANOVA rejects equality of population means at the 5% level.
Method: Kruskal–Wallis. There are no ties. The pooled ranks for Group A are \(1\) through \(5\), for Group B are \(6\) through \(10\), and for Group C are \(11\) through \(15\). Their rank sums are \(15\), \(40\), and \(65\). Thus,
With \(2\) degrees of freedom, the approximate p-value is \(e^{-12.5/2}=0.0019\) to four decimals. Both procedures reject their respective null hypotheses. This agreement is unsurprising: the values increase in a strongly ordered way across groups. Common mistake avoided: the tests’ agreement does not make their null hypotheses interchangeable; ANOVA addresses means, while Kruskal–Wallis addresses distributions.
Worked Example: An Extreme Value Changes the Comparison
A hypothetical claims analyst compares a quantitative response in three independent portfolios, with five observations per portfolio. The data are A: \(1,2,3,4,20\); B: \(5,6,7,8,9\); and C: \(10,11,12,13,14\). For calculation, use the usual ANOVA F reference and the chi-square approximation for Kruskal–Wallis. Then assess how the extreme value affects interpretation.
ANOVA calculation. The means are \(6\), \(7\), and \(12\), and the grand mean is \(125/15=8.3333\). The within-group sums of squares are \(250\), \(10\), and \(10\), respectively, giving \(SS_{\text{within}}=270\). For example, Group A’s sum is \((1-6)^2+(2-6)^2+(3-6)^2+(4-6)^2+(20-6)^2=250\). The between-group sum is
Thus, \(MS_{\text{between}}\approx51.6667\), \(MS_{\text{within}}=270/12=22.5\), and \(F\approx2.2963\), with \(2\) and \(12\) degrees of freedom. The approximate p-value is \(0.1431\), so this reference calculation does not reject equality of means at the 5% level.
Kruskal–Wallis calculation. The pooled ranks for A are \(1,2,3,4,15\), for B are \(5\) through \(9\), and for C are \(10\) through \(14\). The rank sums are \(25,35,60\). Therefore,
The approximate chi-square p-value with \(2\) degrees of freedom is \(e^{-6.5/2}=0.0388\). The approximate procedures therefore give different decisions at the 5% level. The extreme value has substantial influence on the mean and on ANOVA’s squared deviations, but it contributes only the largest rank to Kruskal–Wallis. Since each group is small, both reference approximations warrant caution; the extreme value also makes the ANOVA normal-error assumption questionable. Common mistake avoided: do not report the ANOVA result as reliable merely because its formula can be calculated. Its model conditions still need assessment.
Worked Example: Choosing a Conclusion When Group Shapes May Differ
A hypothetical study compares a quantitative measurement in three independent groups: A: \(1,2,3,20\); B: \(4,5,6,7\); and C: \(8,9,10,11\). The unusual high value in A raises concern about normality. Calculate both statistics and state what each result can establish. Use the chi-square approximation for Kruskal–Wallis, while recognizing that four observations per group make that approximation tentative.
ANOVA calculation. The group means are \(6.5\), \(5.5\), and \(9.5\); the grand mean is \(86/12=7.1667\). The within-group sums of squares are \(245\), \(5\), and \(5\), so \(SS_{\text{within}}=255\). The between-group sum is
There are \(2\) and \(9\) degrees of freedom. Hence \(MS_{\text{between}}\approx17.3333\), \(MS_{\text{within}}=255/9\approx28.3333\), and \(F\approx0.6118\). The approximate p-value is \(0.5638\); this calculation does not reject equality of means at the 5% level. The large within-group variation, including the extreme observation, contributes to the denominator.
Kruskal–Wallis calculation. There are no tied values. The rank sums are \(18\) for A (ranks \(1,2,3,12\)), \(22\) for B (ranks \(4,5,6,7\)), and \(38\) for C (ranks \(8,9,10,11\)). Thus,
The approximate p-value with \(2\) degrees of freedom is \(e^{-4.3077/2}\approx0.1160\). This calculation also does not reject at the 5% level. Neither result proves that the groups are identical. More importantly, Kruskal–Wallis does not convert this comparison into a test of medians unless the population distributions have a common shape and spread. The small group sizes make the chi-square approximation especially tentative.
Common Traps and Practical Interpretation
Choosing between these methods is not a mechanical rule such as “use ANOVA for normal data and Kruskal–Wallis for everything else.” Start by identifying the target of inference. If the scientific question concerns average response and the ANOVA assumptions are plausible, ANOVA directly addresses that question. If the response is ordinal, or ranks are more appropriate than the original measurement scale, Kruskal–Wallis may be suitable. Neither test removes the need to check independence, which comes from the study design.
- Wrong degrees of freedom for Kruskal–Wallis: using \(k\) instead of \(k-1\) compares the statistic with the wrong reference distribution. With three groups, use \(2\) degrees of freedom, not \(3\).
- Calling every Kruskal–Wallis rejection a median difference: the null concerns identical distributions. Differences in spread or shape can also contribute to a rejection.
- Assuming ranks are unaffected by outliers: the exact size of an extreme value does not determine its rank, but that observation still occupies an extreme rank and can affect the statistic.
- Ignoring sample size and ties: the chi-square reference is an approximation. With small samples, or when exact inference matters, check whether an exact or permutation method is appropriate; tied ranks require a correction.
- Treating a non-rejection as proof of equality: a non-significant result means the data did not provide sufficient evidence against that test’s null at the chosen level. It does not establish that the groups are identical.
A clear report names the procedure, its null hypothesis, the relevant conditions, the test statistic and reference distribution, and the conclusion in context. If a Kruskal–Wallis test rejects, further analysis may be needed to identify which groups differ; the overall result alone does not specify the differing pairs.
Check Your Understanding
For each problem, state the method or interpretation that fits the information given. Treat any requested chi-square approximation as an approximation, not an exact small-sample result.
- Three independent groups have normally distributed measurements with a common population variance. The research question concerns average response. Which test directly addresses the question, and what is its null hypothesis? (Answer: One-way ANOVA; \(H_0:\mu_1=\mu_2=\mu_3\).)
- Four groups have independent ordinal ratings, and only the ordering of ratings is meaningful. Which overall test is appropriate, and what degrees of freedom are used for its usual chi-square approximation? (Answer: Kruskal–Wallis; \(4-1=3\) degrees of freedom.)
- A Kruskal–Wallis test rejects when group distributions have clearly different spreads. Can the result be reported simply as evidence that the medians differ? (Answer: No; the test concerns identical distributions, and differing spreads can contribute to rejection.)
- In a three-group ANOVA, \(SS_{\text{between}}=48\), \(SS_{\text{within}}=72\), and each group has \(5\) observations. Calculate \(F\). (Answer: \(df_{\text{between}}=2\), \(df_{\text{within}}=12\), \(MS_{\text{between}}=24\), \(MS_{\text{within}}=6\), so \(F=4\).)
- A Kruskal–Wallis statistic is \(H=7.0\) for five independent groups with no ties. Using the usual chi-square approximation, what degrees of freedom apply, and is the approximate result significant at 5%? (Answer: \(4\) degrees of freedom; the 5% critical value is approximately \(9.488\), so \(7.0<9.488\) and the result is not significant.)