Start With the Measurement and Study Design
A choice between one-way ANOVA and Kruskal–Wallis should not be made from a histogram alone. First ask what the response measures, how the groups were formed, whether observations are independent, and what difference the investigation is meant to detect. Skewness and outliers matter, but they do not override a mismatch between the study design and the test.
One-way ANOVA compares group means under a model for quantitative responses. Kruskal–Wallis uses the order of observations rather than their original numerical distances, so it can be more suitable for ordinal responses or quantitative data with severe skewness or outliers. It is not, however, a universal test of medians or an automatic remedy for poor design.
“Independent groups” means that observations in one group do not create dependence with observations in another group, and that repeated or clustered observations have not been treated as unrelated. For example, measuring the same person under several conditions does not produce independent groups. The ordinary one-way ANOVA and Kruskal–Wallis procedures are not appropriate as if each measurement came from a different person.
For ANOVA, inspect the residuals, which are the observed responses minus their fitted group means. A residual-versus-fitted plot can reveal unequal spread or structure; a normal probability plot can reveal marked departures from normality or unusual observations. Also inspect the original group distributions: residual diagnostics do not replace checking data quality, scale, and design.
What the Diagnostics Can—and Cannot—Tell You
| Diagnostic or design feature | Implication for the choice |
|---|---|
| Quantitative response; independent groups; residuals roughly symmetric with similar spread and no severe outliers | One-way ANOVA is generally reasonable, especially when group sizes are not very small. |
| Ordinal response whose numeric codes do not establish equal distances between categories | Kruskal–Wallis is usually more defensible because it uses ordering, not arithmetic on the codes. |
| Strong skewness or an influential extreme observation in otherwise independent groups | Kruskal–Wallis may be preferable for comparing rank patterns, provided the research question and group distributions support that interpretation. |
| Repeated measurements, matched observations, or substantial clustering | Neither standard independent-groups procedure accounts for the dependence. |
| Strongly different group shapes or spreads when the question is specifically about a common location or median | Neither test automatically answers that question. Kruskal–Wallis can detect distributional differences that are not location differences. |
ANOVA is often reasonably robust to moderate nonnormality, particularly with similar group sizes and no severe outliers. Robustness is not a license to ignore the design or to accept extreme heterogeneity. Conversely, ranks lessen sensitivity to the size of an extreme value, but they do not make a bad measurement reliable or remove dependence between observations.
Why Ranking Limits the Effect of an Extreme Magnitude
Suppose one observation is the largest in the pooled sample. In a rank analysis, it receives the highest rank whether its measured value is just above the others or enormously larger. In a mean-based analysis, its distance from the group mean contributes directly to the mean and to squared deviations. This is why ranks can be less sensitive to the magnitude of an outlier.
whereas increasing its numerical distance can increase its influence on a mean-based calculation. This justification is about sensitivity to magnitude, not a claim that the outlier has no effect: its position in the ordering still matters, and a data error should be investigated rather than hidden by ranking.
Worked Example: Skewness and an Extreme Observation
A hypothetical analyst compares independent claim-processing times, in days, from three separate service teams. The recorded times are:
Team A: 8, 9, 9, 10, 11, 12, 40
Team B: 13, 14, 14, 15, 16, 17, 18
Team C: 18, 19, 20, 20, 21, 22, 23
The analyst asks whether the teams’ response distributions differ. Assume each time belongs to a different randomly selected claim, no claim appears in more than one group, and the value 40 has been checked and is a valid observation.
Method: compare the scale, shape, and design, then choose the procedure. The groups are independent and the response is quantitative. Team A’s sum is \(8+9+9+10+11+12+40=99\), so its mean is \(99/7=14.1429\) days and its median is 10 days. Team B’s sum is 107, giving a mean of \(107/7=15.2857\) and a median of 15. Team C’s sum is 143, giving a mean of \(143/7=20.4286\) and a median of 20.
The ranges are also informative: Team A has range \(40-8=32\), while Team B has range \(18-13=5\) and Team C has range \(23-18=5\). Team A’s observation at 40 is far above its other values. It pulls Team A’s mean above its median and can strongly affect a mean-based comparison. The pattern therefore gives a reason to consider Kruskal–Wallis for the stated question about distributions, after confirming that the extreme value is valid.
Interpretation and check. A rank-based test would reduce the influence of the size of 40 relative to a mean-based method, but Team A’s distribution may differ in shape or spread as well as location. A significant rank-based result would not by itself establish a difference in medians. The recommended choice is conditional on the question being about overall distributional differences; if the target is specifically a location difference despite these shape differences, neither standard procedure automatically answers it. The common mistake avoided is treating “nonparametric” as meaning “tests medians regardless of distribution shape.”
Ordinal Responses: Respect the Scale
An ordinal response records an order, such as “poor,” “fair,” “good,” “very good,” and “excellent.” It is sometimes stored as codes 1 through 5, but those codes do not establish that the gap from 1 to 2 is equal to the gap from 4 to 5. Consequently, a mean or an arithmetic average of category codes can assign meaning to distances the measurement scale does not provide. Kruskal–Wallis can be considered because it uses order, while ties among responses must be handled appropriately in its calculation.
Worked Example: Comparing Ordered Rating Categories
In a hypothetical survey, independent customers rate three different service designs on an ordered five-category scale: 1 = very poor, 2 = poor, 3 = acceptable, 4 = good, and 5 = excellent. The recorded responses are:
Design A: 1, 2, 2, 3, 3, 4
Design B: 2, 3, 3, 4, 4, 5
Design C: 3, 4, 4, 4, 5, 5
Assume each customer rated only one design and was sampled independently. The research question is whether the response distributions differ across designs.
Method: preserve the ordinal scale and compare the category ordering. Each group has six observations, so its two middle categories are the third and fourth ordered responses. For A they are 2 and 3; for B they are 3 and 4; for C they are 4 and 4. If one median category is required, use the explicitly stated lower-middle-category convention: the medians are category 2 for A, category 3 for B, and category 4 for C. Do not average the middle codes to report 2.5, 3.5, or 4.
The category counts also show the ordering without assuming equal category distances. A has two responses in category 2 and two in category 3; B has two in category 3 and two in category 4; C has three in category 4 and two in category 5. These summaries support using an ordinal rank procedure for the distributional question. A one-way ANOVA on codes would treat differences between adjacent codes as equal-sized numerical steps, an assumption not justified by the scale description.
Interpretation and check. The customers are independent across groups, the response has a meaningful order, and the values are not being treated as measurements on an equal-interval scale. Kruskal–Wallis is therefore the more defensible of these two choices, with an appropriate treatment of tied categories in the test calculation. The common mistake avoided is reporting an average of the two central ordinal codes as a median.
When One-Way ANOVA Is a Reasonable Choice
A rank-based procedure is not automatically preferable whenever data are imperfect. If the response is genuinely quantitative and the ANOVA model is a reasonable approximation, ANOVA directly addresses differences in group means. A balanced design with similar residual spreads can tolerate some moderate departure from normality, but strong outliers, severe skewness, or very unequal variability require closer attention.
Worked Example: Diagnostics Support One-Way ANOVA
A hypothetical study compares a quantitative calibration error, in units, for three independent batches. A separate instrument is used for each sampled part, with no repeated part or shared measurement. The observations are:
Batch A: 8, 9, 10, 11, 12
Batch B: 12, 13, 14, 15, 16
Batch C: 16, 17, 18, 19, 20
Method: summarize group centers and spreads, then check the design and residual pattern. The sums are 50, 70, and 90, so the group means are \(50/5=10\), \(70/5=14\), and \(90/5=18\). For each batch, deviations from its mean are \(-2,-1,0,1,2\). The sum of squared deviations is \(4+1+0+1+4=10\); each sample variance is \(10/(5-1)=2.5\), and each sample standard deviation is \(\sqrt{2.5}\approx1.581\).
The response is quantitative, observations are independent, and the three groups have the same observed spread and a symmetric pattern around each mean. These summaries do not reveal a severe outlier or skewness problem. They support one-way ANOVA as a reasonable way to compare group means, subject to checking the residual diagnostics and study design in the full data.
Interpretation and check. This conclusion is about suitability, not a claim that the means are statistically different: no significance test has been carried out here. The common mistake avoided is choosing a rank test solely because it is described as distribution-free, even when the quantitative mean is the intended target and the ANOVA conditions look credible.
When Neither Standard Test Is Credible
A test cannot repair dependence. If the same subjects contribute multiple responses, or if observations are clustered in a way that makes responses within a cluster related, the standard independent-groups versions of both procedures use an inappropriate independence assumption. The analysis must account for the actual design.
A different warning applies when the group distributions have substantially different shapes or spreads. Kruskal–Wallis can still provide evidence that the distributions are not all the same, but that difference may arise from spread or shape rather than a shift in location. If the question is specifically whether groups have equal medians or a common location, unequal shapes can make a simple median interpretation misleading. One-way ANOVA also needs credible residual and variance assumptions for its standard inference. Choose a method matched to the estimand and design rather than forcing either test to answer a question it was not designed to answer.
Worked Example: Repeated Measurements Break Independence
A hypothetical laboratory tests the same five sensors under three temperature settings. Their measured errors are:
Sensor 1: 4, 6, 9
Sensor 2: 5, 7, 8
Sensor 3: 3, 5, 7
Sensor 4: 6, 8, 10
Sensor 5: 2, 4, 6
The columns correspond to the three settings, and each row belongs to one sensor. The research question is whether error changes with setting.
Method: identify the observational unit before selecting a test. There are 15 recorded values but only five independent experimental units. For instance, Sensor 1 contributes all three values 4, 6, and 9. Those readings share the same sensor and so are not three unrelated observations. Treating each setting’s five values as an independent group would discard the within-sensor connection.
The settings are repeated on the same sensors, so the standard one-way ANOVA and the standard independent-groups Kruskal–Wallis procedure are not credible as written. A method for repeated or blocked measurements is needed. The common mistake avoided is counting 15 measurements as 15 independent units merely because they appear as 15 numbers in the data table.
A Practical Decision Sequence
Determine whether groups are genuinely independent. If measurements are paired, repeated, or clustered, do not use the standard independent-groups versions of either test.
Use ANOVA when the response is quantitative and the target is a difference in means. Consider Kruskal–Wallis for ordered responses or when rank-based comparison is appropriate.
Look for severe skewness, influential outliers, unequal spread, and structure. Do not let a single diagnostic plot substitute for checking the data and design.
ANOVA concerns group means under its model. Kruskal–Wallis concerns rank/distributional differences; a median or location interpretation needs suitably comparable group shapes.
Check Your Understanding
For each situation, decide whether one-way ANOVA, Kruskal–Wallis, or neither standard independent-groups procedure is the defensible choice, and identify the key diagnostic or design reason.
- Three independent groups have quantitative repair times. Residual plots show roughly symmetric patterns, similar spreads, and no notable outliers. What method is reasonable if the target is a difference in mean time? (Answer: One-way ANOVA, subject to the stated model and design assumptions.)
- Independent patients each rate one of three plans as poor, fair, good, or excellent. The category order is meaningful, but equal spacing is not established. Which method better respects the scale? (Answer: Kruskal–Wallis, with appropriate handling of tied categories.)
- The same six technicians each operate three machine settings, and each technician’s output is recorded at every setting. Can the ordinary independent-groups Kruskal–Wallis test be used? (Answer: No; the repeated measurements violate its independent-groups design.)
- Three independent quantitative groups have similar medians, but one group has a much larger spread and a long upper tail. The stated target is equality of medians. Is a significant Kruskal–Wallis result automatically evidence that the medians differ? (Answer: No; distributional differences in spread or shape can produce a rank-based difference without a median difference.)
- Three independent groups have balanced sizes and mild residual skewness, with comparable spreads and no severe outliers. The response is quantitative and the target is a difference in means. Which method is generally reasonable, and what should still be checked? (Answer: One-way ANOVA is generally reasonable; still check independence, residual patterns, and whether the variance and normal-error approximations are adequate.)