Interpreting Mann-Whitney Results

What a Mann–Whitney Result Tells You

The Mann–Whitney test compares two independent groups using the ordering of their observations. Its statistic is built from ranks, but the conclusion is not simply “the medians differ.” A result can reflect differences in location, spread, or overall shape. To interpret it well, connect the test statistic to comparisons between pairs of observations, then connect the p-value to the null model used for inference.

Suppose Group A and Group B each contain measurements on the same outcome, such as recovery time in days. The test asks whether the observed ordering of the two groups is unusual if their outcome distributions are the same. A test result can provide evidence that the distributions differ; additional assumptions are needed to describe that difference as a shift in location or a difference in medians.

Definition: For independent samples of sizes $n_A$ and $n_B$, let $R_A$ be the sum of the ranks assigned to Group A after the observations are pooled and ranked. The Mann–Whitney statistic for Group A is $$ U_A=R_A-\frac{n_A(n_A+1)}{2}. $$ With no ties, $U_A$ counts the number of cross-group pairs in which an observation from A is larger than one from B. The complementary statistic is $U_B=n_An_B-U_A$. With ties, average ranks give tied pairs half credit in this count.

The direction depends on how the groups are labelled and on whether larger values represent more or less of the outcome. A large $U_A$ indicates that A tends to have higher observations than B; a small $U_A$ indicates the reverse. For a two-sided test, extremeness is assessed in either direction. The statistic is about cross-group ordering, not the difference between two sample medians.

The Null Model and Its Interpretation

A standard exact Mann–Whitney test uses the null hypothesis that the two population distributions are identical. Under this null, observations are exchangeable with respect to group labels: conditional on the pooled observations, every allocation of the specified numbers of observations to the two groups is equally likely. The alternative for a two-sided test is that the distributions are not identical. In practical analyses, the p-value may be obtained from an exact or approximate rank-sum procedure, with the appropriate method depending on sample size and ties.

Interpretation: A p-value is the probability, assuming the null model and its conditions, of obtaining a test statistic at least as extreme as the one observed. A small p-value is evidence against equality of the distributions. It is not the probability that the null hypothesis is true, nor the probability that the result “happened by chance.”

Under a continuous distribution, there are no ties, and each cross-group pair is either an A observation above a B observation or the reverse. If both groups have the same distribution, either ordering is equally likely for a randomly selected pair. This gives a useful justification for the statistic’s center under the null.

U_A=\sum_{i=1}^{n_A}\sum_{j=1}^{n_B} I(X_i>Y_j), \qquad E(U_A)=\sum_{i=1}^{n_A}\sum_{j=1}^{n_B}P(X_i>Y_j) =\frac{n_An_B}{2}.

Here $X_i$ and $Y_j$ are observations from A and B, and $I(\cdot)$ is 1 when the statement is true and 0 otherwise. Under identical continuous distributions, $P(X_i>Y_j)=1/2$, which gives the expected value shown. If ties are possible, the rank-based statistic gives tied pairs half credit; interpreting its exact null distribution then requires accounting for ties.

The expectation alone does not make every test of $U_A$ a general test of the condition that the pairwise ordering probability equals one half. The standard exact calibration relies on the stronger null of identical distributions and exchangeable labels. When distributions differ in shape or spread, an inference about a narrower “no relative ordering tendency” condition needs its own appropriate justification.

Worked Interpretations

Worked Example: Complete Separation in a Small Sample

Suppose a hypothetical experiment compares a measurement in four independent units per group. The observations are A: 11, 12, 13, 14 and B: 1, 2, 3, 4. Assume the units were independently sampled or assigned in a way that supports the test, and there are no ties. Interpret a two-sided exact test at $\alpha=0.05$.

In the pooled ordering, B has ranks 1 through 4 and A has ranks 5 through 8. Thus $R_A=5+6+7+8=26$, and

U_A=26-\frac{4(4+1)}{2}=26-10=16, \qquad U_B=4(4)-16=0.

Every one of the 16 A–B pairs has the A observation higher. Under the null of identical distributions, there are $\binom{8}{4}=70$ equally likely allocations of four of the eight ranks to A. Only one allocation puts A in the four highest ranks, giving $U_A=16$; only one puts A in the four lowest, giving $U_A=0$. Therefore the two-sided exact p-value is

p=\frac{2}{70}=0.02857.

Because $0.02857<0.05$, the result is statistically significant at the 5% level: the data provide evidence against identical population distributions. In this sample, A’s measurements are all higher than B’s, so the observed ordering favors A. The test does not by itself establish that the population medians differ; a median or location-shift conclusion needs further assumptions about the distributions.

Worked Example: A Near-Extreme Statistic Is Not Quite Significant

In another hypothetical comparison, independent samples of four observations are A: 1, 2, 3, 5 and B: 4, 6, 7, 8. Assume independent observations, no ties, and use a two-sided exact test at $\alpha=0.05$.

The ranks for A are 1, 2, 3, and 5, so $R_A=11$. Therefore,

U_A=11-\frac{4(5)}{2}=11-10=1, \qquad U_B=16-1=15.

Of the 16 cross-group pairs, A is higher in just one: the observation 5 exceeds B’s 4. For four observations in each group, there are 70 possible allocations of ranks under the exact null. One allocation gives $U_A=0$, and one gives $U_A=1$; by symmetry, two allocations are equally extreme in the upper tail. Thus

p=\frac{2+2}{70}=\frac{4}{70}=0.05714.

Since $0.05714>0.05$, this test does not reject the null at the 5% level. The result is not proof that the distributions are identical; it says that this sample does not provide sufficient evidence against the null at the chosen threshold. In particular, “almost significant” should not be reported as significant. The sample ordering favors B, but the p-value measures how unusual the statistic is under the null, not the probability that this observed ordering is false.

Worked Example: Equal Sample Medians, Different Ordering

Consider hypothetical independent samples A: 0, 0, 5, 5, 5, 100 and B: 1, 4, 5, 5, 6, 6. Both sample medians are 5: for each group of six, the middle two observations are 5 and 5. Yet the samples do not have the same pattern of values.

Compare each A observation with each B observation. There are $6\times6=36$ cross-group pairs. The three A values equal to 5 each exceed B’s 1 and 4, contributing $3\times2=6$ pairs. A’s value 100 exceeds all six B values, contributing 6 more. Thus A is higher in 12 pairs. There are also six tied pairs, since each of A’s three 5s ties with each of B’s two 5s. Giving each tie half credit, as average ranks do, produces

U_A=12+\frac{6}{2}=15, \qquad \frac{U_A}{n_An_B}=\frac{15}{36}=0.4167.

The rank statistic reflects the pairwise ordering, including half credit for ties; it is not calculated from the medians. Here the two sample medians agree, while the sample distributions differ in their spread and tail values. This example is descriptive, not a p-value calculation: ties require a tie-aware inferential procedure. It shows why a Mann–Whitney result cannot generally be translated into “the medians differ.”

When a Median Interpretation Is Reasonable

A median-focused interpretation becomes more defensible when the two population distributions have the same shape and spread and differ only by a location shift. In that setting, a distributional difference can reasonably be described as one group tending to have larger or smaller values, and the shift in location is also a shift in the median. The equal-shape condition is substantive: it should be justified by the context and data, not assumed merely because the test uses ranks.

Without that condition, a significant result can reflect other distributional differences. For example, groups might have equal medians but different variability, or might differ in the tails while their centers are similar. Conversely, a modest sample may not provide enough evidence to detect a real difference. The test result should therefore be reported with a description of the observed group distributions and an interpretation that matches the assumptions.

Assumptions and Common Pitfalls

Before interpreting a Mann–Whitney p-value, check the study design and the form of the data. The observations should be independent within and between groups, and the outcome must at least have a meaningful ordering so that ranks are informative. For the standard exact test, the null of identical distributions makes group labels exchangeable conditional on the pooled observations. Random sampling or a suitable randomization design is needed to support the broader population or causal interpretation being made.

If the data contain ties, ranks are typically assigned by averaging the ranks that would otherwise have been occupied. The statistic can still be interpreted with half credit for tied pairs, but its null distribution and p-value calculation need to account for ties. Do not apply a no-ties exact calculation to tied data without checking that the procedure is appropriate.

Pitfall: “The test compares medians.” The statistic compares ranks across groups. Correction: conclude that the test provides evidence of a distributional difference; describe a median or location shift only when the additional shape assumptions are credible.
Pitfall: “A large $U_A$ means A has a significantly higher outcome.” A large statistic gives a direction to the observed pairwise ordering, not statistical significance by itself. Correction: interpret it alongside the p-value and the chosen significance level.
Pitfall: “A p-value above $\alpha$ proves the groups are equal.” Failure to reject is not proof of equality. Correction: say that the data do not provide sufficient evidence against the null at that threshold.
Pitfall: ignoring the group labels or outcome direction. The meaning of “higher” depends on the outcome scale. Correction: name the groups and state what higher values mean in context.

Key takeaway: The Mann–Whitney statistic summarizes cross-group ordering, and its standard test evaluates evidence against identical distributions under the stated assumptions. It is not generally a test of equality of medians alone.

Check Your Understanding

Answer in context, distinguishing what the statistic shows from what the test establishes.

What does $U_A$ count when there are no ties and larger values indicate more of the outcome?
A two-sided Mann–Whitney test gives $p=0.03$ at $\alpha=0.05$. State the conclusion without claiming that the null has a 3% probability of being true.
Why is “the medians differ” not a general interpretation of a significant Mann–Whitney result?
What additional distributional condition makes a location-shift or median interpretation more reasonable?
Two independent groups have identical sample medians but different spreads. Is that incompatible with a Mann–Whitney statistic reflecting a difference? Explain.