Why Interpretation Requires More Than a Small P-Value
The Wilcoxon rank-sum test compares two independent samples by pooling their observations and ranking them. A sufficiently unusual rank sum is evidence against the null model that the two populations have the same distribution. But that conclusion is not automatically the same as saying that one population is shifted to the right of the other. A location-shift interpretation requires a stronger model: the populations must have the same shape and spread, with their difference represented by a translation.
This distinction matters in applications. Suppose a hypothetical study compares processing times under two procedures. A rank-sum result may reflect a change in the typical level, but it may instead reflect differences in skewness, tail behavior, or spread. The test statistic alone does not identify which feature differs.
Under this model, the usual rank-sum hypotheses can be expressed as \(H_0:\Delta=0\) against, for example, \(H_1:\Delta>0\). The interpretation of this alternative as a shift depends on the model being plausible. Without it, a rank-sum test is a test based on relative ranks, not a general-purpose estimator of a difference between medians.
What the Rank-Sum Statistic Measures
Let \(X_1,\ldots,X_{n_X}\) and \(Y_1,\ldots,Y_{n_Y}\) be independent random samples from their respective populations. Pool all \(N=n_X+n_Y\) observations and assign ranks from smallest to largest, using average ranks for ties. Write \(R_Y\) for the sum of the ranks assigned to the \(Y\) observations.
A large \(R_Y\) indicates that \(Y\) observations tend to occupy higher pooled ranks. This is a statement about pairwise ordering. Under a common-shape location-shift model, that ordering can reasonably be interpreted as evidence of a shift. With different shapes or spreads, the same rank pattern can arise for other reasons. Also, a rank-sum test need not detect every possible distributional difference: some alternatives can change spread while leaving the pairwise probability at one half.
For the classical permutation interpretation, the samples must be independent and observations must be exchangeable under the null of identical population distributions. In the continuous case, ties occur with probability zero; if ties occur, midranks and an appropriate tie adjustment are used. A rejection at significance level \(\alpha\) is evidence against the null under the stated testing procedure, not proof of a particular alternative model.
Why a Location Shift Moves the Median Set
The translation model does preserve the median structure, including when a distribution has more than one median. A number \(m\) is a median of \(X\) when \(P(X\leq m)\geq \tfrac12\) and \(P(X\geq m)\geq \tfrac12\). If \(Y=X+\Delta\), then
Therefore \(m+\Delta\) is a median of \(Y\) whenever \(m\) is a median of \(X\). Applying the same argument in reverse shows that every median of \(Y\) is obtained this way. So the set of medians of \(Y\) is the set of medians of \(X\) translated by \(\Delta\). If each population has a unique median, their medians differ by \(\Delta\); if not, corresponding choices differ by \(\Delta\). Selecting arbitrary medians from two sets with multiple values need not give that difference.
Ask whether the populations could reasonably have comparable shapes and spreads, with one distribution translated relative to the other.
If the common-shape model is credible, an upper-tail rejection supports a shift toward larger values in the group with the larger rank sum. Otherwise, describe evidence of a rank or distributional difference without naming it a location shift.
A non-rejection does not establish identical distributions or prove that no shift exists. It means the test did not provide sufficient evidence against its null at the chosen level.
Worked Examples
Worked Example: A Rank Difference Under a Plausible Shift Model
A hypothetical operations study independently samples eight processing times under each of two procedures, in minutes. Procedure X gives \(12,14,15,16,17,18,19,20\); procedure Y gives \(4,5,6,7,8,9,10,11\). Use a two-sided normal approximation at the 5% level, without ties, and interpret the result assuming comparable population shapes and spreads are scientifically reasonable.
Method and pooled ranks. Rank the 16 observations together. The eight Y values receive ranks 1 through 8, so \(R_Y=1+2+3+4+5+6+7+8=36\). The X values receive ranks 9 through 16, so \(R_X=9+10+11+12+13+14+15+16=100\). Check: \(R_X+R_Y=136=16(17)/2\), the sum of all ranks.
Under identical continuous distributions, the mean and variance of \(R_X\) are \(n_X(N+1)/2\) and \(n_Xn_Y(N+1)/12\). Here \(n_X=n_Y=8\) and \(N=16\), giving mean \(8(17)/2=68\) and variance \(8(8)(17)/12=90.6667\). The standard deviation is \(\sqrt{90.6667}=9.5219\).
Test calculation. The observed rank sum is \(100\), which is \(32\) above its null mean. Using a continuity correction, \(z=(100-68-0.5)/9.5219=31.5/9.5219\approx3.308\). The two-sided normal p-value is approximately \(0.0009\) to four decimal places, below \(0.05\); reject the equal-distribution null using this approximation.
Interpretation and trap avoided. Given the stated common-shape, common-spread assumption, the much higher X ranks support a location shift toward larger X values. The rank result alone does not establish that assumption. The common mistake is to report “the X median is larger” solely because the rank-sum test rejects. Without shape comparability, the defensible conclusion is evidence of a distributional or rank difference, not necessarily a median shift.
Worked Example: Same Median, Different Shapes
Consider a hypothetical comparison with ten observations in each group. The observed X values are six zeros and four tens. The observed Y values are six zeros and four negative tens. Both underlying populations are modeled as having probability \(0.6\) at zero and probability \(0.4\) at their respective nonzero value. Use the tie-corrected normal approximation to illustrate what a large rank-sum result can mean.
Check the population medians. For each population, \(P(X\leq0)=0.6\) (or \(P(Y\leq0)=1\)) and the probability of being at least zero is \(0.6\). Thus zero is the unique median in each population. However, the distributions do not have the same shape: the nonzero mass is above zero for X and below zero for Y. They are not related by a location shift.
Rank the pooled observations. The four negative tens receive ranks 1 through 4, with sum \(10\). The twelve zeros receive ranks 5 through 16, whose average rank is \((5+16)/2=10.5\). The four tens receive ranks 17 through 20, with sum \(74\). Therefore \(R_X=6(10.5)+74=137\). The other group has rank sum \(10+6(10.5)=73\); the check \(137+73=210=20(21)/2\) passes.
Under the identical-distribution null, the expected rank sum for X is \(10(21)/2=105\). With tie group sizes \(4,12,4\), the tie-corrected variance is \(100/12\left[21-\{(4^3-4)+(12^3-12)+(4^3-4)\}/\{20(19)\}\right]\). The numerator inside the fraction is \(60+1716+60=1836\), and \(1836/380=4.8316\). The variance is \((100/12)(21-4.8316)\approx134.7368\), with standard deviation \(\sqrt{134.7368}\approx11.6076\).
Using a continuity correction, \(z=(137-105-0.5)/11.6076\approx2.713\). The two-sided normal p-value is approximately \(0.0067\) to four decimal places. This is an approximation; with small samples and many ties, an exact permutation calculation may be preferable.
Interpretation and trap avoided. The observed ranks provide strong evidence against identical distributions under the approximation, even though both modeled populations have the same unique median. The different tail locations and shapes explain why a location-shift interpretation would be wrong. Do not equate “significant rank sum” with “different medians.”
Worked Example: A Small Sample Does Not Establish No Shift
In a hypothetical study, four independent observations from group X are \(2,4,6,8\), and four from group Y are \(5,7,9,11\). Suppose comparable shapes and spreads are reasonable, and test whether Y tends to be larger at the 5% level using a one-sided normal approximation without ties.
Calculate the rank sum. In pooled order the observations are X:2 (rank 1), X:4 (rank 2), Y:5 (rank 3), X:6 (rank 4), Y:7 (rank 5), X:8 (rank 6), Y:9 (rank 7), and Y:11 (rank 8). Thus \(R_Y=3+5+7+8=23\). The remaining ranks sum to \(13\), and \(23+13=36=8(9)/2\).
Under the null, \(E[R_Y]=4(9)/2=18\) and \(\operatorname{Var}(R_Y)=4(4)(9)/12=12\), so the standard deviation is \(\sqrt{12}=3.4641\). For the upper-tail test, \(z=(23-18-0.5)/3.4641=4.5/3.4641\approx1.299\). The one-sided normal p-value is approximately \(0.0969\) to four decimal places, so do not reject at \(0.05\).
Interpretation and trap avoided. The observed Y rank sum is higher, but this small sample does not provide sufficient evidence at the chosen level. Non-rejection is not proof that the populations are identical or that \(\Delta=0\). The correct conclusion is limited to the test and significance level used.
Common Interpretation Errors
- Calling every rejection a median shift. The rank-sum statistic uses pooled ordering. Differences in shape, skewness, or tails can affect that ordering even when medians agree.
- Treating a non-rejection as proof of equality. A test can lack power, particularly with small samples. Failure to reject is not evidence that every feature of the distributions matches.
- Assuming a location shift from the sample alone. Similar-looking sample spreads can be suggestive but do not prove a population model. Use subject-matter knowledge and graphical or descriptive checks where appropriate.
- Using arbitrary medians when medians are not unique. Under a shift, the median sets translate. Only corresponding median choices are guaranteed to differ by the shift amount.
- Claiming the test detects every distributional difference. Rank-sum evidence concerns relative ranks. Some distributional changes may not make one group tend to outrank the other.
Check Your Understanding
For each situation, distinguish the rank-based conclusion from a location-shift conclusion.
- Two independent groups have similar observed shapes and spreads. The second group has a substantially larger rank sum and the test rejects at 5%. State the conditional interpretation and one limitation. (Answer: If the common-shape, common-spread shift model is credible, the result supports a shift toward larger values in the second group; it does not prove that model or the shift.)
- Two populations both have unique median 0, but one puts probability 0.6 at 0 and 0.4 at 8, while the other puts probability 0.6 at 0 and 0.4 at \(-8\). Are they a location-shift pair? What would a rank-sum rejection establish? (Answer: No; their nonzero mass is on opposite sides of zero, so their shapes are not translations. A rejection is evidence against identical distributions, not evidence of a median shift.)
- A test has p-value 0.12 at significance level 0.05, and the common-shape shift model is plausible. What is the decision, and what may not be concluded? (Answer: Do not reject the equal-distribution null; this does not prove that the shift is zero or that the distributions are identical.)
- A study reports a significant rank-sum test and describes the result as “the first population has greater variability.” Is that conclusion justified by the rank-sum result alone? (Answer: No. The result indicates unusual relative ranks under the null, but it does not isolate variability as the cause; assess shape and spread separately.)
- Under a location-shift model, the set of medians of X is \(\{2,5\}\) and \(\Delta=3\). Give the median set for Y, and explain why choosing arbitrary medians matters. (Answer: The median set for Y is \(\{5,8\}\). Corresponding medians differ by 3, but arbitrary choices such as 2 and 8 differ by 6.)