Why Two-Sided Exact Rank-Sum Tests Need a Convention
The rank-sum test compares two independent samples by pooling their observations, ranking the pooled values, and adding the ranks belonging to one sample. For a two-sided test, the observed rank sum can be unusually small or unusually large. The challenge is that a discrete null distribution can have unequal tail probabilities and jumps at its possible values. There is therefore more than one convention for combining the two sides into a p-value.
This tutorial calculates exact two-sided p-values directly from the null allocation distribution. The main convention used is twice the smaller inclusive tail probability, capped at 1. A second exact convention, probability ordering, is also defined and compared. Because the conventions can differ, report which one is being used.
The exchangeability condition holds, for example, when the samples are independent random samples from the same continuous distribution. The rank-sum test can also be viewed as a conditional permutation test: given the pooled observations, the group labels are interchangeable under the null. Independence and a valid sampling or randomization design matter; the calculation does not make dependent observations exchangeable.
The no-ties condition makes the ranks exactly \(1,\ldots,N\). If pooled observations are tied, average ranks may be used, but allocations of those tied ranks require appropriate handling; the simple distinct-rank enumeration below should not be applied without adjustment. These exact calculations also do not use the normal approximation.
Constructing the Exact Null Distribution
Under the null, every subset of \(n_1\) pooled ranks is one possible allocation to sample 1. For each subset, add its ranks. If \(c(r)\) subsets have rank sum \(r\), then the null probability of that sum is \(c(r)/\binom{N}{n_1}\). This count-based method is practical for small samples and makes the meaning of each tail probability explicit.
Here \(P_0\) denotes probability under the null. The possible rank sums are discrete, so inclusive tails include the observed value: \(P_0(R_1\le r_{\mathrm{obs}})\) and \(P_0(R_1\ge r_{\mathrm{obs}})\). Using strict inequalities instead would exclude the observed outcome and would not give the stated exact-tail convention.
Worked Example: A Small Rank Sum and Its Opposite Tail
Two independent groups contain \(n_1=2\) and \(n_2=3\) observations. Assume all five pooled observations are distinct and the null hypothesis makes the group labels exchangeable. Sample 1 receives pooled ranks 1 and 3, so its observed rank sum is \(R_1=1+3=4\). Calculate the two-sided p-value using doubled inclusive tails.
Method. Enumerate the \(\binom{5}{2}=10\) possible pairs of ranks assigned to sample 1. The pairs and their sums are \((1,2):3\), \((1,3):4\), \((1,4):5\), \((1,5):6\), \((2,3):5\), \((2,4):6\), \((2,5):7\), \((3,4):7\), \((3,5):8\), and \((4,5):9\).
The counts by rank sum are therefore 1 at 3, 1 at 4, 2 at 5, 2 at 6, 2 at 7, 1 at 8, and 1 at 9. The lower inclusive tail at the observed value is \(P_0(R_1\le4)=(1+1)/10=2/10\).
The doubled smaller-tail p-value is \(2\min(2/10,9/10)=4/10=0.4000\). As a check, the reflected extreme upper tail is \(P_0(R_1\ge8)=(1+1)/10=2/10\), so adding the lower and opposite upper tails also gives \(4/10\).
Check. The two opposite extreme tails contain two allocations each, and the four allocations are out of ten. The result is \(0.4000\). This method avoids the mistake of counting only outcomes strictly below 4, which would omit the observed rank sum.
Why the Null Distribution Is Symmetric
Symmetry is useful because it identifies the tail on the opposite side of the null center without enumerating it separately. The correct pairing reflects each selected rank within the pooled set; it does not take the complement of the selected set. A complement has \(n_2\) ranks, so it is not generally another allocation of \(n_1\) ranks to sample 1.
The reflection is a one-to-one pairing: reflecting twice returns the original allocation. Consequently, an allocation with sum \(r\) has a partner with sum \(n_1(N+1)-r\), and both allocations have the same null probability. The center is half the sum of each paired pair of rank sums. This argument remains valid whether or not the two sample sizes are equal.
Pool and rank the observations, then calculate \(R_1\) for the sample named in the test.
For distinct ranks under exchangeability, count the \(\binom{N}{n_1}\) allocations to sample 1 and their rank sums.
For doubled inclusive tails, calculate both inclusive tails and double the smaller one, capping at 1. For probability ordering, sum the probabilities of outcomes no more likely individually than the observed outcome.
Confirm that allocation counts sum to \(\binom{N}{n_1}\), and use reflection to check symmetry about \(n_1(N+1)/2\).
Two Exact Two-Sided Conventions
For the doubled inclusive-tail convention, define the exact p-value as twice the smaller of the two inclusive tail probabilities, with a maximum of 1. This convention counts the observed outcome in each tail containing it. Symmetry can simplify the calculation: a low observed rank sum has a reflected high rank sum, with the same tail probability.
A different exact convention uses probability ordering. It sums the null probabilities of all possible rank sums whose individual probability is no greater than the probability of the observed rank sum. This is an exact two-sided rule, but it is not generally the same rule as doubling the smaller inclusive tail. Neither calculation should be presented as though the convention were universal.
Worked Example: Unequal Inclusive Tails in a 2-versus-4 Design
Two independent groups have sizes \(n_1=2\) and \(n_2=4\), with six distinct pooled observations. Under the exchangeable null, sample 1 receives two of the six ranks. Suppose its observed rank sum is \(R_1=10\). Find the doubled inclusive-tail p-value, and verify the allocation counts.
There are \(\binom{6}{2}=15\) allocations. Listing the rank pairs gives the following counts: sum 3 occurs once; 4 occurs once; 5 occurs twice; 6 occurs twice; 7 occurs three times; 8 occurs twice; 9 occurs twice; 10 occurs once; and 11 occurs once. The counts total \(1+1+2+2+3+2+2+1+1=15\).
The lower inclusive tail at 10 contains the allocations with sums 3 through 10: \(1+1+2+2+3+2+2+1=14\). Thus \(P_0(R_1\le10)=14/15\). The upper inclusive tail at 10 contains the sums 10 and 11, so \(P_0(R_1\ge10)=1+1=2\) allocations out of 15.
The smaller inclusive tail is \(2/15\), so the doubled-tail result is \(2(2/15)=4/15=0.2667\), rounded to four decimal places. Reflection maps a sum \(r\) to \(2(7)-r=14-r\); in particular, sums 3 and 11 pair, as do sums 4 and 10. This verifies that the two extreme tails have equal counts.
Check. The lower inclusive tail has 14 allocations, not 13, while the upper inclusive tail has 2. The p-value is \(4/15\). This calculation avoids treating the larger lower tail as the relevant tail or confusing the complementary set of four ranks with another size-two allocation.
Worked Example: Probability Ordering Can Give a Different Answer
Suppose \(n_1=3\), \(n_2=3\), and the six pooled observations have distinct ranks. Under the exchangeable null there are \(\binom{6}{3}=20\) allocations. The observed sample-1 rank sum is 9. Calculate both the doubled inclusive-tail p-value and the probability-ordering p-value.
Method. Enumerating the 20 rank triples gives these counts: sum 6: 1; 7: 1; 8: 2; 9: 3; 10: 3; 11: 3; 12: 3; 13: 2; 14: 1; and 15: 1. They sum to 20, and the distribution is symmetric about \(3(6+1)/2=10.5\).
For doubled inclusive tails, the lower tail through 9 contains \(1+1+2+3=7\) allocations out of 20. The upper tail from 9 contains \(20-(1+1+2)=16\) allocations, since the sums strictly below 9 account for four. Therefore, \(p_{\mathrm{double}}=2\min(7/20,16/20)=14/20=0.7000\).
For probability ordering, the observed sum 9 has point probability \(3/20\). No possible rank sum has a point probability greater than \(3/20\): the largest counts in the table are 3. Thus every possible sum meets the probability-ordering condition, and the p-value is \(20/20=1.0000\).
Check. The two results differ because the conventions include outcomes differently. The doubled-tail convention compares cumulative tails, whereas probability ordering compares the probability at each individual outcome with the observed point probability. Both calculations are exact under their stated conventions; the convention must accompany the reported result.
Common Errors and Practical Checks
An exact p-value is a null probability calculated using a specified rule. It is not the probability that the null hypothesis is true. For the rank-sum test, the enumeration also depends on which group's ranks are being summed: switching from \(R_1\) to the other group's rank sum changes the statistic and its center, although the group labels remain exchangeable under the null.
- Using strict tails. A tail at an observed value includes that value. Replacing \(\le\) or \(\ge\) with a strict inequality omits probability mass and changes the exact convention.
- Assuming two-sided means exactly two times one tail in every discrete calculation. The doubled smaller inclusive-tail rule is a defined convention. State it, and cap the result at 1.
- Pairing allocations with their complements. A complement has size \(n_2\), not necessarily \(n_1\). Use rank reflection \(i\mapsto N+1-i\) to prove symmetry for the distribution of \(R_1\).
- Ignoring ties or dependence. The simple allocation calculation assumes distinct ranks and exchangeability. Ties, paired observations, or other dependence require a method matched to that data structure.
- Reporting an exact p-value without its convention. Doubled inclusive tails and probability ordering can disagree, as the third example shows. Name the rule used.
Check Your Understanding
For each question, use the stated convention and assume independent samples, distinct pooled observations, and exchangeable group labels under the null.
- One group has size 1 and the other has size 4. The observed rank sum for the first group is 1. Calculate the doubled inclusive-tail p-value. (Answer: \(2/5=0.4000\))
- Two groups each have size 2, so the pooled ranks are 1 through 4. The first group's observed rank sum is 5. Calculate the doubled inclusive-tail p-value, using the rank-sum allocation distribution. (Answer: \(1.0000\))
- Group sizes are 2 and 4, and the first group's observed rank sum is 9. Using the 15 possible pairs of ranks from 1 through 6, calculate the doubled inclusive-tail p-value. (Answer: \(8/15=0.5333\))
- Group sizes are 3 and 3, with pooled ranks 1 through 6, and the observed first-group rank sum is 8. Calculate the probability-ordering p-value from the exact allocation counts. (Answer: \(8/20=0.4000\))
- Group sizes are 2 and 5, and the first group receives the two lowest pooled ranks, giving an observed rank sum of 3. Calculate the doubled inclusive-tail p-value from the \(\binom{7}{2}=21\) allocations. (Answer: \(2/21=0.0952\))