Why Invert a Rank-Sum Test?
A two-sample rank-sum test can assess whether two independent populations differ in location, but a test result alone does not say which shift values are compatible with the data. Inverting the test answers that question: it gives a confidence interval for the amount by which one population is shifted relative to the other. The interval is built from all pairwise differences between observations, not just from the difference between the sample medians.
The interpretation requires a specific model. If the two populations have different shapes or spreads, a rank-sum result need not represent a difference in location alone. The procedure below is aligned with the SOA VEE Mathematical Statistics syllabus.
This is a common-shape location-shift model: the populations have the same distributional shape and differ only by translation. The continuous-distribution assumption avoids ties in the rank test under a hypothesized shift. The model does not require normality. If the populations differ in spread or shape, the resulting interval should not automatically be described as an interval for a difference in medians.
From a Rank-Sum Test to Ordered Differences
For a proposed shift \(\Delta_0\), compare each first-sample observation after subtracting \(\Delta_0\) with each second-sample observation. Define
This Mann–Whitney form of the rank-sum statistic counts how many adjusted first-sample observations exceed second-sample observations. Under the null hypothesis \(\Delta=\Delta_0\), the adjusted first sample \(X_i-\Delta_0\) and the second sample \(Y_j\) have the same continuous distribution. Consequently, their pooled ranks have the usual null distribution for the rank-sum statistic.
There are \(N=nm\) pairwise differences. Sort them from smallest to largest:
For a central two-sided test, choose an integer \(r\) so that the rank-sum test accepts when \(r\le U(\Delta_0)\le N-r\). Inverting that acceptance region produces the order-statistic interval
Under the continuous model, equality between a hypothesized shift and an observed pairwise difference has probability zero. The displayed closed-interval convention is therefore appropriate for reporting the endpoints. The value of \(r\) depends on the sample sizes and the desired confidence level; it is not chosen by looking for convenient data values.
Why the Endpoints Have These Indices
The connection follows from counting how many pairwise differences lie above a proposed shift. Write the sorted differences as \(D_{(1)},\ldots,D_{(N)}\). Away from an endpoint, \(U(\Delta_0)\ge r\) means at least \(r\) differences exceed \(\Delta_0\), so \(\Delta_0\) must lie below the upper order-statistic boundary \(D_{(N-r+1)}\). Similarly, \(U(\Delta_0)\le N-r\) means at least \(r\) differences are below \(\Delta_0\), putting \(\Delta_0\) above the lower boundary \(D_{(r)}\). Both acceptance conditions together give the stated interval.
The choice of \(r\) is made from the null distribution of \(U\). Under the continuous equal-distribution null, all assignments of the \(n\) first-sample ranks among the \(n+m\) pooled ranks are equally likely. There are \(\binom{n+m}{n}\) such assignments. The distribution of \(U\) is symmetric: \(U\) and \(N-U\) have the same distribution. Therefore the coverage probability of the central acceptance region is
For a nominal confidence level \(1-\alpha\), select the largest \(r\) for which \(P(U\le r-1)\le\alpha/2\). The resulting coverage is at least \(1-\alpha\). Because \(U\) is discrete, the achieved coverage may be greater than the nominal level. In some very small samples, no nontrivial central interval can attain a requested level; the exact rank-test procedure is then too coarse for that confidence target.
Define the shift as first population minus second population, and check that independent samples from a common-shape location-shift model are reasonable.
Use the exact null distribution for the two sample sizes to select \(r\) for the desired central confidence level.
Form \(X_i-Y_j\) for every pair, giving \(nm\) values, then sort them.
Report \(D_{(r)}\) and \(D_{(nm-r+1)}\), in that order, using the same first-minus-second convention.
Worked Examples
Worked Example: An Exact 80% Interval With Small Samples
A hypothetical study compares a measurement from two independent groups. The first sample is \(A=(13,18)\), and the second is \(B=(10,12,17)\). Assume the samples are independent and come from continuous populations with the same shape, differing only by an additive shift \(\Delta\) of the first population relative to the second. Construct the central exact 80% interval. Use the convention \(D=A-B\).
Method and cutoff. Here \(n=2\), \(m=3\), and \(N=nm=6\). There are \(\binom{5}{2}=10\) possible assignments of the first-sample ranks. The exact null probabilities for \(U=0,1,\ldots,6\) are \(1/10,1/10,2/10,2/10,2/10,1/10,1/10\). For an 80% interval, \(\alpha=0.20\), so each tail must have probability at most \(0.10\). With \(r=1\), \(P(U\le r-1)=P(U=0)=0.10\); the acceptance region is \(1\le U\le5\), with coverage \(0.80\).
Calculate every pairwise difference.
After sorting, the six differences are \(-4,1,1,3,6,8\). Thus \(D_{(1)}=-4\) and \(D_{(6)}=8\). The 80% interval is \([-4,8]\) measurement units.
Check and interpretation. The interval has the required two endpoints because \(r=1\) and \(N-r+1=6\). It includes zero, so zero shift is in the test’s acceptance region. The common mistake is to list the repeated value \(1\) only once or to replace it with another value: there are six pairwise comparisons, and both calculations equal to \(1\) must be retained.
Worked Example: Using a Nontrivial Order-Statistic Cutoff
A hypothetical process study records independent samples \(A=(8,13,22)\) and \(B=(4,10,17)\). Assume continuous populations with a common shape and a location shift \(\Delta\) for the first population relative to the second. Find a central exact 80% interval for \(\Delta\).
Find the cutoff. Now \(n=m=3\), so \(N=9\) and there are \(\binom{6}{3}=20\) rank assignments. The exact probabilities for \(U=0,1,\ldots,9\) have counts \(1,1,2,3,3,3,3,2,1,1\), each divided by 20. For \(r=2\), \(P(U\le r-1)=P(U\le1)=(1+1)/20=0.10\). The central acceptance region \(2\le U\le7\) therefore has probability \(1-2(0.10)=0.80\). The next larger cutoff would make the tail probability too large.
Calculate and sort the differences. The nine values are
In increasing order they are \(-9,-4,-2,3,4,5,9,12,18\). The lower endpoint is \(D_{(2)}=-4\). The upper endpoint is \(D_{(9-2+1)}=D_{(8)}=12\). Hence the interval is \([-4,12]\) measurement units.
Check and interpretation. There are nine values, and the second and eighth are \(-4\) and \(12\), respectively. The interval contains zero, so the data do not rule out zero shift at this test’s corresponding level. The method avoids the trap of using the minimum and maximum differences automatically: those would be the endpoints only when \(r=1\).
Worked Example: Choosing the Cutoff for a 90% Interval
A hypothetical comparison has independent samples \(A=(10,16,27)\) and \(B=(4,12,19)\). Assume both populations are continuous, have the same shape, and differ only by the shift of the first relative to the second. Construct a central exact 90% interval.
Select \(r\). With \(n=m=3\), there are 20 equally likely rank assignments. The probability \(P(U=0)\) is \(1/20=0.05\), so \(r=1\) gives central coverage \(1-2(0.05)=0.90\). A cutoff \(r=2\) would have lower-tail probability \(P(U\le1)=2/20=0.10\), which exceeds the allowed \(0.05\).
Form the differences.
Sorted, these are \(-9,-3,-2,4,6,8,12,15,23\). Because \(r=1\), the endpoints are \(D_{(1)}=-9\) and \(D_{(9)}=23\). The 90% interval is \([-9,23]\) measurement units.
Check and interpretation. The sorted list contains nine values, and its first and ninth values give the interval endpoints. This interval is wide, which reflects the limited information in three observations per group. The median of the nine pairwise differences is \(6\), a point estimate of the shift, but it is not a substitute for the confidence interval.
Interpretation and Common Traps
The interval describes shift values that the central rank-sum tests would not reject. Under the stated model, it is an interval for the common additive shift in population location. Its rank-based construction does not make the location-shift assumption unnecessary: if one group is much more variable or has a different shape, the test may respond to those differences as well.
- Reversing the subtraction reverses the answer. With \(D=A-B\), positive values indicate larger measurements in group A. If instead you calculate \(B-A\), the interval endpoints change sign and their order reverses.
- Using the wrong number of differences changes the endpoints. There are \(nm\) pairwise differences, including repeats if any occur. Do not count only distinct numerical values.
- Confusing the point estimate with the interval. The median pairwise difference is a location estimate; the rank-test cutoff determines the confidence interval endpoints.
- Assuming the nominal confidence is always attained exactly. The rank statistic is discrete. The actual coverage is at least the target when the cutoff rule is applied, and it can exceed that target.
- Overstating what the interval measures. Without a common-shape location-shift model, a rank-sum interval cannot automatically be interpreted as a confidence interval for a difference in medians.
Check Your Understanding
For each problem, use the stated first-minus-second convention and assume independent samples from continuous populations with a common shape and an additive shift.
- Samples are \(A=(14)\) and \(B=(10,12)\). Use \(r=1\). List the pairwise differences and give the interval. (Answer: differences \(2,4\); interval \([2,4]\).)
- Samples are \(A=(9,15)\) and \(B=(6,11,18)\). Use \(r=1\). Sort all pairwise differences and give the interval. (Answer: differences \(-9,-3,-2,3,4,9\); interval \([-9,9]\).)
- Samples are \(A=(8,14,23)\) and \(B=(5,10,17)\). Use \(r=2\). Give the sorted differences and the interval. (Answer: differences \(-9,-3,-2,3,4,6,9,13,18\); interval \([-3,13]\).)
- For samples of sizes \(2\) and \(3\), the pairwise differences are \(-10,-4,-3,3,5,9\). At the proposed shift \(\Delta_0=1\), calculate \(U(\Delta_0)\). (Answer: \(U(1)=3\).)
- For \(n=m=3\), the exact rank-assignment counts for \(U=0,\ldots,9\) are \(1,1,2,3,3,3,3,2,1,1\), out of 20. For a central 80% interval, choose the largest \(r\) with \(P(U\le r-1)\le0.10\), and state the order-statistic indices when \(N=9\). (Answer: \(r=2\); use \(D_{(2)}\) and \(D_{(8)}\).)