Hodges-Lehmann Estimates and Rank-Based Intervals

Why Estimate a Shift in Measurement Units?

A two-sample rank test can assess whether observations in one group tend to be higher than those in another. But a rank statistic alone does not report the size of a difference in the original units. If the groups differ by a reasonably consistent shift, it is useful to estimate that shift directly and to give an interval expressing its uncertainty.

The Hodges-Lehmann estimate does this using every cross-group pair, rather than subtracting just the two sample medians. Its associated rank-based confidence interval is obtained by inverting a two-sample rank-sum test. Both the interpretation and the interval depend on the common-shift model: the two population distributions should have the same shape and spread, with one displaced from the other.

Definition: Let $X_1,\ldots,X_{n_X}$ be observations from one independent group and $Y_1,\ldots,Y_{n_Y}$ observations from another. Define the cross-group differences by $D_{ij}=Y_j-X_i$, for every $i,j$. The two-sample Hodges-Lehmann estimate is the median of the $n_Xn_Y$ values $D_{ij}$. If there is an even number of values, the median is the average of the two middle values after sorting.

The sign convention here is group $Y$ minus group $X$. A positive estimate means that cross-group differences typically favor larger values in group $Y$; a negative estimate means they typically favor larger values in group $X$. The estimate has the same units as the original measurements.

Calculating the Estimate

To calculate the estimate, form all $n_Xn_Y$ differences, sort them, and take their median. Each observation in group $X$ is paired in this calculation with every observation in group $Y$. These are mathematical cross-pairs, not necessarily matched individuals or repeated measurements.

Worked Example: Finding the Median of All Cross-Group Differences

Suppose a hypothetical comparison records completion times, in minutes, for three participants in group $X$ and three in group $Y$. The values are $X=(4,7,9)$ and $Y=(8,11,14)$. Calculate the Hodges-Lehmann estimate for $Y-X$.

The nine cross-group differences are:

\begin{array}{c|rrr} & X_1=4 & X_2=7 & X_3=9\\ Y_1=8 & 4 & 1 & -1\\ Y_2=11 & 7 & 4 & 2\\ Y_3=14 & 10 & 7 & 5 \end{array}

Sorting them gives $-1,1,2,4,4,5,7,7,10$. There are nine values, so the median is the fifth value:

\widehat{\theta}_{HL}=4\text{ minutes}.

The estimate indicates a typical group-$Y$-minus-group-$X$ shift of 4 minutes under a common-shift interpretation. It is not the difference between one specially chosen pair of participants.

When the number of cross-group differences is even, average the two central sorted values. For example, with $X=(2,5)$ and $Y=(7,11)$, the four differences are $5,2,9,6$. In order, they are $2,5,6,9$, so the estimate is $(5+6)/2=5.5$. Reversing the group order reverses every difference and changes the estimate’s sign.

What Does a Common Shift Mean?

Write the common-shift model as $Y$ having the same distribution as $X+\theta$, where $\theta$ is a fixed location shift. This says the distributions have the same shape and spread, but group $Y$ is displaced by $\theta$ measurement units. It does not require the individual observations to be paired.

Under this model, a population version of the pairwise difference has the form $\theta+(X_2-X_1)$, where $X_1$ and $X_2$ are independent draws from the common baseline distribution. Swapping $X_1$ and $X_2$ changes $X_2-X_1$ to its negative without changing its distribution. Thus the difference is symmetric about zero, and, for a continuous distribution, its median is zero. The median pairwise difference is therefore $\theta$.

This explains why the Hodges-Lehmann estimate targets the shift in this model. If the population distributions differ in shape or spread, the median of cross-group differences can still be calculated, but interpreting it as a single shift that describes the full distributions may be misleading.

Model and assumptions: For the usual exact rank-based interval, assume the two groups are independent random samples, and that under a candidate shift $\theta$, the observations $Y_j-\theta$ and $X_i$ come from the same continuous distribution. The continuity assumption means ties have probability zero in the model. Independence and the common-shift model justify the rank-sum reference distribution; without the common-shift model, the interval need not represent a single population shift.

Building a Rank-Based Confidence Interval

A confidence interval for the common shift can be constructed by considering possible values of $\theta$. For each candidate, subtract $\theta$ from the group-$Y$ observations and compare those adjusted values with group $X$. Equivalently, count how many cross-group differences exceed $\theta$. This is a Mann-Whitney rank-sum statistic for the adjusted groups.

Let $N=n_Xn_Y$, and sort all observed cross-group differences as $D_{(1)}\leq \cdots \leq D_{(N)}$. For a candidate shift $\theta$, define $U(\theta)$ as the number of differences greater than $\theta$. Under the null candidate, with independent observations from the same continuous distribution after adjustment, the rank allocation is equally likely among the $\binom{n_X+n_Y}{n_X}$ possible allocations of group labels. This gives the rank-sum reference distribution for $U(\theta)$.

The two tails of this distribution represent candidate shifts that make one adjusted group too consistently larger or too consistently smaller than the other. For a two-sided interval with nominal confidence level $1-\alpha$, choose the largest nonnegative integer $c$ satisfying $2P(U\leq c)\leq\alpha$ under that rank-sum reference distribution. The doubled tail probability is appropriate because the distribution is symmetric about $N/2$.

Rank-based interval: With no ties, the interval obtained by inverting the two-sided rank-sum test is $$ \left[D_{(c+1)},D_{(N-c)}\right], $$ where $c$ is chosen from the null distribution as described above. Because the reference distribution is discrete, the actual confidence level is generally at least the nominal level, rather than necessarily equal to it.

The order-statistic endpoints follow from how $U(\theta)$ changes as $\theta$ increases. Between $D_{(r)}$ and $D_{(r+1)}$, exactly $r$ differences are below $\theta$, so $U(\theta)=N-r$. A candidate is accepted when $c+1\leq U(\theta)\leq N-c-1$. Substituting $N-r$ and rearranging gives $c+1\leq r\leq N-c-1$. Thus the accepted candidate shifts extend from the $(c+1)$th ordered difference to the $(N-c)$th.

Worked Example: Constructing a 90% Rank-Based Interval

Consider hypothetical independent measurements, in millimeters, with group values $X=(0,2.2,5.5,10.1)$ and $Y=(3.2,7.7,12.4,18.9)$. Assume a common shift is a reasonable model and use the exact two-sided rank-sum procedure for a 90% interval.

The 16 differences $Y_j-X_i$, sorted from smallest to largest, are:

-6.9,\,-2.4,\,-2.3,\,1.0,\,2.2,\,2.3,\,3.2,\,5.5,\, 6.9,\,7.7,\,8.8,\,10.2,\,12.4,\,13.4,\,16.7,\,18.9.

There are $N=4\times4=16$ pairwise differences. Under the null shift, there are $\binom{8}{4}=70$ possible assignments of four of the eight pooled ranks to group $Y$. For these sample sizes, the counts of assignments for $U=0,1,\ldots,8$ are $1,1,2,3,5,3,2,1,1$. Therefore, $P(U\leq1)=(1+1)/70=2/70$, and the combined two-tail probability is $4/70\approx0.0571$. For $c=2$, the combined tail probability is $2(1+1+2)/70=8/70\approx0.1143$, which exceeds $\alpha=0.10$. Hence the largest suitable value is $c=1$.

The interval endpoints are the $(c+1)=2$nd and $(N-c)=15$th ordered differences:

[D_{(2)},D_{(15)}]=[-2.4,16.7]\text{ millimeters}.

The interval is conservative for a 90% target: its coverage under the continuous common-shift model is $1-4/70\approx0.9429$. In context, the data give this rank-based interval for the population location shift in group $Y$ relative to group $X$. The interval is wide, so these small samples leave substantial uncertainty about the shift.

Interpreting the Estimate and Interval

The point estimate and interval answer related but different questions. The Hodges-Lehmann estimate summarizes the observed pairwise differences with one number. The interval gives a range of shift values that are not rejected by the corresponding two-sided rank-sum tests at the chosen level, under the stated model and assumptions. It is expressed in the original measurement units, not in rank units.

For the usual exact interval, the nominal confidence level is a repeated-sampling property: if the assumptions hold and the same procedure is applied to many independent samples, the resulting intervals cover the true shift at least as often as the nominal level. It is not a statement that a fixed parameter has a specified probability of lying in the particular observed interval.

A rank-based interval is useful when a shift in location is the scientific target and a normal-model interval is not preferred. Its rank foundation does not make the common-shift assumption unnecessary. If the groups have notably different spreads or shapes, a single shift may not adequately describe their difference; consider whether that summary matches the scientific question before reporting it.

Common Mistakes

Using only the two sample medians. The Hodges-Lehmann estimate is the median of all cross-group differences. It is not generally the difference between the sample medians.
Pairing observations that are not matched. Cross-group differences in this calculation are all possible combinations, not a set of one-to-one paired measurements.
Reversing the direction without changing the sign. With $D_{ij}=Y_j-X_i$, positive values indicate larger measurements in group $Y$. Using $X_i-Y_j$ instead reverses the sign and the interval endpoints.
Interpreting the interval as a difference in medians in every setting. Its common location-shift interpretation requires the distributions to have the same shape and spread apart from a shift.
Treating ties as if the continuous exact calculation applied unchanged. Ties can change the rank-sum reference distribution. Use a method that accounts for ties rather than applying the no-tie exact probabilities without adjustment.

Key takeaway: Form every $Y-X$ cross-group difference and take their median for the Hodges-Lehmann estimate. A rank-based interval can quantify uncertainty in a common shift, provided the independent-sample and common-shift assumptions are reasonable and the rank calculation handles ties appropriately.

Check Your Understanding

Try these questions without looking back.

If group $X$ has three observations and group $Y$ has five, how many cross-group differences are used in the Hodges-Lehmann estimate?
For $X=(2,5)$ and $Y=(7,11)$, list the pairwise differences and calculate the Hodges-Lehmann estimate for $Y-X$.
What does the common-shift model assume about the shapes and spreads of the two population distributions?
Why is the two-sided rank-sum interval based on ordered pairwise differences rather than on the pooled ranks alone?
What changes in the interpretation if the two groups have substantially different spreads?