Two-Sample Spatial Rank Tests

Why Pool Spatial Ranks?

In a two-sample multivariate problem, a direct comparison of group means can be strongly affected by extreme observations. Spatial ranks offer a different comparison: each observation receives a vector score based on the directions from it to the other pooled observations. The two groups can then be compared through their average score vectors.

The geometry is genuinely multivariate. A spatial rank is a vector, not an ordering of observations from smallest to largest. Its direction records where an observation lies relative to the pooled sample, while its length reflects the balance of directions around that observation. Tied observations can therefore share a rank vector, and no arbitrary ordering among them is needed.

Definition: For $x\in\mathbb{R}^p$, define the spatial sign $U(x)=x/\|x\|$ when $x\ne 0$, and $U(0)=0$. Given pooled observations $Z_1,\ldots,Z_N$, their pooled spatial ranks are $$ r_i=\frac{1}{N}\sum_{j=1}^N U(Z_i-Z_j),\qquad i=1,\ldots,N. $$ The self-comparison contributes zero. For two groups of sizes $n_1$ and $n_2$, where $N=n_1+n_2$, define the rank-mean difference $$ D=\bar r_1-\bar r_2,\qquad \bar r_1=\frac{1}{n_1}\sum_{i\in G_1}r_i,\quad \bar r_2=\frac{1}{n_2}\sum_{i\in G_2}r_i. $$

The division by $N$ is a convention that fixes the scale of the ranks; using $N-1$ instead would rescale the scores and the resulting statistic. Here the convention includes the observation itself, with its zero contribution. The pooled ranks depend only on the pooled observations, not on the group labels.

Centering and the Rank-Mean Comparison

A useful fact follows from the antisymmetry $U(x)=-U(-x)$. Each pair of distinct observations contributes opposite vectors to the sum of all ranks, so the pooled ranks are centered exactly, not just on average.

Result: For any fixed pooled observations, $\sum_{i=1}^N r_i=0$. Also, adding the same vector to every pooled observation leaves every spatial rank unchanged.

To verify the first claim, expand the sum and pair the terms:

\sum_{i=1}^N r_i =\frac{1}{N}\sum_{i=1}^N\sum_{j=1}^N U(Z_i-Z_j) =\frac{1}{N}\sum_{i<j}\bigl\{U(Z_i-Z_j)+U(Z_j-Z_i)\bigr\}=0.

When $Z_i=Z_j$, both terms in that pair are zero; otherwise they cancel by antisymmetry. For translation invariance, replacing every $Z_i$ by $Z_i+a$ leaves each difference $(Z_i+a)-(Z_j+a)=Z_i-Z_j$ unchanged. These facts imply that the two group rank sums are opposites. In particular, if $S=\sum_{i\in G_1}r_i$, then

D=\frac{S}{n_1}-\frac{-S}{n_2} =\frac{N}{n_1n_2}S.

Worked Example: A One-Dimensional Pooled Rank Comparison

Suppose group 1 contains $-2$ and $4$, while group 2 contains $0$, $1$, and $5$. In one dimension, $U(x)$ is $1$ for positive $x$, $-1$ for negative $x$, and $0$ at zero. The pooled observations in increasing order are $-2,0,1,4,5$.

For the smallest observation, all four nonzero comparisons point left, so its rank is $-4/5$. Moving from left to right, each rank increases by $2/5$: a comparison with one observation changes from $-1$ to $1$. The pooled ranks are therefore $-4/5,-2/5,0,2/5,4/5$.

Group 1 has ranks $-4/5$ and $2/5$, giving $\bar r_1=(-4/5+2/5)/2=-1/5$. Group 2 has ranks $-2/5,0,4/5$, giving $\bar r_2=2/15$. Thus $D=-1/5-2/15=-1/3$. Equivalently, the group 1 rank sum is $S=-2/5$, and $D=(5/(2\cdot3))S=-1/3$, as required by the centering identity.

Null Behavior Under Exchangeability

The rank difference can be used as a test statistic, but its null interpretation requires care. The exchangeability null says that, under the null, the pooled observations have the same distribution regardless of their group labels. For example, this holds when the two independent samples are drawn from the same distribution. It also holds under an appropriate randomized assignment and a sharp null of no treatment effect.

Equality of two population location parameters alone does not generally imply exchangeability. Two groups can have the same location but different shapes or spreads. In that situation the group labels are not interchangeable, and the exchangeability calculations below do not automatically give a valid null calibration.

Theorem: Fix the pooled observations $Z_1,\ldots,Z_N$, and suppose the $n_1$ group 1 labels are assigned uniformly among the $N$ indexed observations. Treat the resulting ranks $r_i$ as fixed. Then $$ E[D\mid Z_1,\ldots,Z_N]=0,\qquad \operatorname{Cov}(D\mid Z_1,\ldots,Z_N) =\frac{N}{n_1n_2(N-1)}\sum_{i=1}^N r_i r_i^{\mathsf T}. $$

Here the conditioning treats tied observations as indexed units. Thus two units with the same observed value remain distinct members of the pooled sample, even though they have identical spatial ranks. This distinction matters: different allocations of indexed units can produce the same visible group configuration when values are tied.

Proof. Let $I_i$ equal 1 if observation $i$ receives a group 1 label and 0 otherwise. Under uniform assignment of $n_1$ labels,

E[I_i]=\frac{n_1}{N},\quad \operatorname{Var}(I_i)=\frac{n_1n_2}{N^2},\quad \operatorname{Cov}(I_i,I_j)=-\frac{n_1n_2}{N^2(N-1)}\quad(i\ne j).

The off-diagonal covariance follows because $P(I_i=1,I_j=1)=n_1(n_1-1)/(N(N-1))$; subtracting $(n_1/N)^2$ gives the displayed value. Write $S=\sum_i I_i r_i$. Since $\sum_i r_i=0$, its conditional expectation is zero. Let $A=\sum_i r_i r_i^{\mathsf T}$. The sum of the cross-products over $i\ne j$ is $-A$, because

\sum_{i\ne j}r_i r_j^{\mathsf T} =\left(\sum_i r_i\right)\left(\sum_i r_i\right)^{\mathsf T}-A=-A.

Using the indicator variances on the diagonal and covariances off the diagonal therefore gives

\operatorname{Cov}(S\mid Z_1,\ldots,Z_N) =\frac{n_1n_2}{N^2}A +\frac{n_1n_2}{N^2(N-1)}A =\frac{n_1n_2}{N(N-1)}A.

The pooled ranks are fixed under this conditioning, and $D=NS/(n_1n_2)$. Multiplying the covariance of $S$ by $N^2/(n_1^2n_2^2)$ gives the stated covariance of $D$. Its conditional expectation is also zero. $\square$

A Covariance-Standardized Test Statistic

The conditional covariance provides a natural way to scale the group rank difference. Define $A=\sum_i r_i r_i^{\mathsf T}$ and let $V=N A/\{n_1n_2(N-1)\}$, the conditional covariance matrix in the theorem. When $V$ is nonsingular, a quadratic statistic is $W=D^{\mathsf T}V^{-1}D$. If $V$ is singular, use its Moore–Penrose inverse $V^+$, which inverts the covariance on its supported subspace and leaves its null space at zero:

W=D^{\mathsf T}V^+D.

This statistic is large when the observed rank-mean difference is large relative to the variation implied by the pooled ranks. If every rank is zero, then $V=0$ and $D=0$; there is no directional information in these scores. More generally, a singular $V$ indicates that the rank variation lies in a lower-dimensional subspace.

The theorem gives the conditional mean and covariance, not a universal finite-sample distribution for $W$. A test must compare the observed statistic with a suitable null reference calibrated under its design and assumptions. The null moments alone do not justify treating $W$ as a chi-square variable. Bootstrap calibration and permutation calibration are possible approaches in settings where their assumptions hold; exact permutation procedures require additional care and are considered separately.

Worked Example: Standardizing the One-Dimensional Difference

Use the five observations and group labels from the first example. The pooled ranks are $-4/5,-2/5,0,2/5,4/5$, so

Step 1: Calculate the pooled sum of squares. Here $A=\sum_i r_i^2=16/25+4/25+0+4/25+16/25=40/25=8/5$.

Step 2: Calculate the conditional variance. With $N=5$, $n_1=2$, and $n_2=3$, $V=5(8/5)/\{2\cdot3\cdot4\}=1/3$.

Step 3: Calculate the quadratic statistic. Since $D=-1/3$ and $V$ is a positive scalar, $W=D^2/V=(1/9)/(1/3)=1/3$. This gives a standardized observed statistic; it is not, by itself, a p-value or a claim about a chi-square distribution.

Ties and Unequal Group Sizes

A tie in the pooled data is handled directly by the spatial sign: equal observations contribute $U(0)=0$ to one another’s rank scores. Identical observations consequently receive identical spatial ranks. Breaking a tie arbitrarily would replace the geometry of the observed data with an artificial ordering.

Unequal group sizes require no change to the rank definition. They do affect the group averages and the conditional covariance, through the factor $N/(n_1n_2(N-1))$. The covariance adjustment is important: a given rank-sum difference has different variability when divided among groups of different sizes.

Worked Example: Tied Multivariate Observations and Unequal Groups

Take four pooled observations in two dimensions: $Z_1=Z_2=(0,0)$, $Z_3=(1,0)$, and $Z_4=(0,1)$. Let group 1 contain $Z_3$ alone, so $n_1=1$, and let group 2 contain the other three observations, so $n_2=3$.

The tied observations $Z_1$ and $Z_2$ each have rank $(-1/4,-1/4)$: each has two nonzero comparisons pointing in the negative coordinate directions. For $Z_3$, the two comparisons with the tied origin point in direction $(1,0)$, and the comparison with $Z_4$ points in direction $(1,-1)/\sqrt{2}$. Thus $r_3=((2+1/\sqrt{2})/4,-1/(4\sqrt{2}))$. By symmetry, $r_4=(-1/(4\sqrt{2}),(2+1/\sqrt{2})/4)$.

The sum of these four ranks is zero. Since group 1 contains $Z_3$, $D=r_3-\{(r_1+r_2+r_4)/3\}=4r_3/3$. Numerically, $r_3\approx(0.6768,-0.1768)$, so $D\approx(0.9024,-0.2357)$. The group-size factor in the conditional covariance is $N/(n_1n_2(N-1))=4/9$.

For completeness, with $s=1/\sqrt{2}$, summing the four rank outer products gives $$ A=\frac{1}{16} \begin{pmatrix} 7+4s & 1-4s\\ 1-4s & 7+4s \end{pmatrix}, \qquad V=\frac{4}{9}A. $$ The pooled tie has been retained without breaking it, and the smaller group’s size appears explicitly in the covariance. This is the conditional covariance to use in $W=D^{\mathsf T}V^+D$; a numerical reference probability still requires an appropriate calibration.

What the Test Can and Cannot Establish

A two-sample spatial rank statistic is a multivariate comparison built from relative directions in the pooled sample. It can be useful when a mean-based comparison is sensitive to extreme values, and its construction does not require a coordinatewise ranking. But a significant rank difference does not identify a unique cause: differences in location, shape, or other distributional features can affect the pooled ranks.

If the scientific target is a location shift, the model and null hypothesis should state the assumptions that make that interpretation appropriate. If the null is equality of distributions, exchangeability supplies the relevant label-invariance condition. These are not interchangeable statements. A location-only null without exchangeability may still motivate a rank statistic, but it does not justify the conditional null calculation as a test calibration.

Do not break multivariate ties by sorting coordinates. That imposes an arbitrary order that is not part of the spatial-rank definition.
Do not omit the group-size factor. The conditional covariance depends on both $n_1$ and $n_2$, not just the pooled sample size.
Do not infer a universal chi-square reference from the covariance formula. The formula gives two conditional moments; it does not establish the full finite-sample null distribution.
Do not equate equal locations with exchangeability. Equal locations alone do not rule out different spreads or shapes.

Key takeaway: Compute spatial ranks from the pooled observations, compare the two group-average rank vectors, and account for the conditional covariance under exchangeability. Preserve ties, include the actual group sizes, and match the null calibration to the hypothesis and design.

Check Your Understanding

Work through these questions using the definitions and conditional-moment result above.

Why does the sum of the pooled spatial ranks equal zero even when some observations are tied?
What assumption about group labels makes the conditional mean of $D$ equal to zero?
Explain why equal population locations do not, by themselves, imply exchangeability.
How does the factor in $\operatorname{Cov}(D\mid Z_1,\ldots,Z_N)$ change when one group becomes smaller?
Why does the conditional covariance formula not, on its own, justify a chi-square reference distribution for $W$?

What You'll Learn