Worked Problems in Multivariate Rank Testing

Three Testing Problems, Three Sources of Calibration

Rank and depth methods replace raw distances or coordinates with summaries of direction, relative position, or centrality. The calculation is often manageable even when the sampling distribution of the statistic is not. In these worked problems, small sample sizes let us enumerate the relevant sign flips or label assignments exactly.

The assumptions behind those enumerations differ. A one-sample spatial-sign test uses central symmetry to justify flipping directions. Two-sample rank and depth procedures use exchangeability under the null to justify permuting group labels. An exact calculation is only valid when its corresponding assumption is appropriate.

One-Sample Testing with Spatial Signs

Definition: For a hypothesized bivariate center \(\theta_0\), the spatial sign of a nonzero observation \(X_i-\theta_0\) is \(U_i=(X_i-\theta_0)/\|X_i-\theta_0\|\). The one-sample spatial-sign statistic is \(T=\|\sum_{i=1}^n U_i\|\). Large values indicate that the observed directions are unusually aligned.

A sign-flip calibration tests the null that the distribution is centrally symmetric about \(\theta_0\), with independent observations and no probability mass at \(\theta_0\). Under this null, conditional on the unoriented axes of the observations, each direction can be reversed with probability one half. The test compares the observed \(T\) with the values obtained from all \(2^n\) choices of direction flips.

Worked Example: Four Observations Pointing Along Two Axes

Test central symmetry about \((0,0)\) for observations \((2,0),(1,0),(0,3),(0,1)\). The first two spatial signs are \((1,0)\), and the last two are \((0,1)\). Therefore the observed sum of signs is \((2,2)\), giving \(T_{\text{obs}}=\sqrt{2^2+2^2}=2\sqrt{2}\).

For a sign flip, each of the two horizontal signs contributes either \(1\) or \(-1\) to the first coordinate. Thus that coordinate is \(A=\varepsilon_1+\varepsilon_2\), where each \(\varepsilon_i\) is independently \(1\) or \(-1\). The second coordinate is \(B=\varepsilon_3+\varepsilon_4\). Each of \(A\) and \(B\) is nonzero for 2 of its 4 sign choices, and then its absolute value is 2.

The flipped statistic is \(\sqrt{A^2+B^2}\). Its maximum, \(2\sqrt{2}\), occurs precisely when both coordinates are nonzero. There are \(2\times2=4\) such choices among \(4\times4=16\) total sign patterns. Including ties in the upper tail gives the exact p-value \(4/16=0.25\). This is not significant at a 0.05 level; the observed directional alignment is not rare enough under this calibration.

The reason this enumeration is valid can be stated directly. Conditional on the unoriented axes, each of the \(2^n\) sign patterns has probability \(2^{-n}\) under the stated central-symmetry null. For any statistic \(T\), its upper-tail permutation p-value is the fraction of these patterns whose statistic is at least the observed value. The observed pattern is among the patterns being counted, and the resulting exact test is conservative when ties occur.

Result: If the observations are independent, nonzero almost surely, and centrally symmetric about \(\theta_0\), then the conditional sign-flip test using \(T=\|\sum_i U_i\|\) has a valid finite-sample upper-tail p-value. This conclusion does not require a normal distribution.

To see the finite-sample validity, let the \(M=2^n\) possible sign patterns have statistics \(t_1,\ldots,t_M\), and let \(p_j=M^{-1}\#\{k:t_k\ge t_j\}\). Under the null, the observed pattern is uniform over these \(M\) patterns. For any \(\alpha\), order the distinct statistic values from largest to smallest. The patterns whose p-values are at most \(\alpha\) lie in an initial segment of this ordering containing at most \(\alpha M\) patterns: a tied block is included only if its entire upper-tail fraction is at most \(\alpha\). Hence \(P(p\le\alpha\mid\text{unoriented axes})\le\alpha\). Averaging over the axes gives the same unconditional bound. \(\square\)

Two-Sample Testing with Spatial Ranks

Definition: For pooled bivariate observations \(Y_1,\ldots,Y_N\) with no coincident points, the spatial rank of \(Y_i\) is \(R_i=N^{-1}\sum_{j=1}^N (Y_i-Y_j)/\|Y_i-Y_j\|\), with the \(j=i\) term defined as zero. For groups of sizes \(n_1,n_2\), one two-sample statistic is \(T=\|\sum_{i\in G_1}R_i-\sum_{i\in G_2}R_i\|\).

Under the null that the two groups are independent samples from the same distribution, the group labels are exchangeable conditional on the pooled observations. The spatial ranks can therefore be held fixed while all assignments of \(n_1\) pooled points to group 1 are enumerated. The exact upper-tail p-value is the proportion of assignments with statistic at least as large as observed.

Worked Example: Permuting Four Pooled Spatial Ranks

Take pooled points \(Y_1=(1,0),Y_2=(-1,0),Y_3=(0,1),Y_4=(0,-1)\), with groups \(G_1=\{Y_1,Y_3\}\) and \(G_2=\{Y_2,Y_4\}\). For \(Y_1\), the three nonzero unit difference vectors are \((1,0),(1,-1)/\sqrt{2},(1,1)/\sqrt{2}\). Their sum is \((1+\sqrt{2},0)\), so \(R_1=(c,0)\), where \(c=(1+\sqrt{2})/4\). By symmetry, \(R_2=(-c,0),R_3=(0,c),R_4=(0,-c)\).

The group-1 rank sum is \((c,c)\), and the group-2 rank sum is \((-c,-c)\). Thus \(T_{\text{obs}}=\|(2c,2c)\|=2\sqrt{2}c=(1+\sqrt{2})/\sqrt{2}\).

There are \(\binom{4}{2}=6\) assignments to group 1. If the selected pair consists of opposite points, its rank sum is zero and the group difference is zero, because the total rank sum is zero. There are two such pairs: \(\{Y_1,Y_2\}\) and \(\{Y_3,Y_4\}\). Each of the other four pairs consists of perpendicular-axis ranks; its sum has norm \(\sqrt{2}c\), so its group difference has norm \(2\sqrt{2}c=T_{\text{obs}}\). The exact upper-tail p-value is therefore \(4/6=2/3\). These data do not provide evidence against equal group distributions by this statistic.

The permutation argument has a simple justification. Under the null of independent sampling from one common distribution, every assignment of \(n_1\) labels to the \(N\) pooled observations has the same conditional probability, provided the sampling design imposes no further restrictions. Consequently, the observed assignment is uniform among the \(\binom{N}{n_1}\) possibilities. Counting assignments whose statistic is at least the observed statistic gives a valid finite-sample test; counting ties in the tail makes the test conservative rather than anticonservative. \(\square\)

This calibration is for the exchangeability null. A statistic based on spatial ranks may be sensitive to location differences, but a small p-value does not by itself identify the form of a difference, and a non-small p-value does not establish that the distributions are identical.

Depth-Based Comparison of Two Groups

Depth assigns a centrality score relative to a reference distribution. To make the calculation explicit, use empirical halfspace depth. For pooled points \(Y_1,\ldots,Y_N\), define \(D_N(z)\) as the smallest fraction of pooled observations in any closed halfspace containing \(z\). In a group comparison, one can treat these pooled depths as scores and compare the group mean scores. Permutation calibration is justified under exchangeability, just as for the spatial-rank statistic.

Worked Example: Comparing Mean Halfspace Depths

Consider the five pooled points \((0,0),(1,0),(-1,0),(0,1),(0,-1)\). The center has empirical halfspace depth \(3/5\). Indeed, every halfspace containing the center contains at least the center and two of the four vertices; a diagonal halfspace through the center attains that minimum. Each vertex has depth \(1/5\), since a supporting halfspace can contain that vertex alone, and no halfspace containing a pooled point can contain fewer than one pooled observation.

Let group A have size 2 and group B size 3. Suppose group A consists of the center and one vertex. Its mean depth is \((3/5+1/5)/2=2/5\). Group B contains the other three vertices and has mean depth \(1/5\). The observed absolute difference in mean depth is \(2/5-1/5=1/5\).

For exact calibration, hold the five depth scores fixed and assign any two of the five points to group A. There are \(\binom{5}{2}=10\) assignments. Four assignments include the center; in each, the absolute mean-depth difference is \(1/5\). The other six exclude the center: group A then has mean \(1/5\), while group B has mean \((3/5+2(1/5))/3=1/3\), giving absolute difference \(1/3-1/5=2/15\). Since \(1/5>2/15\), the exact upper-tail p-value is \(4/10=0.4\). The observed group A is not unusual under exchangeable labels according to this centrality comparison.

What the Calculations Do—and Do Not—Establish

Each example uses an exact reference distribution, but the reference distribution comes from a different assumption: central symmetry for one-sample direction flips, and exchangeability for two-sample label permutations. A robust-looking statistic does not remove the need to justify its calibration. In particular, permuting labels is not generally valid when the study design or null hypothesis does not make the labels exchangeable.

Depth scores also discard information. A difference in mean depth asks whether the groups differ in average centrality relative to the pooled configuration. It is not an omnibus test for every possible difference between multivariate distributions: groups can have similar average depths while differing in other features. The interpretation must match the statistic actually calculated.

Common mistake: treating the number of permutations as the p-value. The p-value is the fraction of all allowed sign patterns or label assignments whose statistic is at least as extreme as observed. In the spatial-rank example it is \(4/6\), not \(4\) or \(1/6\).
Common mistake: excluding ties from the tail. Exact upper-tail p-values count configurations with statistic equal to the observed statistic. Omitting them can make the p-value too small.
Common mistake: interpreting failure to reject as proof of equality. A non-small p-value says the observed statistic is not sufficiently unusual under the specified calibration; it does not prove the null or rule out differences the statistic is insensitive to.

Key takeaway: First identify the null assumption that makes sign flips or label permutations valid; then calculate the statistic on every allowed configuration and include ties in the upper tail. Interpret the result as evidence about that statistic and null, not as a universal statement about the distributions.

Check Your Understanding

For each question, identify the relevant assumption as well as the calculation or interpretation.

Why does a one-sample spatial-sign test use sign flips rather than arbitrary permutations of observations?
In the four-point spatial-rank example, what would the exact upper-tail p-value be if the observed group assignment had statistic zero?
Why must the pooled spatial ranks remain fixed when group labels are permuted in the worked example?
What feature of the five-point configuration gives the center greater halfspace depth than a vertex?
Does a non-significant depth-based comparison establish that the two group distributions are identical? Explain.