Depth Tests for Two and Multiple Groups

Why Compare Groups by Depth?

In multivariate data, a group can differ from another in location, spread, shape, or tail behavior. A depth function provides an ordering of observations from more central to more peripheral relative to a distribution. Comparing the groups’ positions in a common ordering gives a rank-based way to detect differences without reducing each vector to one arbitrarily chosen coordinate.

The word common matters. If each group is ranked relative to its own depth contours, the ranks can look similar even when the groups occupy different parts of the sample space. For a direct group comparison, first evaluate every observation using a depth function based on the pooled distribution, then compare the resulting pooled depth scores across groups. This tutorial develops two complementary summaries: pooled depth ranks and empirical distributions of pooled depth scores.

The depth function may be, for example, halfspace depth or simplicial depth. The test depends on the chosen depth: different depth functions can order the same observations differently. In the rank tests below, the pooled scores are held fixed while group labels are permuted. Thus, the statistic is driven by how the observed labels are arranged along the pooled depth ordering.

Null Hypotheses and What These Tests Detect

Suppose the groups are independent samples from distributions \(F_1,\ldots,F_k\). A standard null for an exact permutation test is equality of the full distributions:

Under this null, observations are exchangeable across group labels when the group sizes are fixed. A depth-rank test can then be calibrated by comparing the observed statistic with its values under all reallocations of the labels that preserve those sizes.

The chosen statistic describes which features of a difference the procedure is sensitive to. A group with unusually high pooled depth ranks tends to have observations more central in the pooled sample; a group with unusually low ranks tends to have more peripheral observations. A difference in depth distributions can reflect location, scale, or shape differences, but a depth-based statistic is not guaranteed to detect every possible difference between multivariate distributions. In particular, equality of depth-score distributions is generally weaker than equality of the full distributions.

Two Groups: A Pooled Depth Rank Statistic

For two groups, let \(n_1+n_2=N\) and define \(W_1=\sum_{j=1}^{n_1}R_{1j}\), the rank sum for group 1. Under a balanced arrangement of labels, group 1’s expected rank sum is \(n_1(N+1)/2\). A convenient two-sided statistic is the squared departure from this value:

A large value means that group 1 has ranks unusually high or unusually low relative to the pooled ordering. The sign of the unsquared departure indicates direction: a positive departure means higher pooled depth ranks, and a negative departure means lower ranks. The finite-sample test does not need a normal or chi-squared approximation; its reference distribution is obtained by permuting the labels.

Several Groups: Between-Group Variation in Ranks

For \(k\) independent groups, let \(\bar R_g=n_g^{-1}\sum_{j=1}^{n_g}R_{gj}\) be group \(g\)’s mean pooled rank. The overall mean rank is \((N+1)/2\). A direct measure of between-group rank separation is

The statistic weights each group’s squared departure by its size. Large values indicate that at least some group mean ranks are far from the overall mean. A familiar scaled version, when there are no tied depth scores, is

For finite-sample inference in this tutorial, \(H_R\) is calibrated by permutation rather than by relying on a large-sample reference distribution.

To verify the identity, note that \(R_g=n_g\bar R_g\), so

Substituting this expression into \(H_R\) gives

because the second term is \(3(N+1)\) and cancels the final term. This proves the stated identity. \(\square\)

Comparing Empirical Depth Distributions

A mean-rank statistic emphasizes shifts in average depth rank. A more distributional comparison asks whether the groups differ across the full range of pooled depth scores. Let \(\widehat H_g(t)\) be the empirical distribution function of the pooled scores in group \(g\), and let \(\widehat H(t)\) be the empirical distribution function of all pooled scores. One omnibus discrepancy is

The integral is an average squared difference over the observed pooled score distribution. A large value means that one or more groups have an empirical depth-score distribution unlike the pooled distribution. Since \(\widehat H\) is discrete, the integral is a finite weighted sum over the pooled scores. With distinct scores, each observed score has pooled mass \(1/N\). As with the rank statistic, labels are permuted while the pooled scores remain fixed.

Finite-Sample Permutation Calibration

For fixed group sizes, the number of distinct label allocations is \(M=N!/\prod_{g=1}^k n_g!\). For either statistic, calculate its value for the observed labels and for each allocation. For a statistic \(T\) where large values indicate stronger evidence, the nonrandomized upper-tail permutation \(p\)-value is

The use of “greater than or equal to” includes allocations tied with the observed statistic. When there are no ties in the depth scores, the pooled ranks are unchanged by relabelling. If depth scores themselves are tied, a tie-handling rule is needed; the distinct-score calculations above do not specify one.

To prove this, condition on the pooled observations. Under the null, every allocation of the \(N\) observations to labelled groups of the fixed sizes has the same conditional probability, so the observed allocation is uniform over the \(M\) possibilities. The pooled depth scores and the statistic value for each allocation are then fixed. Order those \(M\) statistic values from largest to smallest, retaining repeated values. For an observed allocation, its permutation \(p\)-value is the proportion of all \(M\) values at least as large as its own. The set of allocations whose \(p\)-value is at most \(\alpha\) occupies at most an \(\alpha\) fraction of this ordered list: any tied block at the cutoff is included in full, which can only make the fraction smaller than the cutoff. Uniformity of the observed allocation therefore gives the stated conditional inequality. Averaging over the pooled observations gives the unconditional inequality as well. \(\square\)

Interpretation and a Common Mistake

A depth-rank test gives an interpretable direction: a group’s ranks can be higher or lower than the pooled center-to-outlyingness ordering. The empirical-distribution statistic gives a broader comparison across that ordering, rather than focusing only on mean ranks. Neither summary identifies the particular multivariate feature responsible for a rejection. Graphs of the vectors or of the group-specific depth-score distributions can help describe the observed difference, but they do not change the test’s null calibration.

• Mistake: “The groups have the same distribution of depth scores, so permuting the original group labels must be valid.”
  Correction: Equal depth-score distributions do not generally imply that the original multivariate observations are exchangeable. The exact permutation guarantee here is justified under equality of the full group distributions.
• Mistake: “A large rank statistic proves a location shift.”
  Correction: The statistic detects a difference in the selected depth ordering; changes in spread or shape can also affect that ordering.
• Mistake: “Permuting labels means recomputing a different pooled depth ordering for each allocation.”
  Correction: In the pooled-depth procedures defined here, the pooled sample is fixed, so the pooled scores and ranks stay fixed. Only the group labels change.