Why Invariance Does Not Always Mean Distribution-Free
A rank procedure discards numerical distances and retains order information. In one dimension, this can remove the effect of an unknown continuous distribution: applying the same strictly increasing transformation to every observation leaves all ranks unchanged. In several dimensions, however, there is no single canonical ordering of points. Ranking each coordinate separately preserves marginal orders but does not remove the dependence between coordinates. Affine transformations preserve a different kind of structure, involving linear geometry rather than coordinatewise order.
The useful question is therefore not simply whether a statistic is invariant. It is: what information remains after the transformations act, and does the null model assign that remaining information a parameter-free law? The answer is expressed through a transformation group and its maximal invariant.
Groups, Invariant Statistics, and Maximal Invariants
The orbit of a sample \(x\) is the set of all samples obtainable by transforming \(x\) with elements of \(G\). Any invariant statistic is constant on each orbit. A maximal invariant retains all distinctions between orbits, though it discards distinctions that the group can erase. A test based on a maximal invariant can still have a null distribution that depends on the underlying model; maximality alone does not make that distribution universal.
Increasing Transformations and Univariate Ranks
For a sample of \(n\) distinct real observations, consider the group of strictly increasing bijections from \(\mathbb{R}\) to itself, applied identically to every observation. Let \(R_i\) denote the rank of \(X_i\), with rank 1 assigned to the smallest value. The vector of ranks is invariant because every pairwise ordering is preserved.
It is also maximal. If two samples have the same rank vector, their ordered values occur in the same indexed order. A strictly increasing, piecewise-linear bijection can be constructed to map each ordered value of the first sample to the corresponding ordered value of the second, extending linearly between values and beyond the endpoints. Thus the two samples are in the same orbit. The no-ties condition matters: ties require a group action and a rank convention that explicitly account for tied values.
Worked Example: An Exact Two-Sample Rank Distribution
Suppose four distinct observations are pooled from two groups, each of size two. Under the null hypothesis that all four observations are independent draws from the same continuous distribution, every assignment of two of the four pooled ranks to Group 1 is equally likely. There are \(\binom{4}{2}=6\) assignments.
The possible Group 1 rank sets and their rank sums are \(\{1,2\}\mapsto3\), \(\{1,3\}\mapsto4\), \(\{1,4\}\mapsto5\), \(\{2,3\}\mapsto5\), \(\{2,4\}\mapsto6\), and \(\{3,4\}\mapsto7\). Therefore the rank-sum probabilities are
respectively. This calculation is exact and does not involve the common continuous distribution: the pooled ranks are fixed at \(1,2,3,4\), and exchangeability makes the six group assignments equiprobable. If observations can tie, this argument no longer applies without a specified tie-handling procedure and an appropriate null calibration.
Why the Univariate Rank Law Is Distribution-Free
The central property is exchangeability under the null, not a special shape such as normality. For independent observations with a common continuous distribution, every strict ordering of their labels has the same probability. There are \(n!\) such orderings, they are disjoint, and together they have probability one.
Proof. Continuity implies \(P(X_i=X_j)=0\) for every \(i\ne j\), so there are no ties with probability one. For any permutation \(\pi\) of the indices, independence and identical distributions imply that relabeling the observations leaves their joint distribution unchanged. Hence the probability of the event \(X_{\pi(1)}<\cdots<X_{\pi(n)}\) is the same for every \(\pi\). These \(n!\) ordering events are disjoint and exhaustive up to the probability-zero tie event, so each has probability \(1/n!\). Each ordering corresponds to one rank permutation. Therefore the rank vector is uniform, and any function of it has a law determined by this uniform distribution alone. \(\square\)
Equivalently, for a continuous distribution function \(F\), the probability integral transform gives \(F(X_i)\sim\mathrm{Uniform}(0,1)\), and the transformation preserves ranks almost surely. The exchangeability proof makes explicit why the common null distribution disappears.
Coordinatewise Ranks Retain Dependence
For observations in \(\mathbb{R}^p\), a natural transformation group consists of separate strictly increasing transformations of each coordinate. If \(X_i=(X_{i1},\ldots,X_{ip})\), the transformation has the form \((X_{i1},\ldots,X_{ip})\mapsto(g_1(X_{i1}),\ldots,g_p(X_{ip}))\), with each \(g_j\) strictly increasing. With no ties in any coordinate, the full array of coordinatewise ranks is a maximal invariant: the univariate argument applies separately to each coordinate.
But the joint distribution of these rank arrays is not generally fixed by the marginal distributions. The dependence, or copula, between coordinates determines how their rankings line up across observations. Removing each marginal scale and shape does not remove that dependence.
Worked Example: Same Marginal Rank Patterns, Different Dependence
Take two independent observations from a bivariate model with continuous uniform margins. In one model, let \(X_{i2}=X_{i1}\), so the two coordinates are perfectly concordant. Whichever observation ranks first in coordinate 1 also ranks first in coordinate 2. The two coordinate rank columns therefore always agree.
In a second model, let the two coordinates be independent, with each coordinate still having a continuous uniform distribution. The ordering of the two observations in coordinate 1 and their ordering in coordinate 2 are independent. Each ordering has two possibilities, each with probability \(1/2\), so the rank columns agree with probability \(1/2\) and disagree with probability \(1/2\).
Both models have the same marginal distributions, and both produce coordinatewise ranks. Yet the joint rank-array law differs: agreement has probability 1 in the first model and \(1/2\) in the second. Thus a statistic based on the joint pattern of marginal ranks need not have a distribution-free null law over arbitrary multivariate distributions. Additional assumptions about dependence, or a null calibration that respects it, are needed.
Affine Transformations and Their Maximal Invariant
The affine group on \(\mathbb{R}^p\) consists of transformations \(x\mapsto Ax+b\), where \(A\) is a nonsingular \(p\times p\) matrix and \(b\in\mathbb{R}^p\). It can change location, scale, shear, and orientation, so it preserves neither coordinatewise ranks nor distances in general. Its invariant information describes the configuration of points up to a common affine change of coordinates.
For an ordered sample \(x_0,\ldots,x_{n-1}\), suppose the first \(p+1\) points are affinely independent. Form the matrix \(B=[x_1-x_0,\ldots,x_p-x_0]\), which is then nonsingular. For each remaining point define
These coordinates are a maximal invariant for the affine action on such samples. Under \(y_i=Ax_i+b\), the reference matrix becomes \(AB\) and the displacement becomes \(A(x_i-x_0)\), so the coordinates remain unchanged. Conversely, if two samples have the same coordinates \(u_i\), the unique affine map taking the first \(p+1\) reference points to the second also takes every remaining point to its counterpart, since each is represented by the same coordinates in the reference basis. Samples with affinely dependent reference points require a different choice of reference points or a separate treatment.
Worked Example: Normalizing a Four-Point Configuration
In \(\mathbb{R}^2\), take \(x_0=(2,3)\), \(x_1=(4,3)\), \(x_2=(2,6)\), and \(x_3=(5,5)\). The first three points are not collinear, and
Therefore the invariant coordinate of the fourth point is \(u_3=B^{-1}(x_3-x_0)=(3/2,2/3)^\mathsf{T}\). Now apply \(y=Ax+b\) with \(A=\begin{pmatrix}1&1\\0&2\end{pmatrix}\) and \(b=(-1,4)^\mathsf{T}\). The transformed reference differences form \(B'=\begin{pmatrix}2&3\\0&6\end{pmatrix}\), and \(y_3-y_0=(5,4)^\mathsf{T}\). Solving \(B'u'_3=(5,4)^\mathsf{T}\) gives \(6u'_{32}=4\), hence \(u'_{32}=2/3\), and \(2u'_{31}+3(2/3)=5\), hence \(u'_{31}=3/2\). The normalized coordinates agree exactly, as affine invariance requires.
When Affine Invariance Removes Nuisance Parameters
An affine-invariant statistic can have a nuisance-free law in an affine model, but this is a more limited statement than universal distribution-freeness. Let \(Z\) have a fixed distribution \(P_0\), and suppose the null model consists of distributions of \(X=\mu+AZ\), with \(\mu\) arbitrary and \(A\) nonsingular. For an independent sample transformed by this same affine map, any affine-invariant statistic has the same value as it has on the corresponding sample from \(P_0\).
Proof. Under the specified model, \(X\) has the same distribution as the transformed sample \(\mu+AZ\). By affine invariance, \(T(X)\) consequently has the same distribution as \(T(\mu+AZ)=T(Z)\). The latter law depends on the baseline distribution \(P_0\), but contains neither \(\mu\) nor \(A\). This proves nuisance-parameter freedom within the stated affine family; it does not show that different choices of \(P_0\) produce the same law. \(\square\)
This distinction matters in practice. A model that specifies only an unknown location and scatter matrix may still leave the standardized distributional shape unspecified. Affine invariance removes the affine nuisance parameters, not every possible feature of the data-generating distribution. Nor does it automatically justify a permutation distribution: permutation calibration additionally requires the relevant exchangeability or randomization conditions under the null.
Common Mistake: Treating Invariance as a Distribution-Free Proof
Incorrect reasoning: “The statistic is unchanged by coordinatewise monotone transformations, so its multivariate null distribution is distribution-free.” The bivariate example shows why this fails: the same marginal transformations leave intact the dependence between coordinates, and that dependence changes the joint rank law.
Correction: First identify the group and the maximal invariant. Then identify the null family and determine whether its distributions lie in a single orbit of that group, or otherwise imply a common law for the invariant. A group-invariant statistic has a distribution constant along each model orbit; it need not have the same distribution across distinct orbits. Marginal rank invariance and affine nuisance removal are different claims, and neither should be confused with distribution-freeness over an unrestricted multivariate null.
- Strictly increasing marginal transformations preserve each coordinate's ordering, but not the dependence structure among coordinate rankings.
- A strictly decreasing transformation reverses an ordering; a rank statistic invariant only to increasing transformations need not be invariant to decreasing ones.
- Ties break the simple permutation argument unless the test specifies how ties are handled and how the null law is calibrated.
- Affine invariance removes location and nonsingular linear changes only when the statistic is truly invariant to the full affine group.
The practical lesson is to state the transformation group, its maximal invariant, and the null assumptions together. That identifies precisely what information a rank procedure retains and what its claimed reference distribution is allowed to ignore.
Check Your Understanding
Use the group action and null assumptions—not the word “rank” alone—to assess each claim.
- Why is the rank vector a maximal invariant for a tie-free sample under common strictly increasing transformations?
- Under what assumptions does a rank-based statistic have a null distribution independent of the common univariate distribution?
- Why do coordinatewise ranks fail to determine a universal multivariate null law when the copula is unrestricted?
- For an affinely independent reference set of \(p+1\) points, explain why the coordinates \(B^{-1}(x_i-x_0)\) are unchanged by a nonsingular affine transformation.
- What does affine invariance remove from an affine model, and what distributional feature may remain?