Spatial Sign Covariance and Its Expectation

From Spatial Signs to a Matrix

A spatial sign records direction while discarding distance from a specified center. The outer product of a sign with itself records directional contribution in every pair of coordinates. Averaging these outer products gives a matrix summary of directional balance: the spatial sign covariance matrix. It is useful when the radial magnitudes of observations are highly variable, because no individual observation contributes an outer product with entries larger than one in absolute value.

Let $X$ be a random vector in $\mathbb{R}^p$, with specified center $\mu$. The spatial sign is defined relative to that center; the matrix below describes the distribution of directions around it, not ordinary variation in the original measurement units.

Definition: For $x\in\mathbb{R}^p$, define its spatial sign by $U(x)=x/\|x\|$ when $x\ne 0$, and $U(0)=0$. If $P(X\ne\mu)=1$, the spatial sign covariance matrix about $\mu$ is $$ S_{\mathrm{sign}}(\mu)=E\left[U(X-\mu)U(X-\mu)'\right]. $$ The prime denotes transpose. If $P(X=\mu)>0$, the same expectation is well-defined, but its trace is $P(X\ne\mu)$, rather than one.

The matrix is symmetric and positive semidefinite: for every $a\in\mathbb{R}^p$, $a'S_{\mathrm{sign}}(\mu)a=E[(a'U(X-\mu))^2]\ge 0$. Its trace has an especially direct interpretation. Since $\|U(X-\mu)\|^2=1$ whenever $X\ne\mu$, the trace equals one under the definition's nonzero condition. Thus its eigenvalues are nonnegative and sum to one. They describe how the distribution's directions are spread across the principal axes.

Spherical Distributions: Isotropic Directional Balance

A random vector $Y$ is spherical about zero if $QY$ has the same distribution as $Y$ for every orthogonal matrix $Q$. Assume $P(Y=0)=0$. Then $U(Y)$ is also invariant under every orthogonal transformation, because $U(QY)=QU(Y)$. This rotational symmetry forces the spatial sign covariance to be a scalar multiple of the identity.

Result: If $Y$ is spherical about zero in $\mathbb{R}^p$ and $P(Y=0)=0$, then $$ S_{\mathrm{sign}}(0)=\frac{1}{p}I_p. $$

To see why, write $S=S_{\mathrm{sign}}(0)$. Orthogonal invariance implies $QSQ'=S$ for every orthogonal $Q$. Taking $Q$ to be coordinate sign changes forces every off-diagonal entry of $S$ to be zero. Taking $Q$ to permute coordinates forces all diagonal entries to be equal. Hence $S=cI_p$; because its trace is one, $pc=1$, so $c=1/p$. This argument uses no moment condition on $\|Y\|$: the sign has unit length regardless of the radial distribution.

Worked Example: A Spherical Normal Distribution

Suppose $Y\sim N(0,9I_2)$. The distribution is spherical, so the result gives $S_{\mathrm{sign}}(0)=I_2/2$. The factor 9 in the covariance changes distances from the origin but not directions.

This can also be checked using the angle $\Theta$, which is uniform on $[0,2\pi)$ for a spherical normal vector. Its sign is $(\cos\Theta,\sin\Theta)'$, so the diagonal entries are $E[\cos^2\Theta]=E[\sin^2\Theta]=1/2$, while the off-diagonal entry is $E[\cos\Theta\sin\Theta]=0$. Therefore $$ S_{\mathrm{sign}}(0)= \begin{pmatrix} 1/2&0\\ 0&1/2 \end{pmatrix}. $$

Elliptical Distributions: Shape Determines Directional Balance

A centered elliptical vector with positive definite scatter matrix $\Sigma$ can be represented as $Y=\Sigma^{1/2}Z$, where $Z$ is spherical. The scatter matrix describes the orientation and relative stretching of the distribution; multiplying it by a positive constant does not change the elliptical shape. Assume $P(Z=0)=0$. Since the spatial sign removes the radial length of $Z$, its expectation depends on the shape but not on the radial distribution.

Let $\Sigma=Q\Lambda Q'$, where $Q$ is orthogonal and $\Lambda=\operatorname{diag}(\lambda_1,\ldots,\lambda_p)$, with each $\lambda_i>0$. Orthogonal invariance of $Z$ allows the calculation in these principal coordinates. Write $V=Z/\|Z\|$; for a spherical distribution, $V$ is uniform on the unit sphere. The sign in the principal coordinates is proportional to $\Lambda^{1/2}V$, so the spatial sign covariance has the following form.

Result: For $Y=\Sigma^{1/2}Z$ under the conditions above, the spatial sign covariance has the same principal axes as $\Sigma$. In the eigenbasis of $\Sigma$, it is diagonal, with $$ Q'S_{\mathrm{sign}}(0)Q = \operatorname{diag}(\tau_1,\ldots,\tau_p), \qquad \tau_i=E\left[\frac{\lambda_i V_i^2}{\sum_{k=1}^p\lambda_k V_k^2}\right]. $$ The diagonal entries are nonnegative and sum to one.

For the diagonal form, each off-diagonal entry in the principal coordinates is the expectation of $\sqrt{\lambda_i\lambda_j}V_iV_j/(\sum_k\lambda_kV_k^2)$. Changing the sign of $V_i$ leaves the denominator and distribution unchanged but reverses the numerator. The expectation is therefore zero. Each diagonal entry is the stated $\tau_i$, and summing the fractions gives one for every unit vector $V$.

The ordering of these entries also follows the ordering of the scatter eigenvalues. A larger scatter eigenvalue produces a larger spatial sign covariance eigenvalue. This is a statement about ordering, not a claim that the two sets of eigenvalues are equal or proportional.

Result: If $\lambda_i\ge\lambda_j$, then $\tau_i\ge\tau_j$. For $p\ge2$ and a uniform direction $V$, the inequality is strict when $\lambda_i>\lambda_j$.

To prove the ordering, let $a=V_i^2$, $b=V_j^2$, and $c=\sum_{k\ne i,j}\lambda_kV_k^2$. The distribution of $V$ is unchanged when coordinates $i$ and $j$ are exchanged. Consequently, the expectation of the difference defining $\tau_i-\tau_j$ equals the expectation of its average with the exchanged-coordinate version. Put $D_1=c+\lambda_i a+\lambda_j b$ and $D_2=c+\lambda_i b+\lambda_j a$. The numerator of this average, before division by $2D_1D_2$, is $$ (\lambda_i a-\lambda_j b)D_2+(\lambda_i b-\lambda_j a)D_1 = (\lambda_i-\lambda_j)\{c(a+b)+2(\lambda_i+\lambda_j)ab\}. $$ Both denominators are positive. When $\lambda_i\ge\lambda_j$, the right-hand side is nonnegative, so the average is nonnegative and hence $\tau_i-\tau_j\ge0$. If $\lambda_i>\lambda_j$, then for a uniform direction on a sphere with $p\ge2$, $a,b>0$ with probability one, making the right-hand side strictly positive. This proves the result. $\square$

Worked Example: A Two-Dimensional Elliptical Normal

Let $Y\sim N(0,\operatorname{diag}(4,1))$. The principal axes are the coordinate axes, and the directional representation has $V=(\cos\Theta,\sin\Theta)'$ with $\Theta$ uniform on $[0,2\pi)$. Thus $$ \tau_1=E\left[\frac{4\cos^2\Theta}{4\cos^2\Theta+\sin^2\Theta}\right]. $$

By symmetry over the four quadrants, this expectation is $2/\pi$ times the integral over $0\le\theta\le\pi/2$. Substituting $t=\tan\theta$ gives $$ \tau_1=\frac{2}{\pi}\int_0^\infty\frac{4}{(1+t^2)(4+t^2)}\,dt. $$ Partial fractions give $4/((1+t^2)(4+t^2))=(4/3)/(1+t^2)-(4/3)/(4+t^2)$. Since the two integrals over $0\le t<\infty$ are $\pi/2$ and $\pi/4$, respectively, the integral before multiplication by $2/\pi$ is $(4/3)(\pi/2-\pi/4)=\pi/3$. Therefore $\tau_1=2/3$. The trace is one, so $\tau_2=1/3$, and $$ S_{\mathrm{sign}}(0)= \begin{pmatrix} 2/3&0\\ 0&1/3 \end{pmatrix}. $$ The eigenvalues are ordered as the scatter eigenvalues $4>1$, but they are not $4$ and $1$, nor are they obtained by simply dividing those values by their sum.

Worked Example: Changing Radial Spread Without Changing the Matrix

Let $V=(\cos\Theta,\sin\Theta)'$ with $\Theta$ uniform on $[0,2\pi)$, and let $R$ be independent of $\Theta$, with $P(R=1)=P(R=10)=1/2$. Set $Y=\operatorname{diag}(2,1)RV$. This is an elliptical model with the same shape matrix as in the preceding example, but with a nonconstant radial component.

For every $R>0$, $$ U(Y)=\frac{\operatorname{diag}(2,1)RV}{\|\operatorname{diag}(2,1)RV\|} =\frac{\operatorname{diag}(2,1)V}{\|\operatorname{diag}(2,1)V\|}. $$ The radius cancels exactly. Thus the spatial sign covariance is again $\operatorname{diag}(2/3,1/3)$. In particular, the result does not require $R$ to have a finite second moment; in this example $R$ is bounded, but the cancellation holds for any positive radius.

Interpretation, Uses, and a Common Pitfall

The spatial sign covariance is a directional, scale-free summary. If every centered observation is multiplied by the same positive constant, its sign and the matrix remain unchanged. More generally, the elliptical formula shows why changing the radial distribution while preserving the shape leaves the population matrix unchanged. This can be valuable when radial magnitudes are heavy-tailed or when the scientific question concerns directional concentration rather than the size of deviations.

This matrix is not the ordinary covariance matrix. Ordinary covariance weights directions according to squared distances from the center and requires appropriate second moments for its usual finite-valued definition. Spatial sign covariance instead gives each noncentral observation a unit-length sign before forming its outer product. Its eigenvalues sum to one, unlike covariance eigenvalues, and its numerical eigenvalues need not be proportional to those of the elliptical scatter matrix. Under an elliptical model, it shares their principal axes and preserves their ordering; that is the specific eigenstructure established here.

Common mistake: A reader may see $S_{\mathrm{sign}}$ called a covariance matrix and conclude that its eigenvalues are the covariance eigenvalues, perhaps rescaled to sum to one. That reasoning is incorrect: the definition first divides each centered vector by its own length. For example, scatter eigenvalues $4$ and $1$ produce spatial sign covariance eigenvalues $2/3$ and $1/3$ in the two-dimensional normal example, not $4/5$ and $1/5$. The correction is to use the expectation of the normalized outer product, or the integral formula for $\tau_i$, rather than normalize the ordinary covariance matrix's eigenvalues.

An isotropic spatial sign covariance also does not by itself prove that the full distribution is spherical. For instance, a distribution supported only on coordinate directions has the same spatial sign covariance as a spherical distribution in the example below, while its support is not rotationally invariant.

Worked Example: Isotropic Signs Do Not Establish Sphericity

In two dimensions, let $Y$ take each of $(1,0)',(-1,0)',(0,1)',(0,-1)'$ with probability $1/4$. The signs equal the observations. The outer products are $\operatorname{diag}(1,0)$ for either horizontal point and $\operatorname{diag}(0,1)$ for either vertical point. Averaging gives $$ S_{\mathrm{sign}}(0) =\frac{1}{2}\begin{pmatrix}1&0\\0&0\end{pmatrix} +\frac{1}{2}\begin{pmatrix}0&0\\0&1\end{pmatrix} =\frac{1}{2}I_2. $$ Yet a rotation by $45$ degrees sends the support onto diagonal directions, not the original coordinate axes. The distribution is not spherical. The matrix summarizes second-order directional balance; it does not encode the entire directional distribution.

Check Your Understanding

Use the definitions and results above to answer these questions.

Why does the trace of the spatial sign covariance equal one when $P(X\ne\mu)=1$? What changes if $P(X=\mu)>0$?
For a spherical distribution in $p$ dimensions, why must all diagonal entries of the spatial sign covariance be equal?
An elliptical distribution has scatter eigenvalues $9$, $4$, and $1$. What can be concluded about the ordering and sum of its spatial sign covariance eigenvalues?
Why does changing the radial distribution of an elliptical vector leave its spatial sign covariance unchanged when its shape is fixed?
Does a spatial sign covariance equal to $I_p/p$ prove that a distribution is spherical? Explain using the distinction between a matrix summary and a full distribution.