Singular Elliptical Laws on Subspaces

Why Singular Elliptical Laws Need Lower-Dimensional Coordinates

An elliptical vector is often described by a location and a scatter matrix. If that matrix is positive definite, the vector can have a density over the full ambient space. If the matrix is rank deficient, however, the vector is confined to a lower-dimensional affine subspace. Treating it as though it had an ordinary full-dimensional density can lead to incorrect probability calculations: the subspace has zero ambient volume, even when the entire probability distribution lies on it.

The key is to work with coordinates on the subspace itself. A factorization of the scatter matrix gives a lower-dimensional random vector, and probabilities can be calculated by translating events back to those coordinates.

Definition and Affine Support

Definition: Let \(V\) be a spherically symmetric random vector in \(\mathbb{R}^r\), meaning that \(QV\) has the same distribution as \(V\) for every \(r\times r\) orthogonal matrix \(Q\). Suppose its characteristic function is \(\psi(t^{\mathsf T}t)\), where \(\psi\) is its radial characteristic-function generator. Let \(B\) be a \(p\times r\) matrix of full column rank, and let \(\mu\in\mathbb{R}^p\). The vector \(X=\mu+BV\) is an elliptical vector in \(\mathbb{R}^p\). Its scatter parameter is \(\Sigma=BB^{\mathsf T}\), which is positive semidefinite and has rank \(r\). When \(r<p\), the law is called singular in \(\mathbb{R}^p\).

The characteristic function follows directly from the affine representation:

\varphi_X(t) =E[\exp(i t^{\mathsf T}X)] =\exp(i t^{\mathsf T}\mu)\psi(t^{\mathsf T}\Sigma t), \qquad t\in\mathbb{R}^p.

The range of \(B\), denoted \(\operatorname{Range}(B)\), is an \(r\)-dimensional linear subspace. Since \(X-\mu=BV\), every value of \(X\) lies in the affine subspace \(\mu+\operatorname{Range}(B)\). More precisely, if \(\operatorname{supp}(V)\) denotes the topological support of \(V\), then

\operatorname{supp}(X)=\mu+B\operatorname{supp}(V) \subseteq \mu+\operatorname{Range}(B).

Thus the affine subspace contains the support, but need not equal it. It is the exact support when \(V\) has support \(\mathbb{R}^r\). For instance, a spherical vector supported on a sphere has a smaller support than the whole subspace, even though the subspace still contains all its values.

Intrinsic Coordinates and Probability

Because \(B\) has full column rank, \(B^{\mathsf T}B\) is invertible. Define the left inverse \(B^+=(B^{\mathsf T}B)^{-1}B^{\mathsf T}\). For \(X=\mu+BV\), the intrinsic coordinate is recovered exactly:

B^+(X-\mu) =(B^{\mathsf T}B)^{-1}B^{\mathsf T}BV =V.

Consequently, for any measurable set \(C\subseteq\mathbb{R}^p\),

P(X\in C) = P\bigl(V\in\{v\in\mathbb{R}^r:\mu+Bv\in C\}\bigr).

This is the basic probability rule for a singular elliptical law: take the event of interest in the ambient space, substitute \(X=\mu+Bv\), and calculate under the distribution of \(V\). If an event is inconsistent with the affine support, its probability is zero.

Worked Example: A Degenerate Gaussian on a Plane

Let \(Z=(Z_1,Z_2)^{\mathsf T}\sim N(0,I_2)\), and define \(X=\mu+BZ\), where \(B=\begin{pmatrix}1&0\\1&1\\0&1\end{pmatrix}\) and \(\mu=(\mu_1,\mu_2,\mu_3)^{\mathsf T}\). The scatter matrix and its rank are

\Sigma=BB^{\mathsf T} = \begin{pmatrix} 1&1&0\\ 1&2&1\\ 0&1&1 \end{pmatrix}, \qquad \operatorname{rank}(\Sigma)=2.

Writing \(Z=(Z_1,Z_2)^{\mathsf T}\), the coordinates are \(X_1=\mu_1+Z_1\), \(X_2=\mu_2+Z_1+Z_2\), and \(X_3=\mu_3+Z_2\). Hence the support lies on the plane \(X_2-\mu_2=(X_1-\mu_1)+(X_3-\mu_3)\). The vector has no density with respect to three-dimensional Lebesgue measure, because this plane has zero three-dimensional volume.

To calculate \(P(X_1>\mu_1+1,\ X_3>\mu_3)\), use the intrinsic coordinates. The event is \(Z_1>1,\ Z_2>0\). Independence of the two standard normal coordinates gives

P(X_1>\mu_1+1,\ X_3>\mu_3) =P(Z_1>1)P(Z_2>0) =(1-\Phi(1))\frac12 \approx 0.07933.

The calculation uses a two-dimensional probability, as the construction requires, rather than trying to integrate an ambient three-dimensional density that does not exist.

Intrinsic Density Versus Ambient Density

A singular law can have a density on its supporting subspace. To state the formula, suppose \(V\) has a density \(g\) with respect to \(r\)-dimensional Lebesgue measure. The linear map \(v\mapsto Bv\) multiplies \(r\)-dimensional volume by \(\sqrt{\det(B^{\mathsf T}B)}\). Therefore the density of \(X\) with respect to \(r\)-dimensional Hausdorff measure on \(\mu+\operatorname{Range}(B)\) is

f_{\mathrm{intrinsic}}(x) = \frac{g\bigl(B^+(x-\mu)\bigr)} {\sqrt{\det(B^{\mathsf T}B)}}, \qquad x\in\mu+\operatorname{Range}(B).

To verify the formula, integrate a nonnegative measurable function \(h\) over the subspace and use \(x=\mu+Bv\):

\int_{\mu+\operatorname{Range}(B)}h(x)f_{\mathrm{intrinsic}}(x)\,d\mathcal{H}^r(x) = \int_{\mathbb{R}^r}h(\mu+Bv)g(v)\,dv =E[h(X)].

Here \(\mathcal{H}^r\) is \(r\)-dimensional Hausdorff measure, with its usual normalization agreeing with \(r\)-dimensional volume on an \(r\)-plane. The formula is an intrinsic density, not a density with respect to \(p\)-dimensional Lebesgue measure. When \(r<p\), the law is concentrated on a set of ambient Lebesgue measure zero, so an ambient density cannot represent it.

Worked Example: Intrinsic Density on a Plane

Use the matrix \(B\) from the degenerate Gaussian example, and take \(\mu=0\). Since \(Z\sim N(0,I_2)\), its density is \(g(z)=(2\pi)^{-1}\exp(-z^{\mathsf T}z/2)\). For this matrix,

B^{\mathsf T}B= \begin{pmatrix}2&1\\1&2\end{pmatrix}, \qquad \det(B^{\mathsf T}B)=3, \qquad B^+=(B^{\mathsf T}B)^{-1}B^{\mathsf T}.

For a point \(x\) on the support plane, the intrinsic coordinates are \(B^+x=(x_1,x_3)^{\mathsf T}\), as follows from \(x_1=Z_1\) and \(x_3=Z_2\). Thus the density with respect to area on that plane is

f_{\mathrm{intrinsic}}(x) = \frac{1}{2\pi\sqrt{3}} \exp\left(-\frac{x_1^2+x_3^2}{2}\right).

The factor \(1/\sqrt{3}\) accounts for the area scaling of the map \(z\mapsto Bz\). This density is defined only on the plane and integrates to one with respect to area there.

Linear Projections and Further Rank Reduction

A linear transformation can preserve, reduce, or collapse the dimension of the support. Let \(A\) be a \(q\times p\) matrix and \(b\in\mathbb{R}^q\), and set \(Y=AX+b\). Then \(Y=A\mu+b+ABV\), so the transformed vector is elliptical with scatter parameter \(A\Sigma A^{\mathsf T}=(AB)(AB)^{\mathsf T}\). Its rank is \(\operatorname{rank}(AB)\), which can be smaller than \(r\).

Theorem: Under the conditions above, the characteristic function of \(Y=AX+b\) is \(\varphi_Y(t)=\exp(i t^{\mathsf T}(A\mu+b))\psi(t^{\mathsf T}A\Sigma A^{\mathsf T}t)\). Its support is contained in \(A\mu+b+\operatorname{Range}(AB)\), and equals this affine subspace if \(V\) has support \(\mathbb{R}^r\).

For the characteristic function, start with the scalar exponent and apply spherical symmetry:

\begin{aligned} \varphi_Y(t) &=\exp(i t^{\mathsf T}(A\mu+b)) E\left[\exp\left(i\,((AB)^{\mathsf T}t)^{\mathsf T}V\right)\right]\\ &=\exp(i t^{\mathsf T}(A\mu+b)) \psi\left(((AB)^{\mathsf T}t)^{\mathsf T}((AB)^{\mathsf T}t)\right)\\ &=\exp(i t^{\mathsf T}(A\mu+b)) \psi\left(t^{\mathsf T}A BB^{\mathsf T}A^{\mathsf T}t\right). \end{aligned}

The expectation is the characteristic function of \(V\) evaluated at \((AB)^{\mathsf T}t\). The final support statement follows directly from \(Y=A\mu+b+ABV\); if \(V\) has support \(\mathbb{R}^r\), its image under \(AB\) has support \(\operatorname{Range}(AB)\). \(\square\)

For a scalar projection \(a^{\mathsf T}X\), the relevant scatter is the scalar \(a^{\mathsf T}\Sigma a\). If this equals zero, then \(B^{\mathsf T}a=0\), because \(a^{\mathsf T}\Sigma a=\|B^{\mathsf T}a\|^2\). Therefore \(a^{\mathsf T}X=a^{\mathsf T}\mu\) with probability one: the projection is constant. If the scalar is positive, the projection has the corresponding one-dimensional elliptical distribution.

Worked Example: A Projection That Becomes Constant

Let \(Z\sim N(0,1)\), \(X=(Z,Z,2Z)^{\mathsf T}\), and \(A=\begin{pmatrix}1&-1&0\\0&0&1\end{pmatrix}\). Here \(B=(1,1,2)^{\mathsf T}\), so \(\Sigma=BB^{\mathsf T}\) has rank one. Directly,

AB=\begin{pmatrix}0\\2\end{pmatrix}, \qquad AX=\begin{pmatrix}0\\2Z\end{pmatrix}, \qquad A\Sigma A^{\mathsf T} =(AB)(AB)^{\mathsf T} = \begin{pmatrix}0&0\\0&4\end{pmatrix}.

The first projected coordinate is constant, while the second is \(N(0,4)\). The projected vector is supported on the vertical line \(\{(0,y):y\in\mathbb{R}\}\), not on the whole plane. The rank has stayed one here, but the example also shows why a positive semidefinite projected scatter matrix need not have full rank.

Worked Example: A Uniform Law on a Line

Let \(U\) be uniform on \([-1,1]\), and let \(X=\mu+(3,4)^{\mathsf T}U\) in \(\mathbb{R}^2\). The one-dimensional distribution of \(U\) is spherically symmetric: its law is unchanged by either orthogonal transformation of \(\mathbb{R}\), namely multiplication by \(1\) or \(-1\). The scatter parameter is

\Sigma= \begin{pmatrix}3\\4\end{pmatrix} \begin{pmatrix}3&4\end{pmatrix} = \begin{pmatrix}9&12\\12&16\end{pmatrix}, \qquad \operatorname{rank}(\Sigma)=1.

The support is the line \(\mu+\{(3u,4u)^{\mathsf T}:-1\leq u\leq1\}\). Consider the event \(X_1\geq\mu_1+1\). It is equivalent to \(3U\geq1\), or \(U\geq1/3\). Since the interval \([1/3,1]\) has length \(2/3\) and \([-1,1]\) has length \(2\),

P(X_1\geq\mu_1+1) =P(U\geq1/3) =\frac{2/3}{2} =\frac13.

This calculation is valid despite the fact that \(X\) has no two-dimensional density.

Common Mistake and Practical Checks

A common mistake is to see a \(p\times p\) scatter matrix and assume the law has a density over \(\mathbb{R}^p\). Matrix size alone does not determine the dimension of the distribution: the rank does. If the rank is \(r<p\), probabilities must respect the affine support, and any density must be understood relative to an \(r\)-dimensional measure on that support.

Before calculating, check the factorization and the event in intrinsic coordinates. A useful sequence is:

Factor the scatter matrix.
Write \(\Sigma=BB^{\mathsf T}\) with \(B\) having full column rank \(r\).

State the affine support.
All realizations lie in \(\mu+\operatorname{Range}(B)\); check whether the support of the intrinsic vector fills that subspace.

Translate the event.
Substitute \(X=\mu+Bv\), then calculate under the distribution of \(V\).

Check transformed rank.
For a linear map \(A\), use \(\operatorname{rank}(AB)\) to determine the dimension of the transformed affine support.

This approach also clarifies why singular elliptical laws are useful: they represent exact linear constraints, shared latent variation, and projections whose dimension is lower than the ambient coordinate count. Their geometry is not a technical nuisance; it is part of the probability model.

Check Your Understanding

Answer these questions using the affine representation and support dimension.

If \(X=\mu+BV\), \(B\) is \(p\times r\) with full column rank, and \(r<p\), what affine subspace contains \(X\)?
Why can a singular elliptical vector have an intrinsic density but no density with respect to ambient Lebesgue measure?
For a measurable set \(C\), express \(P(X\in C)\) as a probability involving \(V\).
What determines the rank of the scatter parameter for \(AX+b\), and why can it be smaller than the rank for \(X\)?
If \(a^{\mathsf T}\Sigma a=0\), what can be concluded about \(a^{\mathsf T}X\)?