Means, Covariances, and Scatter Matrices

Why Scatter and Covariance Are Not Always the Same

An elliptical distribution is described by a location and a matrix that determines its shape and scale. It is tempting to identify that matrix with the covariance matrix. This is correct for some familiar distributions, but not in general: the covariance can be a scalar multiple of the scatter matrix, with the scalar determined by the radial distribution. If the relevant radial moment is infinite, the covariance may not exist at all.

The radial-angular representation makes these distinctions explicit. It separates the direction of a random vector, which is uniformly distributed over the sphere, from its nonnegative distance from the center. We will use this representation to determine exactly when the mean and covariance are finite and to calculate them when they exist.

Radial-Angular Representation and Spherical Moments

Definition: Let \(d\geq 1\), let \(\mu\in\mathbb{R}^d\), and let \(\Sigma\) be a positive-definite \(d\times d\) matrix. A full-dimensional elliptical random vector with location \(\mu\) and scatter matrix \(\Sigma\) can be represented as \[ X=\mu+A R U, \] where \(A\) is a nonsingular matrix satisfying \(AA^{\mathsf T}=\Sigma\), \(R\geq 0\) is a random variable, and \(U\) is independent of \(R\) and uniform on the unit sphere in \(\mathbb{R}^d\). The matrix \(\Sigma\) is called a scatter matrix in this representation. The random variable \(R\) is the radius.

The uniform distribution on the sphere is invariant under orthogonal transformations. In particular, reflecting a coordinate changes its sign without changing the distribution, and permuting coordinates leaves the distribution unchanged. These symmetries determine the first two moments of \(U\).

Result: For \(U\) uniform on the unit sphere in \(\mathbb{R}^d\), \[ E[U]=0,\qquad E[UU^{\mathsf T}]=\frac{1}{d}I_d. \]

To see the first identity, reflection symmetry makes each coordinate have the same distribution as its negative, so each coordinate has expectation zero. For the second, reflecting one coordinate shows that the off-diagonal entries of \(E[UU^{\mathsf T}]\) are zero. Permutation symmetry makes all diagonal entries equal. Since \(U^{\mathsf T}U=1\), the trace of \(E[UU^{\mathsf T}]\) is \(E[U^{\mathsf T}U]=1\). Thus each of its \(d\) diagonal entries is \(1/d\). These arguments also apply when \(d=1\), where the sphere consists of \(-1\) and \(1\).

Conditions for a Finite Mean and Covariance

For a finite-dimensional random vector, finite first moments mean that each coordinate has a finite absolute expectation. Equivalently, \(E[\|X\|]\) is finite, for any norm \(\|\cdot\|\) on \(\mathbb{R}^d\). The invertibility of \(A\) lets us compare the size of \(ARU\) to \(R\). In particular, there are constants \(0<c\leq C<\infty\) such that \[ cR\leq \|ARU\|\leq CR. \] One may take \(c\) and \(C\) to be the smallest and largest singular values of \(A\), using the Euclidean norm. Consequently, \(E[\|X-\mu\|]\) is finite exactly when \(E[R]\) is finite. The same comparison, applied to squared norms, shows that \(E[\|X-\mu\|^2]\) is finite exactly when \(E[R^2]\) is finite.

Theorem: For the representation \(X=\mu+ARU\) above, \(E[\|X\|]\) is finite if and only if \(E[R]<\infty\). In that case, \[ E[X]=\mu. \] The covariance matrix exists and is finite if and only if \(E[R^2]<\infty\). When it exists, \[ \operatorname{Cov}(X)=\frac{E[R^2]}{d}\,\Sigma. \]

Proof. Because \(A\) is nonsingular and \(\|U\|=1\), the norm comparisons above imply that \(E[\|X-\mu\|]<\infty\) exactly when \(E[R]<\infty\). Adding the fixed vector \(\mu\) does not change whether the first absolute moments are finite. When \(E[R]<\infty\), independence and boundedness of \(U\) give \[ E[ARU]=A\,E[R]E[U]=0, \] so \(E[X]=\mu\).

For the covariance, finite second moments are equivalent to \(E[\|X-\mu\|^2]<\infty\), which the squared norm comparison shows is equivalent to \(E[R^2]<\infty\). In this case the mean exists as well, since \(R\leq 1+R^2\) for \(R\geq0\). The mean-centered vector is \(ARU\). Independence and the spherical second-moment identity then yield

\begin{aligned} \operatorname{Cov}(X) &=E\big[(ARU)(ARU)^{\mathsf T}\big]\\ &=A\,E[R^2]\,E[UU^{\mathsf T}]\,A^{\mathsf T}\\ &=\frac{E[R^2]}{d}AA^{\mathsf T} =\frac{E[R^2]}{d}\Sigma. \end{aligned}

Every entry in the matrix expectation is integrable because the entries of \(U\) are bounded and \(E[R^2]<\infty\). This establishes the formula and completes the proof. \(\square\)

Reading the Covariance Formula

The theorem separates two roles. The matrix \(\Sigma\) determines the scatter shape and its principal directions; the scalar \(E[R^2]/d\) determines how that scatter is scaled to obtain covariance. In particular, if \(E[R^2]>0\), covariance is a positive scalar multiple of \(\Sigma\). It then has the same eigenvectors and principal directions as \(\Sigma\), and its eigenvalues are those of \(\Sigma\) multiplied by \(E[R^2]/d\).

The positivity condition matters. Since \(R\geq0\), \(E[R^2]=0\) holds exactly when \(R=0\) almost surely. In that case \(X=\mu\) almost surely and its covariance is the zero matrix, not a positive multiple of the positive-definite \(\Sigma\). More generally, \(\Sigma\) specifies the shape parameter in the representation; it need not equal the covariance matrix.

Worked Example: Matching Scatter to Covariance in a Gaussian Model

Let \(Z\sim N_d(0,I_d)\), take \(R=\|Z\|\), and set \(U=Z/\|Z\|\) when \(Z\neq0\). The event \(Z=0\) has probability zero, so the value assigned to \(U\) there does not affect its distribution. Spherical symmetry gives a uniform direction independent of the radius, so \(Z=RU\). Define \(X=\mu+AZ\), where \(AA^{\mathsf T}=\Sigma\).

Since \(Z_1,\ldots,Z_d\) each have variance \(1\), \[ E[R^2]=E[\|Z\|^2]=\sum_{j=1}^d E[Z_j^2]=d. \] Also \(E[R]\leq\sqrt{E[R^2]}=\sqrt d\), by the Cauchy–Schwarz inequality, so the mean exists. The theorem gives \[ E[X]=\mu,\qquad \operatorname{Cov}(X)=\frac{d}{d}\Sigma=\Sigma. \] Thus in this Gaussian construction the scatter matrix equals the covariance matrix. The equality follows from the radial second moment; it is not a universal property of elliptical distributions.

Worked Example: A Fixed Radius in Three Dimensions

Let \(d=3\), let \(R=2\) with probability one, and let \(U\) be uniform on the unit sphere, independent of \(R\). For \(X=\mu+ARU\), where \(AA^{\mathsf T}=\Sigma\), the radial moments are \[ E[R]=2,\qquad E[R^2]=4. \] Therefore the mean and covariance are \[ E[X]=\mu,\qquad \operatorname{Cov}(X)=\frac{4}{3}\Sigma. \] Since \(4/3>0\), the covariance has the same principal directions as \(\Sigma\), but it is not equal to \(\Sigma\).

If instead \(R=0\) with probability one, then \(E[R^2]=0\) and \(X=\mu\) almost surely. The formula gives \(\operatorname{Cov}(X)=0\). This limiting case has no nonzero covariance principal directions, despite the positive-definite scatter matrix used in the representation.

Worked Example: Pareto Radii and Moment Existence

Consider a radius with density \(f_R(r)=\alpha r^{-\alpha-1}\) for \(r\geq1\), where \(\alpha>0\), and zero otherwise. For \(k>0\), direct integration gives \[ E[R^k]=\int_1^\infty r^k\alpha r^{-\alpha-1}\,dr =\alpha\int_1^\infty r^{k-\alpha-1}\,dr. \] The integral is finite exactly when \(k<\alpha\); in that case, \[ E[R^k]=\alpha\left[\frac{r^{k-\alpha}}{k-\alpha}\right]_{1}^{\infty} =\frac{\alpha}{\alpha-k}. \]

For \(\alpha=3\), the mean radius is \(E[R]=3/(3-1)=3/2\), and the second radial moment is \(E[R^2]=3/(3-2)=3\). In dimension \(d=2\), an elliptical vector with this radius has \[ E[X]=\mu,\qquad \operatorname{Cov}(X)=\frac{3}{2}\Sigma. \] For \(\alpha=3/2\), \(E[R]=\frac{3/2}{3/2-1}=3\), so the vector still has mean \(\mu\). But \(E[R^2]=\infty\), because \(2\not<3/2\). Its covariance is not finite. The existence of a mean therefore does not guarantee the existence of a covariance.

Common Mistakes and Why the Distinction Matters

Calling the scatter matrix the covariance automatically. The formula includes the radial factor \(E[R^2]/d\). It equals \(\Sigma\) only when this factor is \(1\).
Assuming symmetry alone guarantees a finite mean. Symmetry identifies the mean as the center only when the first absolute moments exist. If \(E[R]=\infty\), symmetry does not produce an ordinary finite expectation.
Claiming that covariance always has the same principal directions as scatter. This conclusion requires \(E[R^2]>0\). If \(R=0\) almost surely, covariance is zero, so it is not a positive multiple of \(\Sigma\).
Inferring finite covariance from a finite mean. The Pareto example with \(\alpha=3/2\) has a finite mean but an infinite second radial moment. Its covariance does not exist as a finite matrix.

These checks are useful whenever a scatter matrix is used to describe an elliptical model. Before interpreting its entries as variances and covariances, verify that \(E[R^2]\) is finite and determine the value of \(E[R^2]/d\). If that scalar is positive, the covariance and scatter matrices share their principal directions; if it is zero or undefined, that interpretation must be adjusted.

Check Your Understanding

Use the radial-angular representation \(X=\mu+ARU\), with \(AA^{\mathsf T}=\Sigma\), to answer the following.

What conditions on \(R\) ensure that \(E[X]\) exists and equals \(\mu\)?
If \(d=4\) and \(E[R^2]=12\), what is \(\operatorname{Cov}(X)\) in terms of \(\Sigma\)?
Why does \(E[R^2]=0\) prevent covariance from being a positive scalar multiple of a positive-definite scatter matrix?
For a Pareto radius with \(\alpha=5/2\), does the corresponding elliptical vector have a finite mean? Does it have a finite covariance? Explain using radial moments.
Under what condition do covariance and scatter have the same principal directions?