Gaussian Scale Mixtures and Elliptical Tails

Why Mix Gaussian Scales?

A Gaussian model assigns one covariance matrix to an entire population. In some settings, however, observations vary in their overall spread: most may come from relatively concentrated conditions, while a smaller proportion arise under much broader ones. A Gaussian scale mixture represents this variation by drawing a positive scalar and using it to multiply the covariance matrix. The resulting distribution remains elliptical, but its radial density need not have Gaussian decay.

The distinction between the scalar and the matrix matters. The scalar changes spread in every direction at once; the matrix determines the relative shape and orientation of the ellipsoidal contours. How much probability lies far from the center depends on the distribution of the scalar, especially on its large values.

Here \(\Sigma^{1/2}\) is any matrix square root satisfying \(\Sigma^{1/2}(\Sigma^{1/2})'=\Sigma\). For \(x\in\mathbb R^p\), define the squared elliptical radius \(q(x)=(x-\mu)'\Sigma^{-1}(x-\mu)\). The matrix \(\Sigma\) specifies the contour shape; it is not necessarily the covariance matrix of \(X\).

Deriving the Radial Generator

For a fixed value \(w\), the conditional density is a Gaussian density with covariance \(w\Sigma\). Its determinant factor is \(w^{-p/2}|\Sigma|^{-1/2}\), and its exponent contains \(q(x)/w\). Averaging that density over the distribution \(H\) of \(W\) gives the unconditional density.

The expectation defining \(g\) is allowed to be infinite at some points; the density formula holds almost everywhere as a nonnegative mixture density. In particular, a finite density at the center requires \(E[W^{-p/2}]<\infty\). That condition is not needed for the mixture to define a probability distribution.

Derivation. Conditional on \(W=w\), the density at \(x\) is

Since \(|w\Sigma|=w^p|\Sigma|\) and \((w\Sigma)^{-1}=w^{-1}\Sigma^{-1}\), this becomes

Integrating this nonnegative conditional density with respect to the probability distribution \(H\) gives \(f_X(x)=\int f_{X\mid W}(x\mid w)\,dH(w)\). Tonelli’s theorem justifies the integration even if the density is infinite on a set of measure zero, and factoring out \(|\Sigma|^{-1/2}\) yields the stated generator. As a check, integrating the mixture density over \(x\) gives one: Tonelli’s theorem permits reversing the integrals, and every conditional Gaussian density integrates to one. \(\square\)

Because the density depends on \(x\) only through \(q(x)\), its level sets are ellipsoids centered at \(\mu\) whenever the generator’s level sets yield such contours. The Gaussian case is recovered when \(W\) is a constant: for \(W=c\), \(g(q)=(2\pi c)^{-p/2}\exp(-q/(2c))\), corresponding to \(N_p(\mu,c\Sigma)\).

Moments and the Role of the Mixing Distribution

The random-scale representation makes moment conditions direct to derive. Write \(A=\Sigma^{1/2}\), so \(X-\mu=\sqrt{W}AZ\). For any real \(k>0\), the Euclidean norm satisfies \(\|X-\mu\|^k=W^{k/2}\|AZ\|^k\). Independence separates the expectation into a scale factor and a Gaussian factor.

where either side may be infinite. Since \(E[\|\Sigma^{1/2}Z\|^k]\) is finite and strictly positive for every \(k>0\), the \(k\)th absolute radial moment is finite exactly when \(E[W^{k/2}]<\infty\).

Proof. From the representation, \(\|X-\mu\|^k=W^{k/2}\|AZ\|^k\). Both factors are nonnegative and independent. Tonelli’s theorem, or independence applied first to truncations and then monotone convergence, gives \(E[W^{k/2}\|AZ\|^k]=E[W^{k/2}]E[\|AZ\|^k]\), including the possibility of an infinite expectation. A Gaussian vector has finite moments of every positive order, and \(A\) is invertible, so \(\|AZ\|\) is nonzero with probability one and its \(k\)th moment is strictly positive. The finiteness equivalence follows. \(\square\)

In particular, \(E[W]<\infty\) is sufficient and necessary for a finite covariance matrix. The mean exists when \(E[\sqrt{W}]<\infty\); in that case \(E[X]=\mu\), since the independent Gaussian factor has mean zero. When \(E[W]<\infty\), conditioning on \(W\) gives

How Scale Mixing Changes the Tails

The generator shows why large mixing scales matter: a large \(W\) makes the factor \(\exp(-q/(2W))\) decay more slowly as the radius \(q\) increases. In a finite mixture, the largest scale eventually controls the generator’s decay. In a mixture with unbounded \(W\), the far-tail behavior depends on how much probability the mixing distribution assigns to very large scales.

Proof. The generator is the finite sum

Divide this expression by the proposed leading term. The \(j=m\) term becomes one. For \(j<m\), the ratio of the exponential factors is \(\exp[-q(1/(2w_j)-1/(2w_m))]\). Since \(w_j<w_m\), the coefficient in parentheses is positive, so this ratio tends to zero as \(q\to\infty\). There are finitely many such terms, and their sum tends to zero. The full ratio therefore tends to one. \(\square\)

A larger variance of \(W\) does not, by itself, determine the tail class. The upper-tail behavior of the whole mixing distribution matters. For example, a mixing variable bounded above has no arbitrarily large scales, whereas the Pareto mixing variable in the preceding example is unbounded and permits moments to fail at a finite order. Two mixing distributions can have similar low-order summaries but very different probabilities of extremely large scales. Consequently, describe tail behavior through properties such as the support and upper tail of \(W\), rather than through its variance alone.

Why the Construction Matters—and a Common Pitfall

Gaussian scale mixtures retain a simple conditional model while allowing overall spread to vary between observations or latent conditions. This can give an elliptical model a slower-decaying radial density than a single Gaussian, making it useful when extreme observations occur more often than a fixed-scale Gaussian would suggest. The same flexibility also affects moment existence: a sufficiently heavy upper tail for \(W\) can make the covariance or higher moments infinite.

Common mistake: treating \(\Sigma\) as the unconditional covariance. The conditional covariance is \(W\Sigma\), so the unconditional covariance is \(E[W]\Sigma\) only when \(E[W]<\infty\). If \(E[W]\) is infinite, writing down \(E[W]\Sigma\) does not define a finite covariance matrix. Another common error is to say that any nonconstant scale mixture has a particular heavy-tail class. Nonconstancy changes the Gaussian density through averaging, but the precise tail behavior depends on the distribution of \(W\); bounded and unbounded mixing scales need not behave alike.