Scale Mixtures of Normals and the Student Distribution

Why Mix Over the Variance?

A normal model with a fixed variance gives relatively little probability to extreme observations. If its variance itself varies across observations, however, most observations can come from low-variance conditions while a smaller number come from high-variance conditions. Integrating over that uncertainty produces a distribution with heavier tails. A particularly important case is the Student t distribution, obtained by mixing a zero-mean normal law over an inverse-gamma variance.

The construction is useful in its own right: it gives a direct route to the t density and explains why its tails decay as a power rather than exponentially. It also makes precise how the degrees of freedom control the existence of moments.

The Inverse-Gamma Normal Mixture

Definition: For $a>0$ and $b>0$, a random variable $V$ has an inverse-gamma distribution with shape $a$ and scale $b$, written here as $V\sim\operatorname{InvGamma}(a,b)$, if its density is $$ f_V(v)=\frac{b^a}{\Gamma(a)}v^{-a-1}\exp\left(-\frac{b}{v}\right),\qquad v>0. $$ For degrees of freedom $\nu>0$ and scale $s>0$, take $V\sim\operatorname{InvGamma}(\nu/2,\nu s^2/2)$ and, conditionally on $V=v$, let $X\mid V=v\sim N(0,v)$. The marginal distribution of $X$ is the Student t distribution with $\nu$ degrees of freedom and scale $s$, denoted $t_\nu(0,s)$.

The second inverse-gamma parameter is the scale $b=\nu s^2/2$ in the density above; inverse-gamma parameter conventions can differ, so specifying the density avoids ambiguity. The parameter $s$ is the t distribution's scale, not generally the standard deviation of $X$.

Deriving the Student t Density

Write $a=\nu/2$ and $b=\nu s^2/2$. The conditional normal density is $(2\pi v)^{-1/2}\exp(-x^2/(2v))$. By the continuous-mixture density formula, the marginal density at $x$ is the integral of this conditional density against $f_V(v)$.

\begin{aligned} f_X(x) &=\frac{b^a}{\Gamma(a)\sqrt{2\pi}} \int_0^\infty v^{-a-3/2} \exp\left(-\frac{b+x^2/2}{v}\right)\,dv. \end{aligned}

For $c>0$, substitute $u=c/v$ to obtain

\int_0^\infty v^{-q-1}e^{-c/v}\,dv =c^{-q}\Gamma(q),\qquad q>0,

where $\Gamma(q)=\int_0^\infty u^{q-1}e^{-u}\,du$. Apply this identity with $q=a+1/2$ and $c=b+x^2/2$. Since $b=\nu s^2/2$, simplifying gives the density below.

Theorem: If $V\sim\operatorname{InvGamma}(\nu/2,\nu s^2/2)$ and $X\mid V=v\sim N(0,v)$, where $\nu,s>0$, then $$ f_X(x)= \frac{\Gamma((\nu+1)/2)} {\Gamma(\nu/2)\sqrt{\nu\pi}\,s} \left(1+\frac{x^2}{\nu s^2}\right)^{-(\nu+1)/2}, \qquad -\infty<x<\infty. $$ This is the density of $t_\nu(0,s)$.

In particular, the t density is symmetric about zero. Its shape depends on $\nu$: smaller degrees of freedom give heavier tails. As $\nu$ increases, the tail becomes less heavy; for a fixed $s$, the density approaches the $N(0,s^2)$ density.

Worked Example: Calculating a t Density From the Mixture

Let $\nu=5$ and $s=2$. Identify the mixing distribution and calculate the marginal density at $x=2$.

The inverse-gamma shape and scale are $a=\nu/2=5/2$ and $b=\nu s^2/2=5(4)/2=10$. Thus $V\sim\operatorname{InvGamma}(5/2,10)$, and $X\mid V=v\sim N(0,v)$.

The t density has coefficient $$ \frac{\Gamma(3)}{\Gamma(5/2)\sqrt{5\pi}\,2} =\frac{2}{(3\sqrt{\pi}/4)\,2\sqrt{5\pi}} =\frac{4}{3\pi\sqrt{5}}. $$ At $x=2$, the factor involving $x$ is $$ \left(1+\frac{2^2}{5(2^2)}\right)^{-3} =\left(\frac{6}{5}\right)^{-3} =\frac{125}{216}. $$ Therefore $$ f_X(2)=\frac{4}{3\pi\sqrt{5}}\frac{125}{216} =\frac{125}{162\pi\sqrt{5}}. $$

Power-Law Tail Behavior

The density formula shows that the t distribution has polynomial, or power-law, tails. To obtain the exact leading constant, rewrite its density for $x>0$ as

f_X(x)= \frac{\Gamma((\nu+1)/2)} {\Gamma(\nu/2)\sqrt{\nu\pi}\,s} \left(\frac{x^2}{\nu s^2}\right)^{-(\nu+1)/2} \left(1+\frac{\nu s^2}{x^2}\right)^{-(\nu+1)/2}.

The final factor tends to $1$ as $x$ tends to infinity. Consequently,

f_X(x)\sim A x^{-(\nu+1)},\qquad A=\frac{\Gamma((\nu+1)/2)\nu^{\nu/2}s^\nu} {\Gamma(\nu/2)\sqrt{\pi}}, \qquad x\to\infty.

Here $g(x)\sim h(x)$ means $g(x)/h(x)\to1$. The density is symmetric, so the same density asymptotic, with $|x|$ in place of $x$, holds in the left tail. Integrating the asymptotic density gives the one-sided and two-sided survival-tail asymptotics:

P(X>x)\sim \frac{A}{\nu}x^{-\nu}, \qquad P(|X|>x)\sim \frac{2A}{\nu}x^{-\nu}, \qquad x\to\infty.

For justification of the integration, for every $\epsilon>0$ the density ratio $f_X(y)/(Ay^{-\nu-1})$ lies between $1-\epsilon$ and $1+\epsilon$ for all sufficiently large $y$. Integrating those bounds from $x$ to infinity and using $\int_x^\infty y^{-\nu-1}\,dy=x^{-\nu}/\nu$ proves the one-sided equivalence. Symmetry gives the two-sided result.

Worked Example: Tail Asymptotics for Three Degrees of Freedom

Let $X\sim t_3(0,2)$. Calculate the leading density and two-sided tail constants.

Here $\nu=3$ and $s=2$. The density constant is $$ A=\frac{\Gamma(2)3^{3/2}2^3}{\Gamma(3/2)\sqrt{\pi}} =\frac{1\cdot3\sqrt{3}\cdot8}{(\sqrt{\pi}/2)\sqrt{\pi}} =\frac{48\sqrt{3}}{\pi}. $$ Thus $f_X(x)\sim (48\sqrt{3}/\pi)x^{-4}$ as $x\to\infty$, and by symmetry the density behaves as $(48\sqrt{3}/\pi)|x|^{-4}$ in either tail. The two-sided survival constant is $2A/\nu$, so $$ P(|X|>x)\sim\frac{32\sqrt{3}}{\pi}x^{-3}. $$ For example, at a large threshold $x=10$, the asymptotic approximation is $32\sqrt{3}/(1000\pi)$, about $0.0176$. This is an approximation from the tail limit, not the exact probability at $10$.

Which Moments Exist?

The mixture representation gives a direct moment calculation. Let $Z\sim N(0,1)$ be independent of $V$. The conditional normal construction is equivalent to $X=\sqrt{V}\,Z$. For $r>0$, independence and nonnegative integration give $E[|X|^r]=E[V^{r/2}]E[|Z|^r]$, allowing either side to be infinite.

The standard normal absolute moment is $E[|Z|^r]=2^{r/2}\Gamma((r+1)/2)/\sqrt{\pi}$. From the inverse-gamma density, substituting $u=b/v$ gives

E[V^{r/2}] =\frac{b^{r/2}\Gamma(a-r/2)}{\Gamma(a)} \quad\text{if }r<2a.

This expectation is finite exactly when $r<2a$: the integral defining it near $v=\infty$ has integrand proportional to $v^{r/2-a-1}$, which is integrable there exactly under that condition. At $r=2a$ it diverges logarithmically, and for $r>2a$ it also diverges. With $a=\nu/2$ and $b=\nu s^2/2$, the result is as follows.

Theorem: For $X\sim t_\nu(0,s)$ with $\nu,s>0$ and $r>0$, $$ E[|X|^r] = \frac{(\nu s^2)^{r/2}\Gamma((r+1)/2)\Gamma((\nu-r)/2)} {\sqrt{\pi}\,\Gamma(\nu/2)} \quad\text{if }r<\nu. $$ If $r\geq\nu$, then $E[|X|^r]=\infty$. In particular, a signed moment of positive integer order is finite only when that integer is less than $\nu$. When it exists, every odd signed moment is zero by symmetry.

The mean exists as an ordinary finite expectation only for $\nu>1$, in which case symmetry gives $E[X]=0$. The second moment, and hence the variance, exists only for $\nu>2$. In that case,

\operatorname{Var}(X)=E[X^2] =E[V] =\frac{\nu s^2}{\nu-2}.

The equality $E[V]=\nu s^2/(\nu-2)$ follows by evaluating the inverse-gamma moment formula at $r=2$. Since the mean is zero for $\nu>2$, this second moment is the variance. When $\nu\leq2$, the variance is not finite, even though the distribution is still a valid probability law.

Worked Example: Finite Mean but Infinite Fourth Moment

Let $X\sim t_4(0,3)$. Determine its mean, variance, and fourth absolute moment.

Here $V\sim\operatorname{InvGamma}(2,18)$, since $\nu/2=2$ and $\nu s^2/2=4(9)/2=18$. Because $4>2$, the mean exists and symmetry gives $E[X]=0$. The variance is $$ \operatorname{Var}(X)=\frac{4(3^2)}{4-2}=18. $$ For the fourth absolute moment, $r=4=\nu$, which does not satisfy the required strict inequality $r<\nu$. Therefore $E[|X|^4]=\infty$; it is not valid to infer that this moment exists just because the variance is finite.

As a further check, the first absolute moment is finite. Its formula gives $$ E[|X|] =\frac{36^{1/2}\Gamma(1)\Gamma(3/2)} {\sqrt{\pi}\Gamma(2)} =\frac{6\cdot1\cdot(\sqrt{\pi}/2)} {\sqrt{\pi}\cdot1} =3. $$

Common Pitfall: Treating Scale as Standard Deviation

The parameter $s$ in $t_\nu(0,s)$ is a scale parameter, not automatically the standard deviation. The variance is $\nu s^2/(\nu-2)$ only when $\nu>2$; it is infinite when $0<\nu\leq2$. For example, the distribution in the preceding example has scale $3$ but standard deviation $\sqrt{18}$, not $3$.

A related mistake is to say that a t distribution has a mean of zero for every positive $\nu$ simply because its density is symmetric. Symmetry identifies the center, but an ordinary finite expectation requires $E[|X|]<\infty$, which holds only for $\nu>1$. For $\nu\leq1$, the mean is undefined.

Key takeaway: In the inverse-gamma normal mixture, the variance mixing distribution determines the Student t law. Its density decays like $|x|^{-(\nu+1)}$, its survival tails like $x^{-\nu}$, and its absolute moment of order $r>0$ is finite exactly when $r<\nu$.

Check Your Understanding

Use the mixture density, tail, and moment conditions to answer these questions.

For $X\sim t_6(0,2)$, what inverse-gamma shape and scale specify the variance mixture?
For $X\sim t_\nu(0,s)$, which power of $x$ describes the leading decay of the density in the right tail?
Does a $t_2(0,1)$ random variable have a finite mean? Does it have a finite variance?
For $X\sim t_5(0,1)$, which of $E[|X|^3]$ and $E[|X|^5]$ are finite?
Explain why symmetry alone does not guarantee that the mean exists.