Deriving the Multivariate Student T Distribution

Why the Scale Mixture Produces a t Distribution

A multivariate Gaussian model has a fixed covariance matrix, while a multivariate Student t model can be represented as a Gaussian whose scale varies randomly across observations. A small mixing value makes the conditional Gaussian broad; integrating over the mixing distribution gives the t density. This construction is useful because it gives both an explicit density and a clear way to understand the role of the scatter matrix.

We derive the density from the mixture rather than assume its formula. The degrees of freedom control the mixing distribution and, in turn, determine whether the t vector has finite covariance. Throughout, let $p$ be a positive integer, let $\nu>0$, let $\mu\in\mathbb{R}^p$, and let $\Sigma$ be a symmetric positive-definite $p\times p$ matrix.

Definition: Let $W\sim\chi^2_\nu$, and independently let $Z\sim N_p(0,I_p)$. Choose a matrix $A$ such that $AA^\mathsf{T}=\Sigma$, and define $X=\mu+\sqrt{\nu/W}\,AZ$. The distribution of $X$ is the $p$-variate Student t distribution with degrees of freedom $\nu$, location $\mu$, and scatter matrix $\Sigma$, written $X\sim t_p(\nu,\mu,\Sigma)$. Equivalently, conditional on $W$, $X\mid W\sim N_p(\mu,(\nu/W)\Sigma)$.

The scale factor $S=\nu/W$ is positive and is sometimes described as an inverse-chi-square scale. This parameterization fixes its numerator at $\nu$; that detail matters in the density and covariance calculations below. A scatter matrix determines the shape of the conditional Gaussian covariance and the t contours, but it is not generally the covariance matrix of $X$.

Deriving the Density

The chi-square density, for $w>0$, is $f_W(w)=w^{\nu/2-1}e^{-w/2}/(2^{\nu/2}\Gamma(\nu/2))$. For a fixed $w$, the conditional normal density has determinant factor $(w/\nu)^{p/2}|\Sigma|^{-1/2}$ and quadratic form $(w/\nu)q(x)$, where $q(x)=(x-\mu)^\mathsf{T}\Sigma^{-1}(x-\mu)$. Multiplying the two densities and integrating out $W$ gives the marginal density.

Theorem: If $\nu>0$ and $\Sigma$ is positive definite, the density of $X\sim t_p(\nu,\mu,\Sigma)$ is $$ f_X(x)= \frac{\Gamma((\nu+p)/2)} {\Gamma(\nu/2)(\nu\pi)^{p/2}|\Sigma|^{1/2}} \left(1+\frac{q(x)}{\nu}\right)^{-(\nu+p)/2}, \qquad x\in\mathbb{R}^p. $$

Proof. The conditional density is

f_{X\mid W}(x\mid w)= \frac{(w/\nu)^{p/2}} {(2\pi)^{p/2}|\Sigma|^{1/2}} \exp\left(-\frac{wq(x)}{2\nu}\right).

Because $W$ has a density and the integrand is nonnegative, the marginal density is obtained by integrating the product $f_{X\mid W}(x\mid w)f_W(w)$ over $w>0$. Collecting constants and powers of $w$, set $a=(\nu+p)/2$. Then

\begin{aligned} f_X(x) &=\frac{1}{(2\pi)^{p/2}|\Sigma|^{1/2}\nu^{p/2}2^{\nu/2}\Gamma(\nu/2)} \int_0^\infty w^{a-1}\exp\left[-\frac{w}{2}\left(1+\frac{q(x)}{\nu}\right)\right]\,dw. \end{aligned}

For $c>0$, the gamma integral is $\int_0^\infty w^{a-1}e^{-cw}\,dw=\Gamma(a)c^{-a}$, for $a>0$. Here $a=(\nu+p)/2>0$ and $c=(1+q(x)/\nu)/2>0$. Substituting this integral and simplifying the powers of $2$ gives the stated density:

$$ \begin{aligned} f_X(x) &=\frac{\Gamma(a)} {(2\pi)^{p/2}|\Sigma|^{1/2}\nu^{p/2}2^{\nu/2}\Gamma(\nu/2)} \left[\frac{1+q(x)/\nu}{2}\right]^{-a}\\ &=\frac{\Gamma((\nu+p)/2)} {\Gamma(\nu/2)(\nu\pi)^{p/2}|\Sigma|^{1/2}} \left(1+\frac{q(x)}{\nu}\right)^{-(\nu+p)/2}. \end{aligned} $$ The density is normalized because it was obtained by integrating the normalized conditional density against the normalized density of \(W\). \(\square\)

Worked Density Calculations

Worked Example: Evaluating a Bivariate t Density

Let $X\sim t_2(5,\mu,\Sigma)$, with $\mu=(1,-1)^\mathsf{T}$ and $\Sigma=\begin{pmatrix}4&1\\1&2\end{pmatrix}$. Find the density at $x=(2,1)^\mathsf{T}$.

First, $|\Sigma|=4(2)-1(1)=7$, and $\Sigma^{-1}=\frac{1}{7}\begin{pmatrix}2&-1\\-1&4\end{pmatrix}$. The difference from the location is $x-\mu=(1,2)^\mathsf{T}$, so

q(x)= \begin{pmatrix}1&2\end{pmatrix} \frac{1}{7}\begin{pmatrix}2&-1\\-1&4\end{pmatrix} \begin{pmatrix}1\\2\end{pmatrix} =\frac{2-4+16}{7}=2.

The gamma ratio is $\Gamma(7/2)/\Gamma(5/2)=5/2$, using $\Gamma(a+1)=a\Gamma(a)$. Thus the density is

\begin{aligned} f_X(2,1) &=\frac{5/2}{(5\pi)^{1}\sqrt{7}} \left(1+\frac{2}{5}\right)^{-7/2}\\ &=\frac{1}{2\pi\sqrt{7}}\left(\frac{7}{5}\right)^{-7/2}. \end{aligned}

The determinant contributes $\sqrt{7}$, not $7$, because the normalizing factor uses $|\Sigma|^{1/2}$.

Worked Example: The One-Dimensional Scale Parameter

Let $X\sim t_1(3,0,4)$. Here the scalar scatter is $\Sigma=4$, so $|\Sigma|^{1/2}=2$. At $x=2$, the quadratic form is $q(x)=x^2/4=1$. Since $\Gamma(2)=1$ and $\Gamma(3/2)=\sqrt{\pi}/2$, the density is

\begin{aligned} f_X(2) &=\frac{\Gamma(2)} {\Gamma(3/2)(3\pi)^{1/2}\,2} \left(1+\frac{1}{3}\right)^{-2}\\ &=\frac{1}{\pi\sqrt{3}}\left(\frac{4}{3}\right)^{-2} =\frac{9}{16\pi\sqrt{3}}. \end{aligned}

In one dimension, this is the usual t density with scale parameter $\sqrt{4}=2$. That scale parameter is not, by itself, the standard deviation; whether a standard deviation exists depends on $\nu$.

When Is the Scatter Matrix the Covariance?

The mixture representation makes the moment threshold transparent. Conditional on $W$, the mean is $\mu$ and the covariance is $(\nu/W)\Sigma$. To average those conditional covariances, we need the inverse moment $E[1/W]$. If $W\sim\chi^2_\nu$, then $W$ has a gamma density with shape $\nu/2$ and scale $2$. Direct integration gives

E[W^{-1}] =\frac{1}{2^{\nu/2}\Gamma(\nu/2)} \int_0^\infty w^{\nu/2-2}e^{-w/2}\,dw =\frac{1}{\nu-2},\qquad \nu>2.

The integral is finite precisely when its power near zero is integrable, requiring $\nu/2-2>-1$, or $\nu>2$. When finite, the gamma integral evaluates to $2^{\nu/2-1}\Gamma(\nu/2-1)$; using $\Gamma(\nu/2)=(\nu/2-1)\Gamma(\nu/2-1)$ gives the last equality.

Theorem: For $X\sim t_p(\nu,\mu,\Sigma)$, $E[X]=\mu$ when $\nu>1$. The covariance exists and is $\operatorname{Cov}(X)=\frac{\nu}{\nu-2}\Sigma$ when $\nu>2$. For $1<\nu\leq2$, the mean exists but the second centered moments are infinite, so covariance does not exist as a finite matrix.

Derivation. Write $X-\mu=\sqrt{\nu/W}\,AZ$, where $Z$ is independent of $W$, $E[Z]=0$, and $E[ZZ^\mathsf{T}]=I_p$. The univariate scale factor has finite first moment exactly when $\nu>1$: using the same gamma integral gives $E[W^{-1/2}]<\infty$ for $\nu>1$. Thus, for $\nu>1$, the mixture has a finite mean and $E[X-\mu]=0$.

For $\nu>2$, the entries of the second-moment matrix are integrable, and independence gives

\begin{aligned} E[(X-\mu)(X-\mu)^\mathsf{T}] &=\nu A\,E[ZZ^\mathsf{T}]\,A^\mathsf{T}E[W^{-1}]\\ &=\nu A I_p A^\mathsf{T}\frac{1}{\nu-2} =\frac{\nu}{\nu-2}\Sigma. \end{aligned}

Since the mean is $\mu$, this second centered moment is the covariance. To see why the threshold cannot be weakened, take the trace of the nonnegative conditional second-moment matrix. Its unconditional value is $\nu\operatorname{tr}(\Sigma)E[W^{-1}]$, which is infinite for $0<\nu\leq2$, because $\operatorname{tr}(\Sigma)>0$ and the inverse moment diverges at zero. For $1<\nu\leq2$, this shows that a finite mean does not imply finite variance. For $\nu\leq1$, the mean itself is not finite. $\square$

Worked Example: Converting Scatter to Covariance

Suppose $X\sim t_3(4,\mu,\Sigma)$, where $\mu\in\mathbb{R}^3$ and $\Sigma=\operatorname{diag}(1,2,3)$. Since $\nu=4>2$, the covariance exists. The multiplier is $\nu/(\nu-2)=4/2=2$, so

\operatorname{Cov}(X) =2\operatorname{diag}(1,2,3) =\operatorname{diag}(2,4,6).

For the same location and scatter but $\nu=2$, the scatter matrix still defines the density, but the inverse mixing moment $E[1/W]$ diverges. It would therefore be incorrect to report $\Sigma$ as the covariance in that case.

Common Pitfall and Practical Interpretation

A common mistake is to read the matrix after the location parameter in $t_p(\nu,\mu,\Sigma)$ as the covariance automatically. The reasoning might be: “The conditional Gaussian has covariance proportional to $\Sigma$, so the marginal covariance must be $\Sigma$.” The correction is that mixing changes the scale by the random factor $\nu/W$. When $\nu>2$, averaging that factor multiplies $\Sigma$ by $\nu/(\nu-2)$; when $\nu\leq2$, the average is infinite.

The distinction matters when a model description, simulation, or reported summary calls for covariance rather than scatter. If a target covariance $C$ is desired and $\nu>2$, the scatter parameter must be $\Sigma=((\nu-2)/\nu)C$. For $\nu\leq2$, no finite target covariance can be matched by this parameterization. The density remains well-defined for every $\nu>0$, so existence of a density should not be confused with existence of moments.

Check Your Understanding

Use the mixture definition and density derivation to answer the following.

For a specified $q(x)$, identify which factors in the density depend on $x$, and which depend only on $\nu$, $p$, and $\Sigma$.
Why must the gamma integral used in the density derivation have a positive rate parameter?
If $\nu=6$, what is the multiplier relating covariance to scatter?
For which values of $\nu$ does the mean exist but the covariance fail to be finite?
If a t model has covariance $C$ and $\nu=5$, what scatter matrix should be used in its density?