Density of a Nonsingular Gaussian Vector

The Density Formula

For a scalar normal variable, the mean locates the distribution and the variance controls its spread. For a vector, the covariance matrix plays both roles: its off-diagonal entries describe dependence, while its determinant and inverse appear directly in the density. This tutorial focuses on how to state and interpret that density when the covariance is positive definite.

Definition: Let \(\boldsymbol X\) be a \(p\)-dimensional multivariate normal random vector with mean vector \(\boldsymbol\mu\in\mathbb R^p\) and symmetric positive definite covariance matrix \(\Sigma\), written \(\boldsymbol X\sim N_p(\boldsymbol\mu,\Sigma)\). Its density with respect to \(p\)-dimensional Lebesgue measure is

f_{\boldsymbol X}(\boldsymbol x) = \frac{1}{(2\pi)^{p/2}(\det\Sigma)^{1/2}} \exp\!\left\{-\frac12(\boldsymbol x-\boldsymbol\mu)^{\mathsf T} \Sigma^{-1}(\boldsymbol x-\boldsymbol\mu)\right\}, \qquad \boldsymbol x\in\mathbb R^p.

Here \(\Sigma\) is positive definite if \(\boldsymbol v^{\mathsf T}\Sigma\boldsymbol v>0\) for every nonzero \(\boldsymbol v\in\mathbb R^p\). In particular, \(\Sigma^{-1}\) exists and \(\det\Sigma>0\).

The formula has two main parts. The exponent uses the quadratic form \((\boldsymbol x-\boldsymbol\mu)^{\mathsf T}\Sigma^{-1}(\boldsymbol x-\boldsymbol\mu)\), which is the squared Mahalanobis distance from \(\boldsymbol x\) to the mean. It is nonnegative and equals zero only at \(\boldsymbol x=\boldsymbol\mu\). The factor \((\det\Sigma)^{-1/2}\) adjusts the density’s scale: greater covariance volume lowers the density’s overall height. The factor \((2\pi)^{-p/2}\) supplies the corresponding dimension-dependent normalization.

A density is not itself a probability assigned to a point. For a region \(A\subseteq\mathbb R^p\), its probability is the integral of the density over that region. In particular, a vector with this density has probability zero of taking any one specified value, even though the density is strictly positive there.

Reading and Evaluating the Formula

The covariance matrix must be used as a whole. Its determinant is a scalar, whereas its inverse weights differences from the mean in the exponent. Replacing \(\Sigma^{-1}\) with \(\Sigma\), or using \(\det\Sigma\) instead of its square root in the denominator, generally gives a different and incorrect density.

Worked Example: A Diagonal Covariance Matrix

Let \(\boldsymbol X\sim N_2(\boldsymbol\mu,\Sigma)\), where \(\boldsymbol\mu=(1,-1)^{\mathsf T}\) and \(\Sigma=\begin{pmatrix}4&0\\0&1\end{pmatrix}\). Find the density at \(\boldsymbol x=(3,0)^{\mathsf T}\).

The determinant and inverse are \(\det\Sigma=4\) and \(\Sigma^{-1}=\begin{pmatrix}1/4&0\\0&1\end{pmatrix}\). The difference from the mean is \(\boldsymbol x-\boldsymbol\mu=(2,1)^{\mathsf T}\), so the quadratic form is

(\boldsymbol x-\boldsymbol\mu)^{\mathsf T}\Sigma^{-1}(\boldsymbol x-\boldsymbol\mu) = (2,1) \begin{pmatrix}1/4&0\\0&1\end{pmatrix} \begin{pmatrix}2\\1\end{pmatrix} =1+1=2.

Since \(p=2\), the normalizing factor is \(1/(2\pi\sqrt{4})=1/(4\pi)\). Therefore \(f_{\boldsymbol X}(\boldsymbol x)=e^{-1}/(4\pi)\approx 0.02927\). This is a density value, not the probability that \(\boldsymbol X\) equals \((3,0)^{\mathsf T}\).

Worked Example: A Correlated Covariance Matrix

Let \(\boldsymbol X\sim N_2(\boldsymbol 0,\Sigma)\) with \(\Sigma=\begin{pmatrix}1&1/2\\1/2&1\end{pmatrix}\). Evaluate the density at \(\boldsymbol x=(1,1)^{\mathsf T}\).

First, \(\det\Sigma=1-(1/2)^2=3/4\), and direct inversion gives \(\Sigma^{-1}=\frac{4}{3}\begin{pmatrix}1&-1/2\\-1/2&1\end{pmatrix} =\begin{pmatrix}4/3&-2/3\\-2/3&4/3\end{pmatrix}\). Thus

\boldsymbol x^{\mathsf T}\Sigma^{-1}\boldsymbol x = (1,1) \begin{pmatrix}4/3&-2/3\\-2/3&4/3\end{pmatrix} \begin{pmatrix}1\\1\end{pmatrix} =\frac43.

The determinant factor is \((\det\Sigma)^{-1/2}=2/\sqrt3\), so \(f_{\boldsymbol X}(\boldsymbol x)=\frac{1}{2\pi\sqrt{3/4}}e^{-2/3} =\frac{1}{\pi\sqrt3}e^{-2/3}\approx 0.09435\). The off-diagonal entries of the inverse matter in the quadratic form; treating the coordinates as if the covariance were diagonal would not produce this value.

Worked Example: Checking the Three-Dimensional Factors

Suppose \(\boldsymbol X\sim N_3(\boldsymbol 0,\Sigma)\), where \(\Sigma=\operatorname{diag}(1,4,9)\), and evaluate the density at \(\boldsymbol x=(\sqrt3,0,0)^{\mathsf T}\).

The determinant is \(1\cdot4\cdot9=36\), so \((\det\Sigma)^{1/2}=6\). The inverse is \(\operatorname{diag}(1,1/4,1/9)\), and the quadratic form is \(3\). Hence

f_{\boldsymbol X}(\boldsymbol x) = \frac{1}{(2\pi)^{3/2}\,6}\exp(-3/2) \approx \frac{0.22313}{94.50} \approx 0.00236.

This calculation shows why the square root of the determinant belongs in the denominator: the determinant is \(36\), but the factor in the density is \(6\).

Normalization and the Full Support

The formula must integrate to one over \(\mathbb R^p\) to be a probability density. The next result verifies that normalization. It uses the standard multivariate normal density and the change-of-variables formula.

Theorem: If \(\Sigma\) is a symmetric positive definite \(p\times p\) matrix, the displayed Gaussian density integrates to one over \(\mathbb R^p\).

Proof. Positive definiteness gives a symmetric positive definite square root \(\Sigma^{1/2}\), with \(\Sigma^{1/2}\Sigma^{1/2}=\Sigma\). Make the change of variables \(\boldsymbol y=\Sigma^{-1/2}(\boldsymbol x-\boldsymbol\mu)\), or equivalently \(\boldsymbol x=\boldsymbol\mu+\Sigma^{1/2}\boldsymbol y\). The absolute Jacobian determinant for the latter map is \(\det(\Sigma^{1/2})=(\det\Sigma)^{1/2}\). Also, \((\boldsymbol x-\boldsymbol\mu)^{\mathsf T}\Sigma^{-1}(\boldsymbol x-\boldsymbol\mu)=\boldsymbol y^{\mathsf T}\boldsymbol y\). Consequently, the integral of the stated density becomes

\int_{\mathbb R^p} f_{\boldsymbol X}(\boldsymbol x)\,d\boldsymbol x = \int_{\mathbb R^p}(2\pi)^{-p/2} \exp\!\left(-\frac12\boldsymbol y^{\mathsf T}\boldsymbol y\right)\,d\boldsymbol y =1.

The last equality is the normalization of the standard \(p\)-dimensional normal density. The Jacobian cancels the determinant factor in the original formula, which is why the determinant appears with exponent \(-1/2\). \(\square\)

For a density, the support is understood as the topological support of its probability distribution: the set of points for which every open neighborhood has positive probability. Positive definiteness ensures that the displayed density is continuous and strictly positive at every point in \(\mathbb R^p\). This yields the following support statement.

Result: A multivariate normal vector with positive definite covariance has support \(\mathbb R^p\).

Justification. Fix any \(\boldsymbol x_0\in\mathbb R^p\). Its density value is positive. By continuity, there is a sufficiently small open ball centered at \(\boldsymbol x_0\) on which the density is bounded below by, for example, half its value at \(\boldsymbol x_0\). That ball has positive Lebesgue measure, so integrating the density over it gives strictly positive probability. Thus every neighborhood of every \(\boldsymbol x_0\) has positive probability, and the support is all of \(\mathbb R^p\). \(\square\)

What the Formula Tells Us—and a Common Pitfall

The density is largest at the mean. Indeed, positive definiteness of \(\Sigma^{-1}\) implies that the quadratic form in the exponent is nonnegative and equals zero only when \(\boldsymbol x=\boldsymbol\mu\). The exponential is therefore at most one, with equality only at the mean. The maximum density value is \((2\pi)^{-p/2}(\det\Sigma)^{-1/2}\).

The determinant factor also depends on measurement scale. If coordinates are expressed in different units, the numerical entries of \(\Sigma\), its determinant, and the density values change accordingly. Probabilities of corresponding regions remain probabilities computed by integrating the density with respect to the matching volume element. A density value should therefore be interpreted relative to the coordinate units, not as a unit-free probability.

Wrong reasoning: “The density at \(\boldsymbol x\) is \(0.03\), so the probability of exactly observing \(\boldsymbol x\) is about three percent.”
Correction: For a continuous vector, \(P(\boldsymbol X=\boldsymbol x)=0\). A probability is obtained by integrating the density over a region; for a small region, the density near a point can help approximate that region’s probability only when combined with its volume.
Another common error: Using \(\det\Sigma\) rather than \((\det\Sigma)^{1/2}\) in the normalizing factor. For example, in the three-dimensional example above, the determinant is \(36\), but its square root is \(6\), which is the factor required in the density.

Key takeaway: For a positive definite covariance matrix, the Gaussian density is defined throughout \(\mathbb R^p\), has a quadratic exponent weighted by \(\Sigma^{-1}\), and is normalized by \((2\pi)^{p/2}(\det\Sigma)^{1/2}\). Its support is the entire vector space.

Check Your Understanding

Use the density formula and its interpretation to answer these questions.

For a \(p\)-dimensional Gaussian vector, which covariance-matrix quantities appear in the exponent, and which appear in the normalizing factor?
If \(\det\Sigma=25\), what determinant factor appears in the denominator of the density?
Why does positive definiteness ensure that the quadratic form in the exponent is zero only at \(\boldsymbol x=\boldsymbol\mu\)?
What does it mean for the support to be all of \(\mathbb R^p\), and why does strict positivity of the density establish it?
Why is a density value not the probability of observing one exact vector value?

Density of a Nonsingular Gaussian Vector

What You'll Learn