From Gaussian Parameters to a Density and Support
A Gaussian model is specified by a mean vector and a covariance matrix, but those parameters do more than locate and spread the distribution. The covariance matrix determines whether the law has a density on the entire ambient space or is concentrated on a lower-dimensional affine set. For a nonsingular law, its determinant and inverse enter the density formula. For a singular law, the support must be identified before any density statement is made.
The calculations below use \(N_d(\mu,\Sigma)\) for a \(d\)-dimensional Gaussian vector with mean \(\mu\) and covariance matrix \(\Sigma\). The matrix \(\Sigma\) must be symmetric and positive semidefinite. It is nonsingular precisely when it is positive definite. The support of a distribution means its topological support: the smallest closed set that has probability one.
The Two Density Formulas to Keep Distinct
When \(\Sigma\) is positive definite, \(X\) has a density with respect to \(d\)-dimensional Lebesgue measure. The expression involves the determinant, which measures the volume scaling of the covariance transformation, and the inverse covariance, which determines the quadratic distance from the mean.
To derive the formula, choose an invertible matrix \(C\) satisfying \(CC^{\mathsf T}=\Sigma\), such as a Cholesky factor, and write \(X=\mu+CZ\), where \(Z\sim N_d(0,I_d)\). The standard normal density at \(z\) is \((2\pi)^{-d/2}\exp(-z^{\mathsf T}z/2)\). Under the transformation \(x=\mu+Cz\), the absolute Jacobian determinant is \(|C|\), so the transformed density is the standard normal density at \(C^{-1}(x-\mu)\), divided by \(|C|\). Since \(C^{-\mathsf T}C^{-1}=\Sigma^{-1}\) and \(|C|=|\Sigma|^{1/2}\), this is the displayed formula. It is positive at every \(x\), so every open ball has positive probability; the support is \(\mathbb R^d\). \(\square\)
For a singular law, that ambient-space formula cannot be used: \(\Sigma^{-1}\) does not exist, and the distribution assigns probability one to a proper affine subspace. It is possible to describe a density on that subspace, using its intrinsic \(r\)-dimensional volume, where \(r=\operatorname{rank}(\Sigma)\).
For a proof, diagonalize \(\Sigma\) in an orthonormal eigenbasis. In the positive-eigenvalue directions, the random vector has coordinates \(\sqrt{\lambda_i}Z_i\), with independent standard normal \(Z_i\); in the zero-eigenvalue directions it has coordinate zero with probability one. Thus it lies in \(\mu+\operatorname{Range}(\Sigma)\). The vector \((Z_1,\ldots,Z_r)\) has support all of \(\mathbb R^r\), and scaling each coordinate by the positive number \(\sqrt{\lambda_i}\) maps that support onto the stated affine subspace. The change-of-variables Jacobian within that subspace is \(\prod_i\sqrt{\lambda_i}\), giving the displayed density. When \(r<d\), the affine subspace has \(d\)-dimensional Lebesgue measure zero but probability one. If an ambient density existed, its integral over that measure-zero set would be zero, contradicting probability one. \(\square\)
Worked Problems
In each problem, first establish the rank of the covariance matrix. Then choose the appropriate density interpretation and determine the support. This order prevents a common error: trying to invert a singular covariance matrix simply because a familiar Gaussian density formula is available.
Worked Example: A Nonsingular Bivariate Density
Let \(X\sim N_2(\mu,\Sigma)\), where \(\mu=(1,-1)^{\mathsf T}\) and \(\Sigma=\begin{pmatrix}4&1\\1&1\end{pmatrix}\). Find its support and evaluate its density at \(x=(3,0)^{\mathsf T}\).
First check nonsingularity: \[ |\Sigma|=4(1)-1(1)=3>0. \] The upper-left entry is positive and the determinant is positive, so this symmetric matrix is positive definite. Its inverse is \[ \Sigma^{-1}=\frac13 \begin{pmatrix}1&-1\\-1&4\end{pmatrix}. \] The displacement from the mean is \(x-\mu=(2,1)^{\mathsf T}\). The quadratic form is \[ (x-\mu)^{\mathsf T}\Sigma^{-1}(x-\mu) =\frac13(2,1) \begin{pmatrix}1&-1\\-1&4\end{pmatrix} \begin{pmatrix}2\\1\end{pmatrix} =\frac13(2,1)\begin{pmatrix}1\\2\end{pmatrix} =\frac43. \] Therefore \[ f(3,0)=\frac{1}{2\pi\sqrt{3}}\exp\!\left(-\frac23\right). \] Because \(\Sigma\) is positive definite, the support is the whole plane \(\mathbb R^2\).
Worked Example: A Second Bivariate Density Calculation
Suppose \(Y\sim N_2(\nu,\Omega)\), where \(\nu=(0,2)^{\mathsf T}\) and \(\Omega=\begin{pmatrix}4&3\\3&9\end{pmatrix}\). Find the density at \(y=(2,5)^{\mathsf T}\) and state the support.
The determinant is \(4(9)-3^2=27>0\). Since the first diagonal entry is positive, \(\Omega\) is positive definite. Its inverse is \[ \Omega^{-1}=\frac1{27} \begin{pmatrix}9&-3\\-3&4\end{pmatrix}. \] Here \(y-\nu=(2,3)^{\mathsf T}\), so \[ (y-\nu)^{\mathsf T}\Omega^{-1}(y-\nu) =\frac1{27}(2,3) \begin{pmatrix}9&-3\\-3&4\end{pmatrix} \begin{pmatrix}2\\3\end{pmatrix} =\frac1{27}(2,3)\begin{pmatrix}9\\6\end{pmatrix} =\frac{36}{27}=\frac43. \] Thus \[ f(2,5)=\frac{1}{2\pi\sqrt{27}}\exp\!\left(-\frac23\right), \] and the support is \(\mathbb R^2\). As a check on the calculation, the quadratic form is nonnegative, as it must be for a positive-definite inverse covariance matrix.
Worked Example: A Singular Bivariate Law Supported on a Line
Let \(Z\sim N(0,1)\) and define \[ X=\begin{pmatrix}2\\-1\end{pmatrix} +Z\begin{pmatrix}2\\-1\end{pmatrix}. \] Find the covariance rank and support. Then give the density at \(x=(4,-2)^{\mathsf T}\) with respect to length along the support.
The mean is \(\mu=(2,-1)^{\mathsf T}\), and the covariance matrix is \[ \Sigma=\begin{pmatrix}2\\-1\end{pmatrix} \begin{pmatrix}2&-1\end{pmatrix} =\begin{pmatrix}4&-2\\-2&1\end{pmatrix}. \] This matrix has rank one: it is a nonzero outer product. The random vector lies on the line \[ \left\{(x_1,x_2)^{\mathsf T}:x_1+2x_2=0\right\}, \] because \(2+2Z+2(-1-Z)=0\). Its direction vector \(u=(2,-1)^{\mathsf T}/\sqrt5\) is a unit eigenvector with eigenvalue \(5\); indeed, \(\Sigma u=5u\). The support is this entire line, not the whole plane.
The point \((4,-2)^{\mathsf T}\) equals \(\mu+(2,-1)^{\mathsf T}\). Its coordinate along the unit direction, relative to \(\mu\), is \[ u^{\mathsf T}(x-\mu) =\frac{(2,-1)(2,-1)^{\mathsf T}}{\sqrt5} =\sqrt5. \] Using the one-dimensional support density with eigenvalue \(5\), \[ f_{\mathrm{supp}}(4,-2) =\frac{1}{\sqrt{2\pi\cdot5}} \exp\!\left(-\frac{(\sqrt5)^2}{2\cdot5}\right) =\frac{1}{\sqrt{10\pi}}e^{-1/2}. \] This is a density per unit length on the line. There is no density with respect to area in the plane.
Worked Example: A Singular Three-Dimensional Law Supported on a Plane
Let \(Z_1,Z_2\) be independent \(N(0,1)\) variables and set \[ X=\begin{pmatrix}1\\0\\1\end{pmatrix} +Z_1\begin{pmatrix}1\\1\\0\end{pmatrix} +Z_2\begin{pmatrix}1\\-1\\2\end{pmatrix}. \] Find its support and its support density at \(x=(3,0,3)^{\mathsf T}\).
The two direction vectors are nonzero and orthogonal, since \(1-1+0=0\), so the covariance has rank two. A vector perpendicular to both is \(n=(1,-1,-1)^{\mathsf T}\). Since \(n^{\mathsf T}\mu=1-0-1=0\), the support plane is \[ x_1-x_2-x_3=0. \] The unit direction vectors are \(u_1=(1,1,0)^{\mathsf T}/\sqrt2\) and \(u_2=(1,-1,2)^{\mathsf T}/\sqrt6\). The covariance eigenvalues in these directions are \(2\) and \(6\), respectively, because each original direction vector has squared length \(2\) or \(6\).
The displacement from the mean at the specified point is \((2,0,2)^{\mathsf T}\). Its coordinates along \(u_1\) and \(u_2\) are \[ u_1^{\mathsf T}(x-\mu)=\frac{2}{\sqrt2}=\sqrt2, \qquad u_2^{\mathsf T}(x-\mu)=\frac{2+4}{\sqrt6}=\sqrt6. \] The quadratic term is therefore \((\sqrt2)^2/2+(\sqrt6)^2/6=1+1=2\). Since the product of the positive eigenvalues is \(12\), the density with respect to area on the plane is \[ f_{\mathrm{supp}}(3,0,3)=\frac{1}{2\pi\sqrt{12}}e^{-1}. \] It is not a three-dimensional density.
Common Mistakes and a Reliable Check
The most consequential mistake is to apply the nonsingular formula before checking the covariance rank. A zero determinant means that the inverse in that formula does not exist; replacing it by an ordinary inverse calculation or treating the result as a density on all of \(\mathbb R^d\) is invalid. For a singular law, identify the affine support and, if needed, use the density defined relative to volume on that support.
A second frequent error is to confuse a support density with an ambient density. In the line example, the density value is measured per unit length; in the plane example, it is measured per unit area. Neither is a density per unit area in the first case or per unit volume in the second. The dimension of the reference measure is part of what a density means.
Verify symmetry and positive semidefiniteness; determine whether the determinant is positive or the rank is deficient.
For positive definite covariance, use the ambient \(d\)-dimensional Gaussian density. For singular covariance, work on the affine support.
Check the support equation, calculate the displacement coordinates, and ensure the quadratic term is nonnegative.
Check Your Understanding
For each problem, identify which density interpretation applies before calculating.
- For \(\Sigma=\begin{pmatrix}2&0\\0&5\end{pmatrix}\), determine whether a Gaussian law with this covariance is singular and state its support.
- A covariance matrix has eigenvalues \(4,1,0\) in three dimensions. What is the dimension of the support, and can the law have a three-dimensional Lebesgue density?
- For the bivariate law in the first worked example, what determinant and inverse-covariance quadratic form determine the density at \((3,0)^{\mathsf T}\)?
- In the line-supported example, why does a density with respect to length not constitute a density with respect to area in \(\mathbb R^2\)?
- For the three-dimensional example, verify that \((3,0,3)^{\mathsf T}\) lies on the stated support plane.