Elliptical Contours of Gaussian Densities

From Gaussian Density to Contours

A multivariate Gaussian density is largest at its mean and decreases according to a quadratic measure of distance from that point. Sets of points with the same density therefore have a simple geometric description: they are ellipses in two dimensions and ellipsoids in higher dimensions. Their center and size follow directly from the density formula, while their shape is governed by the covariance matrix.

Let \(X\) be a \(p\)-dimensional Gaussian vector with mean \(\mu\) and nonsingular covariance matrix \(\Sigma\). Nonsingularity means that \(\Sigma\) is positive definite, so its inverse exists and the density is defined throughout \(\mathbb{R}^p\).

Definition: For a point \(x\in\mathbb{R}^p\), define its squared Mahalanobis distance from \(\mu\), relative to \(\Sigma\), by \(d^2(x,\mu)=(x-\mu)^{\mathsf T}\Sigma^{-1}(x-\mu)\). A constant-density contour is a set of points \(x\) at which the Gaussian density has a fixed value.

The nonsingular Gaussian density is

f(x)=\frac{1}{(2\pi)^{p/2}|\Sigma|^{1/2}} \exp\left\{-\frac{1}{2}(x-\mu)^{\mathsf T}\Sigma^{-1}(x-\mu)\right\}.

Write \(f(\mu)\) for the maximum density, attained at \(x=\mu\). The density at any other point is its maximum multiplied by \(\exp\{-d^2(x,\mu)/2\}\). Thus the contour at a fraction \(\alpha\) of the maximum density, where \(0<\alpha\leq 1\), satisfies

f(x)=\alpha f(\mu) \quad\Longleftrightarrow\quad d^2(x,\mu)=-2\log(\alpha).

This equation is the central description of Gaussian contours. The value on the right is a squared distance in covariance-scaled coordinates, not an ordinary Euclidean distance.

The Ellipsoid Equation and Its Geometry

Theorem: If \(X\sim N_p(\mu,\Sigma)\) with \(\Sigma\) positive definite, then for each \(c>0\), the density contour at relative height \(\exp(-c/2)\) is \( (x-\mu)^{\mathsf T}\Sigma^{-1}(x-\mu)=c\). The region at least that high is \( (x-\mu)^{\mathsf T}\Sigma^{-1}(x-\mu)\leq c\). The contour is an ellipsoid centered at \(\mu\); for \(p=2\) it is an ellipse.

Proof. Taking the ratio of the density at \(x\) to the density at \(\mu\) cancels the normalizing constant:

\frac{f(x)}{f(\mu)} = \exp\left\{-\frac{1}{2}(x-\mu)^{\mathsf T}\Sigma^{-1}(x-\mu)\right\}.

Setting this ratio equal to \(\exp(-c/2)\) and taking logarithms gives the stated contour equation. Since the exponential is decreasing, the inequality \(f(x)\geq \exp(-c/2)f(\mu)\) is equivalent to the quadratic-form inequality with \(\leq c\). Positive definiteness makes this quadratic form strictly positive for every nonzero displacement \(x-\mu\). Its level sets are ellipsoids, the standard quadratic-form description of ellipses and ellipsoids. The center is \(\mu\), where the quadratic form is zero. \(\square\)

The region in the theorem is a density superlevel set, not automatically a region containing a particular probability. The parameter \(c\) controls the contour's size: increasing \(c\) lowers the density threshold and expands the enclosed region.

Result: If \(c_1,c_2>0\), the contour for \(c_2\) is obtained from the contour for \(c_1\) by scaling every displacement from \(\mu\) by \(\sqrt{c_2/c_1}\). Indeed, the quadratic form is homogeneous of degree two in that displacement.

To see the scaling directly, take any point \(x\) on the \(c_1\) contour and let \(y=\mu+a(x-\mu)\). Then \(d^2(y,\mu)=a^2c_1\). Choosing \(a=\sqrt{c_2/c_1}\) places \(y\) on the \(c_2\) contour. The center stays fixed while every radial distance from it scales by the same factor.

Reading the Covariance from a Contour

The inverse covariance matrix appears in the contour equation because displacement is measured in covariance-scaled units. For a unit direction \(u\), points on the ray from the mean have the form \(x=\mu+ru\). Substitution gives

d^2(\mu+ru,\mu)=r^2u^{\mathsf T}\Sigma^{-1}u, \qquad r_c(u)=\sqrt{\frac{c}{u^{\mathsf T}\Sigma^{-1}u}}.

Here \(r_c(u)\) is the distance from the center to the contour along direction \(u\). The covariance also determines the variance of the scalar projection in that direction: \(\operatorname{Var}(u^{\mathsf T}X)=u^{\mathsf T}\Sigma u\). Together, these facts express the covariance-contour relationship: the covariance describes variability in linear projections, and its inverse determines the quadratic distance used to form the contour.

When \(\Sigma=s^2I\), the equation reduces to \(\|x-\mu\|^2=s^2c\), a sphere of radius \(s\sqrt{c}\) (a circle when \(p=2\)). For a diagonal covariance matrix, the contour equation separates by coordinate, making the different coordinate scales visible. With nonzero off-diagonal covariances, cross-products appear in the equation, so the contour need not be aligned with the coordinate axes.

Worked Example: A Correlated Bivariate Gaussian

Let \(X\sim N_2(\mu,\Sigma)\), with \(\mu=(2,-1)^{\mathsf T}\) and

\Sigma= \begin{pmatrix} 4&3\\ 3&9 \end{pmatrix}.

The determinant is \(4(9)-3(3)=27>0\), and the diagonal entries are positive, so this symmetric matrix is positive definite. Its inverse is

\Sigma^{-1} =\frac{1}{27} \begin{pmatrix} 9&-3\\ -3&4 \end{pmatrix}.

Write \(u=x_1-2\) and \(v=x_2+1\) for the coordinates relative to the mean. Then

d^2(x,\mu)=\frac{9u^2-6uv+4v^2}{27}.

The contour at one fifth of the maximum density has \(\alpha=1/5\), so \(c=-2\log(1/5)=2\log 5\). Its equation is \(9u^2-6uv+4v^2=54\log 5\). At the center \((2,-1)\), the density is \(f(\mu)=1/(2\pi\sqrt{27})\); on this contour it is \(f(\mu)/5\). Along the first coordinate direction, \(v=0\), so \(9u^2=54\log 5\) and \(|u|=\sqrt{6\log 5}\). Along the second coordinate direction, \(u=0\), so \(4v^2=54\log 5\) and \(|v|=\sqrt{(27/2)\log 5}\). These are radial intersections with the coordinate directions; the cross-product term shows that the full contour is not axis-aligned.

Worked Example: A Three-Dimensional Ellipsoid

Suppose \(X\sim N_3(\mu,\Sigma)\), where \(\mu=(1,0,-2)^{\mathsf T}\) and \(\Sigma=\operatorname{diag}(1,4,9)\). With \(u=x_1-1\), \(v=x_2\), and \(w=x_3+2\), the squared distance is

d^2(x,\mu)=u^2+\frac{v^2}{4}+\frac{w^2}{9}.

At \(c=9\), the contour and its enclosed density superlevel region satisfy

u^2+\frac{v^2}{4}+\frac{w^2}{9}=9, \qquad u^2+\frac{v^2}{4}+\frac{w^2}{9}\leq 9.

Setting two coordinates at a time to zero gives the three coordinate-direction radii: \(|u|=3\), \(|v|=6\), and \(|w|=9\). These follow by solving \(u^2=9\), \(v^2/4=9\), and \(w^2/9=9\), respectively. The contour's density is \(\exp(-9/2)\) times the maximum. This example shows how the diagonal variances set different coordinate scales: the coordinate standard deviations are \(1,2,3\), and at this contour the corresponding radii are \(3,6,9\).

Worked Example: Scaling the Covariance

Compare two centered bivariate Gaussian laws with covariance matrices \(\Sigma_1=I\) and \(\Sigma_2=4I\). For a relative density height of one half, \(c=-2\log(1/2)=2\log 2\). Under \(\Sigma_1\), the contour equation is

x_1^2+x_2^2=2\log 2,

so its radius is \(\sqrt{2\log 2}\). Under \(\Sigma_2\), the inverse covariance is \((1/4)I\), and the equation becomes

\frac{x_1^2+x_2^2}{4}=2\log 2,

so its radius is \(2\sqrt{2\log 2}\), twice as large. The maximum densities are not equal: they are \(1/(2\pi)\) and \(1/(8\pi)\), respectively, because the covariance determinants are \(1\) and \(16\). Thus a fixed fraction of each law's maximum gives contours at the same relative height, even though the absolute density values differ.

Density Contours Are Not Probability Contours

A common mistake is to say that the contour at a specified density fraction encloses that same fraction of the probability. For example, a contour at half the maximum density does not generally enclose half the probability. Density is probability per unit volume, while probability is an integral of density over a region. The contour equation alone specifies the height of the density at its boundary; probability content requires integrating over the enclosed region or using an appropriate distributional result.

Another mistake is to read the covariance matrix itself as the matrix in the contour equation. The density uses \(\Sigma^{-1}\), not \(\Sigma\). In one dimension this distinction is easy to check: if \(X\sim N(\mu,\sigma^2)\), then the relative density is \(\exp\{-(x-\mu)^2/(2\sigma^2)\}\), and a fixed-density contour lies farther from \(\mu\) when \(\sigma\) is larger. In multiple dimensions, the same principle holds through the quadratic form and directional radii.

Key takeaway: A nonsingular Gaussian's relative-density contour at height \(\exp(-c/2)\) is the level set of squared Mahalanobis distance \(c\). It is centered at the mean, expands by the factor \(\sqrt{c}\), and has its shape determined by the covariance matrix through its inverse in the contour equation.

Check Your Understanding

Use the density equation and covariance-scaled distance to answer these questions.

For a nonsingular Gaussian, what equation describes the contour at a fraction \(\alpha\) of the maximum density?
Explain why increasing \(c\) expands the ellipsoid, and state the scale factor between contours at \(c_1\) and \(c_2\).
For \(X\sim N_2(0,4I)\), find the radius of the contour at relative density \(\exp(-2)\).
Why does a contour at half the maximum density not, in general, enclose half the probability?
In the contour equation, why does the inverse covariance appear rather than the covariance itself?

Elliptical Contours of Gaussian Densities

What You'll Learn