Ellipsoidal Contours and Principal Axes

From Density Levels to Ellipsoids

For a multivariate elliptical law, the scatter matrix describes how the distribution is stretched and rotated away from its center. The geometry becomes explicit when the density depends on a point only through its scatter-adjusted squared distance from that center. A fixed value of this distance describes an ellipsoidal surface; points inside it form an ellipsoidal region.

The distinction between a density contour and a probability region matters. A density level specifies how high the density is at each point on a surface. It does not, by itself, specify how much probability lies inside that surface. That probability also depends on the radial distribution.

Definition: Let \(X\) be a random vector in \(\mathbb{R}^d\) with a full-dimensional density of the form \(f(x)=|S|^{-1/2}g(q(x))\), where \(\mu\in\mathbb{R}^d\), \(S\) is a symmetric positive-definite \(d\times d\) matrix, \(q(x)=(x-\mu)^\mathsf{T}S^{-1}(x-\mu)\), and \(g\) is a nonnegative function on \([0,\infty)\). Here \(|S|\) is the determinant of \(S\). The set of points with density equal to \(k\) is the density level set at height \(k\). For \(c\geq 0\), the set \(q(x)=c\) is a constant-scatter-distance surface; for \(c>0\), the region \(q(x)\leq c\) is its filled ellipsoid.

The factor \(|S|^{-1/2}\) is constant in \(x\), so the density at a point depends on its location relative to the center through \(g(q(x))\). The assumption that \(S\) is positive definite ensures that \(S^{-1}\) exists and that \(q(x)\geq 0\). In this tutorial, “elliptical density” refers to this nonsingular, full-dimensional setting. If the radial profile \(g\) is not monotone, a single density level need not correspond to a single ellipsoidal surface.

Deriving the Principal Axes

The spectral theorem gives an orthonormal basis of eigenvectors for the symmetric positive-definite matrix \(S\). Write its eigendecomposition as \(S=V\Lambda V^\mathsf{T}\), where the columns \(v_1,\ldots,v_d\) of \(V\) are orthonormal eigenvectors and \(\Lambda\) has positive diagonal entries \(\lambda_1,\ldots,\lambda_d\). In coordinates \(z=V^\mathsf{T}(x-\mu)\), the quadratic form separates into one term for each eigenvector direction.

Theorem: With the notation above, \(q(x)=\sum_{j=1}^d z_j^2/\lambda_j\). Consequently, for \(c>0\), \(q(x)=c\) is an ellipsoid centered at \(\mu\), with principal directions \(v_j\) and semi-axis lengths \(\sqrt{c\lambda_j}\). The region \(q(x)\leq c\) is the filled ellipsoid with those same principal directions and semi-axis lengths.

Proof. Since \(S=V\Lambda V^\mathsf{T}\) and \(V^\mathsf{T}V=I\), its inverse is \(S^{-1}=V\Lambda^{-1}V^\mathsf{T}\). Substituting \(z=V^\mathsf{T}(x-\mu)\) gives

q(x)=(x-\mu)^\mathsf{T}V\Lambda^{-1}V^\mathsf{T}(x-\mu) =z^\mathsf{T}\Lambda^{-1}z =\sum_{j=1}^d\frac{z_j^2}{\lambda_j}.

Therefore \(q(x)=c\) is equivalent to \(\sum_{j=1}^d z_j^2/(c\lambda_j)=1\), the equation of an ellipsoid in orthonormal coordinates. Along the \(j\)th eigenvector direction, all other coordinates are zero, so the surface meets that axis when \(z_j^2/\lambda_j=c\), or \(z_j=\pm\sqrt{c\lambda_j}\). This proves the stated axis lengths and directions. Replacing equality by \(\leq\) gives the filled ellipsoid. \(\square\)

An eigenvalue controls the squared scale along its eigenvector direction: for a fixed \(c\), a larger \(\lambda_j\) gives a longer semi-axis. If eigenvalues are repeated, the corresponding eigenspace gives the same scale in every direction within that subspace, so individual principal directions inside it are not uniquely determined.

Worked Example: Finding the Axes of a Rotated Ellipse

Let the center be \(\mu=(1,-2)^\mathsf{T}\), and let the scatter matrix be \(S=\begin{pmatrix}4&3\\3&4\end{pmatrix}\). Find the equation and semi-axis lengths for the surface \(q(x)=4\).

The unit vector \(v_1=(1,1)^\mathsf{T}/\sqrt{2}\) is an eigenvector because \(Sv_1=7v_1\). The unit vector \(v_2=(1,-1)^\mathsf{T}/\sqrt{2}\) is an eigenvector because \(Sv_2=v_2\). Thus the eigenvalues are \(\lambda_1=7\) and \(\lambda_2=1\). In the corresponding coordinates \(z_1=v_1^\mathsf{T}(x-\mu)\) and \(z_2=v_2^\mathsf{T}(x-\mu)\), the quadratic form and surface equation are

q(x)=\frac{z_1^2}{7}+z_2^2, \qquad \frac{z_1^2}{28}+\frac{z_2^2}{4}=1.

The semi-axis lengths are \(\sqrt{4\cdot7}=2\sqrt{7}\) along \(v_1\) and \(\sqrt{4\cdot1}=2\) along \(v_2\). The longer axis is oriented at \(45\) degrees to the coordinate axes, and the ellipse is centered at \((1,-2)^\mathsf{T}\). As a check, \(|S|=4\cdot4-3\cdot3=7\); the area of the filled ellipse is \(\pi(2\sqrt7)(2)=4\pi\sqrt7\), also \(\pi c\sqrt{|S|}=4\pi\sqrt7\).

When Does a Density Level Give One Ellipsoid?

For a specified density height \(k\), a point belongs to its level set precisely when \(g(q(x))=k|S|^{1/2}\). If \(g\) is strictly decreasing, there can be at most one value \(c\geq0\) that solves this equation. Strict decrease alone does not guarantee that such a value exists: the required height might not be in the range of \(g\).

Key condition: If there is a \(c\geq0\) in the attainable range of \(q\) such that \(g(c)=k|S|^{1/2}\), and \(g\) is strictly decreasing on that range, then \(c\) is unique and the density level set is \(q(x)=c\). If no such \(c\) exists, the level set is empty. If \(g\) is not strictly decreasing, the level equation can have multiple solutions in \(q\), producing multiple concentric ellipsoidal surfaces.

Worked Example: A Density Level with No Solution

Suppose \(S=I\), \(d=2\), and the density is \(f(x)=\pi^{-1}e^{-q(x)}\), where \(q(x)=x_1^2+x_2^2\). Consider the proposed density height \(k=2/\pi\). The level equation would require

\frac{1}{\pi}e^{-q(x)}=\frac{2}{\pi}, \qquad\text{so}\qquad e^{-q(x)}=2.

But \(q(x)\geq0\), so \(e^{-q(x)}\leq1\) for every \(x\). The equation has no solution, and this density level set is empty. By contrast, at height \(k=e^{-1}/\pi\), the equation is \(e^{-q}=e^{-1}\), giving \(q=1\), which is a circle in this example. A decreasing profile guarantees uniqueness only when the proposed level is attained.

Worked Example: A Nonmonotone Profile Produces Several Contours

In two dimensions, take \(S=I\), \(\mu=0\), and define \(f(x)=\pi^{-1}(q(x)-1)^2e^{-q(x)}\), where \(q(x)=x_1^2+x_2^2\). This is a density: polar coordinates and the substitution \(q=r^2\) give

\int_{\mathbb{R}^2}f(x)\,dx =\frac{1}{\pi}\,2\pi\int_0^\infty r(r^2-1)^2e^{-r^2}\,dr =\int_0^\infty(q-1)^2e^{-q}\,dq =2-2+1=1.

Consider the level \(k=0.1/\pi\). Its equation is \((q-1)^2e^{-q}=0.1\). Define \(g(q)=(q-1)^2e^{-q}\). Then \(g(0)=1\), \(g(1)=0\), \(g(3)=4e^{-3}\approx0.199>0.1\), and \(g(q)\) tends to zero as \(q\) tends to infinity. Also \(g'(q)=e^{-q}(q-1)(3-q)\), so \(g\) decreases on \((0,1)\), increases on \((1,3)\), and decreases on \((3,\infty)\). Continuity and these strict monotonicity intervals show that the level equation has exactly one solution in each of \((0,1)\), \((1,3)\), and \((3,\infty)\). Thus this single density height gives three circular contours, not one. The monotonicity condition is substantive, not merely technical.

Contour Scale Is Not Probability Content

The parameter \(c\) specifies a geometric region, \(\{x:q(x)\leq c\}\). It does not specify a probability such as 90 percent until the radial distribution has also been taken into account. This distinction can be stated directly using the radial-angular representation of a nonsingular elliptical vector: write \(X=\mu+RAU\), where \(AA^\mathsf{T}=S\), \(U\) is a unit direction, and \(R\geq0\) is the radial variable.

Result: For \(X=\mu+RAU\) with \(AA^\mathsf{T}=S\), \(R\geq0\), and \(U^\mathsf{T}U=1\), the scatter-distance satisfies \(q(X)=R^2\). Therefore, for \(c\geq0\), \(P\{q(X)\leq c\}=P\{R^2\leq c\}\). The probability inside the ellipsoid is determined by the radial distribution, not by the contour equation alone.

Derivation. Since \(S=AA^\mathsf{T}\), \(S^{-1}=A^{-\mathsf{T}}A^{-1}\). Substituting \(X-\mu=RAU\) into the quadratic form yields

q(X) =R^2U^\mathsf{T}A^\mathsf{T}S^{-1}AU =R^2U^\mathsf{T}A^\mathsf{T}A^{-\mathsf{T}}A^{-1}AU =R^2U^\mathsf{T}U =R^2.

Thus the event \(q(X)\leq c\) is exactly the event \(R^2\leq c\), proving the result. The contour parameter \(c\) is a probability-content threshold only if it is selected using the distribution of \(R\). \(\square\)

Worked Example: A Half-Height Contour That Encloses Half the Probability

Let \(X\sim N(0,S)\) in two dimensions, with \(S=\begin{pmatrix}9&0\\0&1\end{pmatrix}\). Its density is \(f(x)=\{2\pi\sqrt{9}\}^{-1}e^{-q(x)/2}\), where \(q(x)=x_1^2/9+x_2^2\). Relative to its peak density, a point at quadratic distance \(c\) has density ratio \(e^{-c/2}\). The contour at half the peak height therefore satisfies \(e^{-c/2}=1/2\), giving \(c=2\log 2\). Its semi-axis lengths are \(3\sqrt{2\log 2}\) and \(\sqrt{2\log 2}\).

To calculate the probability inside it, standardize with \(Z_1=X_1/3\) and \(Z_2=X_2\). These are independent standard normal variables, and \(q(X)=Z_1^2+Z_2^2\). Their joint density is \((2\pi)^{-1}e^{-(z_1^2+z_2^2)/2}\), so polar integration gives, for \(c\geq0\),

P\{q(X)\leq c\} =\int_0^{2\pi}\int_0^{\sqrt c}\frac{1}{2\pi}e^{-r^2/2}r\,dr\,d\theta =1-e^{-c/2}.

At \(c=2\log 2\), this probability is \(1-e^{-\log2}=1/2\). Here the half-height contour happens to enclose half the probability, as the calculation verifies. That correspondence comes from this particular radial distribution; it is not implied by the contour’s height or its ellipsoidal shape.

Common Pitfalls and Geometric Use

Assuming every positive density height occurs. A strictly decreasing profile has at most one solution to the level equation, but the requested height may not be attained. Check existence before naming a contour.
Calling every level set one ellipsoid. If the profile is nonmonotone, a level can occur at several quadratic distances. Solve for all attainable values of \(q\); each positive solution gives a surface.
Reading eigenvalues as axis lengths. The semi-axis length at level \(c\) is \(\sqrt{c\lambda_j}\), not \(\lambda_j\). The eigenvalue is the squared scale in its eigenvector direction before the level factor is applied.
Interpreting a contour as a probability statement. A density height or a value of \(c\) does not automatically mean a particular enclosed percentage. Probability content requires the radial law.

These geometric calculations are useful for visualizing anisotropy, comparing directions of spread, and describing regions of equal density or equal scatter-distance. In applications, the matrix determines orientation and relative axis scales, while a chosen contour level determines how far the surface extends from the center. A probability interpretation requires an additional calculation rather than a geometric inference.

Key takeaway: A quadratic-distance surface \(q(x)=c\) is an ellipsoid whose principal directions are the eigenvectors of \(S\) and whose semi-axis lengths are \(\sqrt{c\lambda_j}\). A density level corresponds to such a surface only at attainable solutions of the radial-profile equation, and its enclosed probability depends on the radial distribution.

Check Your Understanding

Use the quadratic form, its eigendecomposition, and the radial representation to answer these questions.

If an eigenvalue is multiplied by four while the contour value \(c\) stays fixed, how does the corresponding semi-axis length change?
Why does strict decrease of a radial density profile not guarantee that every proposed positive density height defines a contour?
For a nonmonotone profile, what does each positive solution of the density-level equation represent geometrically?
In a radial-angular representation \(X=\mu+RAU\), explain why \(q(X)=R^2\).
What additional distributional information is required to interpret a contour as enclosing a specified probability?