Geometry of Mean Confidence Ellipsoids

From a Confidence Region to Its Geometry

A confidence ellipsoid is more informative than a list of simultaneous intervals: its shape shows which combinations of mean components are estimated more or less precisely. The covariance matrix determines that shape. Its eigenvectors identify the principal directions, and its eigenvalues determine the lengths of the corresponding axes.

Let \(X_1,\ldots,X_n\) be independent observations from a \(p\)-variate normal distribution with mean vector \(\mu\) and known positive-definite covariance matrix \(\Sigma\). Write \(\bar X\) for the sample mean, and let \(c\) be the \(1-\alpha\) quantile of a chi-square distribution with \(p\) degrees of freedom. The known-covariance confidence ellipsoid is

Under the stated model, \(n(\bar X-\mu)^\mathsf T\Sigma^{-1}(\bar X-\mu)\) has a chi-square distribution with \(p\) degrees of freedom. Thus the region has coverage probability \(1-\alpha\). Here the geometry is considered for the observed value of \(\bar X\); the ellipsoid is centered at that value.

Eigenvectors Set the Orientation; Eigenvalues Set the Axes

Because \(\Sigma\) is symmetric and positive definite, it has an orthonormal eigendecomposition

where the columns \(q_1,\ldots,q_p\) of \(Q\) are unit eigenvectors. Consider a displacement \(d=\mu-\bar X\), expressed in this eigenvector basis as \(d=Qa\). Substituting into the ellipsoid inequality gives

This is the equation of an ellipsoid in coordinates \(a_i\). If all coordinates except \(a_i\) are zero, its boundary satisfies \(n a_i^2/\lambda_i=c\). The semi-axis in direction \(q_i\) therefore has length \(\sqrt{c\lambda_i/n}\). A larger eigenvalue means a longer confidence axis in that principal direction. The eigenvalue is the variance of an observation projected onto that unit eigenvector; the factor \(c/n\) converts that variance scale into the confidence-region scale for the mean.

The calculation above proves the result: in eigenvector coordinates, the inequality is \(\sum_i a_i^2/(\lambda_i c/n)\le 1\), which has semi-axis lengths \(\sqrt{c\lambda_i/n}\) along the coordinate axes. Transforming back by \(Q\) maps those coordinate axes to the eigenvector directions \(q_i\). \(\square\)

If an eigenvalue is repeated, there is no unique choice of individual eigenvectors within its eigenspace. The ellipsoid is nevertheless unambiguous: in the corresponding subspace, all principal semi-axes have the same length. When all eigenvalues are equal, the region is a sphere, and its orientation has no geometric effect.

Radial Distance Is Not the Same as Projected Width

In a principal direction, the distance from the center to the boundary is the corresponding semi-axis length. For a direction that is not a principal direction, two different geometric quantities can be useful: the distance to the boundary along a ray, and the maximum projection of the ellipsoid onto that direction. They generally are not equal.

To obtain the radial distance, substitute \(d=tu\) into the defining inequality. Since \(u\) has unit length and \(\Sigma^{-1}\) is positive definite, the boundary occurs at the positive solution of

For the projection, write any displacement in the ellipsoid as \(d=\sqrt{c/n}\,\Sigma^{1/2}z\), where \(\|z\|\le1\). Then

The first inequality is Cauchy–Schwarz. Equality is attained by choosing \(z\) to be a unit vector in the direction of \(\Sigma^{1/2}u\), so the bound is the actual maximum. This proves both formulas. \(\square\)

The projection formula gives the half-width of the ellipsoid in direction \(u\); its full projected width is twice that value. The radial formula instead restricts attention to points lying on the ray in direction \(u\). Only in a principal direction do the two half-lengths necessarily coincide.

Volume and Overall Scale

The volume combines all axis lengths. The unit ball in \(\mathbb R^p\) has volume \(\pi^{p/2}/\Gamma(p/2+1)\), where \(\Gamma\) is the gamma function. The ellipsoid is the image of that ball under the linear map \(z\mapsto\sqrt{c/n}\,\Sigma^{1/2}z\), followed by a translation to \(\bar X\). A linear map scales volume by the absolute value of its determinant. Since \(\Sigma\) is positive definite, \(\det(\Sigma^{1/2})=\sqrt{\det\Sigma}\). Thus

Equivalently, multiplying the semi-axis lengths gives \((c/n)^{p/2}\sqrt{\prod_i\lambda_i}\), and \(\prod_i\lambda_i=\det\Sigma\). This confirms that the determinant summarizes the ellipsoid’s overall volume scale, while the eigenvectors and individual eigenvalues describe its orientation and axis-specific shape.

Common Pitfall and Interpretation

A common mistake is to use \(\sqrt{(c/n)u^\mathsf T\Sigma u}\) as the distance to the boundary along every unit direction \(u\). That expression is the projected half-width, not generally the radial distance. Substitution of \(tu\) into the ellipsoid equation gives the radial distance \(\sqrt{c/(n\,u^\mathsf T\Sigma^{-1}u)}\). In the correlated example, the two quantities in the first coordinate direction were approximately \(1.024\) and \(1.548\), respectively; treating them as interchangeable would misdescribe the ellipse.

For interpretation, begin with the principal axes rather than the diagonal entries of \(\Sigma\). With nonzero correlations, the axes can be rotated relative to the measured variables, as the second example shows. Eigenvalues describe squared spread along those rotated directions, while the ellipsoid’s semi-axis lengths scale as their square roots. Volume provides a single summary of overall size, but by itself it does not reveal orientation or how that size is distributed across directions.