Principal Axes and Volume of Confidence Ellipsoids

From a Contrast Confidence Region to Its Geometry

A confidence ellipsoid is more than a formula describing which parameter vectors are plausible. Its shape shows which combinations of contrasts are estimated precisely and which directions have greater uncertainty. The eigenvectors of its covariance shape matrix give the principal directions; the corresponding eigenvalues determine the lengths of the axes.

For context, consider the fixed-design normal linear model \(Y=X\beta+\varepsilon\), where \(X\) has full column rank \(p\), \(\varepsilon\sim N_n(0,\sigma^2 I)\), and \(s^2=\mathrm{SSE}/\nu\) with \(\nu=n-p\). Let \(\theta=C\beta\) be an \(r\)-dimensional contrast, where \(C\) has row rank \(r\), and let \(\hat{\theta}=C\hat{\beta}\). Define \(A=C(X'X)^{-1}C'\). The exact \(1-\alpha\) contrast confidence ellipsoid is

Here \(F_{r,\nu}(1-\alpha)\) is the \(1-\alpha\) quantile of the \(F\) distribution with \(r\) and \(\nu\) degrees of freedom. This is the exact region for the stated normal model. Its center is \(\hat{\theta}\). The matrix \(A\) describes the relative shape, while \(s^2\) and the \(F\) cutoff set the overall scale.

For the contrast region, write \(q=rF_{r,\nu}(1-\alpha)\). Its shape matrix is \(H=q s^2 A\), so the region can equivalently be written as \((\theta-\hat{\theta})'H^{-1}(\theta-\hat{\theta})\le 1\). Because \(C\) has full row rank and \(X\) has full column rank, \(A\) is positive definite: for every nonzero \(v\), \(v'Av=(C'v)'(X'X)^{-1}(C'v)>0\). Thus its eigenvalues are positive and its axes have nonzero lengths.

Principal Directions and Axis Lengths

A real symmetric positive definite matrix has an orthonormal eigenbasis. Let the eigenvalues of \(A\) be \(\lambda_1,\ldots,\lambda_r\), with corresponding unit eigenvectors \(u_1,\ldots,u_r\). Along \(u_i\), the shape matrix \(H=q s^2 A\) has eigenvalue \(q s^2\lambda_i\). Therefore the ellipsoid’s semiaxis length in that direction is \(\sqrt{q s^2\lambda_i}\). The full axis length is twice this value.

To verify the result, collect the unit eigenvectors into an orthogonal matrix \(U\). Then \(U'U=I\) and \(A=U\operatorname{diag}(\lambda_1,\ldots,\lambda_r)U'\). Set \(z=U'(\theta-\hat{\theta})\), which expresses displacement from the center in principal coordinates. Substitution in the ellipsoid inequality gives

This is the equation of an ellipsoid whose \(i\)th semiaxis has length \(\sqrt{q s^2\lambda_i}\). This also shows why the eigenvalues are not the lengths themselves: they must first be multiplied by the region’s scale and then square-rooted.

In two dimensions, the eigenvector for the larger eigenvalue gives the direction of the major axis. For a symmetric matrix \(A=\begin{pmatrix}a&b\\b&d\end{pmatrix}\), the angle \(\phi\) of that direction, measured from the first coordinate axis, can be found from \(\phi=\tfrac12\operatorname{atan2}(2b,a-d)\), modulo \(\pi\). When the eigenvalues are equal, every direction is a principal direction, so there is no unique orientation.

Volume of a Confidence Ellipsoid

The same principal-axis representation gives the volume in any dimension. The unit ball in \(r\) dimensions has volume \(\pi^{r/2}/\Gamma(r/2+1)\), where \(\Gamma\) is the gamma function. Scaling each of its axes by the ellipsoid’s semiaxis length multiplies volume by the product of those lengths.

For a derivation, use the eigenvector coordinates and map the unit ball to the ellipsoid by \(\theta=\hat{\theta}+U\operatorname{diag}(\sqrt{q s^2\lambda_1},\ldots,\sqrt{q s^2\lambda_r})w\), where \(w'w\le1\). The absolute determinant of this linear map is \(\prod_{i=1}^r\sqrt{q s^2\lambda_i}=(q s^2)^{r/2}\sqrt{\prod_i\lambda_i}\). Since \(\prod_i\lambda_i=\det(A)\), the change-of-variables formula multiplies the unit-ball volume by exactly this determinant. This proves the stated formula. \(\square\)

The volume depends on the product of the eigenvalues, not their individual ordering. A larger confidence cutoff expands every semiaxis by the square root of the cutoff and expands volume by its \(r/2\) power. The orientation remains fixed because the cutoff scales all eigenvalues equally.

Common Mistakes and Interpretation

A frequent mistake is to report an eigenvalue as an axis length. An eigenvalue is a squared scale in its direction: for the confidence region it must be multiplied by \(q s^2\), and the semiaxis length is the square root of that product. A second mistake is to infer the orientation directly from the diagonal entries of \(A\). When the off-diagonal entries are nonzero, the principal directions can be rotated relative to the coordinate axes; diagonalization, not inspection of the marginal variances alone, determines the orientation.

The axes describe joint uncertainty in the contrast vector. A long axis indicates a direction in which combinations of contrasts can vary more while remaining inside the confidence region. It does not mean every individual coordinate has that much uncertainty. The ellipsoid’s projection onto a coordinate axis and its principal semiaxis are generally different geometric quantities.