Ellipsoidal Versus Rectangular Confidence Regions

Why the Shape of a Joint Region Matters

A simultaneous confidence procedure for a vector does more than assign a confidence level: it describes which combinations of parameter values remain plausible together. An ellipsoidal region expresses that joint plausibility through a single quadratic constraint. A rectangular region expresses it through separate coordinate bounds. Even when both procedures have the same nominal coverage, their shapes can lead to different conclusions about particular coordinates or linear contrasts.

Write $\theta\in\mathbb{R}^q$ for the parameter vector of interest and $\hat{\theta}$ for its estimator. Let $V$ be a positive-definite covariance-scale matrix used to measure uncertainty, and let $c>0$ be a calibrated critical value. The exact source of the calibration depends on the model and procedure; for example, it may be chosen from an exact finite-sample distribution or a valid simultaneous approximation. The comparison below concerns the geometry and implications of the resulting regions.

Definition: An ellipsoidal confidence region has the form $$ \mathcal{E}=\left\{\theta\in\mathbb{R}^q:(\theta-\hat{\theta})'V^{-1}(\theta-\hat{\theta})\le c\right\}. $$ A rectangular confidence region has the form $$ \mathcal{R}=\left\{\theta\in\mathbb{R}^q:|\theta_j-\hat{\theta}_j|\le d_j,\quad j=1,\ldots,q\right\}, $$ where the half-widths $d_j>0$ are chosen to provide the intended simultaneous coverage. If $V$ is a covariance matrix, a common rectangular form is $d_j=c_R\sqrt{V_{jj}}$ for a simultaneous critical value $c_R$.

The ellipsoid uses the off-diagonal entries of $V$ as well as its diagonal entries. Thus its orientation and elongation reflect dependence among coordinates. A rectangle aligned with the coordinate axes has no tilted sides: its bounds are stated separately for each coordinate, although a critical value for its simultaneous coverage may still depend on the joint distribution.

Coverage and Geometry Are Different Questions

Coverage is a probability statement about repeated samples. If $\mathcal{C}$ is a random confidence region, its coverage at $\theta$ is $P_\theta(\theta\in\mathcal{C})$. A procedure has coverage at least $1-\alpha$ when this probability is at least $1-\alpha$ for every parameter value in its stated parameter space. The shape of one realized region, by contrast, describes the parameter values admitted by the observed data. Equal coverage does not make two regions identical, nested, or equally informative for every question.

For a multivariate normal vector with known covariance $V$, the quadratic form based on the centered vector has a chi-squared distribution with $q$ degrees of freedom. In that setting, using $c=\chi^2_{q,1-\alpha}$ gives an ellipsoid with coverage $1-\alpha$. Rectangular simultaneous intervals can instead be calibrated using the distribution of the largest standardized coordinate error, or with a conservative method such as Bonferroni. When covariance is estimated, the appropriate finite-sample calibration may differ. The critical value must match the model and procedure; geometric comparisons alone do not establish coverage.

Worked Example: A 95% Ellipse and a Bonferroni Rectangle

Suppose the estimation error in two coordinates is standard bivariate normal with identity covariance. The 95% ellipsoid is the disk $$ (\theta_1-\hat{\theta}_1)^2+(\theta_2-\hat{\theta}_2)^2\le 5.991, $$ because the 0.95 quantile of a chi-squared distribution with two degrees of freedom is approximately $5.991$. Its radius is $\sqrt{5.991}\approx2.448$.

A two-sided Bonferroni rectangle allocates total error probability $0.05$ across four coordinate tails. Each tail has probability $0.05/4=0.0125$, so each coordinate half-width is $z_{0.9875}\approx2.241$. By the union bound, the probability that any of the four tails is exceeded is at most $4(0.0125)=0.05$; hence this rectangle has coverage at least $0.95$.

Consider a parameter displacement $(2.3,0)$ from the estimate. It is in the ellipse because $2.3^2=5.29<5.991$, but it is outside the rectangle because $2.3>2.241$. Conversely, displacement $(2,2)$ is in the rectangle because both absolute coordinates are less than $2.241$, but outside the ellipse because $2^2+2^2=8>5.991$. Neither realized region contains the other. Their different shapes matter even though each is designed for at least 95% coverage.

The Bonferroni rectangle in this example is guaranteed to cover at least 95%, not necessarily exactly 95%. More generally, if a rectangle is built from coordinate projections of an ellipsoid, it contains that ellipsoid and therefore has coverage at least as large as the ellipsoid’s coverage. That containment is a particular construction, not a universal relationship between all ellipsoids and all simultaneous rectangles.

Projecting an Ellipsoid Onto a Contrast

A linear contrast is a scalar $a'\theta$, where $a\in\mathbb{R}^q$ is fixed. The projection of a region onto this contrast is the set of scalar values $a'\theta$ obtained as $\theta$ ranges over that region. For an ellipsoid, the projection has a simple exact form.

Theorem: Suppose $V$ is positive definite and $c>0$. For any $a\in\mathbb{R}^q$, the projection of $\mathcal{E}$ onto $a'\theta$ is $$ \left[a'\hat{\theta}-\sqrt{c\,a'Va},\ a'\hat{\theta}+\sqrt{c\,a'Va}\right]. $$ For $a=0$, this reduces to the single value $0$.

Proof. Set $x=\theta-\hat{\theta}$, so the ellipsoid constraint is $x'V^{-1}x\le c$, and the contrast displacement is $a'x$. Write $$ a'x=(V^{1/2}a)'(V^{-1/2}x). $$ The Cauchy–Schwarz inequality gives $$ |a'x|^2\le (a'Va)(x'V^{-1}x)\le c\,a'Va. $$ Therefore every contrast value in the projection lies between the stated endpoints. If $a\ne0$, the upper endpoint is attained at $$ x=\frac{\sqrt{c}\,Va}{\sqrt{a'Va}}, $$ for which $x'V^{-1}x=c$ and $a'x=\sqrt{c\,a'Va}$. Its negative attains the lower endpoint. Adding $a'\hat{\theta}$ gives the claimed interval. $\square$

This result explains why ellipsoids can give especially focused intervals for contrasts. Their projected half-width is $\sqrt{c\,a'Va}$, which uses the full covariance structure in the direction $a$. A coordinate projection is the special case $a=e_j$, where $e_j$ is the $j$th coordinate vector; the half-width is then $\sqrt{cV_{jj}}$.

Worked Example: Projecting an Ellipsoid Onto a Difference

Let $$ V=\begin{pmatrix}1&0.8\\0.8&1\end{pmatrix},\qquad \hat{\theta}=\begin{pmatrix}10\\4\end{pmatrix},\qquad c=5.991. $$ The matrix is positive definite: its eigenvalues are $1.8$ and $0.2$, both positive. Consider the difference $a'\theta=\theta_1-\theta_2$, with $a=(1,-1)'$. The estimated difference is $10-4=6$, and $$ a'Va=(1,-1)\begin{pmatrix}1&0.8\\0.8&1\end{pmatrix}\begin{pmatrix}1\\-1\end{pmatrix} =(1,-1)\begin{pmatrix}0.2\\-0.2\end{pmatrix}=0.4. $$

The ellipsoid’s projected interval is therefore $$ 6\pm\sqrt{5.991(0.4)}=6\pm\sqrt{2.3964}\approx6\pm1.548, $$ or approximately $[4.452,7.548]$. The small projected half-width reflects the positive covariance: when both coordinates move together, their difference changes relatively little.

Projecting a Rectangle

A rectangle answers coordinate questions directly: each coordinate lies in its stated interval. Its projection onto an arbitrary contrast is also easy to derive, but it may be wide because it permits every coordinate to take its most unfavorable value at once.

Result: For the rectangle $\mathcal{R}$ with half-widths $d_1,\ldots,d_q$, the projection onto $a'\theta$ is $$ \left[a'\hat{\theta}-\sum_{j=1}^q|a_j|d_j,\ a'\hat{\theta}+\sum_{j=1}^q|a_j|d_j\right]. $$

Derivation. Write $x_j=\theta_j-\hat{\theta}_j$, so $|x_j|\le d_j$. Then $$ \left|\sum_{j=1}^q a_jx_j\right|\le\sum_{j=1}^q|a_j|\,|x_j|\le\sum_{j=1}^q|a_j|d_j. $$ The upper bound is attained by choosing $x_j=d_j$ when $a_j>0$, and $x_j=-d_j$ when $a_j<0$; coordinates with $a_j=0$ do not affect the contrast. Reversing these choices attains the lower endpoint. Thus the displayed interval is the exact projection. $\square$

Worked Example: Coordinate Bounds Can Loosen a Contrast Interval

Continue with the estimate and covariance matrix from the preceding example. The coordinate projections of the ellipsoid have half-width $\sqrt{5.991}\approx2.448$, since $V_{11}=V_{22}=1$. The smallest axis-aligned rectangle formed from those projections has coordinate intervals $[10-2.448,10+2.448]$ and $[4-2.448,4+2.448]$.

Projecting that rectangle onto the difference uses $a=(1,-1)'$, so the half-width is $$ |1|(2.448)+|-1|(2.448)=4.896. $$ Its difference interval is $[6-4.896,6+4.896]=[1.104,10.896]$, much wider than the ellipsoid’s projected interval $[4.452,7.548]$. The rectangular calculation allows both coordinates to move to opposite extremes, even though those corner combinations need not lie in the ellipsoid.

What Each Region Can Answer

An ellipsoid is useful when the central question is joint: which combinations of the coordinates are compatible with the data under the chosen calibration? Its projections also yield simultaneous bounds for specified linear contrasts. If the parameter vector lies in the ellipsoid, every contrast value lies in the corresponding projection; this follows directly from the definition of projection. In particular, the ellipsoid’s coordinate projections give valid coordinate intervals, although their Cartesian product is generally larger than the ellipsoid.

A rectangle is useful when the scientific report is organized coordinate by coordinate: for example, when each component must have an interpretable simultaneous interval. It communicates all coordinate bounds at once without imposing a quadratic tradeoff among coordinates. But a contrast interval obtained by projecting the rectangle may be conservative or wide, because the rectangle contains corner combinations that a covariance-aware ellipsoid excludes.

Neither shape is automatically superior. A calibrated ellipsoid may be compact in directions where the covariance indicates low uncertainty, while a rectangle makes coordinate limits especially transparent. A direct rectangular procedure calibrated for simultaneous coverage can also differ from the rectangle formed by projecting an ellipsoid. The intended inferential questions and the calibration determine which region is appropriate.

Common Mistake: Treating Coordinate Projections as the Ellipsoid

A frequent error is to calculate the ellipsoid’s interval for each coordinate and then assume that every combination inside those coordinate intervals belongs to the ellipsoid. Those intervals are only the ellipsoid’s one-dimensional projections; combining them creates a larger rectangle. In two dimensions, a point can satisfy both coordinate bounds and still violate the ellipsoid’s quadratic constraint. The correction is to check the quadratic form for joint membership, or to use the actual rectangular region if coordinatewise bounds are the object of interest.

A related mistake is to compare regions by their confidence labels alone. Two procedures both described as 95% may have different coverage behavior (one may be conservative), and their realized sets may admit different parameter values. State the calibration, identify the question being answered, and then interpret the geometry.

Key takeaway: An ellipsoid encodes joint uncertainty through a covariance-weighted quadratic constraint; a rectangle encodes simultaneous coordinate bounds. The ellipsoid projects onto a contrast with half-width $\sqrt{c\,a'Va}$, while the rectangle’s projection has half-width $\sum_j|a_j|d_j$. These are different summaries of uncertainty, suited to different questions.

Check Your Understanding

Use the definitions and projection results to answer the following.

For a two-dimensional ellipsoid, what does its coordinate projection tell you, and why does the product of those intervals generally contain more points?
Suppose a contrast is $a'\theta$. Which entries of $V$ affect its ellipsoidal projected half-width?
For a rectangle with half-widths $d_j$, derive the projected half-width for a contrast with coefficients $a_j$.
Can two regions with the same nominal coverage contain different points? Explain using their geometry.
Why can a rectangular projection onto a difference be wider than the ellipsoid’s projection, especially when the coordinates are positively correlated?