Joint Ellipsoids for Vector-Valued Contrasts

Why a Vector of Contrasts Needs a Joint Region

A scalar confidence interval describes uncertainty in one linear contrast of regression coefficients. When the scientific question involves several contrasts at once, separate intervals do not, by themselves, provide a confidence statement about the vector as a whole. Their errors can occur together, and the estimated contrasts can be correlated.

The joint confidence region accounts for both features. Its shape is determined by the covariance matrix of the contrast estimates, while its size is calibrated using an $F$ quantile and the residual degrees of freedom. Under the normal linear model, this construction has exact finite-sample coverage.

Model, Contrasts, and the Estimated Covariance

Assume the fixed-design normal linear model $Y=X\beta+\varepsilon$, where $Y$ is an $n$-vector, $X$ is a known $n\times p$ matrix of full column rank, and $\varepsilon\sim N(0,\sigma^2 I_n)$ with $\sigma^2>0$. Let $\nu=n-p>0$. Write $\hat\beta=(X^\mathsf{T}X)^{-1}X^\mathsf{T}Y$ and $s^2=\mathrm{SSE}/\nu$, where $\mathrm{SSE}=(Y-X\hat\beta)^\mathsf{T}(Y-X\hat\beta)$.

Let $C$ be a known $q\times p$ matrix with full row rank $q$, where $1\leq q\leq p$. The target is the vector of estimable contrasts $\theta=C\beta$, estimated by $\hat\theta=C\hat\beta$. Define

V=C(X^\mathsf{T}X)^{-1}C^\mathsf{T}.

Then $\operatorname{Cov}(\hat\theta)=\sigma^2V$. The matrix $V$ is positive definite: for every nonzero $a\in\mathbb{R}^q$, full row rank of $C$ gives $C^\mathsf{T}a\ne0$, so $a^\mathsf{T}Va=(C^\mathsf{T}a)^\mathsf{T}(X^\mathsf{T}X)^{-1}(C^\mathsf{T}a)>0$. Thus $V^{-1}$ exists and measures squared distance in the covariance-scaled geometry of the contrasts.

Definition: For confidence level $1-\alpha$, the exact joint confidence ellipsoid for $\theta=C\beta$ is the set of all vectors $\theta_0\in\mathbb{R}^q$ satisfying $$ (\hat\theta-\theta_0)^\mathsf{T}V^{-1}(\hat\theta-\theta_0) \leq q s^2 F_{q,\nu;1-\alpha}. $$ Here $F_{q,\nu;1-\alpha}$ is the $1-\alpha$ quantile of the central $F$ distribution with numerator degrees of freedom $q$ and denominator degrees of freedom $\nu$. The unknown parameter vector $\theta$ is covered when it belongs to this random set.

The factor $q$ is part of the cutoff, as is the estimate $s^2$ of the error variance. The numerator degrees of freedom equal the dimension of the contrast vector, not the number of coefficients in the full model. The matrix $V$ is the covariance factor without $\sigma^2$; including $\sigma^2$ in $V$ and also using $s^2$ would count the variance twice.

Exact Coverage

The construction is calibrated by the same quadratic-form $F$ statistic used for a joint test of $q$ linear restrictions. The following argument verifies its coverage directly under the model assumptions.

Theorem: Under the stated normal linear model assumptions, the ellipsoid has exact coverage: $$ P\!\left((\hat\theta-\theta)^\mathsf{T}V^{-1}(\hat\theta-\theta) \leq q s^2 F_{q,\nu;1-\alpha}\right)=1-\alpha. $$

Proof. The least-squares estimator is normal with mean $\beta$ and covariance $\sigma^2(X^\mathsf{T}X)^{-1}$. Therefore $\hat\theta-\theta$ is normal with mean zero and covariance $\sigma^2V$. Since $V$ is positive definite, whitening gives $V^{-1/2}(\hat\theta-\theta)/\sigma\sim N(0,I_q)$. The sum of squares of its $q$ independent standard normal coordinates has a chi-square distribution with $q$ degrees of freedom. Hence

\frac{(\hat\theta-\theta)^\mathsf{T}V^{-1}(\hat\theta-\theta)}{\sigma^2} \sim \chi^2_q.

The residual sum of squares is the squared length of the projection of $Y$ onto the orthogonal complement of the column space of $X$, which has dimension $\nu$. Under the model, $\mathrm{SSE}/\sigma^2\sim\chi^2_\nu$. The fitted and residual projections are onto orthogonal subspaces. For a multivariate normal vector, orthogonal projections are independent; consequently this residual quantity is independent of $\hat\theta$, and therefore of the contrast quadratic form above. Since $s^2=\mathrm{SSE}/\nu$, it follows that

\frac{(\hat\theta-\theta)^\mathsf{T}V^{-1}(\hat\theta-\theta)} {q s^2} = \frac{\chi^2_q/q}{\chi^2_\nu/\nu} \sim F_{q,\nu}.

By the definition of the $1-\alpha$ quantile, the probability that this ratio is at most $F_{q,\nu;1-\alpha}$ is $1-\alpha$. Rearranging that event gives exactly the stated ellipsoid coverage. $\square$

Worked Examples

Worked Example: Checking a Candidate Vector in Two Dimensions

Suppose $q=2$, $\nu=10$, and the estimated contrast covariance factor and residual variance are $V=\frac18\begin{pmatrix}3&-1\\-1&3\end{pmatrix}$ and $s^2=1$. For a 95% region, $F_{2,10;0.95}\approx4.1028$, so the cutoff is $2(1)(4.1028)=8.2056$. Consider the candidate $\theta_0=\hat\theta-(1.2,0.8)^\mathsf{T}$.

The inverse covariance factor is $V^{-1}=\begin{pmatrix}3&1\\1&3\end{pmatrix}$. With $d=\hat\theta-\theta_0=(1.2,0.8)^\mathsf{T}$, the quadratic form is

d^\mathsf{T}V^{-1}d =3(1.2)^2+2(1.2)(0.8)+3(0.8)^2 =4.32+1.92+1.92 =8.16.

Because $8.16\leq8.2056$, this candidate belongs to the ellipsoid. The comparison uses the full covariance geometry: the cross term $2(1.2)(0.8)$ matters, so checking the two coordinates separately would not establish joint membership.

Worked Example: Constructing the Covariance Factor From Contrasts

Suppose $X^\mathsf{T}X=\operatorname{diag}(4,9,16)$, and the two contrasts are $\theta_1=\beta_1+\beta_2$ and $\theta_2=\beta_2-\beta_3$. Thus $C=\begin{pmatrix}1&1&0\\0&1&-1\end{pmatrix}$. Compute the entries of $V=C(X^\mathsf{T}X)^{-1}C^\mathsf{T}$.

First, $(X^\mathsf{T}X)^{-1}=\operatorname{diag}(1/4,1/9,1/16)$. The variance factor for the first contrast is $1/4+1/9=13/36$; for the second it is $1/9+1/16=25/144$. Their covariance factor is $1/9$, since the contrasts share the $\beta_2$ term with coefficient $1$ in each, and the other coefficients do not overlap. Therefore

V= \begin{pmatrix} 13/36&1/9\\ 1/9&25/144 \end{pmatrix}.

If $n-p=10$, the 95% joint region is centered at $\hat\theta=(\hat\beta_1+\hat\beta_2,\hat\beta_2-\hat\beta_3)^\mathsf{T}$ and consists of all $\theta_0$ satisfying $(\hat\theta-\theta_0)^\mathsf{T}V^{-1}(\hat\theta-\theta_0)\leq 2s^2(4.1028)$. The off-diagonal entries are not optional: omitting them would change the covariance geometry and hence the region.

Worked Example: Inside Both Coordinate Projections but Outside the Ellipsoid

Use the covariance factor and 95% cutoff from the first example, so $V^{-1}=\begin{pmatrix}3&1\\1&3\end{pmatrix}$, $s^2=1$, and the cutoff is $8.2056$. The coordinate projections of this ellipsoid have half-width $\sqrt{8.2056(3/8)}\approx1.7542$, as shown below. Take $d=(1.7,1.7)^\mathsf{T}$. Each coordinate has absolute value $1.7<1.7542$, so each lies within its coordinate projection.

But the joint quadratic form is

d^\mathsf{T}V^{-1}d =3(1.7)^2+2(1.7)(1.7)+3(1.7)^2 =8(2.89) =23.12.

Since $23.12>8.2056$, the vector is outside the ellipsoid. Membership in each coordinate projection is not sufficient for joint membership.

Coordinate Projections and Interpretation

A projection answers a different question from the ellipsoid itself: it gives the range of possible values for one coordinate among all points in the joint region. The following calculation gives the projection onto coordinate $j$. Put $K=q s^2F_{q,\nu;1-\alpha}$ and $d=\hat\theta-\theta_0$. For any $d$ satisfying $d^\mathsf{T}V^{-1}d\leq K$, write $d_j=e_j^\mathsf{T}d$, where $e_j$ is the $j$th coordinate vector. By Cauchy–Schwarz,

|d_j| =\left|(V^{1/2}e_j)^\mathsf{T}(V^{-1/2}d)\right| \leq \sqrt{e_j^\mathsf{T}Ve_j}\sqrt{d^\mathsf{T}V^{-1}d} \leq \sqrt{K V_{jj}}.

This bound is attained by taking $d=\pm\sqrt{K/V_{jj}}\,Ve_j$, which satisfies $d^\mathsf{T}V^{-1}d=K$ and has $j$th coordinate $\pm\sqrt{KV_{jj}}$. Thus the projection interval for coordinate $j$ is $\hat\theta_j\pm\sqrt{q s^2F_{q,\nu;1-\alpha}V_{jj}}$. These are projections of the joint ellipsoid, not generally scalar intervals with the same confidence level. Their simultaneous inclusion does not imply that a vector lies in the ellipsoid, as the preceding example demonstrates.

Common Mistakes

Using a scalar $t$ cutoff for a joint vector. A $t$ interval calibrates one contrast. For a $q$-dimensional exact ellipsoid, use the factor $qF_{q,\nu;1-\alpha}$.
Using the wrong numerator degrees of freedom. The numerator degrees of freedom are $q$, the rank and dimension of the contrast vector, not automatically $p$ or $n$.
Dropping covariance terms. The quadratic form involves $V^{-1}$, including its off-diagonal entries. Treating correlated contrasts as independent generally gives a different region.
Confusing coordinate projections with joint membership. A vector can lie within every projected coordinate interval and still fall outside the ellipsoid.
Mixing variance conventions. Here $V$ excludes $\sigma^2$; the estimated scale $s^2$ appears in the cutoff. Do not multiply by an additional variance factor.

Key takeaway: For $q$ full-rank estimable contrasts, the exact normal-theory joint region uses the covariance factor $C(X^\mathsf{T}X)^{-1}C^\mathsf{T}$, the estimated error variance $s^2$, and the $F$ quantile with degrees of freedom $q$ and $n-p$. The resulting quadratic-form constraint captures dependence among contrasts and gives joint—not merely coordinatewise—coverage.

Check Your Understanding

Use the model assumptions and definitions above to answer these questions.

What conditions ensure that $V=C(X^\mathsf{T}X)^{-1}C^\mathsf{T}$ is invertible?
For a vector of $q$ contrasts, what are the numerator and denominator degrees of freedom in the $F$ quantile?
Why does the ellipsoid’s quadratic form include off-diagonal covariance information?
In the first worked example, what would the membership decision be if the candidate difference were $d=(1.2,0.8)^\mathsf{T}$ and the cutoff were $8.2056$?
Explain why being inside every coordinate projection does not guarantee that a contrast vector is inside the joint ellipsoid.