Confidence Ellipsoids for Estimable Matrix Contrasts

Why a Matrix Contrast Needs a Confidence Region

A multivariate regression can produce an estimate of several coefficients across several responses. A scientific question often concerns a particular matrix contrast of those coefficients, rather than the whole coefficient matrix. Its estimate is itself a matrix, so uncertainty must be described jointly: separate intervals for its entries do not, in general, give the desired simultaneous coverage.

A confidence ellipsoid addresses this by combining the estimated covariance of the contrast with a quadratic distance from a candidate value. The size and calibration of the region depend on how many independent predictor and response directions the contrast contains.

Model, Contrast, and Estimated Covariance

Assume the Gaussian multivariate linear model $Y=XB+U$, where $Y$ is $n\times q$, $X$ is a fixed $n\times p$ design matrix of full column rank, and $B$ is $p\times q$. The rows of $U$ are independent $N_q(0,\Sigma)$, with $\Sigma$ positive definite. Write $\hat B=(X'X)^{-1}X'Y$, $P_X=X(X'X)^{-1}X'$, $E=Y'(I-P_X)Y$, and $\nu=n-p$. We require $\nu\ge b$ below.

Let $C$ be an $a\times p$, full-row-rank matrix and $M$ a $q\times b$, full-column-rank matrix. The contrast of interest is $\Psi=CBM$, and its estimate is $\hat\Psi=C\hat B M$. Since $X$ has full column rank, every row of $C$ is estimable. Define

A=C(X'X)^{-1}C', \qquad S=\frac{E}{\nu}, \qquad \hat\Omega=M'SM.

Both $A$ and $\hat\Omega$ are positive definite under these assumptions, with $\hat\Omega$ positive definite with probability one when $\nu\ge b$. The covariance of $\operatorname{vec}(\hat\Psi)$, where $\operatorname{vec}$ stacks matrix columns, is $(M'\Sigma M)\otimes A$. Its estimated counterpart is $\hat\Omega\otimes A$.

Definition: For a proposed value $\Theta$, a covariance-based quadratic distance from the estimate is $$ Q(\Theta)=\operatorname{tr}\!\left\{\hat\Omega^{-1}(\hat\Psi-\Theta)'A^{-1}(\hat\Psi-\Theta)\right\}. $$ A confidence ellipsoid consists of the matrices $\Theta$ whose distance is no larger than a chosen cutoff.

This quadratic form is the squared Mahalanobis distance of the vectorized difference, using the estimated covariance. It incorporates both the correlation among predictor contrasts, through $A$, and the correlation among response contrasts, through $\hat\Omega$.

Worked Example: Two Predictor Contrasts and One Response Contrast

Suppose $a=2$, $b=1$, $\nu=20$, $A=\begin{pmatrix}4&0\\0&1\end{pmatrix}$, and $\hat\Omega=9$. Let $\hat\Psi=(10,2)'$, and evaluate the distance for $\Theta=(8,1)'$.

The difference is $D=\hat\Psi-\Theta=(2,1)'$, and $A^{-1}=\operatorname{diag}(1/4,1)$. Therefore,

Q(\Theta)=\frac{1}{9} \begin{pmatrix}2&1\end{pmatrix} \begin{pmatrix}1/4&0\\0&1\end{pmatrix} \begin{pmatrix}2\\1\end{pmatrix} =\frac{1}{9}\left(1+1\right)=\frac{2}{9}.

Here the contrast has two independent coordinates, so its exact 95% region uses the cutoff $2F_{2,20;0.95}$, where $F_{d_1,d_2;\gamma}$ is the $\gamma$-quantile of an $F$ distribution with $d_1,d_2$ degrees of freedom. The proposed value lies in that region precisely when $2/9\le 2F_{2,20;0.95}$.

Exact Calibration of the Ellipsoid

The confidence cutoff is not generally a standard $F$ quantile when both $a$ and $b$ exceed one. Its exact distribution can nevertheless be specified using independent standard distributions, giving an exact finite-sample region under the Gaussian model.

Theorem: Let $Z$ be an $a\times b$ matrix of independent $N(0,1)$ variables, and independently let $W\sim W_b(I_b,\nu)$, a Wishart random matrix with identity scale and $\nu$ degrees of freedom. If $c_{a,b,\nu;1-\alpha}$ is the $(1-\alpha)$-quantile of $$ \nu\,\operatorname{tr}(W^{-1}Z'Z), $$ then $$ \mathcal C_{1-\alpha} = \left\{\Theta\in\mathbb R^{a\times b}: Q(\Theta)\le c_{a,b,\nu;1-\alpha}\right\} $$ has $P(\Psi\in\mathcal C_{1-\alpha})=1-\alpha$. The region is an ellipsoid in the $ab$ coordinates of $\Theta$.

Proof. Under the model, $C(\hat B-B)M$ is matrix normal with row covariance $A$ and column covariance $\Omega=M'\Sigma M$. Consequently it has the same distribution as $A^{1/2}Z\Omega^{1/2}$. The residual matrix satisfies $E\sim W_q(\Sigma,\nu)$, independently of $\hat B$. Taking the response contrast gives $M'EM\sim W_b(\Omega,\nu)$, so $\hat\Omega$ has the distribution of $W_\Omega/\nu$, where $W_\Omega\sim W_b(\Omega,\nu)$, independently of $Z$.

At the true value $\Theta=\Psi$, substitute these representations into $Q$. Write $W_\Omega=\Omega^{1/2}W\Omega^{1/2}$, where $W\sim W_b(I_b,\nu)$. Using $\operatorname{tr}(RS)=\operatorname{tr}(SR)$ for conformable matrices gives

Q(\Psi) \ \overset{d}{=}\ \nu\operatorname{tr}\!\left[ W_\Omega^{-1}\Omega^{1/2}Z'Z\Omega^{1/2} \right] = \nu\operatorname{tr}(W^{-1}Z'Z).

Thus the distance at the true contrast has the distribution used to define the cutoff. By the quantile definition, $P\{Q(\Psi)\le c_{a,b,\nu;1-\alpha}\}=1-\alpha$, which is exactly the coverage statement. Since $A^{-1}$ and $\hat\Omega^{-1}$ are positive definite, $Q(\Theta)$ is a positive-definite quadratic form in the entries of $\Theta-\hat\Psi$; its bounded sublevel set is an ellipsoid. $\square$

How the Contrast Dimensions Affect the Region

The predictor-contrast dimension $a$ and response-contrast dimension $b$ both matter. They determine the number $ab$ of coordinates in the ellipsoid and enter the exact distribution of its quadratic distance. The residual degrees of freedom $\nu$ also affect its cutoff, because they determine the precision of the estimated response covariance.

Two useful special cases do have familiar $F$ calibrations. If $a=1$, the quadratic distance has the Hotelling $T^2$ distribution in $b$ dimensions, giving

c_{1,b,\nu;1-\alpha} = \frac{b\nu}{\nu-b+1}F_{b,\nu-b+1;1-\alpha}.

If $b=1$, the distance is a ratio of independent chi-square variables and

c_{a,1,\nu;1-\alpha}=aF_{a,\nu;1-\alpha}.

For $a>1$ and $b>1$, the exact cutoff is generally obtained from the stated Wishart-normal distribution rather than by applying either special-case formula. In practice, it can be computed from its distribution or by Monte Carlo simulation: generate independent $Z$ and $W$ as specified, calculate $\nu\operatorname{tr}(W^{-1}Z'Z)$ repeatedly, and use the empirical $(1-\alpha)$-quantile.

Worked Example: One Predictor Contrast Across Two Responses

Suppose $a=1$, $b=2$, and $\nu=10$. Let $A=4$, $\hat\Omega=\begin{pmatrix}4&1\\1&2\end{pmatrix}$, $\hat\Psi=(5,3)$, and consider $\Theta=(4,5)$. Then $D=\hat\Psi-\Theta=(1,-2)$, and

\hat\Omega^{-1} = \frac{1}{7} \begin{pmatrix}2&-1\\-1&4\end{pmatrix}, \qquad D\hat\Omega^{-1}D' = \frac{22}{7}.

Since $A^{-1}=1/4$, the distance is

Q(\Theta)=\frac{1}{4}\frac{22}{7}=\frac{11}{14}.

Here the one predictor contrast leaves a two-dimensional response contrast, so the exact cutoff is $\frac{2(10)}{10-2+1}F_{2,9;0.95}=\frac{20}{9}F_{2,9;0.95}$. The candidate lies in the 95% region if and only if $11/14\le (20/9)F_{2,9;0.95}$.

Worked Example: Two Predictor and Two Response Contrasts

For $a=b=2$, take $A=\operatorname{diag}(1,4)$, $\hat\Omega=\operatorname{diag}(2,1)$, $\hat\Psi=0$, and $\Theta=\begin{pmatrix}-1&0\\0&-2\end{pmatrix}$. The difference is $D=\hat\Psi-\Theta=\operatorname{diag}(1,2)$. First,

D'A^{-1}D = \begin{pmatrix}1&0\\0&2\end{pmatrix} \begin{pmatrix}1&0\\0&1/4\end{pmatrix} \begin{pmatrix}1&0\\0&2\end{pmatrix} = \begin{pmatrix}1&0\\0&1\end{pmatrix}.

Since $\hat\Omega^{-1}=\operatorname{diag}(1/2,1)$, it follows that $Q(\Theta)=\operatorname{tr}\{\operatorname{diag}(1/2,1)\}=3/2$. The exact cutoff is the $(1-\alpha)$-quantile of $ \nu\operatorname{tr}(W^{-1}Z'Z)$ with $Z$ and $W$ both two-dimensional as defined above. Because both contrast dimensions exceed one, neither single-dimension $F$ formula applies.

Redundancy, Interpretation, and a Common Pitfall

If $C$ has redundant rows or $M$ has redundant columns, the displayed coordinates of $CBM$ contain linear dependencies. The region then has lower-dimensional support in the nominal $a\times b$ space. Use independent bases for the predictor and response contrast spaces, express the contrast in those reduced coordinates, and apply the construction with their effective dimensions. Treating redundant entries as independent would make the covariance singular and the stated inverse undefined.

Common mistake: using a separate univariate interval for every entry of $CBM$ and calling their collection a simultaneous confidence region. Each interval may have the desired marginal coverage, but the probability that all intervals cover at once is not thereby set to $1-\alpha$. The ellipsoid instead calibrates one joint quadratic distance using the covariance across all contrast coordinates.

The region is useful when a conclusion concerns the contrast matrix as a whole, including its joint directions and correlations. Its interpretation is direct: values inside the ellipsoid are not ruled out by the specified confidence procedure, while the ellipsoid’s orientation and axis lengths reflect the estimated covariance. Exactness depends on the stated Gaussian model and rank conditions; changing those assumptions may require a different calibration.

Check Your Understanding

Use the definitions and results above to answer these questions.

What roles do $A$ and $\hat\Omega$ play in the quadratic distance?
Why does the ellipsoid have $ab$ coordinates when $C$ has $a$ independent rows and $M$ has $b$ independent columns?
Which exact $F$ calibration applies when $a=1$, and which applies when $b=1$?
Why is it generally incorrect to use either special-case $F$ cutoff when both $a$ and $b$ exceed one?
What adjustment is needed if the contrast matrices contain redundant rows or columns?