Linear Transformations of Confidence Regions

Why Transform a Confidence Region?

A confidence region for a coefficient vector can answer questions about many derived quantities: sums of coefficients, differences, or a vector of contrasts. A linear transformation carries the original region to a region for those quantities. To interpret that image correctly, it is important to distinguish its geometric shape from a region calibrated directly for the transformed parameter.

We work with the fixed-design normal linear model \(Y=X\beta+\varepsilon\), where \(X\) is an \(n\) by \(p\) matrix of full column rank, \(\varepsilon\sim N(0,\sigma^2 I_n)\), and \(\nu=n-p>0\). Write \(\hat{\beta}\) for the least-squares estimator and \(s^2=\operatorname{SSE}/\nu\). The exact \(1-\alpha\) confidence ellipsoid for \(\beta\), with unknown error variance, is

Here \(F_{p,\nu}(1-\alpha)\) is the \(1-\alpha\) quantile of the \(F\) distribution with \(p\) and \(\nu\) degrees of freedom. The estimated covariance matrix of \(\hat{\beta}\) is \(s^2(X'X)^{-1}\). The earlier material on joint contrast ellipsoids gives the corresponding exact construction for a full-rank contrast vector.

The Image Under a Linear Map

Let \(A\) be a specified \(k\) by \(p\) matrix, and let the target parameter be \(\gamma=A\beta\). The image of a set \(\mathcal C\) under \(A\) means the set of all transformed points \(Ab\) for \(b\in\mathcal C\). The center of the image of \(\mathcal C_\beta\) is \(A\hat{\beta}\). The key question is whether the image has full dimension \(k\).

For any fixed matrix \(A\), covariance transforms by multiplication on both sides:

Replacing \(\sigma^2\) by \(s^2\) gives the estimated covariance used in the ellipsoid. This is a covariance transformation, not an arbitrary change in the region's scale: the threshold in the displayed image is still the threshold of the original \(p\)-dimensional region.

Why the Image Is an Ellipsoid

To prove the result, write \(x=V^{-1/2}(b-\hat b)\). Then \(\mathcal E\) corresponds to \(\|x\|^2\leq c\), and the displacement in the image is \(d=\gamma-A\hat b=Bx\), where \(B=AV^{1/2}\). Since \(A\) has full row rank, \(BB'=AVA'\) is invertible. For any \(d\), the vector

satisfies \(Bx_0=d\). Every other solution has the form \(x=x_0+z\), where \(Bz=0\). Also \(x_0'z=d'(BB')^{-1}Bz=0\), so \(\|x\|^2=\|x_0\|^2+\|z\|^2\). Thus \(x_0\) is the solution with smallest squared norm, and

There is a solution with \(\|x\|^2\leq c\) exactly when this minimum is at most \(c\). This proves the stated description of the image. \(\square\)

This result also explains the rank condition. If \(A\) has full row rank, every displacement in the \(k\)-dimensional target space can be reached, and \(AVA'\) is positive definite. If \(A\) is rank-deficient, the image lies in a lower-dimensional affine subspace. In that case \(AVA'\) is singular, and an inverse-based ellipsoid equation in the whole target space is not valid. The image can instead be described on its affine subspace using a generalized inverse and an explicit range restriction.

Projection, Coverage, and Direct Calibration

When \(A\) has fewer rows than the original parameter vector has components, it projects a higher-dimensional region onto a lower-dimensional target. The image of a \(1-\alpha\) confidence region for \(\beta\) is a confidence region for \(A\beta\): whenever \(\beta\in\mathcal C_\beta\), it follows that \(A\beta\in A\mathcal C_\beta\). Consequently,

The inequality can be strict because a projected point may belong to the image even when the original parameter vector is outside the original ellipsoid. Thus an image region is not generally an exact \(1-\alpha\) region for the transformed parameter.

For a full-row-rank \(k\) by \(p\) matrix \(A\), a directly calibrated exact region for \(A\beta\) uses the \(k\)-dimensional threshold instead:

Under the stated normal-model assumptions, this region has exact coverage \(1-\alpha\). Its center and covariance estimate match those of the image, but its threshold is calibrated for dimension \(k\), not \(p\). When \(A\) is square and invertible, \(k=p\) and the image and directly calibrated region coincide. When the map reduces dimension, do not silently replace the image threshold by the direct threshold: that creates a different region.

Common Mistakes and Practical Use

Linear transformations make it possible to report uncertainty for scientifically meaningful combinations of coefficients without changing the underlying model. The transformed covariance reveals how uncertainty and dependence carry over: even if the original coefficient estimates have nonzero covariance, a particular transformation may increase, decrease, or remove covariance between transformed components. The region's orientation and scale are determined by \(AVA'\), while its dimension is determined by the rank of \(A\).

• Using \(A'VA\) instead of \(AVA'\). For an \(k\) by \(p\) map, the transformed covariance must be \(AVA'\), which is \(k\) by \(k\).
• Assuming every image is full-dimensional. Check the rank of \(A\). A rank-deficient map gives a lower-dimensional image and a singular transformed covariance matrix.
• Calling the image exactly calibrated. The image inherits at least the original region's coverage, but projection can make coverage exceed the nominal level. A directly calibrated region uses the dimension of the transformed parameter.
• Changing the threshold while claiming to describe the same image. The image retains the original threshold. Replacing it with a dimension-specific threshold produces a different confidence region.