Joint Confidence Regions for Contrast Vectors

Why a Vector Needs a Joint Confidence Region

When several prespecified contrasts are of interest, separate intervals do not directly describe uncertainty in the contrast vector as a whole. A joint confidence region answers a different question: which entire vectors of contrast values remain compatible with the observed sample at a stated confidence level? Under multivariate normal sampling, this region has an exact finite-sample ellipsoidal form.

Let \(X_1,\ldots,X_n\) be independent observations from a \(p\)-variate normal distribution with mean vector \(\mu\) and positive-definite covariance matrix \(\Sigma\). Let \(A\) be a fixed \(q\times p\) matrix of rank \(q\), where \(1\leq q\leq p\). The target is the \(q\)-dimensional vector \(\theta=A\mu\). In a contrast problem, rows of \(A\) specify the contrasts; the derivation applies to any prespecified full-rank linear map.

Because \(A\) has full row rank and \(\Sigma\) is positive definite, \(A\Sigma A'\) is positive definite. Indeed, for every nonzero \(v\in\mathbb R^q\), \(A'v\ne0\), so \(v'A\Sigma A'v=(A'v)'\Sigma(A'v)>0\). Thus the transformed mean has a nonsingular covariance, and a quadratic distance using the inverse transformed sample covariance is well-defined with probability one when \(n>q\).

The Exact Confidence Ellipsoid

Write \(F_{q,n-q;1-\alpha}\) for the \(1-\alpha\) quantile of an \(F\) distribution with \(q\) and \(n-q\) degrees of freedom. The exact critical value comes from the Hotelling \(T^2\) distribution for the \(q\)-dimensional transformed observations, not from treating the \(q\) estimated coordinates as independent.

Equivalently, \(A S A'\) estimates the covariance of one transformed observation, while \(A S A'/n\) estimates the covariance of \(\hat\theta\). The same region can therefore be written \((\hat\theta-\theta)'(A S A'/n)^{-1}(\hat\theta-\theta) \leq \dfrac{q(n-1)}{n-q}F_{q,n-q;1-\alpha}\). This form makes the scaling by the covariance of the estimator explicit.

Why the Calibration Is Exact

The key point is that the transformed observations \(Y_i=A X_i\) are independent \(N_q(A\mu,A\Sigma A')\) observations. Their sample mean is \(\hat\theta\), and their sample covariance is exactly \(A S A'\). The required calibration is therefore the one-sample Hotelling \(T^2\) result in dimension \(q\). The derivation below gives the distributional argument, including the degrees of freedom.

To verify the ratio claim, condition on \(U=u\ne0\). Rotational invariance of the Wishart distribution permits rotating \(u\) to \(\|u\|e_1\) without changing the distribution of \(W\). Consequently, \(u'W^{-1}u\) has the conditional distribution \(\|u\|^2(W^{-1})_{11}\). Write \(W=Z'Z\), where \(Z\) is an \(m\times q\) matrix of independent standard normal entries, and partition its columns as \(Z=(z_1,Z_2)\). The block inverse formula gives \((W^{-1})_{11}^{-1}=z_1'(I-P_{Z_2})z_1\), where \(P_{Z_2}\) projects onto the column space of \(Z_2\). With probability one, that space has dimension \(q-1\). Conditional on \(Z_2\), \(z_1\) is an independent standard normal vector, so the projection onto the orthogonal complement has squared length distributed as \(\chi^2_{m-q+1}\). Thus \(D=(W^{-1})_{11}^{-1}\sim\chi^2_{m-q+1}\). The conditional distribution of \(D\) does not depend on \(U\), so it is independent of \(\|U\|^2\sim\chi^2_q\). Therefore \(U'W^{-1}U\) has the distribution of \(\chi^2_q/\chi^2_{m-q+1}\), with independent numerator and denominator, or equivalently \(\dfrac{q}{m-q+1}F_{q,m-q+1}\).

Since \(W=(n-1)S_Z\) and \(m=n-1\), the standardized Hotelling statistic is \(n\bar Z'S_Z^{-1}\bar Z=(n-1)U'W^{-1}U\). Substitution of the ratio distribution gives \(n\bar Z'S_Z^{-1}\bar Z\sim\dfrac{q(n-1)}{n-q}F_{q,n-q}\). Undoing the nonsingular standardization leaves the quadratic statistic unchanged: \[ n(\hat\theta-A\mu)'(A S A')^{-1}(\hat\theta-A\mu) \sim \frac{q(n-1)}{n-q}F_{q,n-q}. \] By the definition of the \(1-\alpha\) quantile, the probability that this statistic is at most its stated critical value is \(1-\alpha\). This proves the theorem. \(\square\)

Worked Calculations

Interpretation and a Common Pitfall

The ellipsoid incorporates both the estimated covariance among contrasts and their different scales. Its boundary consists of contrast vectors at the same squared Mahalanobis distance from \(\hat\theta\); directions with more estimated variability extend farther. Exact coverage refers to the entire \(q\)-vector at once. It does not say that each coordinate has a separate 95% marginal interval.

• Pitfall: using \(q\) separate unadjusted intervals as if they were the joint region. This ignores the joint coverage requirement and the dependence between contrast estimates. The correction is to use the quadratic-form region and its \(F_{q,n-q}\) critical value when the stated Gaussian model and full-rank conditions hold.
• Pitfall: using \(S\) in place of \(A S A'\). The target lives in the transformed \(q\)-dimensional space. Its estimated covariance is \(A S A'\), and the covariance of its sample mean is \(A S A'/n\).
• Pitfall: omitting the degrees-of-freedom factor. For unknown \(\Sigma\), the multiplier is \(q(n-1)/(n-q)\), not just a chi-square quantile. A chi-square calibration applies in a different setting, such as when the covariance is known.

The rank condition is substantive: if \(A\) has redundant rows, \(A S A'\) is singular and cannot be inverted as written. The target must first be represented in a nonredundant full-rank coordinate system. The condition \(n>q\) ensures the residual degrees of freedom permit a nonsingular transformed sample covariance with probability one under the model.