Intervals for Linear Contrasts

Why Linear Contrasts Need the Full Covariance Matrix

Many questions about several parameters are questions about one particular combination of them. A treatment comparison may be a difference between two means; a policy summary may be a weighted sum of several effects; a regression question may concern a specified combination of coefficients. These targets are called linear contrasts. Even when the component estimates are individually precise, the uncertainty in their combination depends on how they vary together.

The key calculation is a quadratic form: the contrast weights multiply the entire covariance matrix, including its off-diagonal covariances. Once that variance is calculated, the interval is a scalar asymptotic Wald interval. It gives coverage for the specified contrast, not simultaneous coverage for the whole parameter vector.

Definition: Let $\theta=(\theta_1,\ldots,\theta_p)^T$ be a parameter vector and let $a=(a_1,\ldots,a_p)^T$ be a fixed, known vector of weights. The linear contrast is the scalar parameter $$ \psi=a^T\theta=\sum_{j=1}^p a_j\theta_j. $$ Given an estimator $\hat{\theta}$, the corresponding estimator is $\hat{\psi}=a^T\hat{\theta}$.

A difference is a contrast: to compare $\theta_1$ and $\theta_2$, take $a=(1,-1)^T$. A weighted sum is also a contrast, whether or not its weights add to one. The weights must be specified as part of the estimand: changing them changes the parameter being estimated.

Variance of a Linear Contrast

Write $\operatorname{Cov}(\hat{\theta})$ for the covariance matrix of the estimator vector. Its $(j,k)$ entry is $\operatorname{Cov}(\hat{\theta}_j,\hat{\theta}_k)$. The diagonal entries are component variances; the off-diagonal entries describe dependence between component estimates.

Result: If $\hat{\theta}$ has finite second moments, then $$ \operatorname{Var}(a^T\hat{\theta})=a^T\operatorname{Cov}(\hat{\theta})a. $$

To verify the formula, set $V=\operatorname{Cov}(\hat{\theta})$. Expanding the variance of the weighted sum gives $$ \operatorname{Var}\left(\sum_{j=1}^p a_j\hat{\theta}_j\right) =\sum_{j=1}^p\sum_{k=1}^p a_ja_k\operatorname{Cov}(\hat{\theta}_j,\hat{\theta}_k) =a^TVa. $$ The off-diagonal terms occur in pairs because $V$ is symmetric. For two parameters and a difference, this becomes $$ \operatorname{Var}(\hat{\theta}_1-\hat{\theta}_2) =\operatorname{Var}(\hat{\theta}_1)+\operatorname{Var}(\hat{\theta}_2) -2\operatorname{Cov}(\hat{\theta}_1,\hat{\theta}_2). $$ Thus, treating the estimates as independent is justified only when their covariance is zero.

Worked Example: A Difference Between Two Estimates

Suppose the estimated parameter vector is $\hat{\theta}=(12.4,10.9)^T$, with estimated covariance matrix $$ \hat V= \begin{pmatrix} 0.16&0.06\\ 0.06&0.25 \end{pmatrix}. $$ Construct a 95% asymptotic interval for $\psi=\theta_1-\theta_2$.

The contrast vector is $a=(1,-1)^T$, so $\hat{\psi}=12.4-10.9=1.5$. Its estimated variance is $$ a^T\hat V a=0.16+0.25-2(0.06)=0.29. $$ The estimated standard error is $\sqrt{0.29}\approx0.5385$. Using the standard normal 97.5th percentile, $z_{0.975}\approx1.96$, the interval is $$ 1.5\mathbin{\pm}1.96(0.5385)=1.5\mathbin{\pm}1.0555, $$ or approximately $(0.445,2.555)$. The positive covariance reduces the variance of a difference: when the two estimates tend to move together, subtracting them removes some of that shared variation.

Constructing the Asymptotic Interval

For a standard large-sample interval, assume $p$ is fixed and the estimator satisfies $$ \sqrt{n}(\hat{\theta}-\theta)\xrightarrow{d}N(0,\Sigma), $$ where $\Sigma$ is a finite, symmetric, nonnegative-definite matrix. Suppose also that an estimated covariance matrix $\hat V$ for $\hat{\theta}$ satisfies $n\hat V\xrightarrow{p}\Sigma$. Equivalently, one may estimate $\Sigma$ directly by $\hat{\Sigma}$ with $\hat{\Sigma}\xrightarrow{p}\Sigma$, and use $\hat V=\hat{\Sigma}/n$. For the chosen contrast, require $a^T\Sigma a>0$, so its limiting variance is not zero.

Theorem: Under these conditions, for any fixed nonzero contrast vector $a$, an asymptotic $100(1-\alpha)\%$ confidence interval for $\psi=a^T\theta$, where $0<\alpha<1$, is $$ a^T\hat{\theta}\mathbin{\pm}z_{1-\alpha/2}\sqrt{a^T\hat V a}. $$ Its coverage probability tends to $1-\alpha$ as $n\to\infty$.

The result follows because applying a fixed linear map to a multivariate normal limit gives a scalar normal limit, and the estimated contrast standard error is consistent for the contrast’s asymptotic scale. Here is the coverage argument in detail.

Obtain the contrast’s limiting distribution.
By the continuous mapping theorem, $ \sqrt{n}a^T(\hat{\theta}-\theta)\xrightarrow{d}N(0,a^T\Sigma a)$.

Establish the standard-error limit.
Since $n\hat V\xrightarrow{p}\Sigma$, continuity of the quadratic form and square root gives $n a^T\hat V a\xrightarrow{p}a^T\Sigma a$. The latter limit is positive.

Studentize and use the normal quantile.
Slutsky’s theorem gives $(a^T\hat{\theta}-a^T\theta)/\sqrt{a^T\hat V a}\xrightarrow{d}N(0,1)$. The probability that its absolute value is at most $z_{1-\alpha/2}$ therefore tends to $1-\alpha$, which is exactly the coverage event for the stated interval.

The interval is centered at the estimated contrast and has half-width equal to a normal critical value times its estimated standard error. This is a scalar statement: its coverage concerns the single specified value $a^T\theta$.

Worked Example: A Weighted Sum of Three Parameters

Suppose $\hat{\theta}=(4,2,1.5)^T$ and the estimated covariance matrix is $$ \hat V= \begin{pmatrix} 0.04&0.012&0\\ 0.012&0.09&0.018\\ 0&0.018&0.16 \end{pmatrix}. $$ For weights $a=(0.5,0.3,0.2)^T$, calculate a 95% asymptotic interval for $\psi=0.5\theta_1+0.3\theta_2+0.2\theta_3$.

The point estimate is $$ \hat{\psi}=0.5(4)+0.3(2)+0.2(1.5)=2.9. $$ The diagonal contributions to its variance are $0.5^2(0.04)=0.01000$, $0.3^2(0.09)=0.00810$, and $0.2^2(0.16)=0.00640$. The nonzero off-diagonal contributions are $2(0.5)(0.3)(0.012)=0.00360$ and $2(0.3)(0.2)(0.018)=0.00216$. Hence $$ a^T\hat V a=0.01000+0.00810+0.00640+0.00360+0.00216=0.03026, $$ and the standard error is $\sqrt{0.03026}\approx0.1740$. The 95% interval is $$ 2.9\mathbin{\pm}1.96(0.1740)=2.9\mathbin{\pm}0.3410, $$ or approximately $(2.559,3.241)$.

Worked Example: How Ignoring Covariance Changes a Difference Interval

Suppose $\hat{\theta}=(1.8,1.2)^T$ and $$ \hat V= \begin{pmatrix} 0.25&0.20\\ 0.20&0.36 \end{pmatrix}. $$ For the contrast $\theta_1-\theta_2$, the estimate is $0.6$. The correct estimated variance is $$ 0.25+0.36-2(0.20)=0.21, $$ so the standard error is $\sqrt{0.21}\approx0.4583$. The 95% interval is $0.6\mathbin{\pm}1.96(0.4583)$, or approximately $(-0.298,1.498)$.

If the covariance were incorrectly set to zero, the variance would be $0.25+0.36=0.61$, giving a standard error of about $0.7810$ and an interval of about $(-0.931,2.131)$. That calculation incorrectly assumes independence. The covariance matrix says the estimates move together, which makes their difference less variable than the independence calculation suggests.

Using and Interpreting Contrast Intervals

In many applications, the covariance matrix is reported alongside the vector estimate by regression or likelihood software. To calculate an interval, specify the estimand as a weight vector, multiply the estimate by those weights, and use the full covariance matrix for the standard error. If software reports an estimated asymptotic covariance matrix $\hat{\Sigma}$ for $\sqrt{n}(\hat{\theta}-\theta)$, divide the quadratic form by $n$ before taking its square root.

A contrast interval answers whether particular values of that contrast are compatible with the data at the stated asymptotic confidence level. For example, if a difference interval contains zero, it does not establish that the two component parameters are equal; it indicates that zero is not excluded by this interval. The interval does not itself provide simultaneous coverage for multiple contrasts.

Common Mistake: Dropping the Off-Diagonal Terms

A frequent error is to calculate the contrast variance as the sum of the weighted component variances and leave out all covariance terms. That shortcut is valid only when the relevant estimates are uncorrelated. For a difference, the omitted term is $-2\operatorname{Cov}(\hat{\theta}_1,\hat{\theta}_2)$; for a general contrast, every off-diagonal covariance contributes with its corresponding pair of weights. Use $a^T\hat V a$, rather than relying on an independence assumption that the model has not supplied.

A second check is to ensure the weights match the scientific question. Reversing the order in a difference changes its sign, and changing the contrast vector changes both the estimand and its standard error. State the parameter combination first; then apply the same weights to the estimate and covariance matrix.

Check Your Understanding

Work through these questions using the contrast weights and covariance information explicitly.

For a contrast $a^T\theta$, what matrix expression gives its estimated variance, and which entries of the covariance matrix does it use?
If $\operatorname{Var}(\hat{\theta}_1)=0.09$, $\operatorname{Var}(\hat{\theta}_2)=0.16$, and $\operatorname{Cov}(\hat{\theta}_1,\hat{\theta}_2)=0.03$, calculate the estimated variance of $\hat{\theta}_1-\hat{\theta}_2$.
Why does a confidence interval for one linear contrast not automatically give simultaneous coverage for every component of $\theta$?
Suppose an estimated asymptotic covariance matrix $\hat{\Sigma}$ estimates the covariance of $\sqrt{n}(\hat{\theta}-\theta)$. What variance estimate should be used for $a^T\hat{\theta}$?
When is it valid to omit covariance terms while calculating a contrast standard error?

What You'll Learn