Gaussian Tests for Linear Contrasts

Why Test a Contrast?

A regression model estimates several coefficients, but a scientific question often concerns a particular combination of them. For example, a researcher may ask whether two slopes are equal, or whether a weighted sum of coefficients has a specified value. Such questions are expressed as linear contrasts. The uncertainty in a contrast depends not only on the uncertainties of the individual coefficient estimates, but also on how those estimates covary.

Here we use the Gaussian linear regression model with known positive precision weights and an unknown common dispersion. The results give exact t tests and confidence intervals under the model’s assumptions. They use the coefficient covariance matrix from weighted least squares, together with the residual estimate of dispersion.

Model and Contrast Notation

The coefficient indices here run from \(0\) through \(p-1\). Thus the entries of \(a\) must match the order of the entries in \(\beta\). A zero entry means that the corresponding coefficient does not contribute to the contrast. The weighted least-squares estimate and the residual dispersion estimate are

The estimated covariance matrix of \(\hat{\beta}\) is \(s^2(X^\mathsf{T}WX)^{-1}\). Consequently, the estimated variance of a contrast estimate is obtained by applying the contrast vector to both sides of that matrix. This calculation includes off-diagonal entries: ignoring them can give the wrong standard error.

From Coefficient Covariance to Contrast Standard Error

To verify the covariance calculation, note that \(E[Y]=X\beta\) and \(\operatorname{Cov}(Y)=\phi W^{-1}\). Substituting the weighted least-squares expression gives

For any fixed vector \(a\), the variance of a linear transformation is \(a^\mathsf{T}\operatorname{Cov}(\hat{\beta})a\). Therefore

Replacing the unknown \(\phi\) by \(s^2\) gives the estimated variance \(s^2a^\mathsf{T}Ca\), and taking its nonnegative square root gives the stated standard error. Since \(C\) is positive definite under full column rank, \(a^\mathsf{T}Ca>0\) for every nonzero \(a\). A zero vector defines the identically zero contrast and is not a useful test.

Exact t Tests and Confidence Intervals

For a proof, let \(Z=W^{1/2}X\) and \(\varepsilon^*=W^{1/2}(Y-X\beta)\). Because the responses are independent Gaussian variables with variances \(\phi/w_i\), \(\varepsilon^*\sim N(0,\phi I_n)\). Let \(P=Z(Z^\mathsf{T}Z)^{-1}Z^\mathsf{T}\), the orthogonal projection onto the column space of \(Z\). Direct substitution into the weighted least-squares estimator gives

The first expression is a linear transformation of a Gaussian vector, so \(a^\mathsf{T}\hat{\beta}-a^\mathsf{T}\beta\) is Gaussian with mean zero and variance \(\phi a^\mathsf{T}Ca\). The matrix \(P\) is symmetric and idempotent with rank \(p\); hence \(I-P\) is symmetric and idempotent with rank \(n-p\). The Gaussian quadratic-form result for an idempotent projection therefore gives \(\operatorname{RSS}_W/\phi\sim\chi^2_{n-p}\).

Finally, \(P(I-P)=0\). The coefficient estimation error depends only on the projection of \(\varepsilon^*\) onto the column space of \(Z\), while the residual vector depends only on its orthogonal complement. These are jointly Gaussian linear transformations with zero cross-covariance, so they are independent. Thus the standardized contrast numerator is a standard normal variable independent of \(\operatorname{RSS}_W/\phi\). Since \(s^2=\operatorname{RSS}_W/(n-p)\), their ratio has the Student t distribution with \(n-p\) degrees of freedom. Inverting the two-sided test yields the displayed confidence interval. \(\square\)

The interval is for the fixed coefficient contrast \(a^\mathsf{T}\beta\), not for a future response. Its width is determined by the chosen contrast, the fitted design and weights through \(C\), and the residual dispersion estimate through \(s\).

Writing the Test and Reading the Interval

For a specified null value \(\psi_0\), the test statistic and its null reference distribution are

A two-sided test rejects at level \(\alpha\) when \(|t_{\mathrm{obs}}|>t_{n-p,1-\alpha/2}\). Its p-value is \(2P(T_{n-p}\geq|t_{\mathrm{obs}}|)\). For a one-sided alternative, use the corresponding tail probability and critical value. The confidence interval contains exactly those null values that would not be rejected by the corresponding two-sided test.

If \(\phi\) were known rather than estimated, the contrast standardized by \(\sqrt{\phi a^\mathsf{T}Ca}\) would have a standard normal distribution. The t distribution arises here because the same data estimate dispersion, and the independent residual variance estimate contributes a chi-square denominator. This distinction is why the residual degrees of freedom \(n-p\) are part of the test.

Common Mistake: Dropping Covariances or Misaligning Indices

A frequent error is to calculate the variance of a difference as the sum of the two coefficient variances. In general,

The incorrect reasoning is that a minus sign changes the estimate but not its variance. The correction is to apply \(a^\mathsf{T}\widehat{\operatorname{Cov}}(\hat{\beta})a\), which necessarily includes the cross terms. A second source of error is mismatching a contrast vector and the coefficient order. For instance, with \((\beta_0,\beta_1,\beta_2)\), the vector \((0,1,-1)^\mathsf{T}\) selects \(\beta_1-\beta_2\), not \(\beta_2-\beta_3\). Always write the coefficient order first, then assign one contrast-vector entry to each coefficient.