Testing a General Contrast With a Worked Example

Why Test a General Contrast?

A scientific question about a regression model need not concern a single coefficient. It may ask whether two slopes are equal, whether a weighted sum of effects reaches a target, or whether a difference between two treatment effects is zero. Each such question can be expressed as a test of a linear contrast of the coefficient vector. The exact normal-theory procedure is a t test, provided the contrast is estimable and the model assumptions hold.

This tutorial focuses on carrying out that test: writing the contrast and null value, evaluating the estimate and its standard error, identifying the reference distribution, and calculating the p-value. The test may be two-sided or one-sided, depending on the scientific alternative.

Definition: In the full-rank normal linear model, a scalar linear contrast is \(c^\top\beta\), where \(\beta\) is the coefficient vector and \(c\) is a specified, nonzero vector of constants. A test of a specified value \(\delta_0\) uses \(H_0:c^\top\beta=\delta_0\). The contrast is estimated by \(c^\top\hat{\beta}\).

Model, Estimate, and Standard Error

Assume the fixed-design model \(Y=X\beta+\varepsilon\), where \(X\) is an \(n\times p\) matrix of full column rank, \(n>p\), and the errors are independent with distribution \(N(0,\sigma^2)\), for \(\sigma^2>0\). Write \(s^2=\mathrm{SSE}/(n-p)\) for the residual mean square. Under these assumptions, \(\hat{\beta}=(X^\top X)^{-1}X^\top Y\), and its estimated covariance matrix is \(s^2(X^\top X)^{-1}\).

Let \(A=(X^\top X)^{-1}\). The contrast estimate and its estimated standard error are

\widehat{c^\top\beta}=c^\top\hat{\beta}, \qquad \widehat{\operatorname{SE}}(c^\top\hat{\beta}) =s\sqrt{c^\top A c}.

The quantity \(c^\top A c\) is a scalar. It is nonnegative because \(A\) is positive definite when \(X\) has full column rank; for nonzero \(c\), it is strictly positive. The standard error therefore incorporates the variances and covariances of all coefficient estimates involved in the contrast. In particular, it is generally incorrect to combine coefficient standard errors without accounting for their covariance.

The Exact t Test

For a specified null value \(\delta_0\), the test statistic is the difference between the estimated and hypothesized contrast values, divided by the estimated standard error:

T= \frac{c^\top\hat{\beta}-\delta_0} {s\sqrt{c^\top A c}}.

Under \(H_0:c^\top\beta=\delta_0\), this statistic has a Student t distribution with \(n-p\) degrees of freedom. The finite-sample reference distribution is exact under the stated normal linear model; it is not an approximation based only on a large sample.

Theorem: Under the fixed-design normal linear model above, if \(c\ne 0\) and \(H_0:c^\top\beta=\delta_0\) is true, then \(T\sim t_{n-p}\), where \(T\) is the contrast statistic defined above.

Proof. From the model, \(\hat{\beta}=\beta+(X^\top X)^{-1}X^\top\varepsilon\). Since \(\varepsilon\sim N(0,\sigma^2 I)\), it follows that \(\hat{\beta}\) is normal with mean \(\beta\) and covariance matrix \(\sigma^2 A\). Therefore, under the null,

Z=\frac{c^\top\hat{\beta}-\delta_0} {\sigma\sqrt{c^\top A c}}\sim N(0,1).

To obtain the distribution of the residual variance estimate, define \(H=X(X^\top X)^{-1}X^\top\) and \(M=I-H\). The residual vector is \(MY=M\varepsilon\), because \(MX=0\). Thus \(\mathrm{SSE}=\varepsilon^\top M\varepsilon\). The matrix \(M\) is symmetric and idempotent and has rank \(n-p\). Orthogonal diagonalization of \(M\), together with the spherical normal distribution of \(\varepsilon/\sigma\), gives

\frac{\mathrm{SSE}}{\sigma^2} =\frac{\varepsilon^\top M\varepsilon}{\sigma^2} \sim\chi^2_{n-p}.

The contrast-estimation error is a linear function of \(H\varepsilon\), whereas the residual vector is a linear function of \(M\varepsilon\). Since \(HM=0\), their covariance is zero. These quantities are jointly normal, so zero covariance implies independence. Consequently, \(Z\) is independent of \(U=\mathrm{SSE}/\sigma^2\). Since \(s^2=\mathrm{SSE}/(n-p)\), the statistic can be written

T=\frac{Z}{\sqrt{U/(n-p)}}.

By the definition of the Student t distribution, this ratio is \(t_{n-p}\). This proves the result. \(\square\)

Choosing the p-Value

The alternative hypothesis determines which tail probability measures evidence against the null. Let \(F_{t_{n-p}}\) denote the cumulative distribution function of a \(t_{n-p}\) variable, and let \(t_{\mathrm{obs}}\) be the observed statistic. The p-values are

\begin{aligned} H_1:c^\top\beta>\delta_0 &\quad\Longrightarrow\quad p=P(T\ge t_{\mathrm{obs}}) =1-F_{t_{n-p}}(t_{\mathrm{obs}}),\\ H_1:c^\top\beta<\delta_0 &\quad\Longrightarrow\quad p=P(T\le t_{\mathrm{obs}}) =F_{t_{n-p}}(t_{\mathrm{obs}}),\\ H_1:c^\top\beta\ne\delta_0 &\quad\Longrightarrow\quad p=P(|T|\ge |t_{\mathrm{obs}}|) =2\{1-F_{t_{n-p}}(|t_{\mathrm{obs}}|)\}. \end{aligned}

The two-sided expression uses the symmetry of the central t distribution: the probability of a result at least as far in the lower tail as \(t_{\mathrm{obs}}\) equals the corresponding upper-tail probability. The p-value is a tail probability computed under the null, not the probability that the null hypothesis is true.

Worked Example: Testing a Difference of Two Coefficients

Suppose \(n=20\), \(p=5\), and the fitted model has \(\hat{\beta}_1=2\), \(\hat{\beta}_2=1\), and \(s^2=4\). The corresponding diagonal entries of \(A=(X^\top X)^{-1}\) are \(A_{11}=0.04\) and \(A_{22}=0.21\), with \(A_{12}=0\). Test \(H_0:\beta_1-\beta_2=0\) against \(H_1:\beta_1-\beta_2\ne0\).

Here \(c\) selects \(\beta_1-\beta_2\). The estimated contrast is \(c^\top\hat{\beta}=\hat{\beta}_1-\hat{\beta}_2=2-1=1\). Its estimated variance is

s^2c^\top A c =4(A_{11}+A_{22}-2A_{12}) =4(0.04+0.21-0)=1, \qquad \widehat{\operatorname{SE}}=1.

The residual degrees of freedom are \(n-p=20-5=15\). The observed statistic is \(t_{\mathrm{obs}}=(1-0)/1=1\). For the two-sided alternative,

p=2P(T_{15}\ge 1) =2[1-F_{t_{15}}(1)] =I_{15/16}(7.5,0.5) \approx 0.3331719 \approx 0.3332.

Thus the data do not provide strong evidence against equality of the two coefficients at conventional significance levels. This conclusion concerns the specified difference, not either coefficient considered separately.

Worked Example: A Contrast Tested Against a Nonzero Value

In another fit, let \(n=20\), \(p=5\), \(s^2=4\), \(\hat{\beta}_1=2\), and \(\hat{\beta}_2=4\). Suppose \(A_{11}=0.04\), \(A_{22}=0.09\), and \(A_{12}=0\). Test \(H_0:2\beta_1+\beta_2=7\) against the two-sided alternative.

The contrast vector assigns weights 2 and 1 to the two coefficients, so its estimate is \(2\hat{\beta}_1+\hat{\beta}_2=2(2)+4=8\). Its estimated variance and standard error are

s^2c^\top A c =4\{2^2(0.04)+1^2(0.09)+2(2)(1)(0)\} =4(0.25)=1, \qquad \widehat{\operatorname{SE}}=1.

The statistic compares the estimate with 7, not with zero: \(t_{\mathrm{obs}}=(8-7)/1=1\), with \(20-5=15\) degrees of freedom. Therefore the two-sided p-value is \(2[1-F_{t_{15}}(1)]\approx0.3332\). The estimate is one standard error above the specified null value; the nonzero null value does not change the reference distribution.

Worked Example: A One-Sided Test of a Sum

Consider a model with \(n=6\), \(p=5\), \(s^2=1\), and estimates \(\hat{\beta}_1=1.5\), \(\hat{\beta}_2=1.5\). Suppose the relevant entries of \(A\) are \(A_{11}=A_{22}=0.5\) and \(A_{12}=0\). Test \(H_0:\beta_1+\beta_2=2\) against \(H_1:\beta_1+\beta_2>2\).

The estimated sum is \(1.5+1.5=3\). Its variance estimate is \(s^2(0.5+0.5+2(0))=1\), so its standard error is 1. The observed statistic is \((3-2)/1=1\), and the residual degrees of freedom are \(6-5=1\). For \(T_1\), the standard Cauchy distribution, the upper-tail probability at 1 is

p=P(T_1\ge1) =\frac{1}{2}-\frac{\arctan(1)}{\pi} =\frac{1}{2}-\frac{1}{4} =0.25.

This is the one-sided p-value because only values larger than the null value count as evidence in the stated direction. A two-sided alternative would instead count outcomes at least as far from zero in either direction and would have p-value 0.50 here.

Common Mistakes and a Practical Check

The most consequential error is to use the correct estimated contrast but the wrong standard error or null value. Before evaluating a tail probability, check the contrast weights, the covariance contribution, the subtraction of \(\delta_0\), the residual degrees of freedom, and the direction of the alternative.

Ignoring covariance. For a sum or difference, include the covariance terms in \(c^\top A c\). Omitting them can overstate or understate the standard error.
Testing against zero automatically. The numerator is \(c^\top\hat{\beta}-\delta_0\). A scientifically specified null value may be nonzero.
Using the wrong tail. The alternative determines whether the p-value is upper-tailed, lower-tailed, or two-sided. A one-sided test should not be selected after seeing which direction the estimate happens to point.
Using the wrong degrees of freedom. In this full-rank model, the exact t reference distribution has \(n-p\) residual degrees of freedom.

A reliable calculation follows a fixed sequence: write \(c\) and \(\delta_0\); compute \(c^\top\hat{\beta}\); compute \(s\sqrt{c^\top A c}\); form the t statistic; then use the appropriate \(t_{n-p}\) tail probability. This makes the inferential question and the arithmetic separately auditable.

Key takeaway: An exact test of \(c^\top\beta=\delta_0\) uses \((c^\top\hat{\beta}-\delta_0)/(s\sqrt{c^\top(X^\top X)^{-1}c})\), which has a \(t_{n-p}\) distribution under the normal linear model and the null hypothesis.

Check Your Understanding

Use the model assumptions and formulas in this tutorial to answer the following.

For a contrast \(c^\top\beta\), which matrix expression gives its estimated variance?
Why does the exact reference distribution have \(n-p\), rather than \(n-1\), degrees of freedom?
If the null is \(c^\top\beta=3\), what belongs in the numerator of the test statistic?
For the alternative \(c^\top\beta<\delta_0\), which tail probability gives the p-value?
Explain why a two-sided p-value for an observed statistic \(t_{\mathrm{obs}}\) uses \(|t_{\mathrm{obs}}|\).