Testing Linear Contrasts With T Statistics

Why Test a Linear Combination?

A regression question often concerns more than one coefficient at a time. For example, a researcher might ask whether two adjusted group effects differ, or whether their average equals a specified benchmark. These questions are about a linear combination of regression coefficients, not necessarily about any single coefficient.

Once the combination and its standard error have been specified, the test follows the same basic logic as a coefficient t test: compare the estimate with the value asserted by the null hypothesis, measured in standard-error units. The reference distribution and the tail area depend on the regression model’s residual degrees of freedom and on the alternative hypothesis.

The Model and the Test Statistic

Use the ordinary least-squares regression model with \(n\) observations and \(q\) fitted coefficients, including the intercept if the model has one. For the classical t test, the model must be correctly specified for the question, the errors must have mean zero and constant variance, observations’ errors must be independent, and the errors must be normally distributed for the exact small-sample t reference distribution. The design matrix must have full column rank so the coefficients and requested contrast can be estimated. In symbols, the error assumption is \(\varepsilon\sim N(0,\sigma^2 I)\) conditional on the predictors.

The contrast standard error \(SE(\hat{\theta})\) measures the estimated sampling variability of the whole linear combination. It is not generally obtained by separately testing each coefficient and then combining the test statistics. Use the standard error for the specified contrast, as defined when estimating contrasts.

Under \(H_0\) and the stated classical regression assumptions, this statistic follows a t distribution with \(n-q\) degrees of freedom. The sign of \(t\) indicates whether the estimate is above or below the null value; the alternative determines which part of the t distribution supplies the p-value.

Why the Reference Distribution Is a t Distribution

The t reference distribution accounts for estimating the error variance from the same data. A short derivation shows why the numerator is centered at zero under the null and why the residual degrees of freedom enter the denominator.

Write the regression in matrix form as \(Y=X\beta+\varepsilon\), where \(X\) is the design matrix and \(\beta\) contains the \(q\) coefficients. Under the model assumptions, the least-squares estimate has mean \(\beta\) and covariance matrix \(\sigma^2(X^\mathsf{T}X)^{-1}\). For a fixed contrast vector \(a\), \(\hat{\theta}=a^\mathsf{T}\hat{\beta}\), so it is normally distributed with mean \(\theta=a^\mathsf{T}\beta\) and variance \(\sigma^2a^\mathsf{T}(X^\mathsf{T}X)^{-1}a\).

Under \(H_0\), the contrast is \(\theta_0\). Therefore, if the true \(\sigma\) were known, the difference \(\hat{\theta}-\theta_0\), divided by its true standard deviation, would be standard normal. In practice, \(\sigma^2\) is replaced by the residual mean square \(s^2=SSE/(n-q)\). Under normal errors, \(SSE/\sigma^2\) has a chi-square distribution with \(n-q\) degrees of freedom and is independent of the estimated coefficient vector. Dividing the standard normal quantity by the square root of this independent chi-square quantity divided by its degrees of freedom gives a t statistic. Replacing the true standard deviation with the estimated contrast standard error therefore gives the stated t distribution.

Worked Tests

Choosing and Interpreting the P-Value

The p-value is the probability, assuming the null hypothesis and model assumptions are true, of observing a test statistic at least as extreme as the one calculated, in the direction or directions specified by \(H_a\). For a t statistic \(t\) and \(df=n-q\), the common forms are:

• For \(H_a:\theta\ne\theta_0\), use \(2P(T_{df}\ge |t|)\).
• For \(H_a:\theta>\theta_0\), use \(P(T_{df}\ge t)\).
• For \(H_a:\theta<\theta_0\), use \(P(T_{df}\le t)\).

A p-value is not the probability that \(H_0\) is true, nor is it the probability that random error alone produced the result. It describes how unusual the statistic would be under the null model. A small p-value provides evidence against the null in favor of the stated alternative; it does not measure the practical importance of the contrast.

Common Mistakes

• Testing the wrong quantity. A question about \(\beta_A-\beta_B\) is not answered by testing \(\beta_A=0\) and \(\beta_B=0\) separately. Form the requested contrast and test it directly.
• Using the estimate instead of its distance from the null. The numerator is \(\hat{\theta}-\theta_0\), not merely \(\hat{\theta}\), unless the reference value is zero.
• Using the wrong tail. A directional alternative uses one tail; a two-sided alternative uses both. The p-value must match the alternative stated before examining the data.
• Using the wrong degrees of freedom. In the classical regression t test, use residual degrees of freedom \(n-q\), where \(q\) counts all fitted coefficients, including the intercept when present.
• Treating failure to reject as proof of equality. A p-value above \(\alpha\) means the data do not provide sufficient evidence against the null at that level; it does not prove the null is correct.