Wald Tests for Linear Contrasts

Why Test a Linear Contrast?

A GLM often represents scientific comparisons through several coefficients at once. With treatment coding for a categorical predictor, for example, comparing two nonreference levels generally requires subtracting their coefficients. With an interaction, the treatment difference at a specified covariate value can involve both a main-effect coefficient and an interaction coefficient. In each case, the target is a linear contrast: a specified linear combination of model parameters.

Once that combination is written down, its estimated uncertainty must reflect the covariance among the coefficient estimates. This tutorial develops the Wald test and confidence interval for one scalar contrast. The test is based on the asymptotic normality of the coefficient estimator, as in the earlier material on Wald tests and large-sample likelihood theory.

Definition: Let \(\boldsymbol\beta\) be a \(p\)-dimensional GLM coefficient vector, let \(\mathbf c\) be a fixed, nonzero \(p\)-vector, and let \(d\) be a specified number. The scalar linear contrast is \(\mathbf c^{\mathsf T}\boldsymbol\beta\), and its estimated value is \(\mathbf c^{\mathsf T}\hat{\boldsymbol\beta}\). The hypothesis \(H_0:\mathbf c^{\mathsf T}\boldsymbol\beta=d\) is tested against, for example, the two-sided alternative \(H_A:\mathbf c^{\mathsf T}\boldsymbol\beta\ne d\).

Estimating the Contrast and Its Uncertainty

Suppose \(\widehat{\operatorname{Var}}(\hat{\boldsymbol\beta})=\hat{\boldsymbol\Sigma}\) is the estimated covariance matrix of the fitted coefficient vector. The contrast estimate is a linear transformation of that vector. Its estimated variance is therefore the corresponding quadratic form:

\widehat{\operatorname{Var}}\!\left(\mathbf c^{\mathsf T}\hat{\boldsymbol\beta}\right) =\mathbf c^{\mathsf T}\hat{\boldsymbol\Sigma}\mathbf c, \qquad \widehat{\operatorname{SE}}\!\left(\mathbf c^{\mathsf T}\hat{\boldsymbol\beta}\right) =\sqrt{\mathbf c^{\mathsf T}\hat{\boldsymbol\Sigma}\mathbf c}.

This formula uses the full covariance matrix, not just its diagonal. For two coefficients, for instance,

\widehat{\operatorname{Var}}(\hat\beta_1-\hat\beta_2) =\widehat{\operatorname{Var}}(\hat\beta_1) +\widehat{\operatorname{Var}}(\hat\beta_2) -2\widehat{\operatorname{Cov}}(\hat\beta_1,\hat\beta_2).

The covariance term can increase or decrease the uncertainty of a difference. It cannot generally be omitted. The covariance matrix may be the model-based estimate under the specified GLM assumptions or a consistent sandwich estimate when that is the intended variance estimator. The same contrast calculation applies to either choice.

Worked Example: Comparing Two Factor Levels in Logistic Regression

Consider a logistic regression with an intercept and a three-level factor, using level A as the reference: \(\operatorname{logit}\{P(Y=1)\}=\beta_0+\beta_B I(B)+\beta_C I(C)\). Suppose the fitted coefficients and their estimated covariance matrix are

\hat{\boldsymbol\beta} = \begin{pmatrix}-1.2\\0.5\\0.9\end{pmatrix}, \qquad \hat{\boldsymbol\Sigma} = \begin{pmatrix} 0.025&-0.004&0.002\\ -0.004&0.040&0.015\\ 0.002&0.015&0.090 \end{pmatrix}.

The log-odds difference between level C and level B is \(\beta_C-\beta_B\). Thus \(\mathbf c=(0,-1,1)^{\mathsf T}\), the estimate is \(0.9-0.5=0.4\), and the estimated variance is

\mathbf c^{\mathsf T}\hat{\boldsymbol\Sigma}\mathbf c =0.090+0.040-2(0.015)=0.100.

The standard error is \(\sqrt{0.100}=0.3162\). For \(H_0:\beta_C-\beta_B=0\), the Wald statistic on the standard normal scale is \(z=0.4/\sqrt{0.100}\approx1.26\), giving a two-sided \(p\)-value of approximately \(0.206\). A 95% Wald interval for the log-odds difference is \(0.4\pm1.96(0.3162)=(-0.220,1.020)\), to three decimal places.

Exponentiating expresses the comparison as an odds ratio: \(\exp(0.4)\approx1.49\), with interval \((\exp(-0.220),\exp(1.020))\approx(0.80,2.77)\). The interval includes 1, consistent with the two-sided test not rejecting at the 5% level.

Wald Test for a Scalar Contrast

For a two-sided test of \(H_0:\mathbf c^{\mathsf T}\boldsymbol\beta=d\), the estimated contrast’s distance from the null value is measured in estimated standard errors. Under the regularity conditions below, this standardized difference is asymptotically standard normal under the null.

Theorem: Suppose the parameter is an interior point of the model’s parameter space, the coefficient estimator satisfies \(\sqrt n(\hat{\boldsymbol\beta}-\boldsymbol\beta)\xrightarrow{d}N(\mathbf 0,\boldsymbol\Sigma)\) for a finite positive-definite matrix \(\boldsymbol\Sigma\), and \(n\hat{\boldsymbol\Sigma}\xrightarrow{p}\boldsymbol\Sigma\). For fixed \(\mathbf c\ne\mathbf 0\), if \(H_0:\mathbf c^{\mathsf T}\boldsymbol\beta=d\) holds, then

Z_W= \frac{\mathbf c^{\mathsf T}\hat{\boldsymbol\beta}-d} {\sqrt{\mathbf c^{\mathsf T}\hat{\boldsymbol\Sigma}\mathbf c}} \xrightarrow{d}N(0,1).

The corresponding squared Wald statistic is \(W=Z_W^2\), which converges in distribution to \(\chi^2_1\). For a level-\(\alpha\) two-sided test, reject when \(|Z_W|>z_{1-\alpha/2}\), or equivalently when \(W>\chi^2_{1,1-\alpha}\).

Proof. Under the null, \(\mathbf c^{\mathsf T}\boldsymbol\beta=d\). Applying the fixed linear transformation \(\mathbf c^{\mathsf T}\) to the assumed asymptotic normal limit gives

\sqrt n\bigl(\mathbf c^{\mathsf T}\hat{\boldsymbol\beta}-d\bigr) \xrightarrow{d}N\!\left(0,\mathbf c^{\mathsf T}\boldsymbol\Sigma\mathbf c\right).

Since \(\boldsymbol\Sigma\) is positive definite and \(\mathbf c\ne\mathbf 0\), the limiting variance \(\mathbf c^{\mathsf T}\boldsymbol\Sigma\mathbf c\) is positive. Also, \(n\mathbf c^{\mathsf T}\hat{\boldsymbol\Sigma}\mathbf c \xrightarrow{p}\mathbf c^{\mathsf T}\boldsymbol\Sigma\mathbf c\). Slutsky’s theorem therefore allows division by the estimated standard deviation: the numerator and denominator both have the factor \(1/\sqrt n\), which cancels, and the resulting limit is \(N(0,1)\). Squaring a standard normal variable gives a \(\chi^2_1\) variable, proving the statement about \(W\). \(\square\)

Worked Example: Treatment Difference at a Chosen Covariate Value

Suppose a Poisson GLM has log link and linear predictor \(\log\{\mu\}=\beta_0+\beta_1x+\beta_2T+\beta_3xT\), where \(T=1\) denotes treatment and \(T=0\) denotes control. At \(x=2\), the treatment-minus-control difference in log means is \(\beta_2+2\beta_3\). Suppose \(\hat\beta_2=0.30\), \(\hat\beta_3=-0.10\), and the estimated covariance submatrix for these two coefficients is

\begin{pmatrix} \widehat{\operatorname{Var}}(\hat\beta_2)& \widehat{\operatorname{Cov}}(\hat\beta_2,\hat\beta_3)\\ \widehat{\operatorname{Cov}}(\hat\beta_2,\hat\beta_3)& \widehat{\operatorname{Var}}(\hat\beta_3) \end{pmatrix} = \begin{pmatrix}0.040&-0.005\\-0.005&0.010\end{pmatrix}.

Here \(\mathbf c\) selects \(\beta_2+2\beta_3\), so the estimated log-mean difference is \(0.30+2(-0.10)=0.10\). Its variance and standard error are

0.040+2^2(0.010)+2(1)(2)(-0.005)=0.060, \qquad \sqrt{0.060}=0.2449.

For the null of no treatment difference at \(x=2\), \(z=0.10/\sqrt{0.060}\approx0.41\), with a two-sided \(p\)-value of approximately \(0.683\). The 95% interval for the log-mean difference is \(0.10\pm1.96(0.2449)=(-0.380,0.580)\). On the mean-ratio scale, the estimate is \(\exp(0.10)\approx1.11\), and the interval is approximately \((0.684,1.786)\). This is a comparison at \(x=2\), not a single treatment effect assumed to hold at every \(x\).

Wald Confidence Intervals

The same standardization gives an approximate confidence interval for the contrast. Under the theorem’s assumptions, the interval has asymptotic coverage \(1-\alpha\):

Result: A two-sided Wald confidence interval for \(\mathbf c^{\mathsf T}\boldsymbol\beta\) is the estimated contrast plus or minus a standard normal critical value times its estimated standard error.

\mathbf c^{\mathsf T}\hat{\boldsymbol\beta} \ \pm\ z_{1-\alpha/2} \sqrt{\mathbf c^{\mathsf T}\hat{\boldsymbol\Sigma}\mathbf c}.

Derivation. Write \(\theta=\mathbf c^{\mathsf T}\boldsymbol\beta\) and \(\hat\theta=\mathbf c^{\mathsf T}\hat{\boldsymbol\beta}\). The asymptotic normality and covariance consistency in the theorem imply \((\hat\theta-\theta)/\sqrt{\mathbf c^{\mathsf T}\hat{\boldsymbol\Sigma}\mathbf c}\xrightarrow{d}N(0,1)\). Thus, with probability approaching \(1-\alpha\), this standardized quantity lies between \(-z_{1-\alpha/2}\) and \(z_{1-\alpha/2}\). Rearranging those two inequalities for \(\theta\) yields the displayed interval. \(\square\)

For a positive-valued ratio that equals \(\exp(\theta)\), such as an odds ratio or mean ratio under a log link, first construct the interval for \(\theta\), then exponentiate both endpoints. This produces an interval on the ratio scale that is generally asymmetric around \(\exp(\hat\theta)\).

Worked Example: A Weighted Contrast of Two Logistic Slopes

Consider the logistic regression \(\operatorname{logit}\{P(Y=1)\}=\beta_0+\beta_1x_1+\beta_2x_2\). Suppose \((\hat\beta_1,\hat\beta_2)=(0.80,-0.20)\), and their estimated covariance submatrix is

\begin{pmatrix}0.090&0.018\\0.018&0.040\end{pmatrix}.

The contrast \(\theta=\beta_1-\tfrac12\beta_2\) has estimate \(0.80-\tfrac12(-0.20)=0.90\). Using \(\mathbf c=(1,-\tfrac12)^{\mathsf T}\), its estimated variance is

0.090+\left(-\tfrac12\right)^2(0.040) +2(1)\left(-\tfrac12\right)(0.018) =0.090+0.010-0.018=0.082.

The standard error is \(\sqrt{0.082}=0.2864\). Testing \(H_0:\theta=0\) gives \(z=0.90/\sqrt{0.082}\approx3.14\), with a two-sided \(p\)-value of approximately \(0.0017\). The 95% interval is \(0.90\pm1.96(0.2864)=(0.339,1.461)\), to three decimal places. In this model, \(\theta\) is the change in log odds associated with a covariate change of one unit in \(x_1\) and minus one-half unit in \(x_2\). Its exponent is the corresponding odds multiplier.

Common Mistakes and Interpretation

Using only the variances. The variance of a difference includes minus twice the covariance. Ignoring that term is justified only if the estimated covariance is zero.
Subtracting standard errors. Standard errors do not combine by subtraction. Form the contrast’s variance from the covariance matrix, then take its square root.
Comparing separate significance tests. One coefficient being significant and another not being significant does not test whether their difference is zero. Construct and test the difference directly.
Using the wrong contrast for the coding. Write the model’s linear predictor and identify the parameter combination that represents the scientific comparison. A coefficient alone may compare a level with a reference group, not with another nonreference level.
Ignoring the point at which an interaction is evaluated. With interactions, a treatment difference can depend on covariate values. State the value used to form the contrast.
Overstating the approximation. Wald tests and intervals rely on large-sample approximations and an estimated covariance matrix suitable for the model and sampling assumptions. Their nominal coverage need not be accurate in small or sparse samples.

A linear contrast is still one scalar target, even when it uses several coefficients. Its Wald test has one degree of freedom because it standardizes one estimated quantity. This differs from testing several independent restrictions together, which requires a joint test.

Check Your Understanding

For each question, identify the contrast before calculating its uncertainty.

In a logistic model with level A as the reference, how would you express the log-odds difference between levels B and C?
Why does the variance of \(\hat\beta_1-\hat\beta_2\) contain a covariance term?
A log-link model includes a treatment-by-covariate interaction. What additional information is needed to define the treatment contrast?
What reference distribution does the squared Wald statistic for one scalar contrast have asymptotically under the null?
How should a confidence interval for a log mean ratio be converted to the mean-ratio scale?