Wald Inference for Generalized Linear Model Contrasts

Why Make Inference About Contrasts?

A generalized linear model estimate is often not the scientific quantity of interest by itself. In a treatment study, the target might be a treatment coefficient, a difference between two treatment effects, or a specified combination of several coefficients. A linear contrast expresses such targets as a weighted sum of the model coefficients. Wald inference uses the estimated covariance of that sum to quantify its uncertainty.

The construction is direct, but its interpretation depends on the model parameterization and on large-sample approximations. In particular, two coefficient systems related by a linear change of coordinates give equivalent Wald inference for the same restriction. A nonlinear change of parameter generally does not preserve the finite-sample Wald statistic.

Estimated Information and a Single Contrast

Write \(J_n(\beta)\) for the model's expected Fisher information for \(\beta\) from \(n\) observations. Assume the model is regular, the true coefficient vector is an interior point, and the information is nonsingular. With known dispersion, the usual large-sample covariance estimate for the coefficient estimator \(\hat\beta\) is \(J_n(\hat\beta)^{-1}\). In a GLM, this is the inverse expected information evaluated at the fitted coefficients; for example, with design matrix \(X\), its familiar weighted form is \(X^\mathsf{T}WX/\phi\), where \(W\) contains the GLM weights and \(\phi\) is the dispersion.

For a single contrast \(c^\mathsf{T}\beta\), the estimated contrast is \(c^\mathsf{T}\hat\beta\), and its estimated variance is \(c^\mathsf{T}J_n(\hat\beta)^{-1}c\). This gives the Wald statistic for testing \(H_0:c^\mathsf{T}\beta=d\):

Under the null and the regularity conditions stated below, \(Z_W\xrightarrow{d}N(0,1)\). A corresponding approximate \(100(1-\alpha)\%\) confidence interval for the contrast is

The interval and test are two forms of the same calculation: the null value \(d\) is rejected by a two-sided level-\(\alpha\) Wald test precisely when it falls outside this interval, using the same standard-normal cutoff.

Joint Contrasts and the Wald Confidence Region

For \(q\) restrictions, the estimated covariance matrix of \(C\hat\beta\) is \(C J_n(\hat\beta)^{-1}C^\mathsf{T}\). If \(C\) has rank \(q\), this matrix is nonsingular when the coefficient information is nonsingular. The joint Wald statistic is

For testing \(H_0:C\beta=d\), its asymptotic null reference distribution is \(\chi^2_q\). The corresponding approximate confidence region for the contrast vector \(C\beta\) consists of all \(u\in\mathbb{R}^q\) satisfying

This is an ellipsoid centered at the estimated contrast vector. It accounts for the covariance among the contrasts; separate one-at-a-time intervals do not, by themselves, provide this joint region.

Why the Joint Wald Test Has a Chi-Square Limit

A precise asymptotic statement needs an increasing-information regime. For example, suppose the dimension \(p\) is fixed, the true parameter \(\beta_0\) is interior, the GLM likelihood satisfies the usual differentiability and identifiability conditions, and \(J_n(\beta_0)/n\to J\), where \(J\) is positive definite. Assume the maximum-likelihood estimator is consistent and \(\sqrt n(\hat\beta-\beta_0)\xrightarrow{d}N_p(0,J^{-1})\), and that \(J_n(\hat\beta)/n\xrightarrow{p}J\). These are the regular large-sample conditions behind the following result.

Proof. Set \(V=CJ^{-1}C^\mathsf{T}\). Since \(J\) is positive definite and \(C\) has rank \(q\), \(V\) is positive definite. Under the null, the assumed asymptotic normality gives \(\sqrt n(C\hat\beta-d)\xrightarrow{d}N_q(0,V)\). Also, by consistency of the estimated information and continuity of matrix inversion, \(n C J_n(\hat\beta)^{-1}C^\mathsf{T}\xrightarrow{p}V\). Rewriting the statistic with these scaled quantities and applying Slutsky's theorem, its limit is \(Y^\mathsf{T}V^{-1}Y\), where \(Y\sim N_q(0,V)\).

Write \(Y=V^{1/2}Z\), where \(Z\sim N_q(0,I_q)\). Then \(Y^\mathsf{T}V^{-1}Y=Z^\mathsf{T}Z\), the sum of squares of \(q\) independent standard normal variables. By definition this has a \(\chi^2_q\) distribution, proving the claim. \(\square\)

Linear Changes of Coefficients and Nonlinear Transformations

Wald inference is exactly invariant to a one-to-one affine change of coefficient coordinates, provided the restriction and covariance are transformed consistently. To see this, let \(\gamma=A\beta+b\), where \(A\) is nonsingular. A restriction \(L\gamma=r\) is the same restriction as \(LA\beta=r-Lb\). The transformed estimator is \(\hat\gamma=A\hat\beta+b\), and its covariance estimate is \(A\widehat V A^\mathsf{T}\), where \(\widehat V=J_n(\hat\beta)^{-1}\). The Wald quadratic in the transformed coordinates is

which is precisely the Wald statistic for the equivalent restriction in the original coordinates. The equality follows because both the estimated contrast and its covariance are transformed by the same matrix.

A nonlinear transformation does not generally preserve this finite-sample equality. If \(\theta=g(\beta)\) is scalar and differentiable, the delta method estimates its variance by \(\nabla g(\hat\beta)^\mathsf{T}\widehat V\nabla g(\hat\beta)\). A Wald test or interval built from this local, linear approximation can differ substantially from one formed on the original coefficient scale. The two approaches can have the same large-sample behavior under suitable regularity conditions, but that does not make their finite-sample results identical.

Common Mistakes and Practical Interpretation

• Using the wrong contrast variance. For \(c^\mathsf{T}\beta\), use \(c^\mathsf{T}\widehat Vc\), including covariance terms. Adding only the variances of the coefficients is correct only in special cases, such as when the relevant estimates are uncorrelated.
• Treating several restrictions as separate tests. A joint null \(C\beta=d\) uses the full contrast covariance and a \(\chi^2_q\) reference distribution. Separate one-dimensional tests do not replace this joint test.
• Assuming every reparameterization leaves a Wald result unchanged. Exact invariance holds for equivalent restrictions under an invertible affine transformation. Nonlinear transformations can change finite-sample Wald statistics and intervals.
• Interpreting a Wald interval as exact. Its coverage is approximate and relies on the estimator's large-sample normal approximation and a reliable estimated information matrix. In small samples, poor conditioning or estimates near a parameter-space boundary can make that approximation questionable.

Wald inference is useful because it turns an estimated coefficient vector and its covariance into tests and confidence regions for scientifically chosen combinations of effects. Its main discipline is to specify the contrast first, use its complete covariance, and remember that the result is tied to the scale on which the Wald approximation is constructed.