Joint Wald Tests for Several Coefficients

Why Test Coefficients Jointly?

A model may contain several coefficients that represent one scientific question. For example, a treatment effect may vary across two subgroups, making the treatment coefficient and its interaction coefficient jointly relevant. Testing each coefficient separately does not, in general, test the combined claim that both are zero. A joint Wald test evaluates a set of linear restrictions together, using the covariance among the coefficient estimates.

The key calculation is a squared distance: how far the estimated restrictions are from their hypothesized values, measured in units of their estimated covariance. Because that covariance includes off-diagonal terms, the test can give a different conclusion from tests that consider each coefficient in isolation.

Hypotheses and the Wald Quadratic Form

The vector \(A\hat\beta-c\) contains the estimated departures from all \(q\) restrictions. Its covariance estimate is \(A\widehat V A^\top\), so the inverse covariance scales and combines those departures. In particular, the calculation requires the covariance between the restricted estimates, not just their separate variances.

For a test that \(q\) selected coefficients are zero, \(A\) selects those coefficients and \(c\) is the zero vector. More general choices of \(A\) test sums, differences, or other linear combinations. The requirement that \(A\) have full row rank rules out redundant restrictions. If some proposed restrictions are redundant, reduce them to an independent set before using the formula; the degrees of freedom are the number of independent restrictions.

Reference Distribution and Justification

Suppose the coefficient estimator has the asymptotic distribution \(\sqrt n(\hat\beta-\beta_0)\xrightarrow{d}N(0,\Omega)\), where \(\Omega\) is positive definite, and suppose \(\hat\Omega\xrightarrow{p}\Omega\). Here \(\widehat V=\hat\Omega/n\) estimates the covariance of \(\hat\beta\). In a GLM, \(\Omega\) may be obtained from the model-based information under the appropriate regularity conditions, or from a consistent sandwich covariance estimator when that is the intended covariance. The result below is asymptotic; it does not assert an exact finite-sample chi-square distribution.

Proof. Under the null, \(A\beta_0=c\). Define \(Y_n=\sqrt n(A\hat\beta-c)\) and \(S=A\Omega A^\top\). By the assumed asymptotic normality and the fact that a linear transformation of a multivariate normal vector is multivariate normal,

Since \(\Omega\) is positive definite and \(A\) has full row rank, \(S\) is positive definite: for every nonzero \(u\in\mathbb R^q\), \(A^\top u\ne0\), so \(u^\top S u=(A^\top u)^\top\Omega(A^\top u)>0\). Also \(A\hat\Omega A^\top\xrightarrow{p}S\). The statistic can be written

By continuity of matrix inversion at the positive-definite matrix \(S\), the estimated inverse converges in probability to \(S^{-1}\). Slutsky's theorem and the continuous mapping theorem therefore give \(W\xrightarrow{d}Y^\top S^{-1}Y\), where \(Y\sim N(0,S)\). The vector \(S^{-1/2}Y\) has a \(q\)-dimensional standard normal distribution, and \(Y^\top S^{-1}Y=(S^{-1/2}Y)^\top(S^{-1/2}Y)\) is the sum of squares of \(q\) independent standard normal variables. That sum has a \(\chi^2_q\) distribution. \(\square\)

Worked Calculations

Equivalent Ways to Write the Same Restrictions

A change of coordinates among the restrictions does not change the Wald statistic, provided it is invertible. This is useful when the same hypothesis is written using different but equivalent linear combinations.

Proof. Write \(r=A\hat\beta-c\) and \(M=A\widehat V A^\top\). Under the transformed restrictions, the residual is \(Cr\) and the covariance estimate is \(CMC^\top\). Since \(C\) and \(M\) are invertible, \((CMC^\top)^{-1}=C^{-\top}M^{-1}C^{-1}\). Hence the transformed statistic is

which is the original statistic. \(\square\)

Interpretation and Common Pitfalls

The degrees of freedom are the number of independent restrictions, \(q\), not the total number of coefficients in the fitted model. A test of two selected coefficients in a model with many other coefficients therefore uses a two-degree-of-freedom reference distribution. The covariance calculation still uses the estimates’ covariance under the fitted model, transformed to the restrictions by \(A\widehat V A^\top\).

A common mistake is to test each coefficient separately and treat the results as a joint test—for example, to add individual Wald statistics or to conclude that a joint null is false whenever one individual test rejects. That reasoning does not evaluate the specified joint restriction with its covariance. The correction is to form the full residual vector \(A\hat\beta-c\), transform the full covariance matrix, and calculate the quadratic form. Individual tests can help describe which estimates are notable, but they answer different hypotheses.

Another frequent error is to set the off-diagonal covariances to zero simply because the hypotheses concern different coefficients. Different coefficients can have correlated estimates, and a linear combination can be more or less precisely estimated as a result. Report the restrictions, the Wald statistic, its degrees of freedom, and the asymptotic p-value together. The test evaluates the joint departure from the null; by itself it does not identify which restriction is responsible for that departure.