Wald Tests for Linear Regression Contrasts

Why Test Several Coefficients at Once?

A regression question often concerns a linear combination of coefficients rather than one coefficient in isolation. For example, a researcher may ask whether two treatment effects are equal, whether a group of coefficients is jointly zero, or whether a weighted sum of coefficients has a specified value. A Wald test answers these questions by measuring the estimated departure from the restrictions relative to its estimated covariance.

The key asymptotic fact is that, under the null hypothesis and regular random-design conditions, a Wald statistic for $q$ independent restrictions converges in distribution to a chi-square random variable with $q$ degrees of freedom. The number of restrictions—not the number of regressors—is the degrees of freedom.

Linear Restrictions and the Wald Statistic

Let $Y_i=X_i'\beta_0+u_i$, where $X_i$ is a $p$-dimensional regressor vector, and suppose $(X_i,u_i)$ are independent and identically distributed. Assume $E[X_i u_i]=0$, $Q=E[X_iX_i']$ is finite and positive definite, and $E[\lVert X_i u_i\rVert^2]<\infty$. Let $\Omega=E[X_iX_i'u_i^2]$ be finite and positive definite. These conditions give the random-design OLS limit $\sqrt{n}(\hat{\beta}-\beta_0)\xrightarrow{d}N(0,V)$, where $V=Q^{-1}\Omega Q^{-1}$. We also assume that the covariance estimator used below is consistent for $V$, as holds for a consistent sandwich estimator under its regularity conditions.

Definition: A set of $q$ linear restrictions on the coefficient vector has the form $H_0:R\beta_0=r$, where $R$ is a known $q\times p$ matrix of rank $q$, and $r$ is a known $q$-vector. Write $\hat V_n$ for an estimator satisfying $\hat V_n\xrightarrow{p}V$; thus $\hat V_n/n$ estimates the covariance matrix of $\hat\beta$. The Wald statistic is $$ W_n=n(R\hat\beta-r)'(R\hat V_nR')^{-1}(R\hat\beta-r). $$

The rank condition ensures that the $q$ restrictions are linearly independent. Since $V$ is positive definite, $RVR'$ is positive definite as well: for any nonzero $a\in\mathbb{R}^q$, $R'a\ne0$, so $a'RVR'a=(R'a)'V(R'a)>0$. Consequently, the inverse in the limiting statistic exists. In applications, $R\hat V_nR'$ is invertible with probability approaching one.

Worked Example: Testing One Slope

Suppose $p=2$, $n=100$, and the restriction is that the second coefficient is zero. The estimate is $\hat\beta_2=0.4$, and the estimated asymptotic covariance of $\sqrt n\hat\beta$ has second diagonal entry $4$. Take $R=(0,1)$ and $r=0$.

The estimated variance for the slope estimate itself is $4/100=0.04$. Equivalently, using the Wald formula directly,

W_n=100\frac{(0.4-0)^2}{4} =\frac{100(0.16)}{4}=4.

There is one restriction, so the reference distribution is $\chi^2_1$. The upper-tail probability is approximately $0.0455$. This is an asymptotic p-value; it does not assert that the statistic has an exact chi-square distribution at sample size 100.

Worked Example: Testing Two Coefficients Jointly

Let $p=3$, $n=100$, and test $H_0:\beta_{0,2}=0,\ \beta_{0,3}=0$. Suppose $(\hat\beta_2,\hat\beta_3)'=(0.5,-0.2)'$, and the corresponding block of $\hat V_n$ is $\begin{pmatrix}4&1\\1&9\end{pmatrix}$. The restriction matrix selects the second and third coefficients, so the restriction error is $d=(0.5,-0.2)'$.

The determinant of the covariance block is $4(9)-1(1)=35$, giving

\begin{pmatrix}4&1\\1&9\end{pmatrix}^{-1} =\frac{1}{35}\begin{pmatrix}9&-1\\-1&4\end{pmatrix}.

Therefore,

W_n=100d'\begin{pmatrix}4&1\\1&9\end{pmatrix}^{-1}d =\frac{100}{35}\left[9(0.5)^2-2(0.5)(-0.2)+4(-0.2)^2\right] =\frac{100(2.61)}{35}\approx 7.457.

There are two independent restrictions, so the reference distribution is $\chi^2_2$. Its upper-tail probability at 7.457 is $\exp(-7.457/2)\approx0.024$. The off-diagonal covariance matters: the joint quadratic form accounts for the association between the two coefficient estimates.

Worked Example: Testing a Sum of Coefficients

Suppose $n=100$, $\hat\beta=(0.8,0.5)'$, and the hypothesis is $H_0:\beta_{0,1}+\beta_{0,2}=1$. Let $R=(1,1)$, $r=1$, and $\hat V_n=\begin{pmatrix}1&0.2\\0.2&2\end{pmatrix}$.

The estimated departure from the restriction is $R\hat\beta-r=0.8+0.5-1=0.3$. Its estimated asymptotic variance is

R\hat V_nR' =(1,1)\begin{pmatrix}1&0.2\\0.2&2\end{pmatrix} \begin{pmatrix}1\\1\end{pmatrix} =1+0.2+0.2+2=3.4.

Thus $W_n=100(0.3)^2/3.4=9/3.4\approx2.647$. This is a one-restriction test, so its asymptotic reference distribution is $\chi^2_1$, with upper-tail probability about $0.104$. Omitting the covariance terms would incorrectly replace the variance $3.4$ by $1+2=3$.

Null Calibration: The Chi-Square Limit

Theorem: Under the stated random-design assumptions, if $H_0:R\beta_0=r$ is true, then $$ W_n\xrightarrow{d}\chi^2_q. $$

To prove the result, define $A=RVR'$, which is positive definite by the rank and covariance conditions above. Under the null, $R\beta_0=r$, so the restriction error in the statistic is $R(\hat\beta-\beta_0)$. The OLS limit and the continuous linear transformation rule give

\sqrt n\,R(\hat\beta-\beta_0)\xrightarrow{d}N(0,A).

Consistency of $\hat V_n$ gives $R\hat V_nR'\xrightarrow{p}A$. Because $A$ is positive definite, matrix inversion is continuous at $A$, and hence $(R\hat V_nR')^{-1}\xrightarrow{p}A^{-1}$. Slutsky's theorem now gives the limit of the quadratic form:

W_n =\{\sqrt n\,R(\hat\beta-\beta_0)\}'(R\hat V_nR')^{-1} \{\sqrt n\,R(\hat\beta-\beta_0)\} \xrightarrow{d} Z'A^{-1}Z, \qquad Z\sim N(0,A).

Let $A^{-1/2}$ be the symmetric inverse square root of $A$. Then $A^{-1/2}Z\sim N(0,I_q)$, and $Z'A^{-1}Z=\lVert A^{-1/2}Z\rVert^2$. The squared norm of a $q$-dimensional standard normal vector is, by definition, $\chi^2_q$. This proves the claimed calibration. $\square$

How to Use the Calibration

For a level-$\alpha$ asymptotic test, reject $H_0$ when $W_n$ exceeds the $1-\alpha$ quantile of $\chi^2_q$. Equivalently, report the upper-tail probability computed from that reference distribution. The test measures the discrepancy between $R\hat\beta$ and $r$ in units of their estimated joint covariance; a large value indicates that the discrepancy is large relative to its sampling uncertainty.

This construction can use a heteroskedasticity-robust sandwich covariance estimator or a covariance estimator justified by a homoskedastic model. The chi-square argument requires consistency for the covariance appropriate to the assumed data-generating process. The estimator’s label alone does not guarantee this condition. The earlier material on studentized OLS contrasts treats the one-dimensional case; the matrix quadratic form here extends the same covariance-standardization idea to several restrictions at once.

Result: Under a fixed alternative $R\beta_0\ne r$, if $\hat\beta\xrightarrow{p}\beta_0$ and $\hat V_n\xrightarrow{p}V$, then $W_n\xrightarrow{p}\infty$. Thus the Wald test is consistent against every fixed alternative satisfying the regularity and rank conditions.

For this claim, set $\delta=R\beta_0-r\ne0$. The restriction error $R\hat\beta-r\xrightarrow{p}\delta$, while $(R\hat V_nR')^{-1}\xrightarrow{p}A^{-1}$. Therefore,

\frac{W_n}{n} =(R\hat\beta-r)'(R\hat V_nR')^{-1}(R\hat\beta-r) \xrightarrow{p}\delta'A^{-1}\delta>0.

The strict inequality follows because $A^{-1}$ is positive definite and $\delta\ne0$. Hence for any fixed $M>0$, $P(W_n>M)\to1$: with probability approaching one, $W_n/n$ is bounded below by a positive constant, while $n$ grows without bound. $\square$

Common Mistakes and Practical Interpretation

Using $p$ degrees of freedom automatically. The reference degrees of freedom are $q=\operatorname{rank}(R)$, the number of independent restrictions. A test of two coefficients among ten regressors has two degrees of freedom, not ten.
Ignoring covariance between estimates. For a sum or a joint restriction, the off-diagonal entries of the covariance matrix contribute to the variance of the restriction error. The third worked example shows that dropping them changes the statistic.
Using a non-independent set of restrictions. If the rows of $R$ are linearly dependent, then $R\hat V_nR'$ is singular in the limit. Remove redundant restrictions or express the hypothesis with a full-row-rank matrix before applying this formula.
Calling the chi-square result exact. The theorem establishes convergence in distribution as $n$ grows. A chi-square approximation can be useful in finite samples, but the asymptotic argument does not make it an exact finite-sample test.
Using a covariance estimator inconsistent for the design. The proof depends on $\hat V_n\xrightarrow{p}V$. If the covariance estimate targets the wrong limiting covariance, studentizing the contrast does not repair the mismatch.

The Wald form is convenient because it tests restrictions by evaluating the unrestricted estimate and its covariance. Its interpretation is clearest when the restrictions are scientifically meaningful, the rows of $R$ are independent, and the covariance estimator matches the assumptions of the regression analysis.

Check Your Understanding

Answer each question using the definitions and results above.

If $R$ has four linearly independent rows, what is the asymptotic null reference distribution of the Wald statistic?
Why must $RVR'$ be positive definite under the theorem’s rank and covariance assumptions?
For a test of $H_0:\beta_1+\beta_2=0$, which entries of the estimated covariance matrix affect the estimated variance of the restriction?
What consistency property of $\hat V_n$ is used in the chi-square proof?
Why does the Wald statistic diverge under a fixed alternative, even though the coefficient estimator itself converges to a finite vector?