The Wald Test for Vector Restrictions

Why Test Restrictions Together?

A parameter vector often has several features that must be assessed jointly. For example, a regression analysis might ask whether two coefficients are both zero, or whether a coefficient equals another while a third coefficient is zero. Testing each statement separately does not answer the joint question: the estimates may be correlated, and the joint test must account for that dependence.

The vector Wald test measures how far the estimated parameter lies from the null restrictions, relative to its estimated sampling variability. The resulting statistic is a covariance-adjusted quadratic distance. Its calculation requires the estimate, the restrictions, and the covariance matrix of the estimate.

Definition: Let $\theta$ be a $p$-dimensional parameter and let $\hat{\theta}$ estimate it. A set of $q$ linear restrictions can be written $H_0:R\theta=r$, where $R$ is a known $q\times p$ matrix with linearly independent rows and $r$ is a known $q$-vector. Write $\widehat V$ for an estimated covariance matrix of $\hat{\theta}$, on the scale of $\hat{\theta}$ itself. When $R\widehat V R^\mathsf{T}$ is invertible, the Wald statistic is $$ W=(R\hat{\theta}-r)^\mathsf{T}(R\widehat V R^\mathsf{T})^{-1}(R\hat{\theta}-r). $$

The vector $R\hat{\theta}-r$ contains the estimated departures from the null. Its covariance estimate is $R\widehat V R^\mathsf{T}$, not merely the diagonal variances of those departures. The inverse covariance matrix scales and combines the departures while accounting for their correlations.

Worked Wald Tests

Worked Example: Testing Two Coefficients Jointly

Suppose a large-sample regression analysis estimates two coefficients, $\theta=(\theta_1,\theta_2)^\mathsf{T}$, by $\hat{\theta}=(1.2,-0.4)^\mathsf{T}$. The estimated covariance matrix of this estimate is $$ \widehat V= \begin{pmatrix} 0.36&0.12\\ 0.12&0.25 \end{pmatrix}. $$ Test $H_0:\theta_1=0,\ \theta_2=0$. Here $R$ is the $2\times2$ identity matrix and $r=(0,0)^\mathsf{T}$, so $R\hat{\theta}-r=(1.2,-0.4)^\mathsf{T}$ and $R\widehat V R^\mathsf{T}=\widehat V$.

The determinant of $\widehat V$ is $0.36(0.25)-0.12^2=0.0756$, so its inverse is $$ \widehat V^{-1}=\frac{1}{0.0756} \begin{pmatrix} 0.25&-0.12\\ -0.12&0.36 \end{pmatrix}. $$

Therefore, $$ W=\frac{0.25(1.2)^2+2(-0.12)(1.2)(-0.4)+0.36(-0.4)^2}{0.0756} =\frac{0.36+0.1152+0.0576}{0.0756} \approx 7.048. $$ Under the null, the large-sample reference distribution is $\chi^2_2$. For two degrees of freedom, the upper-tail probability at $w$ is $\exp(-w/2)$, giving $p=\exp(-7.048/2)\approx0.0295$. At significance level $0.05$, reject the joint null.

The off-diagonal covariance matters: omitting it would replace the joint covariance with a different matrix and generally change the statistic. The calculation tests the two restrictions together, not as two separate Wald tests.

Why the Statistic Is a Covariance-Adjusted Distance

A useful way to understand the quadratic form is to ask for the smallest change to the estimate, measured in covariance units, that would satisfy the null restrictions. The following derivation makes this interpretation precise.

Result: Let $V$ be positive definite, let $R$ have $q$ linearly independent rows, and let $d$ be a $q$-vector. Among all $p$-vectors $u$ satisfying $Ru=d$, the minimum of $u^\mathsf{T}V^{-1}u$ is $d^\mathsf{T}(RVR^\mathsf{T})^{-1}d$.

To prove the result, set $S=RVR^\mathsf{T}$. It is positive definite: for nonzero $a\in\mathbb{R}^q$, $R^\mathsf{T}a\ne0$ because $R$ has full row rank, and hence $a^\mathsf{T}Sa=(R^\mathsf{T}a)^\mathsf{T}V(R^\mathsf{T}a)>0$. Thus $S^{-1}$ exists. Define $u_*=VR^\mathsf{T}S^{-1}d$. Then $Ru_*=SS^{-1}d=d$, so $u_*$ satisfies the constraint.

For any other feasible $u$, write $u=u_*+v$, where $Rv=0$. The cross term vanishes because $v^\mathsf{T}V^{-1}u_*=v^\mathsf{T}R^\mathsf{T}S^{-1}d=(Rv)^\mathsf{T}S^{-1}d=0$. Consequently, $$ u^\mathsf{T}V^{-1}u =u_*^\mathsf{T}V^{-1}u_*+v^\mathsf{T}V^{-1}v \geq u_*^\mathsf{T}V^{-1}u_*. $$ Finally, $u_*^\mathsf{T}V^{-1}u_*=d^\mathsf{T}S^{-1}RVR^\mathsf{T}S^{-1}d=d^\mathsf{T}S^{-1}d$, proving the result. $\square$

For a Wald test, take $d=R\hat{\theta}-r$ and $V=\widehat V$. The statistic is the least covariance-scaled adjustment that would move the estimate onto the restriction set. A departure in a direction with large sampling variance counts less than the same-sized departure in a direction estimated precisely.

Worked Example: Equality and Zero Restrictions

Suppose a regression has $\hat{\theta}=(0.5,0.2,-0.4)^\mathsf{T}$ and estimated covariance matrix $$ \widehat V= \begin{pmatrix} 0.04&0&0\\ 0&0.09&0\\ 0&0&0.16 \end{pmatrix}. $$ Test $H_0:\theta_1-\theta_2=0,\ \theta_3=0$. Use $$ R=\begin{pmatrix}1&-1&0\\0&0&1\end{pmatrix},\qquad r=\begin{pmatrix}0\\0\end{pmatrix}. $$ The observed restriction departures are $R\hat{\theta}-r=(0.3,-0.4)^\mathsf{T}$. Multiplying out gives $$ R\widehat V R^\mathsf{T}= \begin{pmatrix}0.13&0\\0&0.16\end{pmatrix}. $$ The first variance is $0.04+0.09=0.13$, since the first two estimates have zero covariance here. Thus $$ W=\frac{0.3^2}{0.13}+\frac{(-0.4)^2}{0.16} =\frac{0.09}{0.13}+1 \approx1.692. $$ With two restrictions, the reference distribution is $\chi^2_2$, and $p=\exp(-1.692/2)\approx0.429$. The data do not provide evidence against the joint null at the $0.05$ level.

Equivalent Ways to Write the Same Restrictions

The numerical form of the restriction equations can change without changing the hypothesis. For example, multiplying an equation by a nonzero constant changes its written form but not the set of parameter values it permits. The Wald statistic respects any invertible recoding of the full restriction vector.

Result: If $A$ is an invertible $q\times q$ matrix, replacing $R\theta=r$ by $AR\theta=Ar$ leaves the Wald statistic unchanged.

To verify this, write $d=R\hat{\theta}-r$ and $S=R\widehat V R^\mathsf{T}$. Under the recoding, the departure vector is $Ad$ and its estimated covariance is $ASA^\mathsf{T}$. Since $A$ is invertible, $$(ASA^\mathsf{T})^{-1}=(A^\mathsf{T})^{-1}S^{-1}A^{-1}.$$ Substituting into the recoded quadratic form gives $$(Ad)^\mathsf{T}(ASA^\mathsf{T})^{-1}(Ad) =d^\mathsf{T}A^\mathsf{T}(A^\mathsf{T})^{-1}S^{-1}A^{-1}Ad =d^\mathsf{T}S^{-1}d.$$ This is the original statistic, as claimed. $\square$

Worked Example: Testing Two Nonzero Targets

Suppose $\hat{\theta}=(0.8,0.5)^\mathsf{T}$ and $$ \widehat V=\begin{pmatrix}0.04&0.01\\0.01&0.09\end{pmatrix}. $$ Test the joint hypothesis $\theta_1+\theta_2=1,\ \theta_1-\theta_2=0$, using $$ R=\begin{pmatrix}1&1\\1&-1\end{pmatrix},\qquad r=\begin{pmatrix}1\\0\end{pmatrix}. $$ The departure vector is $d=(0.8+0.5-1,\ 0.8-0.5)^\mathsf{T}=(0.3,0.3)^\mathsf{T}$. The transformed covariance is $$ S=R\widehat V R^\mathsf{T}=\begin{pmatrix}0.15&-0.05\\-0.05&0.11\end{pmatrix}. $$ Its determinant is $0.15(0.11)-(-0.05)^2=0.014$, so $$ S^{-1}=\frac{1}{0.014}\begin{pmatrix}0.11&0.05\\0.05&0.15\end{pmatrix}. $$ It follows that $$ W=\frac{0.11(0.3)^2+2(0.05)(0.3)(0.3)+0.15(0.3)^2}{0.014} =\frac{0.0324}{0.014}\approx2.314. $$ The $\chi^2_2$ upper-tail p-value is $\exp(-2.314/2)\approx0.314$. The joint null is not rejected at level $0.05$.

Because $R$ is invertible, these restrictions could also be written as the equivalent targets $\theta_1=0.5,\ \theta_2=0.5$. The invariance result shows that using those equations would give exactly the same statistic.

Calibration, Assumptions, and Common Mistakes

For a regular large-sample Wald test, suppose $\hat{\theta}$ is consistent for $\theta$, $\sqrt n(\hat{\theta}-\theta)\xrightarrow{d}N(0,\Sigma)$ with positive-definite $\Sigma$, and $n\widehat V\xrightarrow{p}\Sigma$. Under $H_0:R\theta=r$, with $q$ linearly independent restrictions and the covariance matrix in the stated scale, the statistic is calibrated against $\chi^2_q$. Its p-value is the probability, under that reference distribution, of a value at least as large as the observed $W$. This is a large-sample calibration, not a guarantee of an exact finite-sample distribution.

The scale of the covariance estimate is essential. If software reports an estimate $\widehat\Sigma$ of the covariance of $\sqrt n(\hat{\theta}-\theta)$, rather than the covariance of $\hat{\theta}$, then $\widehat V=\widehat\Sigma/n$. Equivalently, the statistic can be computed as $$ W=n(R\hat{\theta}-r)^\mathsf{T}(R\widehat\Sigma R^\mathsf{T})^{-1}(R\hat{\theta}-r). $$

A frequent error is to treat restrictions as independent just because they are listed separately. Even if the original parameter estimates have a covariance matrix with simple entries, transformed restrictions may be correlated. Always calculate the entire matrix $R\widehat V R^\mathsf{T}$, including its off-diagonal terms, before inverting it. Also check that the restriction rows are linearly independent and that the resulting covariance matrix is invertible; otherwise the displayed Wald formula cannot be used as written.

Joint Wald tests matter whenever the scientific question concerns a collection of restrictions rather than one coefficient at a time. The quadratic form provides one statistic for that collection, while retaining the covariance information needed for a valid large-sample comparison.

Check Your Understanding

Work through these questions using the definitions and calculations above.

For $H_0:\theta_1=0,\theta_2=0$, what are $R$ and $r$ when $\theta=(\theta_1,\theta_2)^\mathsf{T}$?
Why must the covariance matrix in the Wald formula include the covariance between the restriction estimates?
If software reports the estimated covariance of $\sqrt n(\hat{\theta}-\theta)$, how should it be scaled to use the formula based on $\widehat V$?
What is the chi-square degrees of freedom when $R$ has $q$ linearly independent rows and the null is true under the stated regularity conditions?
Why does multiplying the restriction vector by an invertible matrix leave the Wald statistic unchanged?