Joint Restrictions With Robust Wald Tests

Why Test Restrictions Jointly?

A regression analysis may ask whether several coefficients satisfy specified conditions at once. For example, a researcher might ask whether two policy indicators have no association with an outcome, or whether two slopes are equal. Testing each coefficient separately does not, in general, answer whether the full set of conditions holds: the estimates can be correlated, and the joint question concerns their combined distance from the restrictions.

A heteroskedasticity-robust Wald test measures that distance using the estimated covariance of the restricted quantities. Its large-sample reference distribution is a chi-square distribution whose degrees of freedom equal the number of independent restrictions. This is an asymptotic calibration. It differs from the exact finite-sample F calibration available for the classical test under homoskedastic normal errors.

Model and Joint Restrictions

Consider the linear regression model $y_i=x_i'\beta+u_i$, for observations $i=1,\ldots,n$. Here $y_i$ is the outcome, $x_i$ is a $k$-vector of regressors, $\beta$ is a $k$-vector of coefficients, and $u_i$ is the error. The OLS estimator is $\hat{\beta}$. For robust inference, assume observations are independent, the conditional mean satisfies $E[u_i\mid x_i]=0$, the regressor second moments are well behaved and identify the coefficients, and the errors have finite moments sufficient for the relevant large-sample results. The conditional variance of $u_i$ may vary with $x_i$; constant variance is not required.

Write a collection of linear restrictions as $R\beta=r$, where $R$ is a known $q\times k$ matrix of rank $q$, and $r$ is a known $q$-vector. The null hypothesis asserts that all $q$ restrictions hold; the alternative is that at least one does not. For example, setting two coefficients to zero gives two rows in $R$, each selecting one coefficient. The rank condition means the restrictions are independent rather than redundant.

Definition: Let $\widehat V_{\mathrm{rob}}$ be a heteroskedasticity-robust covariance estimator for $\hat{\beta}$. The robust Wald statistic for $H_0:R\beta=r$ is $$ W=(R\hat{\beta}-r)'\left(R\widehat V_{\mathrm{rob}}R'\right)^{-1}(R\hat{\beta}-r). $$ The matrix $R\widehat V_{\mathrm{rob}}R'$ estimates the covariance matrix of the $q$ estimated restrictions. It must be nonsingular for this statistic to be defined.

The statistic is a squared, covariance-scaled distance: a discrepancy counts more when it is large relative to its estimated uncertainty, and correlation among the restricted quantities is taken into account. The robust covariance estimator can, for example, be the HC0 sandwich estimator introduced in earlier material. The test's robust interpretation depends on using a covariance estimator that consistently estimates the sampling covariance under the allowed heteroskedasticity.

Why the Reference Distribution Is Chi-Square

Under the null and the stated regularity conditions, the vector of estimated restrictions, scaled by its covariance, is asymptotically standard multivariate normal. More explicitly, if $S=R\widehat V_{\mathrm{rob}}R'$, then the Wald quadratic form behaves like the sum of squares of $q$ independent standard normal variables. That sum has a chi-square distribution with $q$ degrees of freedom.

Theorem: If $H_0:R\beta=r$ is true, $R$ has rank $q$, and the robust covariance estimator consistently estimates the asymptotic covariance of $\hat{\beta}$, then $$ W \xrightarrow{d} \chi_q^2. $$ Thus the asymptotic p-value is $P(\chi_q^2\geq W_{\mathrm{obs}})$. The degrees of freedom are the number $q$ of independent restrictions, not the number of regressors or the sample size.

A short justification makes the role of $q$ explicit. Under the null, $R\hat{\beta}-r$ is asymptotically centered normal with covariance estimated by $S$. Standardizing by the inverse square root of $S$ gives an asymptotic $q$-vector of independent standard normal variables. The Wald statistic is the squared length of that standardized vector. Since the sum of squares of $q$ independent standard normal variables has distribution $\chi_q^2$, the stated limit follows.

The result is asymptotic, not an assertion that the statistic has an exact chi-square distribution in a finite sample. A small p-value indicates that, if all restrictions were true and the model and inference assumptions held, a statistic at least this large would be unusual under the asymptotic reference distribution. It is not the probability that the null is true.

Worked Example: Testing Two Coefficients Jointly

Suppose a hypothetical regression estimates two policy coefficients as $\hat{\beta}_1=1.000$ and $\hat{\beta}_2=1.000$, in outcome units per policy-unit. The robust estimated covariance matrix for these two coefficients is diagonal, with variances $1.000$ and $1.400$. Test $H_0:\beta_1=0,\ \beta_2=0$ using the asymptotic robust Wald test.

Set up the restrictions. Here $R$ selects the two coefficients and $r=(0,0)'$. The estimated restriction vector is $R\hat{\beta}-r=(1,1)'$, and its estimated covariance is $$ S=R\widehat V_{\mathrm{rob}}R'= \begin{pmatrix}1.000&0\\0&1.400\end{pmatrix}. $$

Calculate the statistic. Because $S$ is diagonal, its inverse has diagonal entries $1$ and $1/1.4$: $$ W=(1,1) \begin{pmatrix}1&0\\0&1/1.4\end{pmatrix} \begin{pmatrix}1\\1\end{pmatrix} =1+\frac{1}{1.4}=1.7143. $$

There are $q=2$ independent restrictions, so the reference distribution is $\chi_2^2$. For two degrees of freedom its upper-tail probability at $w$ is $\exp(-w/2)$, giving $$ P(\chi_2^2\geq1.7143)=\exp(-1.7143/2)\approx0.4244. $$ At a 5% significance level, the test does not reject the joint null. This result is not evidence that both coefficients are exactly zero; it says the observed joint discrepancy is not unusually large under this asymptotic test.

Interpreting Joint Tests and Covariance

The Wald statistic uses the covariance of the restrictions, not just their individual standard errors. When estimated restrictions are correlated, the off-diagonal entries of $R\widehat V_{\mathrm{rob}}R'$ affect the distance. Consequently, adding separate squared t statistics is generally not a valid way to obtain the joint Wald statistic. That shortcut agrees with the Wald calculation only in special cases, such as an appropriate diagonal restriction covariance matrix.

The test is also invariant to equivalent ways of expressing the same restrictions. If the rows of $R$ and the entries of $r$ are transformed together by a nonsingular $q\times q$ matrix, the null set is unchanged and the Wald quadratic form is unchanged. This is why the substantive content of the restrictions, and their rank, matter more than their particular algebraic representation.

Worked Example: A Different Joint Discrepancy

In another hypothetical regression, the estimated deviations from two null restrictions are $1.000$ and $0.500$. Suppose their robust covariance matrix is diagonal with entries $1.000$ and $2.1875$. Test both restrictions jointly.

The restriction covariance is nonsingular, and the statistic is $$ W= \begin{pmatrix}1&0.5\end{pmatrix} \begin{pmatrix}1&0\\0&1/2.1875\end{pmatrix} \begin{pmatrix}1\\0.5\end{pmatrix} =1+\frac{0.25}{2.1875} =1.1143. $$

Again $q=2$, so the asymptotic p-value is $$ P(\chi_2^2\geq1.1143)=\exp(-1.1143/2)\approx0.5728. $$ At the 5% level, there is insufficient evidence to reject the joint restrictions. The second discrepancy contributes relatively little because its estimated variance is comparatively large. This is a statement about the scaled joint discrepancy, not a claim that either coefficient has a particular probability of being null.

Robust Wald Calibration and the Classical F Test

The classical homoskedastic F test and the robust Wald test address related restriction hypotheses, but their calibrations rely on different assumptions. Under the classical linear model with a correctly specified conditional mean, homoskedastic errors, and normally distributed errors conditional on the regressors, the usual RSS-based statistic for $q$ restrictions has an exact finite-sample $F(q,n-k)$ distribution under the null. Here $n-k$ is the residual degrees of freedom in a model with $k$ estimated coefficients.

The robust Wald statistic instead uses a heteroskedasticity-robust covariance estimate. Its large-sample reference is $\chi_q^2$. Equivalently, $W/q$ has the large-sample reference $\chi_q^2/q$, often denoted $F(q,\infty)$. Replacing the infinite denominator degrees of freedom with $n-k$ to form a finite-sample F approximation is a convention sometimes used by software, but it does not make the robust test an exact classical F test. The exact result for the classical F statistic depends on the homoskedastic normal model and its conventional variance estimate.

Worked Example: Classical F Calibration Under Its Assumptions

Consider hypothetical data with $n=80$ observations and $k=5$ estimated coefficients. For a joint test of $q=2$ restrictions, suppose the restricted and unrestricted residual sums of squares are $RSS_R=120$ and $RSS_U=110$, respectively. For this example, assume the classical homoskedastic normal linear model.

The classical statistic is $$ F=\frac{(RSS_R-RSS_U)/q}{RSS_U/(n-k)} =\frac{(120-110)/2}{110/(80-5)} =\frac{5}{110/75} =3.4091. $$

Under the stated classical assumptions and the null, this statistic has the exact $F(2,75)$ distribution. The numerator degrees of freedom equal the number of restrictions, while the denominator degrees of freedom equal the unrestricted residual degrees of freedom. If errors are heteroskedastic, that exact calibration is not justified merely because the formula can still be computed; a robust Wald procedure uses a robust covariance estimate and asymptotic chi-square calibration instead.

Common Mistakes and Practical Checks

Using the wrong degrees of freedom. The robust Wald reference is $\chi_q^2$, with $q$ equal to the rank of the restriction matrix. It is not $\chi_k^2$ unless the number of independent restrictions happens to equal the number of coefficients.
Adding separate test statistics without checking covariance. The joint statistic must use the full restriction covariance matrix, including off-diagonal elements when present.
Calling the robust test an exact F test. Robust Wald inference is generally asymptotic. The classical exact F result requires the classical homoskedastic normal assumptions.
Interpreting non-rejection as proof of the null. A large p-value means the data do not provide sufficient evidence against the joint restrictions at the chosen significance level; it does not establish that the restrictions are true.
Using a singular restriction covariance. The inverse in the Wald formula requires independent restrictions with a nonsingular estimated covariance. Redundant restrictions should be removed or expressed using a full-rank system.

Key takeaway: For $q$ independent linear restrictions, compare the robust covariance-scaled discrepancy $W$ with $\chi_q^2$ asymptotically. The restriction count determines the degrees of freedom; the heteroskedasticity-robust covariance determines how discrepancies and their dependence are scaled.

Check Your Understanding

Work through these questions using the definitions and assumptions stated above.

A test imposes three independent coefficient restrictions. Which asymptotic reference distribution applies to its robust Wald statistic, and what determines its degrees of freedom?
Why does the joint Wald statistic generally require the full covariance matrix of the estimated restrictions rather than only their individual standard errors?
A robust Wald test has $W=6.00$ for two restrictions. Write the expression for its asymptotic p-value without evaluating it.
What assumptions permit the usual RSS-based classical F statistic to have an exact $F(q,n-k)$ null distribution?
Explain why using a finite denominator degrees-of-freedom approximation with a robust covariance estimate does not produce an exact classical F test.