Deriving Heteroskedasticity-Robust Wald Tests

Why a Robust Wald Test Has a Chi-Square Limit

A Wald test asks how far an estimated coefficient vector lies from a set of values permitted by a null hypothesis, measured relative to its estimated sampling uncertainty. When errors are heteroskedastic, the uncertainty must be estimated with a heteroskedasticity-robust covariance estimator. The central result is that, under the null and suitable large-sample conditions, the resulting quadratic distance has an asymptotic chi-square distribution.

The result is not a finite-sample guarantee. It is an approximation whose justification depends on asymptotic normality, a consistent covariance estimator, and restrictions that are genuinely independent. These conditions determine both the formula and the appropriate degrees of freedom.

Model, Null, and Wald Statistic

Let observations $(y_i,x_i)$, $i=1,\ldots,n$, be independent and identically distributed, with $x_i$ a $k$-vector of regressors. Consider the linear regression representation $y_i=x_i'\beta_0+u_i$, where $\beta_0$ is the population coefficient vector and $E[x_i u_i]=0$. Conditional variances of $u_i$ may depend on $x_i$; homoskedasticity is not required. The OLS estimator is denoted $\hat\beta$.

A null hypothesis of $q$ linear restrictions is written $H_0:R\beta_0=r$. Here $R$ is a fixed $q\times k$ matrix, $r$ is a fixed $q$-vector, and $\beta_0$ is $k$-dimensional. The rows of $R$ must be linearly independent, so $\operatorname{rank}(R)=q$. The alternative is that at least one of these restrictions does not hold.

Definition: Let $\widehat V$ be a heteroskedasticity-robust estimator of the covariance matrix of $\hat\beta$. The robust Wald statistic for $H_0:R\beta_0=r$ is $$ W=(R\hat\beta-r)'[R\widehat V R']^{-1}(R\hat\beta-r). $$ If the assumptions below hold and $H_0$ is true, $W$ converges in distribution to a chi-square random variable with $q$ degrees of freedom.

The matrix $R\widehat V R'$ estimates the covariance matrix of the restricted estimates $R\hat\beta$. It is $q\times q$, and its inverse scales the discrepancy $R\hat\beta-r$ by its uncertainty, accounting for covariance among the restrictions. The number $q$, not the number of coefficients $k$, determines the chi-square degrees of freedom.

Regularity Conditions and the Robust Covariance

The asymptotic argument can be stated in terms of the estimator and its covariance, then connected to OLS. Suppose:

The observations are independent and identically distributed, and the relevant moments are finite enough for the law of large numbers and central limit theorem to apply.
The regressor second-moment matrix $Q=E[x_i x_i']$ is finite and positive definite, and the OLS moment condition $E[x_i u_i]=0$ holds.
The score covariance $\Omega=E[x_i x_i'u_i^2]$ is finite and positive definite, allowing heteroskedasticity across observations.
The robust covariance estimator is consistent in the sense that $n\widehat V$ converges in probability to $V=Q^{-1}\Omega Q^{-1}$.
The restriction matrix $R$ has full row rank $q$. Consequently, $RVR'$ is positive definite and nonsingular.

Under these conditions, the OLS estimator is asymptotically normal:

\sqrt{n}(\hat\beta-\beta_0)\xrightarrow{d}N(0,V), \qquad n\widehat V\xrightarrow{p}V.

The robust covariance estimator need not assert that the variance of the error is constant. For example, the HC0 sandwich estimator has the form $(X'X)^{-1}\left(\sum_{i=1}^n x_i x_i'\hat u_i^2\right)(X'X)^{-1}$, where $X$ is the regressor matrix and $\hat u_i$ is the OLS residual. Under the conditions above, it estimates the covariance of $\hat\beta$ consistently in large samples. The Wald result depends on this consistency, not on a particular finite-sample correction.

Derivation of the Chi-Square Reference Distribution

Under $H_0$, $R\beta_0=r$. Therefore the restricted estimation error is $R\hat\beta-r=R(\hat\beta-\beta_0)$. Applying the fixed linear transformation $R$ to the asymptotic normal limit gives a $q$-dimensional normal limit with covariance $RVR'$.

\sqrt{n}(R\hat\beta-r) =R\sqrt{n}(\hat\beta-\beta_0) \xrightarrow{d}N(0,RVR').

The estimated covariance for this scaled vector is $R(n\widehat V)R'$, which converges in probability to $RVR'$. Since $RVR'$ is positive definite, it has an inverse square root. Standardizing the vector by that inverse square root gives a limiting $q$-vector of independent standard normal variables. The estimated covariance can be used in place of the limiting covariance because it converges to it.

Form the scaled discrepancy.
Under the null, $\sqrt{n}(R\hat\beta-r)$ converges to a normal vector with covariance $RVR'$.

Estimate its covariance.
The matrix $R(n\widehat V)R'$ converges in probability to $RVR'$, so substituting it preserves the limiting distribution.

Take the squared length.
For a standard normal $q$-vector $Z$, $Z'Z$ is the sum of $q$ independent squared standard normals, hence has a $\chi_q^2$ distribution.

In particular, the original statistic can be rewritten as

W= \{\sqrt{n}(R\hat\beta-r)\}' \{R(n\widehat V)R'\}^{-1} \{\sqrt{n}(R\hat\beta-r)\}.

The limiting quadratic form is the squared length of the standardized normal vector. Therefore, under $H_0$,

W\xrightarrow{d}\chi_q^2.

This derivation uses asymptotic normality, covariance consistency, and nonsingularity. If the restrictions are redundant, $R$ has rank below $q$, the restriction covariance is singular, and the displayed inverse does not exist. The number of independent restrictions, rather than the number of written equations, is essential.

Worked Example: Two Restrictions With Correlated Estimates

Suppose a hypothetical regression estimates two contrasts, and the null sets both to zero. The estimated contrast vector is $d=R\hat\beta-r=(1.20,-0.40)'$. Suppose its robust covariance estimate is $S=R\widehat V R'=\begin{pmatrix}0.25&0.05\\0.05&0.16\end{pmatrix}$. The entries are in squared contrast units. The determinant is $0.25(0.16)-0.05^2=0.0375>0$, so $S$ is invertible.

For a symmetric $2\times2$ matrix, the inverse is its adjugate divided by its determinant. Thus

S^{-1}=\frac{1}{0.0375} \begin{pmatrix} 0.16&-0.05\\ -0.05&0.25 \end{pmatrix} = \begin{pmatrix} 4.2667&-1.3333\\ -1.3333&6.6667 \end{pmatrix}.

The statistic is

\begin{aligned} W&=d'S^{-1}d\\ &=(1.20,-0.40) \begin{pmatrix} 4.2667&-1.3333\\ -1.3333&6.6667 \end{pmatrix} \begin{pmatrix} 1.20\\-0.40 \end{pmatrix}\\ &=(1.20,-0.40) \begin{pmatrix} 5.6533\\-4.2667 \end{pmatrix} \approx 8.4907. \end{aligned}

There are $q=2$ independent restrictions, so the asymptotic reference distribution is $\chi_2^2$. For two degrees of freedom, the upper-tail probability at $w$ is $\exp(-w/2)$. The asymptotic p-value is therefore $P(\chi_2^2\ge 8.4907)=\exp(-8.4907/2)\approx0.0143$. At significance level $\alpha=0.05$, this is evidence against the joint null; it does not establish which particular restriction fails.

Worked Example: A Smaller Wald Statistic

In another hypothetical analysis, let the estimated discrepancy be $d=(0.20,-0.20)'$ and its robust covariance estimate be $S=\begin{pmatrix}0.04&0.01\\0.01&0.09\end{pmatrix}$. The determinant is $0.04(0.09)-0.01^2=0.0035>0$. Inverting gives

S^{-1}=\frac{1}{0.0035} \begin{pmatrix} 0.09&-0.01\\ -0.01&0.04 \end{pmatrix} = \begin{pmatrix} 25.7143&-2.8571\\ -2.8571&11.4286 \end{pmatrix}.

Multiplying by the discrepancy and taking the dot product,

\begin{aligned} S^{-1}d&= \begin{pmatrix} 25.7143&-2.8571\\ -2.8571&11.4286 \end{pmatrix} \begin{pmatrix}0.20\\-0.20\end{pmatrix} = \begin{pmatrix}5.7143\\-2.8571\end{pmatrix},\\ W&=(0.20,-0.20) \begin{pmatrix}5.7143\\-2.8571\end{pmatrix} \approx1.7143. \end{aligned}

With two restrictions, $W$ is compared with $\chi_2^2$, giving $P(\chi_2^2\ge1.7143)=\exp(-1.7143/2)\approx0.4244$. At the 0.05 level, the data do not provide sufficient evidence to reject the joint null. This conclusion is not proof that both restrictions are true.

Worked Example: One Linear Restriction

Suppose a hypothetical regression tests a single contrast, $H_0:R\beta_0=r$. Its estimated discrepancy is $R\hat\beta-r=0.30$, and the robust estimated variance of that contrast is $R\widehat V R'=0.0144$. The scalar Wald statistic is

W=\frac{(0.30)^2}{0.0144} =\frac{0.09}{0.0144} =6.25.

Here $q=1$, so the reference distribution is $\chi_1^2$. Since a squared standard normal has this distribution, $P(\chi_1^2\ge6.25)=2[1-\Phi(2.5)]\approx0.0124$, where $\Phi$ is the standard normal cumulative distribution function. This is an asymptotic p-value; at $\alpha=0.05$, the null is rejected under the stated approximation.

Interpretation and Common Pitfall

The Wald statistic is dimensionless: discrepancies are divided by their estimated uncertainty through the inverse covariance matrix. Its p-value is the probability, assuming the null and the model’s regularity conditions, of obtaining a statistic at least as large as the observed one under the asymptotic reference distribution. It is not the probability that the null hypothesis is true.

Common mistake: Counting the number of coefficient values involved and using that number for the degrees of freedom. For instance, two restrictions on a four-coefficient regression still give $q=2$, provided the two rows of $R$ are independent. Conversely, writing the same restriction twice does not create two independent restrictions. Determine the rank of $R$, verify that the restriction covariance is nonsingular, and use the number of independent restrictions for the chi-square degrees of freedom.

Another error is to treat the asymptotic chi-square result as exact in a small sample. The result is a large-sample approximation. Robust covariance estimation protects the variance calculation against heteroskedasticity under the stated conditions, but does not remove the need for adequate sample information or justify using the approximation without regard to the design.

Key takeaway: Under a true full-rank null, asymptotic normality of the estimator and consistency of its robust covariance estimator make the standardized restriction error asymptotically standard normal. Its squared length is therefore asymptotically $\chi_q^2$, where $q=\operatorname{rank}(R)$.

Check Your Understanding

Use the conditions and derivation above to answer these questions.

In $H_0:R\beta_0=r$, what determines the asymptotic chi-square degrees of freedom?
Why is $RVR'$ required to be nonsingular for the usual Wald statistic?
Under the null, what is the limiting distribution of $\sqrt{n}(R\hat\beta-r)$?
If three equations in a restriction matrix have rank two, should the usual test use two or three degrees of freedom? Explain.
What does a p-value of 0.04 mean for a robust Wald test, and what does it not mean?