Robust Wald Tests for Vector Parameters

Why Robust Estimation Needs Robust Testing

A robust estimator can limit the influence of unusual observations on a parameter estimate. But a hypothesis test also depends on how uncertain that estimate is. If its covariance is estimated incorrectly, the resulting test can be too liberal or too conservative, even when the point estimate is robust. The Wald test combines a vector estimate, a set of linear restrictions, and a covariance estimate to quantify how far the estimate lies from the null hypothesis.

The key is to use the asymptotic covariance of the robust estimator, which typically reflects both the estimator's sensitivity to observations and the variability of its estimating contributions. Under regularity conditions, this yields a chi-square test of a fixed number of restrictions. The result is asymptotic: it does not promise exact calibration in a particular finite sample.

Setup and Definition

Let $\theta\in\mathbb{R}^p$ be the vector parameter of interest, and let $\hat{\theta}$ be a robust estimator based on $n$ independent observations. A linear null hypothesis can be written $H_0:A\theta=b$, where $A$ is a fixed $q\times p$ matrix, $b\in\mathbb{R}^q$, and $A$ has row rank $q$. Thus there are $q$ linearly independent restrictions.

Assume the estimator has an asymptotic linear representation $\sqrt{n}(\hat{\theta}-\theta)\xrightarrow{d}N(0,V)$, where $V$ is a positive-definite $p\times p$ matrix. A common sufficient form of this representation is $\sqrt{n}(\hat{\theta}-\theta)=n^{-1/2}\sum_{i=1}^n\operatorname{IF}(X_i;F)+o_p(1)$, where the influence-function vector has mean zero and finite covariance $V=E[\operatorname{IF}(X;F)\operatorname{IF}(X;F)']$. Here $F$ denotes the sampling distribution and $X_i$ are independent observations from it. Let $\hat V$ consistently estimate $V$, so $\hat V\xrightarrow{p}V$.

Definition: The robust Wald statistic for $H_0:A\theta=b$ is $$ W_n=n(A\hat{\theta}-b)'(A\hat V A')^{-1}(A\hat{\theta}-b). $$ It is defined when $A\hat V A'$ is invertible. Under the null and the stated asymptotic conditions, its reference distribution is chi-square with $q$ degrees of freedom.

The factor $n$ is present because $V$ estimates the covariance of $\sqrt{n}(\hat{\theta}-\theta)$, not the covariance of $\hat{\theta}$ itself. If software instead reports $\hat C$, an estimate of $\operatorname{Cov}(\hat{\theta})$, then typically $\hat C=\hat V/n$, and the equivalent statistic is $(A\hat{\theta}-b)'(A\hat C A')^{-1}(A\hat{\theta}-b)$. Check which covariance convention is being used before applying a formula.

Deriving the Chi-Square Calibration

Theorem: Suppose $A$ has row rank $q$, $A\theta=b$, $\sqrt{n}(\hat{\theta}-\theta)\xrightarrow{d}N(0,V)$, and $\hat V\xrightarrow{p}V$, where $A V A'$ is positive definite. Then $$ W_n\xrightarrow{d}\chi^2_q. $$

Proof. Under $H_0$, $A\theta=b$, so $\sqrt{n}(A\hat{\theta}-b)=A\sqrt{n}(\hat{\theta}-\theta)$. By the assumed convergence and the continuous mapping theorem, this converges in distribution to $Z\sim N(0,\Sigma)$, where $\Sigma=A V A'$. Meanwhile, $A\hat V A'\xrightarrow{p}\Sigma$. Since $\Sigma$ is positive definite, matrix inversion is continuous at $\Sigma$, giving $(A\hat V A')^{-1}\xrightarrow{p}\Sigma^{-1}$. Slutsky's theorem now gives $W_n\xrightarrow{d}Z'\Sigma^{-1}Z$. Write $Z=\Sigma^{1/2}U$, where $U\sim N(0,I_q)$. Then $Z'\Sigma^{-1}Z=U'U=\sum_{j=1}^q U_j^2$, which has a chi-square distribution with $q$ degrees of freedom. $\square$

This derivation shows what the calibration depends on: asymptotic normality of the estimator, consistent estimation of its covariance, and nondegenerate uncertainty in the tested directions. Robustness by itself does not supply any of these conditions automatically. For example, an estimator with an influence function of infinite variance does not meet the finite-covariance condition used here.

Where the Sandwich Covariance Comes From

For many robust estimators defined by estimating equations, the asymptotic covariance has a sandwich form. To be precise, suppose $\hat{\theta}$ solves an equation based on contributions $\psi(X_i,\theta)\in\mathbb{R}^p$, with $E[\psi(X,\theta)]=0$. Suppose the derivative matrix $B=E[\partial\psi(X,\theta)/\partial\theta']$ is nonsingular and the relevant second moments are finite. Under the usual differentiability and regularity conditions for linearizing the estimating equation, the influence function is $-B^{-1}\psi(X,\theta)$. Its covariance is therefore $$ V=B^{-1}M B^{-T},\qquad M=E[\psi(X,\theta)\psi(X,\theta)']. $$ The sample version replaces the population derivative and second-moment matrices by consistent estimates. The two outside factors account for how changes in the estimating equation translate into changes in the estimate; the middle factor measures the variability of the contributions.

In a correctly specified model, a model-based covariance may sometimes simplify to a familiar expression. Robust estimation generally does not justify assuming that simplification. The sandwich estimate allows the variability represented by $M$ to differ from the derivative represented by $B$, an important distinction when observations depart from the model used for calibration.

Worked Example: Testing Two Restrictions With a Robust Covariance

Suppose $p=3$, $n=100$, and the robust estimate is $\hat{\theta}=(1.1,1.9,0.4)'$. Test $H_0:\theta_1+\theta_2=3,\ \theta_3=0$. Let $A=\begin{pmatrix}1&1&0\\0&0&1\end{pmatrix}$, $b=(3,0)'$, and suppose the estimated asymptotic covariance is $\hat V=\begin{pmatrix}4&1&0\\1&3&0\\0&0&1\end{pmatrix}$.

First calculate the restriction residual: $A\hat{\theta}-b=(1.1+1.9-3,\ 0.4)'=(0,0.4)'$. Next, $A\hat V A'=\begin{pmatrix}4+3+2(1)&0\\0&1\end{pmatrix} =\begin{pmatrix}9&0\\0&1\end{pmatrix}$. Its inverse is $\begin{pmatrix}1/9&0\\0&1\end{pmatrix}$. Thus $$ W_n=100(0,0.4) \begin{pmatrix}1/9&0\\0&1\end{pmatrix} \begin{pmatrix}0\\0.4\end{pmatrix} =100(0.16)=16. $$

There are $q=2$ independent restrictions, so the reference distribution is $\chi^2_2$. Its upper-tail probability at 16 is $e^{-16/2}=e^{-8}\approx0.000335$. On this asymptotic calibration, the observed estimate is far from the null restrictions.

Worked Example: Why the Sandwich Changes the Test

Consider a two-dimensional robust estimating-equation estimator with derivative matrix $B=\begin{pmatrix}2&0\\0&1\end{pmatrix}$ and variability matrix $M=\begin{pmatrix}4&1\\1&4\end{pmatrix}$. The sandwich covariance is $$ V=B^{-1}M B^{-T} =\begin{pmatrix}1/2&0\\0&1\end{pmatrix} \begin{pmatrix}4&1\\1&4\end{pmatrix} \begin{pmatrix}1/2&0\\0&1\end{pmatrix} =\begin{pmatrix}1&0.5\\0.5&4\end{pmatrix}. $$

Suppose $n=100$, $\hat{\theta}_1=0.2$, and the restriction is $H_0:\theta_1=0$, so $A=(1,0)$. The sandwich variance in this tested direction is $A V A'=1$. Therefore, $W_n=100(0.2)^2/1=4$, with a $\chi^2_1$ reference distribution.

If one incorrectly used the model-based simplification $B^{-1}$ as the covariance, the variance in the tested direction would be $0.5$, and the statistic would instead be $100(0.2)^2/0.5=8$. The discrepancy is not caused by changing the estimate or the null; it comes from changing the uncertainty estimate. A covariance formula must be justified for the estimator and sampling conditions at hand, not chosen because it is simpler.

Equivalent Restrictions and Redundancy

The same hypothesis can be written in different coordinates. If $C$ is an invertible $q\times q$ matrix, then $A\theta=b$ and $CA\theta=Cb$ express the same restrictions. Their Wald statistics agree, so the test depends on the restriction space rather than a particular full-rank representation.

Result: For invertible $C$, the Wald statistic computed with $CA$ and $Cb$ equals the statistic computed with $A$ and $b$, whenever the required covariance matrices are invertible.

Proof. Put $r=A\hat{\theta}-b$ and $S=A\hat V A'$. The transformed residual is $Cr$, and its covariance matrix is $CSC'$. Since $C$ and $S$ are invertible, $(CSC')^{-1}=C^{-T}S^{-1}C^{-1}$. Consequently, $(Cr)'(CSC')^{-1}(Cr)=r'C'C^{-T}S^{-1}C^{-1}Cr=r'S^{-1}r$. The common factor $n$ leaves the statistics equal. $\square$

Worked Example: Detecting a Redundant Restriction

Suppose $\theta\in\mathbb{R}^2$, and the stated null consists of $\theta_1+\theta_2=0$ and $2\theta_1+2\theta_2=0$. The second equation is twice the first, so these are not two independent restrictions. The matrix with both rows has rank one, and the corresponding two-by-two restriction covariance is singular.

Use the single independent restriction $A=(1,1)$, $b=0$. Let $\hat{\theta}=(0.2,-0.1)'$, $n=50$, and $\hat V=\begin{pmatrix}1&0.3\\0.3&2\end{pmatrix}$. Then $A\hat{\theta}-b=0.1$, and $A\hat V A'=1+2+2(0.3)=3.6$. The statistic is $W_n=50(0.1)^2/3.6=0.1389$, approximately, with one degree of freedom. The upper-tail probability is approximately $0.709$. Counting the duplicated equation as a second restriction would make the inverse in the Wald formula undefined, not produce a valid two-degree-of-freedom test.

Interpretation and Common Pitfalls

The statistic measures a squared, covariance-scaled distance between the estimated restrictions and their null values. A large value is evidence against $H_0$ relative to the uncertainty in the tested directions. The chi-square degrees of freedom are the number of independent restrictions, not automatically the dimension $p$ of $\theta$.

Using the wrong covariance scale. If $\hat V$ estimates the covariance of the root-$n$ limit, retain the factor $n$. If using an estimate of $\operatorname{Cov}(\hat{\theta})$, do not multiply by $n$ again.
Assuming a robust point estimate guarantees a robust test. The test also needs a valid covariance estimate and the asymptotic conditions that justify its reference distribution.
Using redundant constraints. Check that $A$ has full row rank. Reduce equivalent restrictions to an independent set before inverting their covariance matrix.
Treating the chi-square approximation as exact. It is a large-sample result. A small sample can yield poor calibration even when the asymptotic argument is valid.

A useful practical check is to identify the covariance convention, verify the rank of the restriction matrix, and make sure the tested covariance $A\hat V A'$ is nonsingular. These checks address the most common algebraic and inferential failures without changing the hypothesis being tested.

Check Your Understanding

Work through these questions to check your understanding of robust Wald tests.

What assumptions give a chi-square reference distribution for a Wald test of $q$ linear restrictions?
If $\hat V$ estimates the covariance of $\sqrt{n}(\hat{\theta}-\theta)$, why does the Wald statistic include $n$?
In a sandwich covariance, what roles do the derivative matrix and the variability matrix play?
What should be done if two rows of the restriction matrix are linearly dependent?
Why can a robust estimator still lead to a poorly calibrated Wald test?