Heteroskedasticity-Robust First-Stage Tests

Why Robust First-Stage Testing Matters

A first-stage relevance test asks whether the excluded instruments help explain the endogenous regressor after accounting for included controls. The conventional first-stage F test uses a homoskedastic variance calculation. If the first-stage error variance differs across observations, that calculation can be wrong: the reported test statistic and its reference distribution may no longer have their stated calibration. A heteroskedasticity-robust Wald test changes the estimated covariance matrix used for the test, allowing the error variance to vary across observations.

This adjustment addresses a particular problem in inference; it does not turn a test of zero coefficients into a complete measure of instrument strength. A small p-value is evidence against the null that all excluded-instrument coefficients are zero. It does not by itself show that the instruments are strong enough for reliable conventional 2SLS inference.

First-Stage Model and Joint Null

Consider one endogenous regressor \(D_i\), a vector of \(q\) excluded instruments \(Z_i\), and a vector of included exogenous controls \(X_i\). The first-stage linear regression is

D_i = X_i'\gamma + Z_i'\pi + u_i.

The vector \(\pi\) contains the coefficients on the excluded instruments, and \(q\) is their number. Conditional relevance in this linear first stage is tested by the joint null \(H_0:\pi=0\), against \(H_a:\pi\ne 0\). Rejection says that the excluded instruments have some explanatory contribution conditional on the included controls. It does not say that every instrument is individually relevant, nor does it establish the other conditions needed for an instrumental-variables causal interpretation.

Definition: A heteroskedasticity-robust first-stage Wald test evaluates \(H_0:\pi=0\) using an estimated covariance matrix that allows the first-stage error variance to differ across observations. For \(q\) restrictions, its Wald statistic is compared asymptotically with a chi-square distribution with \(q\) degrees of freedom.

Let \(W_i=(X_i',Z_i')'\) be the full first-stage regressor vector, and let \(\hat\theta=(\hat\gamma',\hat\pi')'\) be the OLS estimate. One common heteroskedasticity-consistent covariance estimate, HC0, is

\widehat{\operatorname{Var}}_{\mathrm{HC0}}(\hat\theta) = (W'W)^{-1} \left(\sum_{i=1}^n W_i W_i'\hat u_i^2\right) (W'W)^{-1},

where \(W\) is the matrix with rows \(W_i'\), and \(\hat u_i\) is the first-stage residual for observation \(i\). HC1 and other variants apply finite-sample adjustments or leverage corrections. The particular variant should be reported; the robust test is not defined by the word “robust” alone.

Let \(\widehat V_\pi\) be the \(q\times q\) block of this covariance estimate corresponding to \(\hat\pi\). The joint Wald statistic is

W_{\mathrm{rob}} = \hat\pi'\widehat V_\pi^{-1}\hat\pi.

Under the null and the regularity conditions described below, \(W_{\mathrm{rob}}\) has an asymptotic chi-square distribution with \(q\) degrees of freedom. Thus the asymptotic p-value is \(P(\chi_q^2\ge W_{\mathrm{rob}})\), where \(\chi_q^2\) denotes a chi-square random variable with \(q\) degrees of freedom. Some software reports \(W_{\mathrm{rob}}/q\) as a robust Wald F statistic and uses a finite-sample F reference distribution. That convention should be identified explicitly: the asymptotic chi-square version is the direct Wald result, and software’s finite-sample reference can differ.

Why the Sandwich Covariance Allows Unequal Variance

The key point is that the variance contribution of observation \(i\) depends on its squared residual \(\hat u_i^2\). Under heteroskedasticity, observations need not contribute equally to the uncertainty in \(\hat\theta\). The sandwich estimate weights each observation’s regressor information \(W_iW_i'\) by its own squared residual rather than imposing one common error variance.

For a short justification, suppose the linear conditional mean is correctly specified, observations are independent, the regressors have full column rank, and suitable moment and leverage conditions hold. The OLS estimation error can be written as

\hat\theta-\theta = \left(\frac{W'W}{n}\right)^{-1} \left(\frac{1}{n}\sum_{i=1}^n W_i u_i\right).

The middle sum has asymptotic covariance determined by the average of \(W_iW_i'u_i^2\), not generally by one common variance multiplied by the average of \(W_iW_i'\). Replacing the unknown errors with residuals gives the sandwich estimate. If the covariance estimate is consistent and the null restrictions are true, the estimated coefficient difference, standardized by its covariance, yields the asymptotic chi-square Wald statistic. This is why unequal conditional error variance is permitted without changing the OLS coefficient estimate.

The test still requires a suitable sampling framework. The HC covariance above is designed for independent observations; it does not account for dependence among observations, such as repeated observations within a person or shared shocks within a group. The first-stage mean must also be adequately specified for the coefficient test to have its intended interpretation. Extreme leverage, small samples, or problematic moments can make the asymptotic approximation poor.

Worked Example: One Excluded Instrument

Suppose hypothetical independent data produce a first-stage coefficient of \(\hat\pi=0.18\) units of \(D\) per one-unit increase in a single excluded instrument, conditional on the included controls. The heteroskedasticity-robust standard error is \(0.075\) units. Test \(H_0:\pi=0\) at \(\alpha=0.05\).

There is one restriction, so the robust Wald statistic is the squared robust z statistic:

W_{\mathrm{rob}} = \left(\frac{0.18}{0.075}\right)^2 = (2.40)^2 = 5.76.

Under \(H_0\), the asymptotic reference distribution is \(\chi_1^2\). The p-value is \(P(\chi_1^2\ge5.76)\approx0.0164\). Because \(0.0164<0.05\), reject the null of a zero first-stage coefficient. In this hypothetical sample, there is evidence that the instrument is relevant in the specified first-stage linear model, conditional on the controls. The result does not establish that the instrument is sufficiently strong for all IV inference, or that it satisfies the other conditions for a valid instrument.

Several Excluded Instruments

With multiple instruments, the test is joint. It uses the full covariance matrix of the coefficient estimates, including their covariances. Testing each coefficient separately and then counting significant results is not the same procedure: it does not test the stated joint null at the stated level. Nor can the joint test generally be reconstructed by adding separate squared t statistics while ignoring covariance.

Worked Example: A Joint Test With Two Instruments

Suppose a hypothetical first stage has two excluded-instrument coefficients, \(\hat\pi_1=0.12\) and \(\hat\pi_2=0.08\), measured in units of \(D\) per instrument unit. Suppose their estimated robust covariance matrix is

\widehat V_\pi = \begin{pmatrix} 0.0064 & 0.0032\\ 0.0032 & 0.0064 \end{pmatrix}.

Test \(H_0:\pi_1=\pi_2=0\). The inverse covariance matrix is

\widehat V_\pi^{-1} = \begin{pmatrix} 208.333 & -104.167\\ -104.167 & 208.333 \end{pmatrix}.

Using \(\hat\pi=(0.12,0.08)'\), the statistic is

W_{\mathrm{rob}} = \begin{pmatrix}0.12&0.08\end{pmatrix} \begin{pmatrix} 208.333 & -104.167\\ -104.167 & 208.333 \end{pmatrix} \begin{pmatrix}0.12\\0.08\end{pmatrix} \approx 2.333.

For example, the quadratic form inside the multiplication is \(208.333(0.12^2+0.08^2-0.12\times0.08)\approx2.333\). The asymptotic reference is \(\chi_2^2\), giving \(P(\chi_2^2\ge2.333)=\exp(-2.333/2)\approx0.311\). At the 0.05 level, do not reject the joint null. This result does not show that both coefficients are exactly zero; it says the data do not provide strong evidence against the joint zero-coefficient null using this test.

When the Conventional First-Stage F Can Mislead

The conventional homoskedastic F test relies on a variance calculation that treats the first-stage errors as having a common conditional variance. If that assumption fails, its estimated standard errors and reference distribution may not deliver the intended rejection rate. In particular, error variance related to instrument values or other regressors can make the conventional test too liberal or too conservative. The direction of the distortion is not determined simply by knowing that heteroskedasticity exists.

A robust Wald test replaces the conventional covariance estimate while keeping the same first-stage OLS coefficients and the same joint null. The following hypothetical output illustrates why the distinction can matter.

Worked Example: Conventional and Robust Results Differ

Suppose a hypothetical first-stage regression with one excluded instrument reports \(\hat\pi=0.18\). Its conventional homoskedastic standard error is \(0.040\), while its HC robust standard error is \(0.090\). For simplicity, suppose the conventional test uses a large residual degrees of freedom.

The conventional statistic is approximately

F_{\mathrm{conv}} = \left(\frac{0.18}{0.040}\right)^2 = 4.50^2 = 20.25.

With one restriction and large residual degrees of freedom, this corresponds to a p-value below \(0.001\). The robust Wald statistic is instead

W_{\mathrm{rob}} = \left(\frac{0.18}{0.090}\right)^2 = 2.00^2 = 4.00,

which has an asymptotic \(\chi_1^2\) p-value of approximately \(0.0455\). At \(\alpha=0.05\), the robust test just rejects the zero-coefficient null, while the conventional result appears far more decisive. The estimates are identical; the difference arises from the covariance estimates. These hypothetical figures illustrate a possible pattern, not a universal direction of change. Whether either test is well calibrated depends on its assumptions and, for the robust test, on the adequacy of the asymptotic approximation.

Assumptions, Diagnostics, and Limits

A heteroskedasticity-robust Wald test is appropriate only when its assumptions and scope match the data. Before interpreting it, check that the first-stage equation includes the intended controls and excluded instruments, that the design matrix has full rank, and that the observations can reasonably be treated as independent for the chosen covariance estimator. Check for influential observations and unusual leverage, since basic HC estimates can perform poorly when a few observations dominate the design. If observations are dependent within groups, an independent-observation HC covariance is not a remedy for that dependence.

Also distinguish a test of relevance from a diagnostic of strength. The null \(H_0:\pi=0\) concerns exact zero coefficients. With a large sample, even a small conditional relationship can produce a small robust p-value. Conversely, failure to reject does not prove irrelevance. The robust Wald statistic and its p-value do not, by themselves, quantify the bias or size distortion that weak instruments can cause in 2SLS inference. Stock-Yogo critical values are derived for particular homoskedastic settings, so they should not be treated as direct critical values for this heteroskedasticity-robust Wald statistic.

A useful report states the first-stage outcome, the excluded instruments and included controls, the joint null, the robust covariance variant, the statistic and reference distribution, and the p-value. For multiple instruments, report the number of restrictions. A conventional F statistic may also be shown for comparison, but label its homoskedastic assumption clearly rather than presenting it as interchangeable with the robust test.

Common pitfall: “The robust Wald test rejects, so the instruments are strong and 2SLS inference is safe.” The test only rejects the joint null that the excluded-instrument coefficients are zero, under its assumptions. It is not a general guarantee of adequate instrument strength or valid IV inference.

Check Your Understanding

Try each question, then check the brief solutions.

A first stage has three excluded instruments. What is the joint relevance null, and what is the asymptotic degrees of freedom for its robust Wald test?
For one excluded instrument, \(\hat\pi=0.10\) and its robust standard error is \(0.05\). Calculate the Wald statistic and state the approximate asymptotic p-value.
Why can a conventional homoskedastic first-stage F test be misleading when the first-stage errors are heteroskedastic?
A robust test rejects \(H_0:\pi=0\). Does that establish that the instruments are strong enough for reliable conventional 2SLS inference? Explain.

Solutions. (1) The null is that all three excluded-instrument coefficients equal zero; the asymptotic reference distribution is \(\chi_3^2\). (2) \(W_{\mathrm{rob}}=(0.10/0.05)^2=4.00\); against \(\chi_1^2\), the p-value is approximately \(0.0455\). (3) The conventional test uses a common-variance calculation; unequal error variances can make its estimated uncertainty and resulting test calibration incorrect. (4) No. Rejection is evidence against zero coefficients in the first-stage model, not a guarantee of adequate strength or of the other conditions needed for valid IV inference.