Heteroskedasticity-Robust 2SLS Inference

Why Robust Inference Matters in 2SLS

Two-stage least squares (2SLS) estimates relationships when some regressors are endogenous, using instruments to isolate variation in those regressors. Inference then depends not only on the 2SLS coefficient estimate, but also on how its uncertainty is calculated. The conventional 2SLS covariance estimator assumes errors have a common variance. If the error variance differs across observations, that estimator generally does not provide valid standard errors.

A heteroskedasticity-robust covariance estimator allows the error variance to vary across observations while retaining the 2SLS coefficient estimate. It is a sandwich estimator: one matrix describes how the instruments and regressors map disturbances into coefficient uncertainty, another estimates the variation in instrument-weighted residuals, and the outer matrices combine those pieces. Its validity still requires appropriate instrument conditions, sufficient instrument relevance, and independent observations for the version developed here.

Definition: Let $y$ be an $n\times1$ outcome vector, $X$ an $n\times k$ matrix of included regressors (including any endogenous regressors), and $Z$ an $n\times l$ matrix of instruments, including the included exogenous regressors. The 2SLS estimator is $$ \hat{\beta}_{2SLS}=(X'P_ZX)^{-1}X'P_Zy,\qquad P_Z=Z(Z'Z)^{-1}Z'. $$ Write $x_i'$ and $z_i'$ for row $i$ of $X$ and $Z$, and let $\hat{u}_i=y_i-x_i'\hat{\beta}_{2SLS}$ be the structural 2SLS residual. The heteroskedasticity-robust sandwich covariance estimator is $$ \widehat{\operatorname{Var}}_{rob}(\hat{\beta}_{2SLS}) =(X'P_ZX)^{-1} \left(\sum_{i=1}^n x_i z_i'(Z'Z)^{-1} \left[\sum_{j=1}^n z_j z_j'\hat{u}_j^2\right] (Z'Z)^{-1}z_i x_i'\right) (X'P_ZX)^{-1}. $$

Assumptions and the Sandwich Derivation

The estimator is intended for the usual 2SLS setting: the instruments satisfy the relevant exogeneity conditions, and they provide enough independent variation to identify the coefficients. More precisely, for the population moment condition, $E[z_i u_i]=0$; the matrix of instrument-regressor moments has full column rank for the $k$ coefficients; and the instrument second-moment matrix is nonsingular. The observations are independent, with suitable finite moments so sample moments converge and the variance is finite. The error variance may depend on the instruments or other observed characteristics: constant variance is not required. Errors across observations are assumed not to be dependent for this estimator.

These conditions support a large-sample approximation. They do not make the estimator an exact finite-sample covariance formula. In particular, inference relies on a sufficiently informative first stage; heteroskedasticity-robust standard errors do not correct weak-instrument problems.

The derivation starts from the estimation error. Since $y=X\beta+u$,

\hat{\beta}_{2SLS}-\beta =(X'P_ZX)^{-1}X'P_Zu.

If errors are independent across observations but may have different variances, the conditional covariance of $u$, given the design, is diagonal with entries $\operatorname{Var}(u_i\mid X,Z)$. The covariance of the estimation error therefore has the form

(X'P_ZX)^{-1}X'P_Z\operatorname{Var}(u\mid X,Z)P_ZX(X'P_ZX)^{-1}.

The robust estimator replaces the unknown diagonal variances with squared structural residuals. To see how this produces the sandwich, substitute $P_Z=Z(Z'Z)^{-1}Z'$. The middle matrix becomes

X'Z(Z'Z)^{-1} \left(\sum_{i=1}^n z_i z_i'\hat{u}_i^2\right) (Z'Z)^{-1}Z'X.

This is the matrix expression inside the definition. The outer matrices are both $(X'P_ZX)^{-1}$, which is symmetric. With consistent residuals and the stated moment and independence conditions, this sandwich estimates the large-sample covariance even when the variances differ across observations. The estimate is often called the HC0 robust covariance estimator; finite-sample adjustments are separate choices.

Worked Calculations

Worked Example: A Just-Identified Scalar 2SLS Estimate

Consider a hypothetical no-intercept model with one endogenous regressor $x$ and one instrument $z$. The observations are $z=(1,2,3)$, $x=(1,2,1)$, and $y=(2,5,4)$. The instrument is relevant in this sample because $\sum_i z_i x_i=8\ne0$. The 2SLS estimate is

\hat{\beta}_{2SLS} =\frac{\sum_i z_i y_i}{\sum_i z_i x_i} =\frac{1(2)+2(5)+3(4)}{1(1)+2(2)+3(1)} =\frac{24}{8}=3.

The structural residuals are $y_i-\hat{\beta}_{2SLS}x_i=(-1,-1,1)$. They satisfy the sample instrument moment condition: $\sum_i z_i\hat{u}_i=1(-1)+2(-1)+3(1)=0$. For this scalar, just-identified case, the robust variance estimator simplifies to

\widehat{\operatorname{Var}}_{rob}(\hat{\beta}_{2SLS}) =\frac{\sum_i z_i^2\hat{u}_i^2}{(\sum_i z_i x_i)^2} =\frac{1(1)+4(1)+9(1)}{8^2} =\frac{14}{64}=0.2188.

Thus the robust standard error is $\sqrt{0.2188}=0.4677$ outcome units per unit of $x$. For comparison, the conventional homoskedastic estimate uses $s^2=\sum_i\hat{u}_i^2/(n-k)=3/(3-1)=1.5$, where $k=1$. Here $X'P_ZX=(\sum_i z_i x_i)^2/\sum_i z_i^2=64/14$, so

\widehat{\operatorname{Var}}_{hom}(\hat{\beta}_{2SLS}) =\frac{s^2}{X'P_ZX} =\frac{1.5}{64/14}=0.3281,\qquad SE_{hom}=\sqrt{0.3281}=0.5728.

The two standard errors differ because the robust calculation weights each squared residual by $z_i^2$, rather than imposing one common error variance. With only three observations, these values illustrate the calculations; they should not be taken as evidence that one estimate is generally larger or smaller.

Worked Example: Robust Covariance With Two Instruments

Consider a second hypothetical no-intercept model with one regressor and two instruments. Let $z_1=(1,1,1,1)'$, $z_2=(1,-1,1,-1)'$, and $Z=[z_1,z_2]$. These columns are linearly independent and $Z'Z$ is diagonal with entries 4 and 4. Let $x=(3,1,3,1)'$, $y=(5.5,0.5,3.5,2.5)'$. In this example, $x=2z_1+z_2$, so $P_Zx=x$, and $Z$ has full column rank. The 2SLS estimate is

\hat{\beta}_{2SLS} =\frac{x'P_Zy}{x'P_Zx} =\frac{x'y}{x'x} =\frac{30}{20}=1.5.

The structural residuals are $\hat{u}=y-1.5x=(1,-1,-1,1)'$. Both sample instrument moments are zero: $z_1'\hat{u}=0$ and $z_2'\hat{u}=0$. Since $P_Zx=x$, the middle of the robust sandwich reduces to $\sum_i x_i^2\hat{u}_i^2$. Therefore

\widehat{\operatorname{Var}}_{rob}(\hat{\beta}_{2SLS}) =\frac{\sum_i x_i^2\hat{u}_i^2}{(x'P_Zx)^2} =\frac{9(1)+1(1)+9(1)+1(1)}{20^2} =\frac{20}{400}=0.0500, $$ $$ SE_{rob}=\sqrt{0.0500}=0.2236.

The residual sum of squares is 4. Using $n-k=3$, the conventional variance is $s^2/(x'P_Zx)=[4/3]/20=0.06667$, giving $SE_{hom}=0.2582$. The instrument and regressor dimensions, residuals, and sums used here make the robust comparison explicit; the example does not establish a general ordering between robust and conventional standard errors.

Worked Example: Equal Residual Sum of Squares, Different Robust Standard Error

Return to the first example’s $z=(1,2,3)$ and $x=(1,2,1)$, but suppose the outcome is constructed as $y=3x+u$ with $u=(1,-2,1)/\sqrt{2}$. This is a hypothetical data construction. The instrument moment is $z'u=(1-4+3)/\sqrt{2}=0$, so the 2SLS estimate remains 3. The residual sum of squares is $(1+4+1)/2=3$, the same as in the first example. Consequently, the conventional variance remains $0.3281$ and its standard error remains $0.5728$.

The robust numerator, however, changes because the squared residuals fall differently across the instrument values:

\sum_i z_i^2\hat{u}_i^2 =1\left(\frac12\right)+4(2)+9\left(\frac12\right) =13, $$ $$ \widehat{\operatorname{Var}}_{rob}(\hat{\beta}_{2SLS}) =\frac{13}{8^2}=0.2031,\qquad SE_{rob}=\sqrt{0.2031}=0.4507.

The contrast shows what the robust estimator responds to: not just the total residual variation, but its distribution across observations with different instrument values.

Choosing and Interpreting the Standard Error

The conventional and robust calculations answer different variance-model questions. Under homoskedastic errors, the conventional covariance estimator uses one common variance estimate multiplied by $(X'P_ZX)^{-1}$. The robust estimator instead estimates the variation of the instrument-weighted residual contributions observation by observation. When the constant-variance assumption is plausible, the conventional estimator can be appropriate; when heteroskedasticity is possible, the robust estimator avoids relying on that assumption. In finite samples they can differ in either direction.

Estimator	Error structure it allows	Main qualification
Conventional 2SLS covariance	Common error variance across observations	Can misstate uncertainty if variance differs across observations
Heteroskedasticity-robust 2SLS covariance	Different variances across independent observations	Does not address dependence between observations or weak instruments
Cluster-robust or HAC covariance	Some forms of within-group or serial dependence	Requires a dependence structure and corresponding method appropriate to the data

A robust standard error quantifies sampling uncertainty under its assumptions; it does not change the 2SLS coefficient estimate. For example, a coefficient measured in outcome units per unit of an exposure remains in those units when its standard error changes. A confidence interval or test built using this standard error still relies on the relevant large-sample approximation and instrument conditions.

Diagnostics, Limitations, and Common Mistakes

Heteroskedasticity-robust inference is not a remedy for every problem in an IV analysis. Inspect residuals against fitted values or relevant covariates for patterns in their spread, and check for unusually influential observations: the squared residual terms in the sandwich can make individual observations consequential. Such checks can reveal instability, but they cannot prove that the instrument is exogenous. Instrument relevance and the credibility of the IV assumptions require separate assessment.

Using the wrong residuals. The sandwich uses $\hat{u}_i=y_i-x_i'\hat{\beta}_{2SLS}$, the structural 2SLS residuals. It does not use first-stage residuals.
Assuming “robust” means universally valid. This estimator allows heteroskedasticity across independent observations. If observations have within-group or serial dependence, its independence assumption fails; a covariance method designed for that dependence is needed.
Assuming a smaller standard error is evidence of a better model. Robust and conventional estimates may differ in either direction. Neither comparison verifies instrument validity or relevance.
Treating ordinary second-stage OLS output as 2SLS inference. The 2SLS coefficient can be computed by regressing on fitted regressors, but that regression’s ordinary standard error does not automatically give the 2SLS covariance estimator. Use the 2SLS sandwich based on the instruments and structural residuals.

The practical choice is therefore not “robust is always correct” versus “conventional is always correct.” It is whether the variance assumptions and dependence structure underlying the reported covariance estimator match the data, while the separate identification assumptions remain credible.

Check Your Understanding

Work each question, then compare your reasoning with the solutions below.

In a scalar, just-identified 2SLS model, $\sum_i z_i x_i=10$ and $\sum_i z_i^2\hat{u}_i^2=25$. Calculate the robust variance and standard error.
Why can two data sets with the same residual sum of squares have different robust 2SLS standard errors?
Does using a heteroskedasticity-robust covariance estimator make inference valid when observations within each school are dependent? Explain.
Which residuals enter the heteroskedasticity-robust 2SLS sandwich, and why?

Solutions. 1. The variance is $25/10^2=0.2500$, so the standard error is $\sqrt{0.2500}=0.5000$. 2. The robust “meat” weights squared residuals by instrument contributions, so their allocation across observations matters, not only their sum. 3. No. The estimator here assumes independent observations; dependence within schools requires a covariance method suited to that structure. 4. Use structural residuals $\hat{u}_i=y_i-x_i'\hat{\beta}_{2SLS}$, because the sandwich estimates the variation in the instrument-weighted structural disturbances that generate 2SLS estimation error.