Introduction
In two-stage least squares (2SLS), the second stage regresses the outcome on treatment values predicted from the instruments and included covariates. This produces the 2SLS coefficient, but it does not make the fitted treatment an ordinary observed regressor. The fitted values were estimated using the same sample, and the outcome equation is an instrumental-variables model. Consequently, the usual OLS standard error from that second-stage regression generally does not estimate the uncertainty of the 2SLS coefficient.
The key distinction is between the second-stage coefficient and the variance calculation. Regressing the outcome on first-stage fitted regressors reproduces the 2SLS coefficient algebraically. But the usual OLS standard error treats that regression as if it were an ordinary regression with fixed regressors and ordinary regression errors. The correct 2SLS variance instead follows the instrument-based estimating equations and uses the structural residuals.
Set-Up: Instruments, Fitted Values, and 2SLS
Let \(y\) be an \(n\)-vector of outcomes, \(X\) an \(n\)-by-\(k\) matrix of regressors in the structural outcome equation, and \(Z\) an \(n\)-by-\(L\) matrix of instruments. The columns of \(X\) can include endogenous regressors and included exogenous covariates; \(Z\) includes the instruments and the included exogenous covariates. Assume the instrument matrix has sufficient rank to identify the coefficients. Write \(P_Z=Z(Z'Z)^{-1}Z'\) for the projection matrix onto the columns of \(Z\), and \(\widehat X=P_ZX\) for the first-stage fitted regressors.
The second-stage OLS coefficient is the same estimator. In fact, OLS of \(y\) on \(\widehat X\) gives
because \(P_Z\) is symmetric and idempotent, so \(P_Z'=P_Z\) and \(P_ZP_Z=P_Z\). This equality explains why a hand-calculated second-stage coefficient can be right even when the OLS standard error printed alongside it is not.
Worked Example: The Coefficient Can Be Right While the Standard Error Is Not
Consider four hypothetical observations, with an intercept and one instrument \(z\). The first-stage regression predicts treatment \(D\), and the outcome model uses \(D\) as its regressor.
| Observation | \(z\) | Observed \(D\) | Fitted \(\widehat D\) | \(y\) |
|---|---|---|---|---|
| 1 | -3 | 3 | 2 | 5 |
| 2 | -1 | 1 | 4 | 5 |
| 3 | 1 | 9 | 6 | 15 |
| 4 | 3 | 7 | 8 | 15 |
The fitted values come from the first-stage regression \(\widehat D=5+z\). Regressing \(y\) on an intercept and \(\widehat D\) gives slope \(2\): the fitted line is \(y=2\widehat D\). Thus the second-stage coefficient is the 2SLS coefficient.
The second-stage residuals are \(y-2\widehat D=(1,-3,3,-1)\). Their sum of squares is \(1+9+9+1=20\). The centered fitted-treatment values are \((-3,-1,1,3)\), whose squared deviations sum to \(20\). The usual OLS calculation, using \(n-2=2\) residual degrees of freedom, gives $$ s^2=\frac{20}{2}=10,\qquad SE_{\text{naive}}(\widehat\beta)=\sqrt{\frac{10}{20}}=0.7071. $$ This is the standard error from treating the second-stage regression as an ordinary OLS regression. It is not, in general, a valid 2SLS standard error.
Why the Ordinary OLS Standard Error Does Not Apply
The standard OLS variance formula is derived for an outcome equation whose regressors are the regressors being used in the variance calculation, under the model's corresponding error assumptions. In the second-stage regression, the regressor is instead a fitted value constructed from \(Z\), while the structural equation is \(y=X\beta+u\). The structural error is \(u=y-X\beta\), not \(y-\widehat X\beta\). In the example, those two residual vectors are different: at \(\beta=2\), the second-stage residuals are \((1,-3,3,-1)\), whereas the structural residuals are \((-1,3,-3,1)\).
There is also a difference in the uncertainty calculation. The 2SLS estimator is driven by the instrument-based moments: instruments must be uncorrelated with the structural error in the population. Under heteroskedasticity, the variability of those moments depends on the structural residuals and the instruments. The ordinary second-stage OLS formula uses neither the right residuals nor the right moment variability. In particular, it treats the fitted regressor as though it were an ordinary fixed regressor in a correctly specified outcome regression.
This is not a claim that every naive standard error is too small. Depending on the data and error pattern, the ordinary second-stage formula can be larger or smaller than the appropriate 2SLS standard error. The problem is that it is generally not estimating the right variance, not that it always errs in one direction.
The 2SLS Variance Formulas
For a homoskedastic structural error with variance \(\sigma^2\), the conventional homoskedastic 2SLS variance estimate has the form
where \(\widehat\sigma^2\) estimates the structural error variance, commonly using the sum of squared structural residuals \(\sum_i\widehat u_i^2\) with a degrees-of-freedom adjustment. This formula relies on the homoskedastic variance specification; it should not be used as though homoskedasticity were automatic.
With heteroskedasticity, a common heteroskedasticity-consistent estimator, HC0, is
Here the diagonal matrix contains squared structural residuals. Software may use a small-sample adjustment to HC0; that changes the finite-sample estimate but not the central distinction from the naive second-stage OLS calculation.
Why the Sandwich Formula Follows
The 2SLS estimator solves the sample moment equation \(X'P_Z(y-X\beta)=0\). Since \(P_Z=Z(Z'Z)^{-1}Z'\), these are weighted combinations of the instrument moments \(Z'(y-X\beta)\). A small change in the sample moments changes the coefficient through the inverse matrix \((X'P_ZX)^{-1}\); the variability of the moments is determined by the squared structural errors and the instruments. The sandwich formula records both parts: the inverse matrices are the outer “bread,” and the residual-weighted instrument variability is the “meat.”
The corresponding finite-sample HC0 expression can be seen by writing \(\widehat X=P_ZX\). Its middle term is
where \(\widehat X_i\) is row \(i\) of \(\widehat X\). The outside factors are \((\widehat X'\widehat X)^{-1}\), because \(\widehat X'\widehat X=X'P_ZX\). Therefore the sandwich uses fitted regressors in the instrument-based score contribution but weights that contribution by the structural residual. It is not the ordinary OLS sandwich for the regression of \(y\) on \(\widehat X\), whose residual would instead be \(y-\widehat X\widehat\beta\).
Worked Example: A Robust 2SLS Standard Error From the Same Data
Return to the four observations above. The 2SLS coefficient is \(2\), and the structural residuals are \(y-2D=(-1,3,-3,1)\). With an intercept and one fitted treatment regressor, the slope component of the HC0 variance uses the centered fitted-treatment values \((-3,-1,1,3)\). Its scalar bread is their sum of squares, \(A=20\), and its meat is $$ B=\sum_i(\widehat D_i-\overline{\widehat D})^2\widehat u_i^2 =9(1)+1(9)+1(9)+9(1)=36. $$ Thus $$ \widehat{\operatorname{Var}}_{\mathrm{HC0}}(\widehat\beta_{2SLS}) =\frac{B}{A^2}=\frac{36}{20^2}=0.09, \qquad SE_{\mathrm{HC0}}(\widehat\beta_{2SLS})=\sqrt{0.09}=0.3000. $$ For these hypothetical data, the naive OLS standard error was \(0.7071\), while the robust 2SLS standard error is \(0.3000\). The difference illustrates why the naive calculation is not generally valid; it does not establish that the naive standard error must be larger in other data.
Worked Example: Calculating a Robust Standard Error From Summary Quantities
Suppose a hypothetical just-scalar 2SLS analysis, after accounting for the intercept, has \(A=\sum_i(\widehat D_i-\overline{\widehat D})^2=80\). Suppose the structural residuals give \(B=\sum_i(\widehat D_i-\overline{\widehat D})^2\widehat u_i^2=500\). These are summary quantities from the fitted analysis, not estimates from the second-stage OLS residuals.
The HC0 slope variance and standard error are $$ \widehat{\operatorname{Var}}_{\mathrm{HC0}}(\widehat\beta_{2SLS}) =\frac{B}{A^2} =\frac{500}{80^2} =\frac{500}{6400} =0.078125, $$ $$ SE_{\mathrm{HC0}}(\widehat\beta_{2SLS}) =\sqrt{0.078125}=0.2795. $$ The standard error is in the units of the coefficient: outcome units per treatment unit. For example, if the outcome is measured in days and treatment in milligrams, the standard error is in days per milligram.
What to Report and Common Pitfalls
For a 2SLS analysis, report the coefficient from the IV estimator and a standard error calculated for that estimator under a stated variance assumption: homoskedastic, heteroskedasticity-robust, or, where appropriate, cluster-robust. The previous material on cluster-robust inference addresses dependence within clusters; changing the clustering adjustment does not make the naive second-stage OLS formula appropriate. The underlying variance calculation must still correspond to 2SLS.
- Wrong reasoning: “The fitted values are the second-stage regressors, so the usual OLS standard error is valid.” Correction: The coefficient matches 2SLS, but the structural error and instrument-based moment variability differ from those assumed in the ordinary second-stage OLS variance calculation.
- Wrong reasoning: “Use the second-stage residuals in the 2SLS robust formula.” Correction: Use \(\widehat u_i=y_i-X_i'\widehat\beta_{2SLS}\), the residual from the structural equation with the original regressors.
- Wrong reasoning: “The naive standard error always understates uncertainty.” Correction: Its direction of error is not fixed; it is generally invalid because it targets the wrong variance calculation.
- Wrong reasoning: “The fitted treatment is measured without uncertainty because it is a computed value.” Correction: It depends on an estimated first stage. The 2SLS variance accounts for the estimation through the instrument-based projection and moment calculation; it is not obtained by treating the fitted values as externally observed data.
The practical check is simple: if the reported coefficient is 2SLS but the reported standard error comes from an ordinary OLS regression on first-stage fitted values, verify the calculation. A proper 2SLS variance estimator uses the 2SLS structure, the instruments, and the structural residuals.
Check Your Understanding
Answer each question by identifying the estimator or variance component involved.
- Why does OLS of the outcome on first-stage fitted regressors produce the 2SLS coefficient?
- In a 2SLS analysis, how is a structural residual different from a second-stage regression residual?
- What does the heteroskedasticity-robust 2SLS “meat” use in addition to fitted regressors?
- Can the naive second-stage OLS standard error be assumed to be too small? Explain.
- Which variance formula would you use if you are willing to assume homoskedastic structural errors, and what changes if heteroskedasticity is allowed?