Why the Details of Implementation Matter
Two-stage least squares (2SLS) is defined by a particular sequence of projections and regressions. The first stage predicts each endogenous regressor using the instruments and included exogenous covariates. The second stage uses those predictions in place of the endogenous regressors, while retaining the included exogenous covariates. That sequence produces the 2SLS coefficient estimates, but it does not make the predicted values ordinary observed regressors for purposes of inference.
A practical implementation must therefore get two things right: which variables are predicted and carried into the second stage, and which information is retained for calculating uncertainty. A regression package’s 2SLS or instrumental-variables command handles both stages as one estimator. If the stages are run manually, the researcher must preserve the original instruments and use an inference procedure designed for 2SLS.
This setup assumes a linear structural equation \(y=W\gamma+D\beta+u\), where \(\beta\) is the coefficient vector of interest. For a causal interpretation, the instruments must satisfy the relevant exogeneity and exclusion assumptions: conditional on included covariates, they must not be related to the structural error, and they must affect the outcome through the endogenous regressors rather than through another pathway. The instruments must also be sufficiently relevant to identify the coefficients. In matrix terms, the population rank condition must hold; in a sample calculation, the corresponding matrices must have the required rank for the estimator to exist. These conditions justify the IV design, but they do not guarantee that a particular data set will give precise estimates.
What Goes Into Each Stage
In the first stage, regress each column of \(D\) on all columns of \(H=[W\ Z]\). This means that each endogenous regressor is predicted using both the excluded instruments and the included exogenous covariates. The resulting fitted columns form \(\hat D\). In the second stage, regress \(y\) on the original \(W\) columns and these fitted endogenous columns. Do not replace \(W\) with its predictions, and do not put the observed endogenous columns \(D\) back into the second-stage regressor list.
The reason \(W\) appears in both stages is that it is part of the structural model, not an excluded instrument. It is also part of the set of variables used to construct the first-stage predictions. With multiple endogenous regressors, each one gets its own first-stage fitted column. The predicted columns are then used together in the second stage; predicting one endogenous regressor does not make the others exogenous.
List the endogenous regressors \(D\), included exogenous covariates \(W\), and excluded instruments \(Z\). Check that the instrument set and covariates match the intended structural model.
For each column of \(D\), run a first-stage regression on all columns of \([W\ Z]\). Save the fitted values, not just the first-stage coefficients or a fitted value for one selected regressor.
Regress \(y\) on \([W\ \hat D]\). The coefficients on \(\hat D\) are the 2SLS coefficients on \(D\); the coefficients on \(W\) are also estimated as part of the structural model.
Retain the original instrument matrix \(H\), the structural residuals, and the design information required by the chosen 2SLS variance estimator. Do not report the ordinary OLS standard errors from the second regression as if that regression had observed regressors.
Worked Calculations
Worked Example: One Endogenous Regressor and One Instrument
Suppose three hypothetical observations have no included covariates other than an intercept. The instrument values are \(z=(-1,0,1)'\), the endogenous regressor values are \(D=(-2,1,3)'\), and the outcome values are \(y=(-3,2,5)'\). The first-stage regression includes an intercept and \(z\).
The mean of \(D\) is \(2/3\), and the mean of \(z\) is zero. The slope in the first stage is \(z'(D-\bar D)/z'z=5/2=2.5\), because \(z'D=5\) and \(z'z=2\). Thus the fitted values are \(\hat D=(2/3-2.5,\ 2/3,\ 2/3+2.5)'=(-11/6,\ 2/3,\ 19/6)'\).
The second-stage regression includes an intercept and \(\hat D\). The slope is the centered cross-product divided by the centered sum of squares. Since the fitted values have mean \(2/3\), their centered values are \((-2.5,0,2.5)'\). The outcome mean is \(4/3\), so the centered outcomes are \((-13/3,2/3,11/3)'\). Therefore,
The second-stage intercept is \(4/3-(1.6)(2/3)=4/15\), or about \(0.267\). The calculation illustrates how the fitted endogenous values enter the coefficient regression. It does not establish that the instrument is valid or strong: those are assumptions and diagnostics that must be considered separately.
Worked Example: Predicting Multiple Endogenous Regressors
Suppose there are two excluded instruments \(z_1,z_2\), with \(z_1'z_1=z_2'z_2=1\) and \(z_1'z_2=0\). Let \(v\) and \(w\) be vectors orthogonal to both instruments. Consider two endogenous regressors \(D_1=z_1+2z_2+v\) and \(D_2=2z_1-z_2+w\), with no additional covariates in this simplified calculation.
Projecting each regressor onto the instrument span removes its component orthogonal to that span. The first-stage fitted columns are therefore \(\hat D_1=z_1+2z_2\) and \(\hat D_2=2z_1-z_2\). Now suppose the hypothetical outcome is \(y=1.5D_1-0.5D_2+e\), where \(e\) is orthogonal to \(z_1\) and \(z_2\). Its projection onto the instrument span is \(0.5z_1+3.5z_2\): substituting the definitions gives \(1.5(z_1+2z_2)-0.5(2z_1-z_2)=0.5z_1+3.5z_2\).
The second-stage coefficients \(b_1,b_2\) must satisfy \(b_1\hat D_1+b_2\hat D_2=0.5z_1+3.5z_2\). Equating the coefficients on the orthogonal instrument vectors gives \(b_1+2b_2=0.5\) and \(2b_1-b_2=3.5\). Solving the first equation for \(b_1=0.5-2b_2\) and substituting into the second gives \(1-5b_2=3.5\), so \(b_2=-0.5\) and \(b_1=1.5\). Both endogenous regressors must be predicted and entered together; using a fitted value for only one would not implement the specified two-regressor model.
Second-Stage Residuals Are Not Structural Residuals
A central implementation trap is to treat the residuals from the second-stage coefficient regression as the structural residuals. The second-stage fitted values use \(\hat D\), whereas the structural model contains the observed \(D\). For a given coefficient vector, the structural residual is \(y-W\hat\gamma-D\hat\beta\), not \(y-W\hat\gamma-\hat D\hat\beta\). This distinction matters because the instruments and structural residuals, rather than the second-stage OLS residuals alone, are needed in 2SLS inference.
Worked Example: Checking the Structural Residual
Take mutually orthogonal hypothetical vectors \(z_1,z_2,v\), each with squared length 4. Let \(D=z_1+z_2+v\) and \(y=3z_1+z_2+2v\), with the two instruments \(z_1,z_2\). Projecting \(D\) onto their span gives \(\hat D=z_1+z_2\). The second-stage coefficient through the origin is
Using the observed \(D\), the structural residual at this estimate is \(y-2D=(3z_1+z_2+2v)-2(z_1+z_2+v)=z_1-z_2\). In contrast, the second-stage regression residual is \(y-2\hat D=(3z_1+z_2+2v)-2(z_1+z_2)=z_1-z_2+2v\). The \(v\) terms cancel in the structural residual but not in the second-stage regression residual.
This example also shows why an overidentified sample need not have each instrument’s residual product equal to zero. Here \(z_1'(y-2D)=4\) and \(z_2'(y-2D)=-4\), while \(\hat D'(y-2D)=(z_1+z_2)'(z_1-z_2)=0\). The 2SLS coefficient condition is satisfied even though the two separate instrument-residual products are nonzero.
Why the Original Instruments Still Matter
The second-stage coefficient regression can reproduce the 2SLS coefficient estimates when set up correctly, but its fitted values do not replace the original instruments for inference. The \(\hat D\) columns are calculated from \(H\), so their sampling variation and dependence on the instruments matter. Treating them as fixed observed data and applying the usual OLS standard-error formula ignores that first-stage construction. Correct 2SLS inference uses the original instrument design and structural residuals under the assumptions appropriate to the chosen variance estimator.
The practical rule is simple: use the fitted values to form the second-stage coefficient regression, but use a dedicated 2SLS procedure to calculate standard errors, tests, and intervals. Do not calculate a standard error from the second-stage regression and assume it is valid merely because the point estimates agree. The details of homoskedastic, robust, and other 2SLS variance estimators depend on the sampling and error structure.
Diagnostics, Alternatives, and Limitations
Before interpreting a result, check that the software’s specification and the manual stages match. Confirm that every endogenous regressor has a first-stage regression; all intended included covariates appear in each first stage and in the second-stage regression; and the intended excluded instruments are included in every first stage. Inspect rank and first-stage relevance diagnostics, but do not regard a strong first stage as proof of instrument validity. Relevance can be assessed from the data; exclusion and exogeneity generally require substantive justification.
A common mistake is to run the first stage using only \(Z\), omitting \(W\), then regress \(y\) on \(\hat D\) without \(W\). This changes the projection and the structural specification. The correction is to use \(H=[W\ Z]\) to construct \(\hat D\), and then regress on \([W\ \hat D]\). Another mistake is to use \(D\) rather than \(\hat D\) in the second stage; that is not 2SLS and generally returns an OLS regression with endogenous regressors. A third mistake is to use the ordinary second-stage OLS standard errors; use a 2SLS inference procedure instead.
The main alternative is to estimate the model directly with an instrumental-variables or 2SLS command, specifying the outcome, included covariates, endogenous regressors, and excluded instruments. This is usually safer because the software performs the coefficient estimation and inference consistently with the specified estimator. Manual stages can be useful for learning, checking predictions, or debugging a model, but they are easier to mis-specify. Neither approach repairs invalid instruments, weak identification, an incorrect functional form, or a mismatch between the assumed model and the data. In particular, implementing the algebra correctly does not by itself establish a causal interpretation.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- A model includes one endogenous regressor, an intercept, and two exogenous covariates. What variables belong in its first-stage regression, and what regressors belong in its second-stage coefficient regression?
- In the orthogonal-vector example, what is the structural residual at the estimated coefficient of 2, and what is the second-stage regression residual?
- Why is a strong first stage not enough to establish that an instrument supports a causal interpretation?
- Why should the original instruments be retained after the fitted endogenous values have been created?
Solutions. (1) Regress the endogenous variable on the intercept, both exogenous covariates, and both excluded instruments. Regress the outcome on the intercept, both original exogenous covariates, and the fitted endogenous value. (2) The structural residual is \(y-2D=z_1-z_2\); the second-stage regression residual is \(y-2\hat D=z_1-z_2+2v\). They differ because the structural equation uses observed \(D\), not \(\hat D\). (3) Strength concerns relevance, not whether the instrument is exogenous or affects the outcome only through the endogenous regressor. Those assumptions require separate justification. (4) The fitted values were constructed from the instruments, and valid 2SLS inference must account for that instrument-based estimation and use structural residuals; ordinary OLS standard errors treating the fitted values as observed regressors do not do so.