Residual Inclusion And Two-Stage Least Squares

Why the Same Coefficient Can Come From Different Regressions

In a linear instrumental-variables model, two familiar-looking procedures can produce the same estimate of the coefficient on an endogenous regressor. One regresses the outcome on the regressor’s first-stage fitted values; the other includes the original regressor and its first-stage residual. The equality is a precise consequence of the first-stage projection, not evidence that the two regressions have the same fitted values or the same uncertainty.

This distinction matters in practice. The fitted-value regression is a way to obtain the 2SLS point estimate, but its ordinary OLS standard error is generally not the 2SLS standard error. Residual inclusion also gives the same coefficient in the linear case under the stated setup, but its ordinary OLS standard error is not automatically the appropriate IV standard error either.

Model and Conditions

Let \(y_i\) be an outcome, \(x_i\) an endogenous regressor, and \(W_i\) a vector of included exogenous regressors, including an intercept. The structural equation is

Let \(Q_i\) be a vector of excluded instruments and \(Z_i=(W_i',Q_i')'\) the full instrument vector. For the standard linear IV interpretation, the instruments must satisfy the relevant exclusion and exogeneity conditions, expressed as \(E[Z_i u_i]=0\), and the instruments must be relevant enough to identify the coefficients. In particular, the instrumented variation in \(x_i\), after accounting for \(W_i\), must be nonzero. The required rank condition must hold for all coefficients being estimated.

For residual inclusion to represent a linear control-function model, an additional condition is needed: the conditional mean of the structural error must be representable by a linear function of the first-stage residual, so that the outcome equation can be written as \(y_i=W_i'\gamma+\beta x_i+\lambda v_i+\epsilon_i\), with the remaining error satisfying the needed conditional mean-zero condition. The IV moment conditions alone do not guarantee this conditional representation. The algebraic coefficient equality below follows from the projection, while a causal interpretation depends on the substantive IV assumptions.

Since \(W_i\) is included in \(Z_i\), its fitted value in the first stage is itself. The first-stage residual is therefore orthogonal in the sample to every column of \(Z\), including \(W\) and the fitted values \(\hat{x}\). In matrix notation, if \(P_Z=Z(Z'Z)^{-1}Z'\), then \(\hat{x}=P_Zx\), \(\hat{v}=(I-P_Z)x\), and \(\hat{x}'\hat{v}=0\), as well as \(W'\hat{v}=0\).

Why the Coefficients Are Equal

The key identity is \(x=\hat{x}+\hat{v}\). It makes the regressor \(x\) in the residual-inclusion regression equal to the fitted regressor plus the residual. Because \(\hat{v}\) is orthogonal to both the included regressors and \(\hat{x}\), adding \(\hat{v}\) as a control does not change the slope on \(\hat{x}\).

To justify the result, first remove the linear projection on \(W\) from each variable. Denote the resulting variables with a tilde. Since \(\hat{v}\) is orthogonal to \(W\), this residualization leaves \(\hat{v}\) unchanged; since \(x=\hat{x}+\hat{v}\), it gives \(\tilde{x}=\widetilde{\hat{x}}+\hat{v}\). Also \(\widetilde{\hat{x}}'\hat{v}=0\). The coefficient on \(x\) in the regression that also controls for \(\hat{v}\) is therefore

The expression on the left is the partial-regression formula for the residual-inclusion coefficient: controlling for \(\hat{v}\) removes the \(\hat{v}\) part of \(x\), leaving \(\widetilde{\hat{x}}\). The expression on the right is the slope in the fitted-value regression. Orthogonality is essential: without it, adding a control could change the slope.

Why Standard Errors Differ

The coefficient equality does not make the regressions interchangeable for inference. The fitted-value regression omits \(\hat{v}\), whereas residual inclusion adds it. Their residual sums of squares and residual degrees of freedom can therefore differ. More importantly, neither regression’s ordinary OLS standard error automatically accounts for the sampling variation relevant to the IV estimator.

In a homoskedastic linear IV model, let \(X=[W,x]\), \(k\) be the number of columns of \(X\), and \(\hat{u}=y-X\hat{\theta}_{2SLS}\) be the structural residuals, calculated using the original \(x\). A conventional homoskedastic 2SLS covariance estimator is

For heteroskedasticity-robust inference, the covariance estimator uses a 2SLS sandwich based on the structural residuals and instrument moments, rather than the ordinary OLS covariance formula from either auxiliary regression. In either case, compute the 2SLS estimate and its covariance consistently with the chosen assumptions. Do not substitute the fitted-value regression’s residuals for the structural residuals simply because its coefficient is correct.

Using Reported Output Carefully

In applied work, software may report a 2SLS estimate and robust standard error directly, or it may be used to run a first stage followed by a second regression. These workflows should not be confused. If the second-stage regression is ordinary OLS on \(\hat{x}\), its point estimate can equal 2SLS, but its default standard error is generally not the IV standard error. A residual-inclusion regression has a different set of regressors; its ordinary OLS standard error also requires assumptions specific to that control-function regression.

Common Mistakes

• Assuming equal coefficients mean equal regressions. The residual-inclusion regression adds \(\hat{v}\), so its fitted values, residual sum of squares, and residual degrees of freedom can differ from those of the fitted-value regression.
• Using ordinary second-stage OLS standard errors as 2SLS standard errors. The second-stage coefficient can be correct while its default OLS covariance calculation is not the appropriate IV calculation.
• Assuming residual inclusion automatically gives valid inference. The coefficient equivalence is an algebraic fact under the specified first-stage projection. Interpretation of the control-function equation and its standard errors requires appropriate conditional-mean and sampling assumptions.
• Forgetting included exogenous regressors in the first stage. The first stage must use the full instrument set, including the included regressors \(W\); their presence supplies the orthogonality used in the equivalence.
• Assuming one standard error must be larger. The differences depend on the data, model, and covariance assumptions. There is no universal ordering between the naive OLS calculations and the appropriate 2SLS standard error.