Why OLS on Fitted Values Gives 2SLS Coefficients

Why the Second Regression Can Recover the 2SLS Coefficients

Two-stage least squares is often described as two regressions: first predict each endogenous regressor from the instruments and included exogenous covariates, then regress the outcome on those predictions and the included covariates. This procedure can seem puzzling. The second regression uses generated fitted values rather than the observed endogenous regressors, so why should its coefficients correspond to a familiar IV estimator?

The answer is an exact finite-sample algebraic identity. Under the right construction, OLS on the fitted regressors gives the same coefficient vector as 2SLS. This identity concerns coefficient estimates; it does not make the fitted regressors observed data, establish that the instruments are valid, or make every way of calculating uncertainty appropriate.

Setup and Conditions

Suppose there are \(n\) observations, an outcome vector \(y\), a matrix \(D\) of endogenous regressors, and a matrix \(W\) of included exogenous covariates. The columns of \(W\) may include an intercept. Write the full regressor matrix as \(X=[D\;W]\). Let \(Q\) contain the excluded instruments, and let the full instrument matrix be \(Z=[Q\;W]\): the included exogenous covariates are instruments for themselves as well as regressors.

Definition: When \(Z\) has full column rank, the projection matrix onto the column space of \(Z\) is \(P_Z=Z(Z'Z)^{-1}Z'\). The first-stage fitted regressor matrix is \(\hat X=P_ZX\). Since every column of \(W\) is in the column space of \(Z\), \(P_ZW=W\), so \(\hat X=[P_ZD\;W]=[\hat D\;W]\).

The standard 2SLS coefficient estimator is

\hat\beta_{\mathrm{2SLS}}=(X'P_ZX)^{-1}X'P_Zy.

For this expression to be defined as written, the matrix \(X'P_ZX\) must be nonsingular. This is the relevant sample rank condition: the instrumented variation must identify all coefficients in \(X\). The two-stage OLS calculation also requires \(\hat X\) to have full column rank. These conditions are equivalent here because \(\hat X'\hat X=X'P_ZX\).

The algebraic equality below does not require instrument exogeneity or an exclusion restriction. Those assumptions matter for interpreting 2SLS as a causal estimator, not for proving that two formulas produce the same coefficients. The regressions must use the same observations, the same instrument set, and the same included covariates; this discussion assumes ordinary unweighted least squares.

The Equality, Derived

The second-stage OLS regression uses \(y\) as its outcome and \(\hat X\) as its design matrix. Its coefficient vector minimizes the sum of squared residuals \((y-\hat Xb)'(y-\hat Xb)\), and, when \(\hat X\) has full column rank, its normal equations give \(\hat b_{\mathrm{OLS}}=(\hat X'\hat X)^{-1}\hat X'y\). The following identities establish that this is the 2SLS formula.

Use projection properties.
\(P_Z\) is symmetric and idempotent: \(P_Z'=P_Z\) and \(P_ZP_Z=P_Z\). These follow from its definition and the fact that \(Z'Z\) is symmetric.

Calculate the second-stage cross-product.
Since \(\hat X=P_ZX\), \(\hat X'\hat X=X'P_Z'P_ZX=X'P_ZX\).

Calculate the outcome cross-product.
\(\hat X'y=X'P_Z'y=X'P_Zy\).

Substitute into the OLS formula.
\(\hat b_{\mathrm{OLS}}=(\hat X'\hat X)^{-1}\hat X'y=(X'P_ZX)^{-1}X'P_Zy=\hat\beta_{\mathrm{2SLS}}\).

Thus, the equality follows from the projection matrix, not from a claim that the fitted regressors equal the observed regressors. In general, \(\hat D\ne D\). Rather, 2SLS uses the part of each regressor that lies in the instrument space, and the second-stage OLS calculation uses exactly that projected regressor matrix.

Worked Examples

Worked Example: Four Observations With an Included Covariate

Consider four hypothetical observations. Let the intercept vector and three contrast vectors be

\mathbf 1=(1,1,1,1)',\quad a=(1,1,-1,-1)',\quad b=(1,-1,1,-1)',\quad c=(1,-1,-1,1)'.

Each vector has squared length \(4\), and each pair has inner product \(0\). For example, \(\mathbf 1'a=1+1-1-1=0\), while \(\mathbf 1'\mathbf 1=1+1+1+1=4\). Let the included covariate be \(a\), the excluded instrument be \(b\), and the endogenous regressor be \(d=3\mathbf 1+2a+b+2c\). The instrument space is spanned by \(\mathbf 1,a,b\), so the first-stage fitted regressor is \(\hat d=3\mathbf 1+2a+b\).

In observation order, \(d=(8,2,0,2)'\) and \(\hat d=(6,4,2,0)'\). Set the hypothetical outcome to \(y=7\mathbf 1+5a+6b+3c=(21,3,5,-1)'\). The second-stage regressors are \(\mathbf 1,a,\hat d\). Since \(\hat d=3\mathbf 1+2a+b\), the fitted outcome from coefficients \((-11,-7,6)'\), in that order, is

-11\mathbf 1-7a+6\hat d =-11\mathbf 1-7a+6(3\mathbf 1+2a+b) =7\mathbf 1+5a+6b.

The residual is therefore \(y-(7\mathbf 1+5a+6b)=3c\). It is orthogonal to each second-stage regressor: \(c'\mathbf 1=0\), \(c'a=0\), and \(c'\hat d=3c'\mathbf 1+2c'a+c'b=0\). Hence the OLS normal equations hold, and the coefficients are \(-11\) for the intercept, \(-7\) per unit of \(a\), and \(6\) per unit of the fitted endogenous regressor. The 2SLS coefficient vector is exactly the same. This calculation also checks the rank condition: \(\mathbf 1,a,\hat d\) are linearly independent because \(\hat d\) has a nonzero \(b\) component.

Worked Example: A Scalar Instrument and Intercept

Suppose three hypothetical observations have instrument values \(z=(-1,0,1)'\), endogenous regressor values \(d=(0,1,3)'\), and outcome values \(y=(1,2,8)'\). Include an intercept in both stages. The first-stage intercept is \(\bar d=4/3\), and its slope on \(z\) is

\frac{z'd}{z'z}=\frac{3}{2}=1.5, \qquad \hat d=\frac{4}{3}+1.5z =\left(-\frac{1}{6},\frac{4}{3},\frac{17}{6}\right)'.

For the second stage, the intercept and \(\hat d\) span the same space as the intercept and \(z\). The OLS slope on \(z\) is \(z'y/z'z=7/2\). Since \(\hat d\) changes by \(1.5\) for each one-unit change in \(z\), the slope on \(\hat d\) is \((7/2)/1.5=7/3\). The intercept is the mean outcome minus the slope times the mean fitted regressor:

\bar y-\frac{7}{3}\bar{\hat d} =\frac{11}{3}-\frac{7}{3}\frac{4}{3} =\frac{5}{9}.

Directly, the just-identified 2SLS slope is \(z'y/z'd=7/3\), and its intercept is \(\bar y-(7/3)\bar d=5/9\). Both calculations give an intercept of \(5/9\) and a slope of \(7/3\), with the slope measured in outcome units per unit of \(d\). Here \(z'z=2\), \(z'd=3\), and the intercept and \(z\) have full rank, so the required calculations are defined.

Worked Example: Omitting an Included Covariate Changes the Result

Return to the four-observation data, where \(a\) is an included covariate. Suppose someone regresses \(y\) on an intercept and \(\hat d\) but accidentally omits \(a\). The mean of \(\hat d\) is \(3\), and the mean of \(y\) is \(7\). Centering both variables gives \(\hat d-3\mathbf 1=2a+b\) and \(y-7\mathbf 1=5a+6b+3c\). Using the orthogonality and squared lengths already checked,

\hat\beta_{\hat d} =\frac{(\hat d-3\mathbf 1)'(y-7\mathbf 1)} {(\hat d-3\mathbf 1)'(\hat d-3\mathbf 1)} =\frac{4(2\cdot5+1\cdot6)}{4(2^2+1^2)} =\frac{64}{20} =\frac{16}{5}=3.2.

This is not the correct 2SLS coefficient of \(6\) from the model that includes \(a\). The discrepancy is not a failure of the equivalence theorem: the mistaken regression does not use the full second-stage design matrix. To reproduce the intended 2SLS coefficients, the second stage must include the included exogenous covariates as well as the fitted endogenous regressors.

Implementation Checks and Limits

The equivalence provides a useful check on a hand implementation. If coefficient estimates from a direct 2SLS calculation and a correctly constructed two-regression procedure differ beyond numerical rounding, inspect the design matrices before interpreting the result. In particular, verify that each endogenous regressor was projected on the full instrument set, including every included exogenous covariate, and that the second-stage regression includes those covariates. Also verify that both stages use the same observations and the same intercept convention.

Rank matters as well. If an instrument or regressor is redundant, an inverse in the displayed formulas may not exist. If the projected regressors do not have full column rank, the second-stage coefficients are not uniquely determined by ordinary least squares. Software may drop columns or use a generalized inverse, so check the reported design and rank rather than assuming the formulas apply unchanged.

A further pitfall is to mistake equality of coefficients for equality of all outputs from two software procedures. OLS on fitted values reproduces the 2SLS coefficient estimates under the stated conditions. Ordinary OLS standard errors from that second regression are generally not the appropriate 2SLS standard errors. The generated fitted regressors are constructed from the instruments, and the uncertainty calculation must reflect the IV estimator. Coefficient equality alone does not justify reusing the second-stage OLS standard errors.

Direct matrix 2SLS and the two-regression computation are alternative ways to obtain the point estimates. The matrix form makes the projection and rank conditions transparent; the two-regression form can be convenient for inspecting first-stage fitted values. Neither computation, by itself, establishes the substantive IV assumptions needed for a causal interpretation. Nor does this algebraic result determine the appropriate inference procedure.

Key takeaway: With the same observations and instrument set, and with included exogenous covariates retained in the second stage, OLS of \(y\) on the first-stage fitted regressors gives the 2SLS coefficient vector. The identity follows because \(P_Z\) is symmetric and idempotent; it is a statement about coefficients, not ordinary second-stage standard errors.

Check Your Understanding

Answer each question, then compare your reasoning with the brief solutions.

In one sentence, explain why \(P_ZW=W\) when the columns of \(W\) are included in \(Z\).
Which two projection-matrix properties give \(\hat X'\hat X=X'P_ZX\)?
What condition makes the second-stage OLS coefficient vector unique?
A researcher obtains different coefficients from direct 2SLS and from the two regressions. Name two implementation details to check.
Does the coefficient equivalence justify reporting ordinary OLS standard errors from the second-stage regression? Why or why not?

Solutions. (1) Each column of \(W\) already lies in the column space onto which \(P_Z\) projects, so projection leaves it unchanged. (2) Symmetry, \(P_Z'=P_Z\), and idempotence, \(P_ZP_Z=P_Z\). (3) The fitted-regressor matrix \(\hat X\) must have full column rank, equivalently \(X'P_ZX\) must be nonsingular. (4) Check that the first stages use the full instrument set, including \(W\); that the second stage includes \(W\); and that both stages use the same observations and intercept convention. (5) No. The identity establishes equality of point estimates only; the ordinary second-stage OLS standard errors are generally not the appropriate 2SLS standard errors.