2SLS Normal Equations and Moment Conditions

Why the 2SLS Equations Matter

Two-stage least squares (2SLS) is often introduced through a projection: project the regressors onto the space spanned by the instruments, then use those projected regressors in the estimator. That formula also tells us which sample equations the estimate satisfies. The distinction matters because in an exactly identified model those equations can be written as zero instrument-residual moments, whereas in an overidentified model 2SLS generally sets only particular combinations of those moments to zero.

The statements here are algebraic properties of the 2SLS estimator. They do not establish that an instrument is valid for causal interpretation. Instrument validity and the causal assumptions of instrumental-variable analyses are separate from whether a sample estimator exists and solves its defining equations.

Setup and 2SLS Normal Equations

Let \(y\) be an \(n\times 1\) outcome vector, \(X\) an \(n\times k\) matrix of regressors, and \(Z\) an \(n\times L\) matrix of instruments. The structural linear model is \(y=X\beta+u\), where \(\beta\) is a \(k\times 1\) coefficient vector and \(u\) is the residual vector at a proposed coefficient value. Assume the columns of \(Z\) are linearly independent, so \(Z'Z\) is invertible. Define the projection onto the column space of \(Z\) by \(P_Z=Z(Z'Z)^{-1}Z'\). The estimator discussed here requires \(X'P_ZX\) to be invertible.

When an intercept is part of the structural regression, it is included as a column of \(X\), and the instrument set must include that column as an instrument. Examples below that omit an intercept say so explicitly. The number of instruments is \(L\), while the number of structural regressors is \(k\). Exact identification means \(L=k\); overidentification means \(L>k\).

The normal equations follow directly by multiplying the estimator formula by \(X'P_ZX\). Equivalently, 2SLS minimizes the projected-residual quadratic \((y-X\beta)'P_Z(y-X\beta)\). Since \(P_Z\) is symmetric, differentiating this expression with respect to \(\beta\) gives \(-2X'P_Z(y-X\beta)\); setting the derivative to zero yields the stated equations. In the earlier material on the projection-matrix formula for 2SLS, the same projection defines the estimator; here the focus is what its normal equations imply about sample moments.

Exactly Identified Models: All Instrument Moments Are Zero

When \(L=k\), suppose additionally that \(X'Z\) is nonsingular. This is the rank condition that ensures the instruments provide enough independent information about the regressors in this exactly identified setup. The normal equations can be factored as

Here \(X'Z(Z'Z)^{-1}\) is a square, nonsingular matrix: each factor is square and nonsingular under the stated conditions. Therefore the product can equal zero only if \(Z'(y-X\hat{\beta}_{2SLS})=0\). Thus every sample instrument-residual moment is zero. Conversely, solving these moment equations gives \((Z'X)\hat{\beta}=Z'y\), so \(\hat{\beta}=(Z'X)^{-1}Z'y\). Under the same rank conditions, this solution is the 2SLS estimator.

Overidentified Models: Projected Residuals, Not Every Moment

If \(L>k\), there are more instrument-residual moments than coefficient equations. The 2SLS normal equations remain

The left side has \(k\) components, while \(Z'(y-X\hat{\beta}_{2SLS})\) has \(L\) components. In general, 2SLS makes the specified \(k\) combinations zero, not all \(L\) moments individually. The fitted regressors \(P_ZX\) are the columns of \(X\) projected onto the instrument space, and the normal equations say these fitted regressors are orthogonal to the 2SLS residual vector. They do not say that the residual vector is orthogonal to every instrument separately.

A useful equivalent characterization is minimization: 2SLS chooses \(\beta\) to minimize \((y-X\beta)'P_Z(y-X\beta)\). This objective measures the squared length of the residual after projection onto the instrument space. Its first-order conditions are the projected-residual normal equations above. Because there are more instruments than coefficients in an overidentified model, setting every instrument moment to zero would impose more equations than unknown coefficients and usually has no exact solution.

Rank Failure and What the Equations Cannot Guarantee

The equation \(X'P_ZX\hat{\beta}=X'P_Zy\) defines a unique estimate only if \(X'P_ZX\) is nonsingular. In an exactly identified model this requires the instruments to be sufficiently related to the regressors. If the rank condition fails, there may be no unique solution even if the instrument columns themselves are linearly independent. Overidentification does not automatically prevent this failure: there still must be enough projected variation in \(X\) to identify all \(k\) coefficients.

Diagnostics, Comparisons, and a Common Mistake

A practical algebraic check is to calculate the rank of \(Z\), inspect whether \(X'P_ZX\) is nonsingular, and then verify the equations appropriate to the identification status. In exact identification, calculate the full vector \(Z'(y-X\hat{\beta})\); it should be zero apart from numerical rounding. In overidentification, check \(X'P_Z(y-X\hat{\beta})\), or equivalently \((P_ZX)'(y-X\hat{\beta})\); individual entries of \(Z'(y-X\hat{\beta})\) need not vanish.

Ordinary least squares uses the normal equations \(X'(y-X\hat{\beta}_{OLS})=0\). The 2SLS equations instead use \(X'P_Z(y-X\hat{\beta}_{2SLS})=0\). Confusing these equations can lead to incorrect checks or interpretations. In particular, orthogonality to the fitted regressors \(P_ZX\) is not the same as orthogonality to every instrument when there are more instruments than regressors.

These equations describe the sample estimator, not a test of instrument validity. A small or zero sample moment does not by itself prove the assumptions needed for a causal interpretation. Likewise, when the instrument matrix or projected regressors fail the rank conditions, the algebra cannot deliver a unique 2SLS coefficient vector. Diagnostics for weak instruments and statistical inference require additional results beyond these normal equations.