Overidentified IV And Instrument Weighting

Why More Instruments Change the Estimation Problem

In an instrumental-variables model, instruments provide directions of variation that can help identify coefficients on endogenous regressors. When the number of independent instruments exceeds the number of coefficients to be identified, the population moment conditions contain more restrictions than are needed for identification. In a sample, however, those restrictions will generally not all hold exactly at one coefficient vector. Two-stage least squares (2SLS) gives a systematic way to use the predictive information in the whole instrument set.

The key is to distinguish three ideas: how many instruments are listed, how many independent instrument directions they provide, and how many coefficients must be identified. A count alone is not enough: redundant instruments do not create additional independent restrictions, and having many instruments does not guarantee that the coefficients are identified.

Model, Moment Conditions, and Identification

For clarity, write the structural model after accounting for any included exogenous controls:

y=X\beta+u.

Here \(y\) is an \(n\times1\) outcome vector, \(X\) is an \(n\times k\) matrix of endogenous regressors, \(\beta\) is a \(k\times1\) coefficient vector, and \(u\) is the structural disturbance. Let \(Z\) be an \(n\times q\) matrix of excluded instruments. In the population, the instrument-validity moment conditions are \(E[z_i u_i]=0\), one for each instrument component. This is an orthogonality condition; it is not necessarily a zero-covariance statement unless suitable mean-zero conditions also hold.

Definition: The model is overidentified when it has more linearly independent valid moment conditions than coefficients to identify. With \(q\) independent instrument moments and \(k\) coefficients, the number of overidentifying restrictions is \(q-k\), provided \(q>k\) and the identification rank condition holds.

The population rank condition for identifying all \(k\) coefficients is

\operatorname{rank}\big(E[z_i x_i']\big)=k.

This says that the instruments’ population cross-moments with the endogenous regressors span \(k\) distinct coefficient directions. It does not say that the \(q\) instruments themselves are linearly independent. Nor does it establish instrument validity: validity is the separate orthogonality condition \(E[z_i u_i]=0\).

If the model includes exogenous controls, they can be included as their own instruments. Equivalently, one can partial out those controls and work with residualized \(y\), \(X\), and excluded instruments. The \(q\) and \(k\) counts here refer to the excluded instrument moments and endogenous-regressor coefficients after that adjustment; the included controls do not create excluded overidentifying restrictions.

Worked Example: Counting Independent Restrictions

Suppose there are two endogenous regressors and three excluded instruments. Assume the three instrument variables provide three linearly independent moment conditions, and suppose

E[z_i x_i'] = \begin{pmatrix} 1&0\\ 0&1\\ 1&1 \end{pmatrix}.

The matrix has rank 2: its first two rows already span both coefficient directions. Thus the rank condition for identifying the two coefficients holds. Under the stated assumption that all three instrument moments are linearly independent, the number of overidentifying restrictions is \(q-k=3-2=1\).

The independence assumption matters. If instead the third instrument were exactly the sum of the first two, the listed set would contain only two independent instrument directions. The cross-moment matrix could still have rank 2 and identify both coefficients, but there would not be three independent moments and hence not one extra restriction. Identification rank and the count of independent moments answer different questions.

2SLS as Projection onto the Instrument Space

In a sample, the columns of \(Z\) span the instrument space: all linear combinations of the observed instrument columns. Assume \(Z\) has full column rank \(q\), so its projection matrix is

P_Z=Z(Z'Z)^{-1}Z'.

This matrix projects each column of a data matrix onto the column space of \(Z\). It is symmetric and idempotent: \(P_Z'=P_Z\) and \(P_Z^2=P_Z\). In particular, \(P_ZX\) contains the fitted values from projecting each endogenous regressor on all the instruments. The sample rank condition is \(\operatorname{rank}(Z'X)=k\), which ensures the projected regressors provide \(k\) distinct directions.

Definition: The 2SLS estimator, when the required inverses exist, is

\hat{\beta}_{2SLS}=(X'P_ZX)^{-1}X'P_Zy.

It is the ordinary least-squares coefficient from regressing \(y\) on the projected regressors \(P_ZX\), not from regressing \(y\) on \(X\) as if the endogenous regressors were exogenous.

The projection combines information from the entire instrument space. It does not pick a single instrument, and it does not generally give every instrument’s sample moment equation its own exactly satisfied coefficient restriction. The estimator uses the \(k\) coefficient directions available through \(X\), while an overidentified model supplies more than \(k\) instrument moments.

Worked Example: Computing 2SLS by Projection

Consider four hypothetical observations, one endogenous regressor \(x\), and two instruments. There are no additional controls. Let the instrument columns and the outcome and regressor vectors be

z_1= \begin{pmatrix}1\\1\\0\\0\end{pmatrix}, \qquad z_2= \begin{pmatrix}0\\0\\1\\1\end{pmatrix}, \qquad x= \begin{pmatrix}2\\0\\1\\1\end{pmatrix}, \qquad y= \begin{pmatrix}3\\1\\4\\2\end{pmatrix}.

The two instrument columns are nonzero and orthogonal, with \(z_1'z_1=z_2'z_2=2\). They are therefore linearly independent, and

P_Z=\frac{z_1z_1'}{2}+\frac{z_2z_2'}{2}.

Projecting \(x\) onto the instrument space replaces its values within each instrument group by that group’s mean:

P_Zx= \begin{pmatrix}1\\1\\1\\1\end{pmatrix}.

The projected regressor is not zero, so the sample rank condition holds for this one-coefficient example. Similarly,

P_Zy= \begin{pmatrix}2\\2\\3\\3\end{pmatrix}.

With one coefficient, the 2SLS formula reduces to a scalar ratio. Substituting the data gives

\hat{\beta}_{2SLS} =\frac{x'P_Zy}{x'P_Zx} =\frac{(2)(2)+(0)(2)+(1)(3)+(1)(3)} {(2)(1)+(0)(1)+(1)(1)+(1)(1)} =\frac{10}{4}=2.500.

The calculation uses the variation in \(x\) that is projected onto the two-instrument space. It does not establish that the instruments are valid in a population; that conclusion requires the model’s substantive validity assumptions, not just this sample projection.

How Instrument Weighting Appears in 2SLS

The projection formula also makes the combination of instrument information explicit in a simple case. If the instrument columns are mutually orthogonal, then the projection is a sum of projections onto each instrument column. For one endogenous regressor and \(q\) such instruments, 2SLS can be written as

\hat{\beta}_{2SLS} = \frac{\displaystyle\sum_{j=1}^{q} \frac{(z_j'x)(z_j'y)}{z_j'z_j}} {\displaystyle\sum_{j=1}^{q} \frac{(z_j'x)^2}{z_j'z_j}}.

When \(z_j'x\ne0\), define the instrument-specific ratio \(r_j=(z_j'y)/(z_j'x)\). Then the same formula is a weighted average of these ratios, with weights \(w_j=(z_j'x)^2/(z_j'z_j)\):

\hat{\beta}_{2SLS} = \frac{\sum_{j=1}^{q}w_jr_j}{\sum_{j=1}^{q}w_j}.

The weights are nonnegative and reflect how strongly each instrument projects onto \(x\), relative to its scale. This weighted-average expression relies on orthogonal instrument columns. With correlated instruments, the projection still defines 2SLS, but the corresponding combination accounts for their joint geometry rather than simply averaging separate ratios.

Worked Example: Predictive Contributions from Two Orthogonal Instruments

Consider a hypothetical sample with one endogenous regressor and two orthogonal instrument columns. Suppose the relevant cross-products are \(z_1'z_1=4\), \(z_1'x=4\), \(z_1'y=12\), \(z_2'z_2=1\), \(z_2'x=1\), and \(z_2'y=0\). Both instruments have nonzero sample predictive association with \(x\), so each contributes to the projected regressor.

The individual ratios are \(r_1=12/4=3\) and \(r_2=0/1=0\). Their weights are

w_1=\frac{4^2}{4}=4, \qquad w_2=\frac{1^2}{1}=1.

Thus the 2SLS coefficient is

\hat{\beta}_{2SLS} =\frac{4(3)+1(0)}{4+1} =\frac{12}{5}=2.400.

The result lies between the two instrument-specific ratios, with more weight on the first instrument because it contributes more to the projected variation in \(x\). This arithmetic describes the projection’s weighting; it does not by itself establish validity or justify interpreting either ratio as a causal effect.

Why the Sample Moments Do Not All Vanish

For a just-identified model, the number of independent sample moment equations matches the number of coefficients, so the equations can often be solved exactly. In an overidentified model, there are more instrument moment equations than coefficients. Sample variation will generally prevent a single \(\hat{\beta}\) from making every component of \(Z'(y-X\hat{\beta})\) equal zero.

2SLS instead satisfies its projected normal equations:

X'P_Z(y-X\hat{\beta}_{2SLS})=0.

To verify this, substitute the estimator and use \(P_Z'=P_Z\):

\begin{aligned} X'P_Z(y-X\hat{\beta}_{2SLS}) &=X'P_Zy-X'P_ZX(X'P_ZX)^{-1}X'P_Zy\\ &=X'P_Zy-X'P_Zy\\ &=0. \end{aligned}

These are \(k\) equations in \(k\) coefficient directions. They do not imply \(Z'(y-X\hat{\beta}_{2SLS})=0\) component by component when there are more than \(k\) independent instruments. The remaining discrepancy is the sample manifestation of the extra restrictions. Overidentification makes those restrictions available for assessment; it does not ensure that they are true.

Common Pitfalls

Counting listed instruments as independent moments. Three instrument columns do not automatically mean three independent restrictions. Check for linear redundancy in the instrument set; the cross-moment rank condition alone does not establish instrument independence.
Confusing identification with validity. The condition \(\operatorname{rank}(E[z_i x_i'])=k\) concerns identification. Instrument validity requires the orthogonality conditions \(E[z_i u_i]=0\), which are distinct.
Calling orthogonality zero covariance without qualification. The condition \(E[z_i u_i]=0\) is a zero product expectation. It equals zero covariance only when the relevant means make that equality hold.
Assuming 2SLS satisfies every sample moment exactly. In an overidentified model, 2SLS satisfies the projected normal equations, not generally every component of \(Z'(y-X\hat{\beta})=0\).
Treating a projection as proof of causality. 2SLS uses instrument-predictable regressor variation. A causal interpretation still depends on the structural model and valid instrument assumptions.

Key takeaway: Overidentification requires more independent valid moment conditions than coefficients, alongside the rank condition for identification. 2SLS combines the instruments by projecting the endogenous regressors onto their joint span and fitting the outcome using that projected variation.

Check Your Understanding

Answer each question using the definitions and projection logic developed above.

A model has two endogenous regressors and four linearly independent valid instrument moments. How many overidentifying restrictions does it have, assuming the rank condition holds?
Why does a rank-two cross-moment matrix for three instruments and two regressors not, by itself, prove there are three independent moment conditions?
In the 2SLS formula, what role does \(P_ZX\) play, and what condition is needed for the projected regressors to identify all coefficients?
Why does 2SLS generally not set every component of \(Z'(y-X\hat{\beta}_{2SLS})\) to zero in an overidentified sample?
What additional assumption is needed before \(E[z_i u_i]=0\) can be described as zero covariance?