Why More Instruments Can Increase Bias
Adding valid instruments can seem like an unambiguous improvement: the instruments provide more information about an endogenous regressor. But in a finite sample, each additional instrument also gives the first stage another direction in which to fit the observed regressor. If those directions mostly capture sample noise, the fitted regressor can inherit a component correlated with the structural error. The resulting 2SLS estimate can move toward an endogeneity-driven benchmark rather than toward the structural coefficient.
The key is a projection. 2SLS uses the part of the endogenous regressor lying in the instrument space. As that space grows, it can contain more of the regressor’s noise, not just more of its systematic first-stage signal. This tutorial derives that mechanism in a simple model. Its bias formula is a ratio-of-expectations approximation, not an exact expression for the expectation of the 2SLS estimator.
A Projection Representation of 2SLS
Consider a model with one endogenous regressor and no additional controls:
Here \(y\) and \(x\) are \(n\)-vectors, \(Z\) is an \(n\times K\) matrix of excluded instruments with full column rank, and \(\mu\) is the systematic first-stage component. The vectors \(v\) and \(u\) are the first-stage disturbance and structural error. Assume the instruments are valid: \(E[Z'u]=0\). The first-stage disturbance can nevertheless be correlated with the structural error, so \(E[v_i u_i]=\sigma_{vu}\) need not be zero. This correlation is the source of endogeneity.
For the calculation below, take \(Z\) as fixed, assume observations are independent across \(i\), and assume \(E[v]=E[u]=0\), \(\operatorname{Var}(v_i)=\sigma_v^2\), and \(\operatorname{Cov}(v_i,u_i)=\sigma_{vu}\), with constant variances and covariance across observations. Define the projection onto the instrument space as \(P_Z=Z(Z'Z)^{-1}Z'\). Its rank and trace are both \(K\), and \(P_Z\mu=\mu\) because \(\mu\) lies in the column space of \(Z\).
The formula makes the risk of overfitting visible. Even if the population instruments are valid, the realized projection \(P_Zv\) need not be zero. With more instrument directions, more of the sample variation in \(v\) can be included in \(P_Zx\). Since \(v\) and \(u\) are correlated, this projected noise can also contribute to \(x'P_Zu\).
Expected Projection Components and the Bias Approximation
Expand the numerator and denominator of the estimation error:
The assumptions imply \(E[\mu'u]=0\) and \(E[\mu'v]=0\). They also give the two projection calculations needed to compare expected numerator and denominator:
For example, the first equality follows by writing \(v'P_Zu=\sum_{i,j}v_i(P_Z)_{ij}u_j\). Independence across observations and zero means make the \(i\ne j\) terms have expectation zero, while \(E[v_i u_i]=\sigma_{vu}\); the remaining terms sum to \(\sigma_{vu}\sum_i(P_Z)_{ii}\). The variance calculation is analogous. Since \(P_Z\mu=\mu\), it follows that \(E[x'P_Zx]=\mu'\mu+K\sigma_v^2\).
Define the first-stage signal measure \(\lambda=\mu'\mu/\sigma_v^2\), when \(\sigma_v^2>0\). The approximation can then be written as
The factor \(K/(\lambda+K)\) shows how the approximation depends on instrument count relative to signal. With signal held fixed, increasing \(K\) raises that factor. The quantity \(\sigma_{vu}/\sigma_v^2\) is the noise-only endogeneity slope: it is the slope associated with the covariance between the noise component \(v\) and \(u\), scaled by the variance of \(v\). It is not generally the OLS coefficient shift.
Worked Example: Adding a Noise Direction to the Projection
Worked Example: Adding a Noise Direction to the Projection
Consider a hypothetical sample of four observations. Write sample vectors in an orthonormal basis \(e_1,e_2,e_3,e_4\), and suppose \(x=2e_1+e_2+e_3\) and \(u=0.6(e_2+e_3)+e_4\). Let the first instrument span \(e_1\); then consider adding a second instrument spanning \(e_2\). The population first-stage signal in this illustration is \(2e_1\), while \(e_2+e_3\) is a particular realized noise component. The calculation illustrates a sample projection, not a claim that a valid instrument must have a nonzero sample covariance with the error in every realization.
One instrument. The projection is \(P_1=e_1e_1'\), so \(P_1x=2e_1\) and \(P_1u=0\). The contribution to the 2SLS estimation error is $$ \frac{x'P_1u}{x'P_1x} =\frac{(2e_1)'0}{(2e_1)'(2e_1)} =\frac{0}{4}=0. $$
Two instruments. The expanded projection is \(P_2=e_1e_1'+e_2e_2'\). Thus \(P_2x=2e_1+e_2\) and \(P_2u=0.6e_2\). Using the orthonormality of the basis, $$ \frac{x'P_2u}{x'P_2x} =\frac{(2e_1+e_2)'(0.6e_2)}{(2e_1+e_2)'(2e_1+e_2)} =\frac{0.6}{4+1}=0.12. $$
The added direction has captured a component of the realized noise in \(x\), and that component is aligned with \(u\). The resulting estimation error is \(0.12\) in this sample. The example does not say that every added valid instrument increases bias in every realization; it shows the projection mechanism that can generate many-instrument bias on average.
Worked Example: Comparing Instrument Counts and Benchmarks
Worked Example: Comparing Instrument Counts and Benchmarks
Suppose a hypothetical design has \(n=100\), \(\mu'\mu=40\), \(\sigma_v^2=1\), and \(\sigma_{vu}=0.6\). Compare the ratio-of-expectations bias approximation for \(K=20\) and \(K=80\), and calculate the corresponding OLS ratio-of-expectations benchmark. These values describe the stated model and are illustrative, not empirical estimates.
For \(K=20\), the approximation is $$ \frac{K\sigma_{vu}}{\mu'\mu+K\sigma_v^2} =\frac{20(0.6)}{40+20(1)} =\frac{12}{60}=0.2000. $$
For \(K=80\), it is $$ \frac{80(0.6)}{40+80(1)} =\frac{48}{120}=0.4000. $$
For comparison, the OLS slope is \(\hat{\beta}_{OLS}=\beta+x'u/(x'x)\). Under the same assumptions, \(E[x'u]=n\sigma_{vu}=60\) and \(E[x'x]=\mu'\mu+n\sigma_v^2=40+100=140\). Its ratio-of-expectations shift is therefore $$ \frac{n\sigma_{vu}}{\mu'\mu+n\sigma_v^2} =\frac{100(0.6)}{40+100} =\frac{60}{140}=0.4286. $$
The noise-only endogeneity slope is \(\sigma_{vu}/\sigma_v^2=0.6000\), which differs from the OLS benchmark \(0.4286\) in this example. As the instrument count approaches the full \(n\)-dimensional projection in this no-control setup, \(P_Z\) approaches the identity, and the 2SLS estimator becomes OLS; the ratio-of-expectations expressions then agree. With fewer instruments, the approximation depends on both \(K\) and the signal, so it should not be described as generally moving toward the OLS shift at the noise-only slope.
Worked Example: In-Sample First-Stage Fit Under No Signal
Worked Example: In-Sample First-Stage Fit Under No Signal
Consider a hypothetical first stage with \(n=50\) observations and no population first-stage signal, so \(x=v\). Suppose \(v\sim N(0,\sigma_v^2 I)\), and the instrument space is a fixed \(K\)-dimensional subspace with \(K<n\). There is no intercept in this setup. The in-sample first-stage \(R^2\) is \(v'P_Zv/v'v\).
Choose an orthonormal basis whose first \(K\) directions span the instrument space. In that basis, the coordinates of \(v\) are independent normal variables with the same variance. The expected share of the total squared length in the first \(K\) coordinates is \(K/n\): by symmetry, each of the \(n\) coordinates contributes the same expected share, and all \(n\) shares sum to one. Thus \(E[R^2]=K/n\).
With \(K=5\), the expected in-sample \(R^2\) is \(5/50=0.1000\). With \(K=20\), it is \(20/50=0.4000\). The expected fitted noise sum of squares is \(K\sigma_v^2\), changing from \(5\sigma_v^2\) to \(20\sigma_v^2\). No population signal has been added; the larger instrument space simply captures a larger share of the realized noise. A high in-sample first-stage fit, on its own, therefore does not establish that the instruments provide strong population information about \(x\).
Interpretation and Practical Implications
The projection argument identifies a finite-sample trade-off. More instruments can include more first-stage signal, which may be useful, but they also add directions in which sample noise can be fitted. If that noise is correlated with the structural error, its contribution to the 2SLS numerator can pull the estimate away from \(\beta\). The approximate shift depends on the balance between \(K\sigma_v^2\), the projected noise variation, and \(\mu'\mu\), the signal.
Instrument count alone does not determine whether an instrument set is appropriate. Validity remains essential: an instrument correlated with \(u\) in the population violates the exclusion restriction, a different problem from finite-sample overfitting. Nor does the calculation imply that one should discard valid, informative instruments mechanically. It explains why a large instrument set relative to the available signal and sample can make conventional 2SLS behave poorly, and why an apparently impressive in-sample first stage may be misleading.
The formulas here use one endogenous regressor, no additional controls, and a homoskedastic covariance structure. With included controls, the corresponding projection argument is applied after residualizing the variables with respect to those controls, using the rank of the residualized instrument space. Under heteroskedasticity, the simple covariance calculations above do not directly supply the same approximation. In all cases, distinguish an approximation to bias from an exact expectation, and keep instrument validity, relevance, and finite-sample overfitting conceptually separate.
Common Mistake
Check Your Understanding
Use the projection representation and assumptions in this tutorial to answer the following questions.
- Why does \(E[v'P_Zu]\) equal \(K\sigma_{vu}\) under the stated assumptions?
- For \(\mu'\mu=30\), \(\sigma_v^2=2\), \(\sigma_{vu}=0.5\), and \(K=10\), calculate the ratio-of-expectations bias approximation.
- In what sense can additional instruments overfit the endogenous regressor even when they have no population first-stage signal?
- Why is \(\sigma_{vu}/\sigma_v^2\) not generally the OLS ratio-of-expectations shift?
- Does a rise in in-sample first-stage \(R^2\) establish that the added instruments are valid or informative in the population? Explain.