Many-Instrument Asymptotics and First-Stage Overfitting

Why the Number of Instruments Matters

Two-stage least squares uses instruments to construct a fitted version of the endogenous regressor. With a fixed, modest instrument set, this projection can isolate variation associated with the instruments. But as the number of instruments grows, the projection space also becomes better able to fit random variation in the sample. That fitted noise can be correlated with the structural error, even when the instruments themselves are valid.

The issue is not simply that a model has “many” instruments in an informal sense. The central comparison is between the dimension of the instrument space and the sample size. If that dimension is a nonnegligible fraction of the sample, a nonnegligible fraction of the endogenous regressor’s noise can be projected into its fitted values. This tutorial derives that mechanism in a simple linear model, then distinguishes the implications for consistency and for conventional inference.

Projection, Leverage, and a Working Model

For clarity, suppose exogenous controls have already been partialled out. Consider one endogenous regressor \(x\) and one structural outcome \(y\):

Here \(\beta\) is the structural coefficient, \(u\) is the structural error, \(\pi\) describes the first-stage component explained by the instruments, and \(v\) is the remaining first-stage disturbance. Assume the instruments are valid in the sense that \(E(u\mid Z)=E(v\mid Z)=0\). Endogeneity is allowed: \(u\) and \(v\) can be correlated. For the derivation below, assume observations are independent across \(i\), the conditional variances are constant, \(E(v_i^2\mid Z)=\sigma_v^2\), and \(E(v_i u_i\mid Z)=\sigma_{vu}\). Take \(K<n\), and assume the relevant quadratic forms concentrate around their means as \(n\) grows.

These conditions describe a stylized setting for isolating the mechanism. In particular, the instruments are valid, but the first-stage disturbance can share variation with the structural error. Define the first-stage signal \(Q_n=\pi'Z'Z\pi\), and suppose \(Q_n/n\to q>0\). This expresses strong aggregate first-stage signal on the sample-size scale.

How the Instrument Space Fits Noise

Write \(x=Z\pi+v\). Since \(P_ZZ=Z\), the projected regressor is \(P_Zx=Z\pi+P_Zv\). The term \(Z\pi\) is the signal, while \(P_Zv\) is projected first-stage noise. Its expected squared length, conditional on \(Z\), is

To see why, expand \(v'P_Zv=\sum_{i,j}v_i(P_Z)_{ij}v_j\). Independence across observations makes the conditional expectations of the off-diagonal terms zero; the diagonal terms sum to \(\sigma_v^2\sum_i h_{ii}\). Thus each additional independent instrument direction contributes, on average, another \(\sigma_v^2\) of projected noise. Equivalently, \(E[(P_Zv)_i^2\mid Z]=\sigma_v^2 h_{ii}\). The leverage identifies where projected noise is concentrated, while the trace identity gives its total expected amount.

The instrument count therefore controls noise fitting even if the instruments are valid. Validity means the instruments are uncorrelated with the structural disturbance in the specified sense; it does not mean the sample projection is incapable of fitting chance variation in \(v\).

Deriving the Many-Instrument Bias Mechanism

The numerator and denominator of the estimation error can be expanded using \(x=Z\pi+v\). Because \(P_ZZ=Z\) and \(P_Z\) is symmetric,

Conditional on \(Z\), the first numerator term has expectation zero by instrument validity. The second has expectation \(\sigma_{vu}\operatorname{tr}(P_Z)=K\sigma_{vu}\): expanding it into entries leaves only the diagonal contributions \(E(v_i u_i\mid Z)h_{ii}\). In the denominator, the cross term has expectation zero, and \(E(v'P_Zv\mid Z)=K\sigma_v^2\). Hence

These are expectations of the numerator and denominator separately; in general, their ratio is not the exact finite-sample expectation of the estimator. Under the concentration conditions stated above, however, their leading orders describe the probability limit. In particular, with \(K/n\to\kappa\), \(Q_n/n\to q>0\), and \(K=O(n)\), the centered fluctuations of these quadratic and linear forms are of smaller order than \(n\). Dividing numerator and denominator by \(n\) yields the limit:

The extra term is nonzero when the instrument count is a positive fraction of the sample and the first-stage disturbance is correlated with the structural error. It arises even though the instruments are valid: the sample projection fits some of \(v\), and that fitted component is associated with \(u\) through \(\sigma_{vu}\). When \(\kappa=0\), this particular probability-limit bias disappears under the stated strong-signal conditions.

Consistency and the Rate Needed for Conventional Inference

A vanishing \(K/n\) removes the nonzero probability-limit bias in this model, but that fact alone does not guarantee that conventional inference is centered accurately at the usual square-root sample-size rate. When \(K/n\) is small, the leading bias is approximately \(K\sigma_{vu}/(nq)\). Multiplying by \(\sqrt n\) gives an approximate scaled bias of

Thus, for this component of bias to be negligible relative to an estimator’s usual \(1/\sqrt n\) sampling scale, a sufficient rate in this stylized calculation is \(K/\sqrt n\to0\), along with the other regularity conditions required by the relevant central limit theorem and variance estimator. If \(K/n\to0\) but \(K/\sqrt n\) does not, the estimator can be consistent while the bias remains important relative to a conventional standard error. These are distinct claims: consistency concerns whether the estimation error tends to zero, while conventional centered inference concerns its size and distribution after scaling.

Conventional 2SLS standard errors and normal-reference tests are typically justified under fixed-instrument asymptotics, or under growing-instrument conditions strong enough to preserve the relevant centering and variance approximation. If the bias is of constant order while a conventional standard error shrinks at order \(1/\sqrt n\), the confidence interval can become increasingly concentrated away from the true coefficient. A robust variance calculation may address certain forms of heteroskedasticity, but it does not by itself remove this many-instrument bias or make an invalid centering valid.

Common Mistake: Treating Leverage as Treatment Variation

A tempting but incorrect explanation is that a high \(h_{ii}\) means observation \(i\) has a large amount of treatment variation that the instruments explain. That confuses a property of the projection design with a realized value of the regressor. The diagonal \(h_{ii}\) is determined by the rows of \(Z\) and the instrument space; changing \(x_i\) while holding \(Z\) fixed does not change \(h_{ii}\). It describes how the projection can weight observation \(i\), not how much treatment variation that observation actually contains.

A related mistake is to say that valid instruments cannot produce overfitting. Validity is a population condition about the relationship of instruments to the structural error. Overfitting here is a finite-sample projection phenomenon: a higher-dimensional instrument space captures more of the realized first-stage noise. The resulting projected noise can carry the disturbance correlation between \(v\) and \(u\) into the 2SLS numerator.