Many Instruments and First-Stage Overfitting

Why a Large Instrument Set Can Mislead

A first stage can look impressive because it fits the observed endogenous regressor well. But fit is not the same as reliable instrument strength. If many excluded instruments are included, they can explain sample-specific noise in the endogenous regressor, even when they contain little genuine predictive information. That overfitting can make an unadjusted first-stage $R^2$ rise while the evidence of useful first-stage signal remains weak.

The concern is not simply that a model has “too many” instruments according to a universal number. It is the size of the instrument set relative to the independent information in the sample, the strength of the first-stage signal, and the validity of the instruments. A large set can also worsen the finite-sample behavior of two-stage least squares (2SLS): when the first stage fits noise, the second stage may inherit a component of the endogeneity that instruments were meant to remove.

Definition: In a linear IV model, let $Y$ be the outcome, $D$ the endogenous regressor, $W$ a vector of included exogenous controls, and $Z$ a vector of $K$ excluded instruments. The structural equation and first stage are $$ Y = \beta D + W'\gamma + u,\qquad D = Z'\pi + W'\delta + v. $$ The instruments must be relevant for $D$, satisfy the exclusion restriction (they affect $Y$ through $D$, not directly), and be exogenous to the structural error $u$. The rank condition requires enough independent instrument variation to identify the endogenous coefficient. “Many instruments” describes a setting where $K$ is large relative to the sample’s independent information; there is no universal cutoff that defines it.

The first-stage strength statistic should assess the excluded instruments’ contribution after accounting for $W$, not merely the fit of the full first-stage regression. In the simple case with one endogenous regressor, independent observations, homoskedastic errors, and an intercept but no other controls, the classical first-stage $F$ statistic for $K$ excluded instruments can be written in terms of the partial $R^2$:

F = \frac{R^2_{\mathrm{partial}}/K}{(1-R^2_{\mathrm{partial}})/(n-K-1)}.

Here $R^2_{\mathrm{partial}}$ is the fraction of the residual variation in $D$, after removing the intercept and included controls, explained by the excluded instruments. With additional controls, the denominator degrees of freedom are reduced by the number of estimated control parameters as well. The formula makes an important point: the numerator’s explained variation is assessed relative to the number of excluded instruments, and the denominator depends on the remaining degrees of freedom.

How Extra Instruments Inflate Fit

In an ordinary least-squares regression, adding a predictor cannot reduce the in-sample $R^2$. This is a mathematical property of least squares, not evidence that the new predictor has real predictive value beyond the sample. When instruments are numerous, some can align with random fluctuations in $D$. The resulting increase in $R^2$ may therefore reflect fitting noise rather than a stronger population relationship.

A simple benchmark clarifies the issue. Suppose, after removing the intercept and controls, the instruments are unrelated to the first-stage disturbance and have rank $K$. Under a null first stage, projection onto their span assigns $K$ dimensions to fitting the residual variation. The expected fraction explained is approximately $K/(n-r)$, where $r$ is the number of dimensions already used for the intercept and controls. This is a benchmark for the unadjusted fit under no signal, not a universal correction to apply to an observed $R^2$. As $K$ grows, a nontrivial amount of apparent fit can arise by chance alone.

Worked Example: More Fit, Less First-Stage Evidence

Consider a hypothetical sample of $n=120$ independent observations, an intercept, one endogenous regressor, and two alternative instrument sets. The smaller set has $K=5$ instruments and partial $R^2=0.10$; the larger set has $K=30$ instruments and partial $R^2=0.24$. Calculate the classical first-stage $F$ statistic for each set.

For $K=5$, the denominator degrees of freedom are $120-5-1=114$. Therefore, $$ F = \frac{0.10/5}{0.90/114} = \frac{0.020}{0.007895} \approx 2.533. $$

For $K=30$, the denominator degrees of freedom are $120-30-1=89$. Thus, $$ F = \frac{0.24/30}{0.76/89} = \frac{0.008}{0.008539} \approx 0.9368. $$

The larger set explains more residual variation in the sample, but its $F$ statistic is lower. The additional fit is spread across many more instruments and is not strong evidence that the set supplies a concentrated first-stage signal. These calculations use the classical statistic’s independent, homoskedastic-error assumptions; with heteroskedasticity or clustering, a robust or cluster-robust test is needed instead.

Why Many Instruments Can Bias 2SLS in Finite Samples

The finite-sample problem can be seen by partialling out $W$, leaving a regression with no included controls. Let $P_Z$ denote projection onto the instrument space, and write the residualized first stage as $D=Z\pi+v$. The 2SLS estimation error is

\hat{\beta}_{2SLS}-\beta = \frac{D'P_Zu}{D'P_ZD}.

Even if instruments are exogenous, the first-stage disturbance $v$ can be correlated with the structural error $u$; that is the reason $D$ is endogenous. Under the null of no first-stage signal, $D=v$. If errors are homoskedastic and independent across observations, and $Z$ has rank $K$, then

E[v'P_Zu] = K\sigma_{vu}, \qquad E[v'P_Zv] = K\sigma_v^2,

where $\sigma_{vu}$ is the covariance of $v$ and $u$, and $\sigma_v^2$ is the variance of $v$. These relationships follow from the trace of the projection matrix: $E[v'P_Zu]=\sigma_{vu}\operatorname{tr}(P_Z)=K\sigma_{vu}$, and $E[v'P_Zv]=\sigma_v^2\operatorname{tr}(P_Z)=K\sigma_v^2$.

This calculation shows the mechanism: projecting onto more instrument dimensions captures more of the shared sample noise in $v$ and $u$. The ratio of expected numerator to expected denominator is $\sigma_{vu}/\sigma_v^2$, the OLS endogeneity component in the no-signal case. This ratio is an intuition, not an exact expression for the expected 2SLS estimate: in general, the expectation of a ratio is not the ratio of expectations. With a nonzero first-stage signal, the signal adds to the denominator, but the noise contribution still grows with the number of instruments. This helps explain why finite-sample 2SLS estimates can be biased toward OLS, especially when the first stage is weak and the instrument count is large relative to the sample.

Worked Example: A Projection-Noise Approximation

Consider a hypothetical residualized model with $n=200$ observations and $K=40$ independent instrument dimensions. Suppose $E[Z'Z]=200I$, the first-stage coefficient vector satisfies $\pi'\pi=0.10$, the first-stage disturbance variance is $\sigma_v^2=1$, and the covariance between the first-stage and structural errors is $\sigma_{vu}=0.60$. Use the ratio of expected moments as a bias approximation.

The expected signal contribution to $D'P_ZD$ is $n\pi'\pi=200(0.10)=20$. The projection-noise contribution is $K\sigma_v^2=40(1)=40$, so the expected denominator is $20+40=60$. The expected numerator is $K\sigma_{vu}=40(0.60)=24$. The ratio-of-moments approximation is therefore $$ \frac{E[D'P_Zu]}{E[D'P_ZD]} \approx \frac{24}{60}=0.400. $$

This is an approximate finite-sample endogeneity component, not an exact expected bias or a confidence statement. It illustrates how a large instrument dimension can contribute substantially to the projected noise when the signal is limited. Under the same simplified variance setup, the population OLS endogeneity component is $\sigma_{vu}/(\pi'\pi+\sigma_v^2)=0.60/1.10\approx0.545$, so the approximation is in the OLS direction.

Diagnosing and Responding to a Large Instrument Set

A useful assessment considers the instrument count alongside the strength statistic and the design. Report $K$, the sample size and its independent units, the partial first-stage fit, and a first-stage strength statistic appropriate to the error structure. For clustered data, the number of independent clusters matters; a large observation count does not compensate automatically for few independent clusters. The classical $F$ statistic is not justified under arbitrary heteroskedasticity or dependence, so use a robust or cluster-robust procedure when those features are present. The Effective F Statistic and other weak-instrument diagnostics address inferential concerns that a raw first-stage $R^2$ cannot resolve.

There is no rule that a particular instrument-to-observation ratio makes an analysis invalid. Concern increases when $K$ consumes a substantial share of available degrees of freedom, the first-stage signal is weak, or the estimate changes materially across defensible instrument sets. Adding instruments also adds identifying assumptions: each excluded instrument must satisfy exogeneity and exclusion, and the rank condition must hold. A large set does not make invalid instruments valid.

Possible responses depend on the scientific design. If there is a principled reason to retain only a subset, a smaller set can reduce overfitting, but selecting instruments solely because they give a favorable estimate or test result undermines the analysis. Limited-information maximum likelihood (LIML) and Fuller estimators are alternatives sometimes considered because their finite-sample behavior can be less sensitive to many-instrument bias than 2SLS; they do not repair invalid instruments or guarantee strong identification. Weak-instrument-robust inference is another approach when conventional 2SLS inference is unreliable. These alternatives answer different concerns, so the analysis should explain the estimand, assumptions, and diagnostic evidence rather than treating a single statistic as a pass/fail certificate.

Worked Example: Choosing What to Report

A hypothetical study has 180 independent observations and 25 excluded instruments for one endogenous treatment. The classical first-stage output reports a full-model $R^2$ of 0.42, while the partial $R^2$ for the excluded instruments is 0.18. Explain what the analyst should and should not conclude.

The full-model $R^2$ includes any fit from included controls and is not the relevant measure of the excluded instruments’ contribution. Even the partial $R^2$ must be read with the instrument count. With an intercept and no other controls, the corresponding classical statistic would be $$ F = \frac{0.18/25}{0.82/(180-25-1)} = \frac{0.0072}{0.005325} \approx 1.352. $$

Thus the partial fit of 0.18 by itself should not be described as proof of strong instruments. The calculated statistic uses homoskedastic, independent errors; if the study has heteroskedasticity or clustering, the analyst should report an appropriate robust strength diagnostic instead. The analyst should also disclose the instrument count and sample’s independent units, assess whether every instrument has a defensible exclusion argument, and examine whether conclusions depend on a scientifically justified alternative set. The information given does not establish that the instruments are valid, and it does not support a causal conclusion by itself.

Common Mistakes and Limitations

“The $R^2$ increased, so the instruments got stronger.” In-sample $R^2$ cannot decrease when predictors are added. Examine the excluded instruments’ partial fit and a strength statistic that accounts for their number.
“A large instrument count is always invalid.” There is no universal cutoff. The concern depends on instrument count relative to independent sample information, signal strength, error dependence, and instrument validity.
“A high $F$ proves the exclusion restriction.” Strength diagnostics assess relevance under their assumptions; they do not test whether instruments are exogenous or excluded from the outcome equation.
“A robust standard error fixes many-instrument bias.” Robust standard errors address specified variance or dependence patterns. They do not by themselves remove finite-sample estimator bias or weak identification.

Diagnostics are evidence about particular problems, not a substitute for design reasoning. In particular, an apparently strong first stage cannot validate the assumptions that give an IV estimate a causal interpretation.

Check Your Understanding

Answer each question, then compare with the solutions.

Why can a first-stage $R^2$ rise when instruments that add no population signal are included?
With $n=100$, an intercept, $K=9$, and partial $R^2=0.09$, calculate the classical first-stage $F$ statistic.
What is the projection-noise mechanism that can move finite-sample 2SLS estimates toward OLS?
Name two reasons a classical first-stage $F$ statistic may be unsuitable in a particular study.
Does a strong first-stage diagnostic establish instrument validity? Explain.

Solutions. (1) Least-squares $R^2$ cannot fall when predictors are added; extra instruments can fit sample noise, so the increase need not represent population signal. (2) The denominator degrees of freedom are $100-9-1=90$, giving $F=(0.09/9)/(0.91/90)=0.010/0.01011\approx0.9890$. (3) If $v$ and $u$ are correlated, projection onto more instrument dimensions captures more shared sample variation; that can place an endogenous component into the fitted treatment and pull 2SLS toward OLS. (4) Heteroskedastic errors violate the classical variance assumption; clustered or otherwise dependent observations violate the independent-error calculation. In either case, a suitable robust procedure is needed. (5) No. A strength diagnostic concerns relevance, not whether instruments satisfy exogeneity and exclusion.