Why Manage a Large Instrument Set?
Adding instruments can increase the information available about an endogenous treatment, but it is not automatically beneficial. With many instruments relative to the sample size, two-stage least squares (2SLS) can overfit the first stage and become biased toward the ordinary least-squares estimate. Many-instrument asymptotics and LIML address aspects of this problem, but they do not answer a different practical question: which instruments should enter the analysis, and in what form?
Instrument reduction involves a trade-off. A smaller set may reduce overfitting and simplify the scientific argument, but it can also discard useful variation and make the treatment weakly identified. More fundamentally, keeping fewer instruments does not make the retained instruments valid. The exclusion restriction and other instrument assumptions still require substantive support.
For clarity, the equations below use variables residualized with respect to \(X\), so the displayed relationships omit \(X\). In that notation, a key validity condition is \(E[Zu]=0\), while relevance requires the instruments to predict \(D\) sufficiently to identify \(\beta\). The exact assumptions needed for causal interpretation depend on the design. In particular, a strong first stage is not a substitute for exogeneity or exclusion.
Four Strategies and Their Distinct Roles
There are four related but non-equivalent responses to a large candidate set. Theory-based restriction chooses instruments using substantive knowledge. First-stage screening chooses using observed predictive performance. Principal-component reduction replaces the original instruments with selected linear combinations. Jackknife IV (JIVE), by contrast, changes how fitted treatment values are constructed; it can help address first-stage overfitting without reducing the number of instruments.
| Strategy | What is changed? | Potential benefit | Main caution |
|---|---|---|---|
| Theory-based restriction | Keep a substantively defensible subset | Can avoid questionable exclusion restrictions and reduce dimension | Omitting relevant instruments may weaken identification |
| First-stage screening | Keep instruments or groups that predict treatment in the sample | Can remove evidently uninformative directions | Selection uses data and does not establish validity or guarantee ordinary inference |
| Principal components | Replace instruments with selected linear combinations | Can compress highly correlated instruments | Directions explaining little instrument variance may predict treatment well |
| JIVE | Use leave-one-out first-stage predictions in an IV estimator | Reduces the own-observation overfitting mechanism | Does not repair invalid instruments or remove weak identification |
Theory-Based Restriction: Start With the Scientific Argument
A theory-based restriction begins with the proposed mechanisms linking each candidate instrument to treatment and outcome. An instrument should be retained only if the design gives a credible reason to believe it shifts treatment and affects the outcome through treatment, rather than through a separate path. If a candidate has a plausible direct path to the outcome, or shares an unaddressed cause with the structural error, dropping it may be preferable to relying on it merely because it improves the first stage.
A fixed subset of valid instruments remains valid as a subset, provided the relevant moment conditions hold. But restriction can lower the rank or strength of the first stage. If the retained instruments barely predict treatment after conditioning on \(X\), estimates may be imprecise and conventional approximations may be unreliable. The choice is therefore not “validity versus relevance” in the abstract: it is a substantive judgment about credible restrictions, followed by a separate assessment of identification strength.
Worked Example: Restricting on a Plausible Exclusion Concern
Suppose a hypothetical study has four candidate instruments: distance to a clinic, historical clinic openings, a local outreach campaign, and a weather-related travel disruption. Researchers are estimating the effect of treatment \(D\) on outcome \(Y\). Before seeing first-stage results, they argue that distance and historical openings can affect treatment access but have no plausible direct pathway to \(Y\) after conditioning on \(X\). The outreach campaign may also affect health through information, and the travel disruption may affect health directly.
The proposed theory-based set is therefore \(Z_R=(Z_{\text{distance}},Z_{\text{openings}})\), not all four candidates. This decision uses the stated causal argument, not a calculation that proves the exclusion restriction. The retained instruments must still be relevant conditional on \(X\), and the claim that their other pathways are adequately controlled remains an assumption.
If the partial first-stage \(R^2\) for the four-instrument set were 0.18, but were 0.04 for the restricted set in this hypothetical analysis, that drop would be a warning that the restriction may leave little predictive information. It would not justify restoring an instrument with a serious exclusion concern: a stronger first stage cannot compensate for invalidity. The researchers would report the weaker identification and reconsider the design or the strength of their scientific argument.
First-Stage Screening: Predictiveness Is Not Validity
Screening uses observed first-stage associations to keep instruments that appear to predict \(D\). A researcher might examine joint tests, partial \(R^2\), or robust first-stage diagnostics for prespecified instrument groups. Screening can reveal that some candidates contribute little measurable predictive information. Yet a sample association can be noisy, and a candidate that looks strong in the first stage can still violate exclusion or exogeneity.
The inferential concern is that the same data are used to choose instruments and estimate the treatment effect. A selected set is data-dependent: conventional standard errors and confidence intervals calculated as if that set had been fixed in advance may not reflect the selection step. This concern does not mean that first-stage screening automatically changes a candidate instrument’s underlying causal pathway. It means that selection and inference must be considered together. The detailed treatment of inference after instrument selection is a separate topic.
Worked Example: Screening a Candidate Set
In a hypothetical sample, investigators prespecify four instrument groups and calculate robust partial first-stage statistics conditional on \(X\). The reported values are 12.4, 7.1, 2.0, and 0.8, respectively. They propose retaining the first two groups under a screening rule declared before estimating the outcome equation.
The arithmetic decision is simple: the first two reported statistics exceed the investigators’ prespecified screening threshold of 5, while the other two do not. The threshold here is a hypothetical design choice, not a universal weak-instrument cutoff. The statistics indicate evidence about predictive contribution under the fitted first stage; they do not show that the first two instruments satisfy exclusion.
The researchers should then assess identification strength using diagnostics appropriate to their setting and report that the instrument set was screened. If they instead tried many thresholds or many subsets and reported only the set producing the preferred treatment estimate, the selection could distort ordinary inference. Screening also does not guarantee that the retained set is strongly identifying in the population: sample estimates of predictive strength are uncertain.
Principal Components: Compressing Variation Can Lose Signal
Principal-component reduction forms new instruments as linear combinations of the original instruments. Let \(Z\) be an \(n \times L\) matrix of centered candidate instruments, and let \(A\) be an \(L \times K\) matrix whose columns define \(K\) combinations. The reduced instrument matrix is \(Z^*=ZA\), with \(K<L\). Principal components usually choose directions that explain large amounts of variation in \(Z\), not directions that best predict \(D\).
If the transformation \(A\) is fixed in advance and \(E[Zu]=0\), then the transformed instruments satisfy the corresponding moment condition. Indeed, the condition follows by multiplying the original moments by \(A'\):
This short argument establishes validity of fixed linear combinations under the original moment condition; it does not establish relevance. If the principal components are computed from \(Z\) alone, a conditional exogeneity condition such as \(E[u\mid Z,X]=0\) supports validity of those data-dependent transformations. Under only an unconditional moment restriction, one should not assume that every transformation estimated from the data automatically preserves that restriction. In either case, component selection and inference require care.
Worked Example: A Low-Variance Component Can Carry First-Stage Signal
Consider two centered, uncorrelated candidate instruments whose principal-component directions are \(PC_1\) and \(PC_2\). In a hypothetical sample, their variances are 9 and 1, so \(PC_1\) explains \(9/(9+1)=0.90\), or 90%, of the total instrument variance. Suppose their sample covariances with treatment are 0.2 and 0.8, respectively.
Keeping only \(PC_1\) would retain the direction with the most instrument variance, but it would discard the direction with the larger observed covariance with treatment. For a simple comparison, the squared covariances are \(0.2^2=0.04\) for \(PC_1\) and \(0.8^2=0.64\) for \(PC_2\). These are not complete first-stage diagnostics, since they do not account for treatment variance, controls, or sampling uncertainty, but they show why variance explained is not the same as predictive relevance.
The investigators could retain both components, or choose a reduction rule that also examines the treatment first stage and is specified transparently. Either choice requires checking the resulting first-stage strength. Keeping a component does not make it valid, and discarding a component can weaken identification even if the retained components explain most of the variation in \(Z\).
JIVE: Change the Prediction, Not Necessarily the Instrument Set
In ordinary 2SLS, each observation contributes to the first-stage fit used to construct its own fitted treatment. With many instruments, that own-observation contribution can make fitted values track observed treatment too closely, which is one mechanism behind many-instrument bias. JIVE reduces this particular feedback by predicting each observation’s treatment using a first stage estimated without that observation.
For a basic illustration with one instrument \(z_i\) and no controls, the leave-one-out fitted treatment is \(\widehat D_i^{(-i)}=z_i\left(\sum_{j\ne i}z_jD_j\right)/\left(\sum_{j\ne i}z_j^2\right)\). The corresponding JIVE coefficient is the IV ratio using these predictions as the instrumented treatment. With multiple instruments and controls, the first-stage regression is likewise refitted leaving observation \(i\) out, with the relevant covariate adjustment.
Worked Example: A Leave-One-Out IV Calculation
Take a hypothetical four-observation dataset with one instrument and no controls. The instrument values are \(z=(1,2,3,4)\), treatment values are \(D=(1,2,2,4)\), and outcome values are \(Y=(2,3,4,7)\). Compute each leave-one-out first-stage prediction from the other three observations.
For observation 1, the other observations give \(\sum z_jD_j=4+6+16=26\) and \(\sum z_j^2=4+9+16=29\), so \(\widehat D_1^{(-1)}=1(26/29)=0.8966\). For observation 2, the corresponding sums are 23 and 26, giving \(\widehat D_2^{(-2)}=2(23/26)=1.7692\). For observation 3, they are 21 and 21, giving \(\widehat D_3^{(-3)}=3(21/21)=3.000\). For observation 4, they are 11 and 14, giving \(\widehat D_4^{(-4)}=4(11/14)=3.1429\).
The JIVE ratio is
The estimate is about 1.786 outcome units per treatment unit under this illustrative calculation. It is not evidence by itself that the instrument is valid or that the estimate is precise. The example shows the leave-one-out construction; in an actual analysis, the estimator’s sampling uncertainty and the validity and strength of the instruments still need appropriate treatment.
Choosing a Strategy Without Confusing Its Purpose
These approaches address different problems. Theory-based restriction makes a substantive choice about which exclusion arguments are credible. Screening and principal components are ways to use the observed candidate set more selectively or compactly, but both can sacrifice relevant variation and create data-dependent choices. JIVE targets own-observation overfitting in the first stage; it is not a rule for deciding which causal assumptions are plausible.
A concise justification should state how the instrument set was chosen, what evidence supports relevance, and why the retained instruments are believed to satisfy the design’s validity assumptions. It should also make clear whether transformations or screening used the same sample as the outcome analysis. When many candidate sets were considered, reporting only the preferred one can conceal uncertainty from that choice.
For each candidate, explain the proposed effect on treatment and assess plausible pathways to the outcome outside treatment.
Distinguish a prespecified theory-based subset from a rule based on sample first-stage performance or principal components.
Evaluate the reduced or transformed first stage, use inference appropriate to the estimator and design, and disclose data-dependent choices.
Common Mistake
Wrong reasoning: “The selected instruments have the largest first-stage statistics, so they are the most valid.” A first-stage statistic concerns predictive evidence for treatment; it does not test whether an instrument has a direct effect on the outcome or is associated with the structural error. Similarly, retaining principal components that explain nearly all instrument variance does not guarantee that they retain the treatment-predictive directions.
Correction: Treat validity, relevance, and inference as separate questions. Use substantive design knowledge to defend validity, diagnose the strength of the actual retained first stage, and account for data-dependent choices when drawing conclusions. No reduction method can turn an implausible exclusion restriction into a credible one.
Check Your Understanding
Use the distinctions among validity, relevance, and inference to answer these questions.
- Why might a theory-based restriction be preferable to retaining an instrument solely because it improves the first stage?
- What does a large first-stage statistic tell you, and what does it not establish about an instrument?
- How can principal-component truncation discard a direction that is useful for predicting treatment?
- Under what moment condition does a fixed linear transformation of valid instruments preserve the moment restriction?
- What feature of JIVE addresses own-observation first-stage overfitting, and what important IV problem does JIVE not solve?