Lasso Instrumental Variables with Many Instruments

Why Select Instruments in the First Stage?

Instrumental-variables analysis can become difficult when the candidate instrument set is large. Some instruments may contain little information about the endogenous regressor, while others may be redundant or invalid. Lasso offers a way to estimate a sparse first-stage relationship when the number of candidates is large relative to the sample size. But its selection criterion is predictive: it identifies instruments useful for predicting the endogenous regressor, not instruments that satisfy the assumptions needed for causal interpretation.

That distinction matters twice. First, a selected instrument can be strongly predictive and still violate the exclusion restriction or exogeneity. Second, using the same sample to select instruments and then treating the selected set as fixed in ordinary two-stage least squares (2SLS) can make conventional inference misleading.

Structural Model and Instrument Conditions

Suppose the outcome \(y_i\) depends on an endogenous regressor \(d_i\) and observed controls \(x_i\). Let \(z_i\) be a vector of \(p\) candidate instruments, where \(p\) may be large relative to \(n\). A linear structural equation and first stage are

y_i = \beta d_i + x_i'\gamma + u_i, \qquad d_i = z_i'\pi + x_i'\delta + v_i.

Here \(\beta\) is the structural coefficient of interest, \(u_i\) is the structural error, and \(v_i\) is the first-stage error. Endogeneity means that \(d_i\) may be correlated with \(u_i\), so ordinary least squares for the structural equation need not identify \(\beta\).

Definition: Instrument validity requires the instruments to be excluded from the structural equation except through \(d_i\), and to be exogenous in the sense that \(E[z_i u_i\mid x_i]=0\). Instrument relevance requires that the instruments predict \(d_i\) conditional on \(x_i\); for a scalar \(d_i\), the conditional first-stage coefficient vector must not be zero. Validity and relevance are separate requirements.

These conditions are assumptions about the data-generating process and the research design. A large first-stage coefficient does not establish exogeneity or exclusion. Conversely, a plausible validity argument does not ensure that the instruments are sufficiently informative about \(d_i\).

Lasso as a Sparse First-Stage Procedure

To describe the selection step, first remove the linear contribution of the controls from \(d\) and the instrument columns, or equivalently include the controls in the regression without penalizing their coefficients. Let \(\tilde d\) and \(\tilde Z\) denote the resulting residualized outcome and instrument matrix. A common Lasso estimate is

\hat\pi = \arg\min_{b\in\mathbb{R}^{p}} \left\{ \frac{1}{2n}\|\tilde d-\tilde Zb\|_2^2 +\lambda\|b\|_1 \right\}.

The tuning parameter \(\lambda\) controls the penalty. The selected set is \(\hat S=\{j:\hat\pi_j\ne 0\}\). Instruments with zero fitted coefficients are omitted from the selected first stage; those with nonzero coefficients are retained. Controls are normally kept regardless of this instrument selection.

A sparse first-stage assumption says that \(d_i\) can be approximated well using a relatively small number of the candidate instruments, possibly with approximation error if the relationship is not exactly sparse. Lasso can be useful when this assumption is reasonable and the instrument design satisfies suitable conditions, such as restricted-eigenvalue or compatibility conditions. The earlier material on Lasso oracle inequalities explains why these conditions support prediction and estimation guarantees. Those guarantees do not, by themselves, establish instrument validity or valid post-selection inference.

Lasso shrinks coefficients toward zero, and its selected set can change with the sample or the tuning parameter. Its primary aim here is a sparse predictive first stage. It is not a test of whether an instrument is exogenous, and a nonzero Lasso coefficient is not a certificate that the instrument is strong enough for reliable IV inference.

Worked Example: Lasso Selection With Orthogonal Instruments

Consider a hypothetical residualized first stage with five standardized, mutually orthogonal candidate instruments. Suppose each instrument has sample variance one, so \(\tilde z_j'\tilde z_j/n=1\), and the sample correlations with \(\tilde d\), written as \(\tilde z_j'\tilde d/n\), are \(0.42,\ 0.18,\ 0.07,\ -0.31,\ 0.12\). Use \(\lambda=0.15\).

With orthogonal standardized columns, the Lasso solution for coefficient \(j\) is the soft-thresholded correlation:

\hat\pi_j = \operatorname{sign}(c_j)\max(|c_j|-\lambda,0), \qquad c_j=\frac{\tilde z_j'\tilde d}{n}.

For the first instrument, \(\hat\pi_1=0.42-0.15=0.27\). For the second, \(\hat\pi_2=0.18-0.15=0.03\). The third is set to zero because \(0.07<0.15\). For the fourth, \(\hat\pi_4=-(0.31-0.15)=-0.16\). The fifth is set to zero because \(0.12<0.15\). Thus \(\hat S=\{1,2,4\}\).

This calculation identifies instruments that help predict the first-stage residual in this illustration. It does not show that instruments 1, 2, and 4 satisfy \(E[z_i u_i\mid x_i]=0\), nor does it establish that their combined first stage supports accurate inference for \(\beta\). Those require separate arguments.

From the Selected Set to 2SLS

After selecting instruments, a common procedure estimates the structural coefficient by 2SLS using the selected columns. To write the estimator simply, suppose the controls have already been partialled out. Let \(\tilde Z_{\hat S}\) be the residualized matrix of selected instruments, and let \(P_{\hat S}\) be its projection matrix when it has full column rank. With one endogenous regressor,

P_{\hat S} = \tilde Z_{\hat S} (\tilde Z_{\hat S}'\tilde Z_{\hat S})^{-1} \tilde Z_{\hat S}', \qquad \hat\beta_{\text{2SLS}} = \frac{\tilde d'P_{\hat S}\tilde y} {\tilde d'P_{\hat S}\tilde d}.

The estimator uses the part of \(\tilde d\) projected onto the selected instruments to estimate its relationship with \(\tilde y\). If the selected instrument set were fixed in advance, valid, and sufficiently relevant, standard IV arguments could justify estimation and inference under their corresponding assumptions. But when \(\hat S\) is chosen from the same observations, it is a random set. Applying fixed-set reasoning without accounting for that randomness can fail.

Worked Example: Predictive Selection Is Not a Validity Test

Consider a hypothetical study of the effect of a training program \(d_i\) on later earnings \(y_i\). Researchers consider two candidate instruments. Instrument \(z_{1i}\) is a randomized offer to attend a training-information session; instrument \(z_{2i}\) is distance to the nearest training center. Suppose a Lasso first stage selects both. In a hypothetical first-stage regression, their estimated coefficients are \(0.34\) (standard error \(0.10\)) and \(0.28\) (standard error \(0.09\)).

The corresponding coefficient-to-standard-error ratios are \(0.34/0.10=3.40\) and \(0.28/0.09\approx3.11\). These results indicate predictive first-stage associations in this sample. They do not establish that either instrument is valid. The offer might affect earnings through channels other than training, and distance might be associated with local labor-market opportunities that directly affect earnings.

The design argument for the offer would need to address exclusion and exogeneity; the distance instrument would need its own such argument. Lasso selection cannot settle either issue. Nor do these two ratios alone determine the strength of the full selected-instrument set: strength depends on the joint first stage, controls, and the relevant sampling uncertainty. The lesson is not that the instruments are invalid, but that first-stage prediction and instrument validity answer different questions.

Why Same-Sample Selection Can Distort Inference

The problem is more than the familiar fact that Lasso coefficients are shrunk. Re-estimating 2SLS on the selected set removes that particular shrinkage from the second-stage estimator, but it does not make the selected set fixed. When selection uses first-stage associations, and the endogenous regressor is related to the structural error, the selection event can also be related to sample instrument-error associations.

A simple Gaussian approximation illustrates the mechanism. Suppose, under a null first stage, \(m\) candidate instruments are valid and have no population predictive effect. Define standardized sample scores \(A_j\) for instrument \(j\)'s association with \(d\), and \(B_j\) for its association with the structural error \(u\). If \(v\) and \(u\) are correlated, then the scores can be approximately jointly normal with correlation \(\rho\), even though each instrument is valid in the population. For simplicity, take the score pairs to be independent across candidates and write

B_j=\rho A_j+\varepsilon_j, \qquad E[\varepsilon_j\mid A_1,\ldots,A_m]=0.

Suppose selection keeps the candidate with the largest absolute first-stage score, \(j^*=\arg\max_j |A_j|\). Conditional on all the first-stage scores, the expected structural-error score for the selected instrument is

E[B_{j^*}\mid A_1,\ldots,A_m] = \rho A_{j^*}.

Multiplying by \(A_{j^*}\) and taking expectations gives

E[A_{j^*}B_{j^*}] = \rho E[A_{j^*}^{\,2}].

When \(\rho\ne0\), the expected selected-sample product need not be zero, despite population instrument validity. Choosing the largest absolute score also makes \(E[A_{j^*}^{\,2}]\) larger than the corresponding squared score for a prespecified candidate. This is a selection effect, not evidence that the original instruments were invalid in the population. It shows why ordinary calculations that condition on the selected set as though it had been fixed in advance can misrepresent uncertainty.

The illustration is not a general formula for the bias of every post-Lasso 2SLS estimator. It isolates one route by which same-sample selection affects the moments used for IV inference. Other concerns include weak selected first stages and, when the number of instruments remains large relative to the sample, finite-sample many-instrument bias. Selection uncertainty can be negligible under stronger conditions, for example if selection is sufficiently stable and the resulting set behaves like a fixed set. It should not simply be assumed away.

Worked Example: A Selected Score and Its Error Association

Continue the Gaussian illustration, and suppose \(\rho=0.50\). Imagine that among four candidates the largest absolute first-stage score is \(A_{j^*}=2.40\). Given that value, the conditional expected structural-error score is

E[B_{j^*}\mid A_{j^*}=2.40] = 0.50(2.40) = 1.20.

Thus the selection rule has chosen a first-stage score whose expected error score is also positive in this illustrative model. The original instruments can still be valid in the population; the association arises after selecting by a statistic related to \(d\), when the first-stage and structural errors are correlated. This calculation diagnoses a possible inference problem. It is not an estimate of a causal effect and does not imply that every Lasso-IV application has this amount of distortion.

Practical Safeguards and a Common Mistake

A defensible Lasso-IV analysis keeps the questions separate and reports the role of each assumption. It describes the candidate instrument set and the substantive validity argument, states how controls were handled, documents how the penalty and selected set were obtained, and evaluates the strength of the relevant first stage. It also explains how inference accounts for instrument selection and any weak-instrument concerns. Procedures such as sample splitting or other selection-aware approaches may be relevant, but none can repair an instrument that violates exclusion or exogeneity.

Common mistake: “Lasso retained these instruments because they predict treatment, so they are valid and ordinary 2SLS standard errors apply.” This confuses prediction with identification and treats a sample-selected instrument set as fixed. The correction is to justify validity from the design, assess relevance separately, and use inference whose assumptions address how the instruments were selected.

Lasso is valuable when a sparse first stage is plausible and many candidates make unregularized estimation unstable. Its contribution is disciplined prediction and screening. The causal interpretation still rests on instrument assumptions, and uncertainty still depends on the complete procedure—not only the final 2SLS regression.

Check Your Understanding

Answer each question by distinguishing what the first stage shows from what the IV assumptions require.

What does a nonzero Lasso coefficient in the first stage indicate, and what does it not establish?
State the separate roles of instrument relevance and the condition \(E[z_i u_i\mid x_i]=0\).
In the orthogonal-design example, why were instruments 3 and 5 assigned zero Lasso coefficients?
Explain how choosing an instrument using its sample association with \(d_i\) can affect its sample association with \(u_i\) when \(v_i\) and \(u_i\) are correlated.
Why might ordinary fixed-instrument 2SLS standard errors be unreliable after selecting instruments on the same sample?