Jackknife IV for Many Instruments

Why Remove an Observation’s Own Influence?

With many instruments, ordinary two-stage least squares (2SLS) can fit the endogenous regressor unusually closely in the sample. The fitted value for observation \(i\) partly reflects that observation’s own first-stage disturbance. If this disturbance is correlated with the structural error, the fitted value can inherit a spurious relationship with that error. Jackknife instrumental variables (JIVE) targets this source of finite-sample bias by predicting each observation’s endogenous regressor without using that observation’s first-stage outcome.

The key operation is leave-one-out prediction, not deletion of an observation from the structural outcome equation. Each observation remains in the final estimating equation; only its contribution to its own first-stage fitted value is removed. This can reduce many-instrument bias, but does not validate instruments, guarantee a strong first stage, or ensure precise inference.

Setup and Definition

Consider one endogenous regressor and a structural equation \(y_i=\beta d_i+u_i\), after partialling out any included exogenous regressors. Here \(y_i\) is the outcome, \(d_i\) is the endogenous regressor, \(\beta\) is the structural coefficient, and \(u_i\) is the structural error. Let \(Z\) be the \(n\times K\) matrix of excluded instruments, with full column rank and \(K<n\). For clarity, use \(y,d,Z\) for the residualized variables after controlling for included exogenous regressors.

The first stage regresses \(d\) on \(Z\). Its fitted values are \(P_Zd\), where \(P_Z=Z(Z'Z)^{-1}Z'\) is the projection matrix. Write \(h_{ii}\) for the \(i\)th diagonal element of \(P_Z\), called the leverage of observation \(i\). The full-sample fitted value \((P_Zd)_i\) uses \(d_i\), including its first-stage disturbance.

The leave-one-out expression can be written using only off-diagonal entries of \(P_Z\): \(\tilde d_i^{\,J}=\sum_{j\ne i}P_{Z,ij}d_j/(1-h_{ii})\). Thus the prediction uses the instruments for observation \(i\), but not its own observed \(d_i\). In matrix form, \(\tilde d^{\,J}=Ld\), where \(L_{ii}=0\) and \(L_{ij}=P_{Z,ij}/(1-h_{ii})\) for \(j\ne i\). Generally \(L\) is not itself a projection matrix.

Why the Leave-One-Out Step Reduces a Bias Source

To isolate the mechanism, condition on the instruments and suppose the first stage has the form \(d=m+v\), where \(m=Z\pi\) is the instrument-predicted component and \(v\) is the first-stage disturbance. Suppose observations are independent, \(E[u_i\mid Z]=0\), and \(E[v_i u_i\mid Z]=\sigma_{uv}\). For this calculation, also take \(\operatorname{Var}(v_i\mid Z)=\sigma_v^2\) and a common covariance \(\sigma_{uv}\) across observations. These conditions describe a simple homoskedastic benchmark, not every application.

The 2SLS estimation error is \(\hat\beta_{2SLS}-\beta=d'P_Zu/(d'P_Zd)\). Since \(P_Zm=m\), the expected numerator conditional on \(Z\) is \[ E[d'P_Zu\mid Z]=E[(m+v)'P_Zu\mid Z]=\sigma_{uv}\operatorname{tr}(P_Z)=K\sigma_{uv}. \] The last equality follows because \(m'P_Zu\) has conditional expectation zero and only the diagonal terms \(P_{Z,ii}v_i u_i\) contribute to the expectation of \(v'P_Zu\). The projection matrix has trace \(K\), its rank. Consequently, if \(\sigma_{uv}>0\), this component of the 2SLS numerator has positive expectation. It is the own-observation terms \(P_{Z,ii}v_i u_i\) that produce it.

For JIVE, \(\tilde d^{\,J}=Lm+Lv\), and every diagonal element of \(L\) is zero. Under the stated independence and conditional-mean assumptions, \[ E[(\tilde d^{\,J})'u\mid Z] =\sum_i\sum_{j\ne i}L_{ij}E[d_j u_i\mid Z]=0. \] The signal terms have zero expectation because \(E[u_i\mid Z]=0\); the cross-observation disturbance terms have zero expectation because observations are independent. Thus JIVE removes this particular expected numerator term. This is the central jackknife argument.

Zero expected numerator does not imply exact finite-sample unbiasedness: the estimator is a ratio with a random denominator, and numerator and denominator can be dependent. The result identifies a bias mechanism and its removal, rather than proving that every JIVE estimate is unbiased.

A Balanced Many-Instrument Comparison

The following benchmark makes the comparison concrete. Suppose \(n=100\), \(K=20\), all leverages equal \(h_{ii}=K/n=0.20\), and \(\|m\|^2=100\). Let \(\sigma_v^2=1\) and \(\sigma_{uv}=0.50\). This is a hypothetical design with a non-negligible instrument-to-observation ratio. Assume the instruments are valid in the sense used above. In this balanced design, \(Lm=m\): because \(P_Zm=m\) and \(P_{Z,ii}=h\), the \(i\)th element of \(Lm\) is \((m_i-hm_i)/(1-h)=m_i\).

In a general design, the JIVE denominator’s expected signal component is \(m' Lm\), which need not equal \(\|m\|^2\). If it is small or close to zero, the ratio can be unstable even though the own-observation numerator component has been removed. Relevance and a denominator well away from zero remain essential.

Bias Reduction Can Cost Variance

A leave-one-out fitted value relies on predictions from other observations, so it can be noisier than the full-sample fitted value. The next calculation compares a leading variance component in the same balanced design, but now assumes \(u\) is independent of \(v\), with \(\operatorname{Var}(u_i)=\sigma_u^2\). This independence assumption simplifies the variance calculation and is not the correlated-error case used to display the bias mechanism.

Treating the estimator’s denominator as approximately its expected signal component, its leading variance is the variance of the numerator divided by the squared signal denominator. For 2SLS, the numerator is \((m+P_Zv)'u\). Independence gives \(\operatorname{Var}((m+P_Zv)'u\mid Z)=\sigma_u^2(\|m\|^2+\sigma_v^2\operatorname{tr}(P_Z^2))=\sigma_u^2(\|m\|^2+K\sigma_v^2)\), since \(P_Z^2=P_Z\).

For JIVE the numerator is \((m+Lv)'u\), so its variance is \(\sigma_u^2(\|m\|^2+\sigma_v^2\operatorname{tr}(LL'))\). In a balanced design with leverage \(h\), each row of \(L\) has squared entries summing to \((h-h^2)/(1-h)^2=h/(1-h)\). Hence \(\operatorname{tr}(LL')=nh/(1-h)=K/(1-K/n)\). When \(K/n>0\), this exceeds \(K\), showing the additional first-stage prediction noise in this benchmark.

Conditions, Interpretation, and Common Pitfalls

The leave-one-out construction requires \(h_{ii}<1\) for every observation being used; otherwise the displayed prediction is undefined. In practice, very high leverage can also make the factor \(1/(1-h_{ii})\) large and the prediction unstable. JIVE also requires valid excluded instruments, adequate first-stage signal, and a nonzero estimating denominator. Its basic leave-one-out justification relies on observations being independent, or on a design with suitable conditions ensuring that an observation’s excluded first-stage disturbance is uncorrelated with other observations’ structural errors. Clustered or dependent data require methods and assumptions that account for that dependence.

A useful distinction is that JIVE addresses the reuse of observation \(i\)’s first-stage disturbance in its own predicted value. It does not repair an exclusion restriction violation, eliminate all finite-sample bias, or make weak instruments informative. In particular, when the denominator is close to zero, the estimator can be erratic. The Anderson–Rubin approach is a separate weak-instrument-robust inference method; replacing 2SLS with JIVE is not a substitute for such inference.

• Mistake: treating zero expected numerator as exact unbiasedness. A ratio with random numerator and denominator need not have expectation equal to the ratio of their expectations. The correction is to describe the jackknife result as removing a particular leading own-observation bias component.
• Mistake: assuming JIVE must be more precise because it reduces bias. Leave-one-out predictions can be noisier, as the variance calculation shows. Bias and variance are distinct properties and should both be considered.
• Mistake: assuming leave-one-out prediction solves weak identification. If the instruments provide little signal, the JIVE denominator can still be small. Removing self-influence does not create instrument relevance.