Finite-Sample Bias of 2SLS Toward OLS

Why Finite-Sample Bias Matters

Valid instruments can identify a structural effect even when the endogenous regressor is correlated with the structural error. But validity does not make 2SLS exactly unbiased in a finite sample. When the first stage is weak, the estimator can be pulled toward the OLS estimate, which is itself affected by endogeneity. This is a statement about the estimator's finite-sample distribution, not a claim that every realized 2SLS estimate lies between the truth and OLS.

A useful leading approximation makes the mechanism visible: for a fixed number of instruments, the bias increases with instrument count and decreases with first-stage strength. The approximation has conditions and a range of usefulness. In particular, it should not be treated as an exact bias formula in the local-to-zero weak-instrument limit.

Model and Bias Target

Consider the scalar structural and first-stage equations

y=\beta x+u,\qquad x=Z\pi+v.

Here $Z$ is a fixed $n\times K$ instrument matrix of full column rank, $\pi$ is the first-stage coefficient vector, and $K$ is fixed. Assume the rows of $(u,v)$ are independent and identically distributed bivariate normal with mean zero and covariance matrix

\begin{pmatrix} \sigma_u^2 & \sigma_{uv}\\ \sigma_{uv} & \sigma_v^2 \end{pmatrix}, \qquad \sigma_v^2>0.

The instruments are exogenous in this model: they are fixed, while the disturbances have mean zero conditional on $Z$. Endogeneity corresponds to $\sigma_{uv}\ne 0$. Write $P_Z=Z(Z'Z)^{-1}Z'$. The 2SLS estimator and its estimation error are

\hat\beta_{\mathrm{2SLS}}=(x'P_Zx)^{-1}x'P_Zy, \qquad \hat\beta_{\mathrm{2SLS}}-\beta =\frac{x'P_Zu}{x'P_Zx}.

The bias discussed here is $E[\hat\beta_{\mathrm{2SLS}}]-\beta$, when this expectation exists. It is a repeated-sampling average, not a description of the error in any one sample.

Definition: Let $Q=Z(Z'Z)^{-1/2}$, so $Q'Q=I_K$, and set $\gamma=(Z'Z)^{1/2}\pi$. The projected first-stage signal is $Q\gamma=Z\pi$, with squared length $A=\gamma'\gamma=\pi'Z'Z\pi$. The concentration parameter in this fixed-design Gaussian model is $\lambda=A/\sigma_v^2$.

The quantity $A$ depends on both the first-stage coefficients and the design scale. In particular, $\gamma$ and $A$ do not automatically grow with $n$ just because $n$ grows. The scaling of $Z$ and the coefficients must be specified.

Deriving the Leading Bias

Because $P_Z=QQ'$, define $a=Q'v$, $b=Q'u$. Both are $K$-vectors, and $x'P_Zu=(\gamma+a)'b$, $x'P_Zx=(\gamma+a)'(\gamma+a)$. Joint normality gives the conditional mean $E[b\mid a]=(\sigma_{uv}/\sigma_v^2)a$. Therefore, for $K\ge 3$, the bias can be written as

E[\hat\beta_{\mathrm{2SLS}}]-\beta = \frac{\sigma_{uv}}{\sigma_v^2} E\left[\frac{\gamma'a+a'a}{A+2\gamma'a+a'a}\right].

The restriction $K\ge 3$ ensures the ratio has an integrable singularity near $\gamma+a=0$ under the Gaussian density. We now expand this expression for large $A$, holding $K$, $\sigma_v^2$, and $\sigma_{uv}$ fixed.

Theorem: Under the fixed-design Gaussian model above, with $K\ge 3$, as $A\to\infty$, $$ E[\hat\beta_{\mathrm{2SLS}}]-\beta = \frac{(K-2)\sigma_{uv}}{A} +o(A^{-1}). $$ Equivalently, the leading term is $(K-2)\sigma_{uv}/(\lambda\sigma_v^2)$.

Proof. Put $h=\gamma'a$, $s=a'a$, and $N=h+s$, so the ratio inside the expectation is $N/(A+2h+s)$. Since $a\sim N(0,\sigma_v^2 I_K)$,

E[h]=0,\quad E[s]=K\sigma_v^2,\quad E[h^2]=A\sigma_v^2,\quad E[hs]=0,\quad E[s^2]=K(K+2)\sigma_v^4.

Expand the denominator about $A$. On the event $\|a\|\le\sqrt A/2$, the denominator is at least $A/4$, and the exact identity

\frac{1}{A+t}=\frac{1}{A}-\frac{t}{A^2} +\frac{t^2}{A^2(A+t)},\qquad t=2h+s,

shows that the final term contributes $o(A^{-1})$ after multiplication by $N$ and taking expectations. For example, Gaussian moment bounds give an $O(A^{-3/2})$ bound on this event. The complementary event has Gaussian probability exponentially small in $A$; the possible singularity at $a=-\gamma$ is integrable for $K\ge3$, so its contribution is also $o(A^{-1})$. Thus

E\left[\frac{N}{A+2h+s}\right] = \frac{E[N]}{A}-\frac{E[N(2h+s)]}{A^2} +o(A^{-1}).

Now $E[N]=K\sigma_v^2$, and

E[N(2h+s)] =E[2h^2+3hs+s^2] =2A\sigma_v^2+K(K+2)\sigma_v^4.

Consequently, the first term is $K\sigma_v^2/A$, while the part of the second term of order $A^{-1}$ is $2\sigma_v^2/A$. The remaining part is order $A^{-2}$. Multiplying the resulting expansion by $\sigma_{uv}/\sigma_v^2$ proves the theorem. $\square$

This is a leading-order expansion as signal grows; it is often called a Nagar-type bias approximation. It is not an exact formula for arbitrary $A$. In particular, a small $\lambda$ does not justify using an expansion derived for large $A$.

Interpreting the Formula

For $K\ge3$, the leading term has the sign of $\sigma_{uv}$, the same sign as the endogeneity component of OLS bias. Increasing $K$ while holding $A$ fixed increases the coefficient $K-2$. Increasing $A$, or equivalently $\lambda$ at fixed $\sigma_v^2$, reduces the leading term. These comparisons are about the approximation under its stated model; changing instruments can also change $A$, so instrument count alone does not determine the bias.

The OLS probability limit under a fixed-strength design can help interpret the phrase “toward OLS.” If $A/n\to\kappa>0$, then the OLS estimation error converges in probability to $\sigma_{uv}/(\sigma_v^2+\kappa)$. When the signal per observation $\kappa$ is small relative to $\sigma_v^2$, this is approximately $\sigma_{uv}/\sigma_v^2$. Comparing that approximation with the leading 2SLS bias gives the familiar heuristic

\frac{\text{leading 2SLS bias}}{\text{OLS bias}} \ \approx\ \frac{K-2}{\lambda}.

More generally, using the OLS probability limit rather than its weak-signal approximation gives the ratio $(K-2)(\sigma_v^2+\kappa)/A$, since $A=n\kappa$. Thus $(K-2)/\lambda$ is a useful heuristic in a weak-signal comparison, not a universal identity between the two biases.

Worked Example: Comparing Bias and OLS Under a Weak Signal

Suppose $K=4$, $\sigma_v^2=1$, $\sigma_{uv}=0.6$, and $A=40$. The concentration parameter is $\lambda=40/1=40$. The leading 2SLS bias is

$(4-2)(0.6)/40=1.2/40=0.03.$

If $n=200$, then $\kappa=A/n=0.2$. The OLS probability-limit bias is $0.6/(1+0.2)=0.5$. Their ratio is $0.03/0.5=0.06$. The weak-signal heuristic gives $(K-2)/\lambda=2/40=0.05$, close but not identical: the OLS probability limit includes the signal contribution $\kappa$ in its denominator.

Worked Example: Instrument Count at Fixed Signal

Compare two designs with the same $A=80$, $\sigma_v^2=1$, and $\sigma_{uv}=0.5$, but different instrument counts. With $K=3$, the leading bias is $(3-2)(0.5)/80=0.00625$. With $K=7$, it is $(7-2)(0.5)/80=2.5/80=0.03125$.

The second leading term is five times the first because the instrument count has increased while the total projected signal $A$ is held fixed. These are leading approximations, not exact finite-sample expectations. The calculation also does not say that adding instruments must raise bias in every comparison: if adding instruments changes the signal $A$, that change enters the formula too.

Asymptotic Scaling: Bias Is Not Inconsistency

The design normalization determines whether $A$ grows with sample size. Suppose the instruments are unnormalized and $Z'Z/n\to\Sigma_Z$, where $\Sigma_Z$ is positive definite, while the coefficient vector $\pi\ne0$ is fixed. Then

\frac{A}{n} =\pi'\frac{Z'Z}{n}\pi \longrightarrow \kappa=\pi'\Sigma_Z\pi>0.

Hence $A$ and $\lambda=A/\sigma_v^2$ grow at order $n$. The leading bias is then order $1/n$. This conclusion depends on fixed coefficients on unnormalized instruments and the stated design scaling; it does not follow merely from having more rows in an orthonormalized $Q$.

Result: Under fixed $K$, the fixed-strength design conditions just stated, and the Gaussian disturbance assumptions, $\hat\beta_{\mathrm{2SLS}}\xrightarrow{p}\beta$. Its finite-sample bias can be nonzero even though it is consistent.

Proof. Write $\hat\beta_{\mathrm{2SLS}}-\beta=(\gamma+a)'b/\|\gamma+a\|^2$. Since $A/n\to\kappa>0$, $\|\gamma\|=O(\sqrt n)$, while $a=Q'v=O_p(1)$ and $b=Q'u=O_p(1)$ for fixed $K$. The numerator is $O_p(\sqrt n)$, and the denominator is $A+2\gamma'a+a'a=n\kappa+O_p(\sqrt n)$, hence is order $n$ in probability. Their ratio is $O_p(n^{-1/2})$, which converges in probability to zero. Therefore $\hat\beta_{\mathrm{2SLS}}\xrightarrow{p}\beta$. $\square$

This fixed-strength case is different from a local-to-zero sequence. If, for example, $\pi_n=c/\sqrt n$ with $Z'Z/n\to\Sigma_Z$, then $A\to c'\Sigma_Zc$, a finite constant, and $\lambda$ does not diverge. The large-$A$ bias expansion does not describe that limiting regime. Weak-instrument asymptotics can then produce a nondegenerate limit for the estimation error, so the estimator need not converge in probability to $\beta$. That is inconsistency, a property of an asymptotic sequence; finite-sample bias is a property of an estimator's expectation at a given sample size. Neither term should be used as a synonym for the other.

Worked Example: Fixed Coefficients and Growing Signal

Let $K=4$, $\sigma_v^2=1$, $\sigma_{uv}=0.4$, and suppose $Z'Z/n\to I$ with fixed $\pi=(0.5,0,0,0)'$. Then $A/n\to\pi'\pi=0.25$. At $n=400$, $A$ is approximately $100$, so the leading bias is $(4-2)(0.4)/100=0.008$. At $n=1600$, $A$ is approximately $400$, and the leading bias is $0.8/400=0.002$.

The concentration parameters are approximately $100$ and $400$, respectively. Their growth here follows from the fixed-coefficient, unnormalized-design scaling $Z'Z/n\to I$; it would not follow from $Q'Q=I_K$ alone with a fixed $\gamma$.

Common Mistakes

Treating the expansion as exact. The expression $(K-2)\sigma_{uv}/A$ is a leading term for large $A$, not a guaranteed accurate numerical bias when instruments are extremely weak.
Assuming the sign proves every estimate is pulled toward OLS. The sign describes the leading average bias under this model. It does not order individual realizations of 2SLS, OLS, and $\beta$.
Letting $A$ grow without specifying a design. With $A=\pi'Z'Z\pi$, growth follows under fixed $\pi$ and $Z'Z/n\to\Sigma_Z>0$. It does not follow from an orthonormal basis by itself.
Confusing bias with inconsistency. Nonzero finite-sample bias is compatible with consistency. A local-to-zero sequence can instead prevent convergence to the true parameter, and the large-signal bias expansion is not its analysis.

Key takeaway: Under the stated Gaussian scalar model, the leading 2SLS bias is $(K-2)\sigma_{uv}/A$. More instruments raise this leading term when signal is held fixed; greater signal lowers it. Whether the bias vanishes with sample size depends on the design and coefficient scaling.

Check Your Understanding

Work through these questions using the assumptions and distinctions developed above.

In the fixed-design model, what are $A$ and $\lambda$, and why does $Q'Q=I_K$ alone not imply that either grows with $n$?
For $K=5$, $A=60$, and $\sigma_{uv}=-0.3$, calculate the leading bias term and state its direction relative to $\beta$.
Why is $(K-2)/\lambda$ a heuristic for the ratio of 2SLS bias to OLS bias rather than an exact identity?
Under fixed $\pi\ne0$ and $Z'Z/n\to\Sigma_Z>0$, what happens to $A$ and the leading bias as $n$ increases?
How does a local-to-zero sequence $\pi_n=c/\sqrt n$ change the order of $A$, and why does that put the sequence outside the large-$A$ bias expansion?