Fuller Estimation and Weak-Instrument Bias

Why Modify LIML?

With weak instruments, an estimator can be sensitive to small-sample variation in the first stage. Two-stage least squares (2SLS) can have substantial finite-sample bias toward the ordinary least-squares estimate. LIML often behaves better in this setting, but its finite-sample distribution can still be dispersed, and its mean may fail to exist in some designs. Fuller estimation modifies LIML to improve its finite-sample behavior while retaining the k-class structure.

The modification is made to LIML’s k parameter, not by averaging the LIML and 2SLS coefficients. The adjustment moves that parameter downward, generally toward the 2SLS value of one. How much the coefficient itself moves depends on the observed cross-products; it need not move monotonically toward 2SLS in every data set.

Definition and Notation

Consider a linear instrumental-variables model with one endogenous regressor $x$, outcome $y$, included exogenous controls, and excluded instruments. Partial out the included controls from the variables and instruments. Let $N$ be the resulting sample dimension and $q$ the rank of the excluded-instrument space. Write $P_Z$ for the orthogonal projection onto that space. Assume the relevant design matrices have the ranks needed to define the estimates below.

For a scalar k-class estimator, a convenient expression using the partialled-out data is

\hat\beta(k) = \frac{x'(I-kM_Z)y}{x'(I-kM_Z)x} = \frac{x'y-kx'P_Zy}{x'x-kx'P_Zx}, \qquad M_Z=I-P_Z,

whenever the denominator is nonzero. At $k=1$, this is 2SLS, because $I-P_Z=M_Z$ is the residual-maker for projection onto the instrument space. LIML selects its own data-dependent k parameter, denoted $\hat k_{\mathrm{LIML}}$.

Definition: Under the convention used here, the Fuller estimator with adjustment parameter $\alpha>0$ uses $$ \hat k_F(\alpha)=\hat k_{\mathrm{LIML}}-\frac{\alpha}{N-q}, \qquad \hat\beta_F(\alpha)=\hat\beta\!\left(\hat k_F(\alpha)\right). $$ The adjustment is defined when $N>q$ and the resulting k-class denominator is nonzero. Fuller(1) and Fuller(4) refer to $\alpha=1$ and $\alpha=4$, respectively.

The denominator $N-q$ is the residual dimension after accounting for the excluded-instrument space in the partialled-out sample. Stating the dimension convention matters: formulas written with the original sample size may count included controls as instruments and express the same residual degrees of freedom differently.

Worked Example: Computing Fuller(4) From Cross-Products

Suppose the partialled-out cross-products are $x'x=100$, $x'P_Zx=80$, $x'y=50$, and $x'P_Zy=42$. Suppose $N=100$, $q=5$, and the fitted LIML parameter is $\hat k_{\mathrm{LIML}}=1.20$. Calculate 2SLS, LIML, and Fuller(4).

For 2SLS, $k=1$, so $$ \hat\beta(1)=\frac{50-42}{100-80}=\frac{8}{20}=0.4. $$

For LIML, $$ \hat\beta(1.20)=\frac{50-1.20(42)}{100-1.20(80)} =\frac{-0.4}{4}=-0.1. $$ The denominator is small, so this estimate is particularly sensitive to changes in $k$.

For Fuller(4), the adjustment is $4/(100-5)=4/95$, giving $$ \hat k_F(4)=1.20-\frac{4}{95}=\frac{22}{19}. $$ Therefore $$ \hat\beta_F(4) =\frac{50-42(22/19)}{100-80(22/19)} =\frac{26/19}{140/19} =\frac{13}{70} \approx 0.186. $$ Here the Fuller estimate lies between the LIML and 2SLS estimates. That ordering is an outcome of these particular cross-products, not a general ordering theorem.

What the Adjustment Does to the Coefficient

The k parameter is not the coefficient itself. The following identity makes the distinction precise. Let $A=x'x$, $B=x'P_Zx$, $C=x'y$, and $D=x'P_Zy$. For two values $k$ and $k-\delta$, assume both denominators in the expression for $\hat\beta$ are nonzero. Direct subtraction gives

Result: $$ \hat\beta(k-\delta)-\hat\beta(k) = \frac{\delta(AD-BC)} {(A-kB+\delta B)(A-kB)}. $$ Thus, for $\delta>0$, the direction of the coefficient change depends on $AD-BC$ and the signs of the two denominators.

To verify the identity, put both fractions over a common denominator. Their numerator difference is $$ (C-kD+\delta D)(A-kB)-(C-kD)(A-kB+\delta B). $$ The terms involving $C-kD$ cancel, leaving $\delta\{D(A-kB)-B(C-kD)\}=\delta(AD-BC)$. Dividing by the product of the denominators proves the result. $\square$

This calculation explains why a downward change in $k$ does not guarantee that the coefficient moves toward the 2SLS coefficient. It also explains why estimates can move sharply when a k-class denominator is near zero: the denominator of the change formula then amplifies even a small adjustment.

Worked Example: Comparing Fuller(1) and Fuller(4)

Use the cross-products and LIML output in the preceding example. Fuller(1) subtracts $1/95$, so $$ \hat k_F(1)=1.20-\frac{1}{95}=\frac{113}{95}. $$ Its coefficient is $$ \hat\beta_F(1) =\frac{50-42(113/95)}{100-80(113/95)} =\frac{4/95}{460/95} =\frac{1}{115} \approx 0.0087. $$

Fuller(4) instead subtracts $4/95$, producing $\hat\beta_F(4)=13/70\approx0.186$. In this example, the larger adjustment moves the coefficient closer to the 2SLS value $0.4$ and farther from the LIML value $-0.1$. Fuller(1) and Fuller(4) are not interchangeable: $\alpha$ controls the k adjustment, and the effect on the coefficient depends on the sample’s cross-products.

Finite-Sample Bias and Variance

The motivation for Fuller’s adjustment is finite-sample performance. Under the classical linear IV conditions used for LIML, the LIML k parameter has a small-sample bias contribution related to estimating the nuisance variation in the structural residuals. Subtracting $\alpha/(N-q)$ corrects the k parameter by an amount of the same order as that contribution. Common choices such as Fuller(1) and Fuller(4) trade off the strength of that correction.

In many weak-instrument designs, Fuller estimates have less finite-sample bias and lower variance than LIML. They can also have more stable finite-sample moments in settings where the LIML coefficient has a very heavy-tailed distribution or an undefined mean. These are operating properties under specified models and designs, not guarantees for every realization. Increasing $\alpha$ does not universally lower either bias or variance, and finite-sample mean squared error depends on the instrument strength, number of instruments, and error distribution as well as the adjustment.

The comparison with 2SLS also requires care. Fuller changes a LIML k parameter in the direction of one; it does not remove the weak-instrument bias problem by definition, nor does it make the resulting coefficient equal to an average of LIML and 2SLS. In some samples its coefficient is nearer to 2SLS, in others not. The earlier material on finite-sample bias of 2SLS explains why that estimator can be pulled toward OLS; Fuller’s rationale is to moderate LIML’s small-sample behavior without simply replacing it by 2SLS.

Worked Example: How the Adjustment Shrinks With Sample Size

Suppose the partialled-out sample has $N=500$, the excluded-instrument rank is $q=8$, and $\hat k_{\mathrm{LIML}}=1.05$. For Fuller(4), $$ \frac{\alpha}{N-q}=\frac{4}{492}\approx0.00813, \qquad \hat k_F(4)\approx1.05-0.00813=1.04187. $$ For Fuller(1), the adjustment is $1/492\approx0.00203$, and the adjusted parameter is approximately $1.04797$.

With fixed $q$ and fixed $\alpha$, the adjustment tends to zero as $N$ increases. For example, if $N=1000$ with the same $q=8$, the Fuller(4) adjustment is $4/992\approx0.00403$, about half as large. This calculation concerns the k parameter; the size of the corresponding coefficient change also depends on how sensitive the k-class coefficient is in the observed design.

Asymptotic Equivalence When the Instrument Count Is Fixed

The shrinking adjustment yields a useful formal comparison. In the scalar case, suppose $q$ and $\alpha$ are fixed as $N$ increases. Assume the normalized cross-products $A/N,B/N,C/N,D/N$ are bounded in probability, and that the two denominators $A-kB$ and $A-kB+\delta B$, divided by $N$, are bounded away from zero in probability for $k=\hat k_{\mathrm{LIML}}$ and $\delta=\alpha/(N-q)$. These conditions rule out a k-class denominator that is arbitrarily close to zero with non-negligible probability.

Theorem: Under these conditions, $$ \hat\beta_F(\alpha)-\hat\beta_{\mathrm{LIML}}=O_p(N^{-1}). $$ In particular, the Fuller adjustment is asymptotically negligible at the usual $N^{-1/2}$ scale.

By the cross-product identity, the absolute difference is $$ \left|\hat\beta_F(\alpha)-\hat\beta_{\mathrm{LIML}}\right| = \frac{\delta\,|AD-BC|} {|(A-kB+\delta B)(A-kB)|}. $$ Since $q$ and $\alpha$ are fixed, $\delta=\alpha/(N-q)=O(N^{-1})$. Boundedness in probability of the normalized cross-products implies $AD-BC=O_p(N^2)$. The denominator conditions imply that the product in the denominator is of order $N^2$ in probability. Combining these orders gives $O_p(N^{-1})$, as claimed. Since $\sqrt{N}\,O_p(N^{-1})=o_p(1)$, the difference is negligible at the $N^{-1/2}$ scale. $\square$

This result is conditional on fixed instrument rank and regular denominators. It does not say that Fuller and LIML have identical finite-sample behavior, nor does it cover designs in which the number of instruments grows with the sample size. A small change in $k$ can matter substantially in a finite sample when the coefficient is highly sensitive to $k$.

Practical Interpretation and Common Pitfall

Fuller estimation is useful when LIML’s finite-sample behavior is a concern and a specified Fuller adjustment is appropriate for the design. The value of $\alpha$ should be reported, along with the instrument count convention and the estimator used. In applications, it is informative to compare Fuller and LIML estimates with 2SLS, but estimate comparisons alone do not establish instrument validity or reliable inference under weak identification.

Pitfall: “A larger Fuller adjustment must move the coefficient toward 2SLS.” This incorrectly treats the coefficient as a monotone function of $k$. The change identity shows that its direction depends on the data and denominator signs. Compare the computed coefficients, rather than inferring their order from the k parameters alone.
Pitfall: “Fuller is just LIML with a smaller reported standard error.” Fuller changes the point estimator’s k parameter. Its finite-sample bias and variance can change, but the adjustment is not itself a variance formula or an inference procedure.
Pitfall: “Fuller eliminates weak identification.” It does not. A Fuller estimate can still be sensitive to the data, and weak-identification concerns remain relevant to inference.

Key takeaway: Fuller(α) replaces the LIML k parameter by $\hat k_{\mathrm{LIML}}-\alpha/(N-q)$. The adjustment can improve finite-sample operating properties, but its effect on the coefficient is design-dependent, and it does not by itself resolve weak-identification inference.

Check Your Understanding

Use the definitions and calculations above to answer these questions.

In the stated convention, what quantity is subtracted from the LIML k parameter to form Fuller(4)?
Why can lowering $k$ fail to move the coefficient toward the 2SLS estimate?
For fixed $q$ and $\alpha$, how does the Fuller adjustment change as $N$ increases?
What denominator condition is important when comparing Fuller and LIML estimates?
Why is it incorrect to describe Fuller as a general solution to weak identification?