Why Naive Two-Stage Substitution Fails In Nonlinear Models

Why Replacing the Regressor Is Not a General Nonlinear IV Strategy

In a linear instrumental-variables model, a fitted value from the first stage has a special role: under the usual projection construction, regressing the outcome on fitted regressors produces the 2SLS coefficient. That fact can suggest a tempting general recipe: predict an endogenous regressor, then insert that prediction into any outcome model. In a nonlinear model, this substitution is generally not justified. A conditional mean of a regressor is not the regressor itself, and a nonlinear response to the mean need not equal the conditional mean response.

There is a second problem. If the outcome error remains related to the actual endogenous regressor, replacing that regressor with a first-stage prediction does not, by itself, model or remove that dependence. A residual-inclusion approach can address it only when the first-stage residual contains the relevant conditional information about the outcome error and the outcome equation represents that information adequately.

Definition: Let \(D\) be an endogenous regressor, \(Z\) an instrument, and \(X\) observed exogenous covariates. The first-stage conditional mean is \(m(Z,X)=E[D\mid Z,X]\), and the first-stage residual is \(V=D-m(Z,X)\). Thus \(E[V\mid Z,X]=0\). A naive two-stage substitution replaces \(D\) in a nonlinear outcome function \(f(D,X;\beta)\) with an estimate of \(m(Z,X)\), without separately establishing that this replacement identifies or correctly specifies the outcome model.

Two Distinct Reasons Substitution Can Fail

First, nonlinear functions do not generally commute with conditional expectation. In general, \(f(E[D\mid Z,X],X;\beta)\) is not equal to \(E[f(D,X;\beta)\mid Z,X]\). The first quantity evaluates the response at the conditional mean of \(D\); the second averages responses across the conditional distribution of \(D\). They answer different questions.

Second, consider the structural conditional-mean model

Y=f(D,X;\beta)+\varepsilon.

If \(\varepsilon\) is related to \(D\), then evaluating \(f\) at a fitted value does not automatically make the outcome error conditionally mean zero. The actual outcome was generated using \(D\), and the dependence between \(D\) and \(\varepsilon\) may still matter for estimation. A first-stage prediction can summarize how \(D\) varies with \(Z\) and \(X\), but it need not summarize all the information about \(\varepsilon\) carried by the difference \(D-m(Z,X)\).

Worked Example: Curvature Changes the Conditional Mean

Suppose, hypothetically, that at a fixed value of \(Z\) and \(X\), the endogenous regressor is \(D=1\) with probability \(1/2\) and \(D=-1\) with probability \(1/2\). Its first-stage conditional mean is \(m(Z,X)=0\). Let the outcome conditional mean be \(f(D)=\exp(\beta D)\), with \(\beta=\log 2\), and suppose there is no outcome error.

The mean response at the two possible values of \(D\) is

E[\exp(\beta D)\mid Z,X] =\tfrac12\exp(\log 2)+\tfrac12\exp(-\log 2) =\tfrac12(2)+\tfrac12(0.5)=1.25.

Substituting the first-stage mean into the response instead gives

\exp\{\beta m(Z,X)\} =\exp\{(\log 2)(0)\}=1.

The two results differ: the average response is \(1.25\), whereas the response evaluated at the average regressor is \(1\). This discrepancy arises from nonlinearity even though this example has no endogeneity in the outcome error. It is therefore not, on its own, a correction for endogeneity.

A useful contrast is a linear structural conditional mean, \(f(D,X;\beta)=\beta D+X^\top\gamma\). Its conditional mean over \(D\), holding \(X\) fixed, is \(\beta E[D\mid Z,X]+X^\top\gamma\), because expectation passes through a linear function. That algebra helps explain why fitted values appear in linear 2SLS. It does not imply that the same substitution works for a general nonlinear function. Nor does it make ordinary second-stage standard errors valid when the fitted values are estimated.

What a Residual Control Must Represent

Residual inclusion takes a different approach. It keeps the actual \(D\) in the structural response and adds information from the first-stage residual. To describe when this is justified, suppose the first stage correctly represents \(m(Z,X)=E[D\mid Z,X]\), so that \(V=D-m(Z,X)\). Suppose also that the outcome model is \(Y=f(D,X;\beta)+\varepsilon\). Instrument validity requires an appropriate exogeneity condition, such as \(E[\varepsilon\mid Z,X]=0\), together with an exclusion restriction: \(Z\) does not enter the structural response directly.

For residual inclusion to correct the conditional mean, a further condition is needed: the conditional mean of the outcome error given the endogenous regressor and the instruments must depend on those variables through the residual and observed covariates. Formally, for some correctly specified function \(h\), require

E[\varepsilon\mid D,Z,X]=h(V,X).

Then iterated conditioning gives \(E[\varepsilon\mid D,X,V]=h(V,X)\), so the outcome conditional mean becomes

E[Y\mid D,X,V]=f(D,X;\beta)+h(V,X).

This equation explains both the potential value and the limitation of residual inclusion. The actual endogenous regressor remains in the nonlinear response. The residual is included to model the part of the conditional outcome error represented by \(h\). If \(h(V,X)\) is assumed to be \(\rho V\), that is a substantive linear conditional-error assumption, not an automatic consequence of estimating a first stage.

Key condition: Residual inclusion is justified for a conditional-mean model only when the first-stage residual is a sufficient control for the relevant conditional error mean, and the included function of that residual is correctly specified. A residual is not a universal correction for endogeneity.

Worked Example: A Valid Linear Residual Control

Consider a hypothetical model with \(D=Z+V\), where \(Z\) and \(V\) are independent, \(E[V]=0\), and \(Z\) varies in the population. Then \(E[D\mid Z]=Z\), so the first-stage residual is exactly \(V=D-Z\). Let

Y=\exp\{(\log 2)D\}+0.8V+u,

where \(E[u\mid D,Z,V]=0\). Assume the instrument affects the outcome only through \(D\) and the displayed error component, and note that \(E[0.8V+u\mid Z]=0\) under the stated assumptions. The conditional error mean is \(h(V)=0.8V\), so a residual-inclusion conditional mean retains the nonlinear term in actual \(D\) and adds \(0.8V\).

At \(Z=0\), the first-stage prediction is \(\widehat D=0\). Suppose \(V\) takes \(-0.5\) and \(0.5\), each with probability \(1/2\). The corresponding actual regressor values are \(D=-0.5\) and \(D=0.5\). Their conditional outcome means are

\begin{aligned} V=-0.5:\quad &\exp\{(\log 2)(-0.5)\}+0.8(-0.5) =2^{-0.5}-0.4\approx 0.3071,\\ V=0.5:\quad &\exp\{(\log 2)(0.5)\}+0.8(0.5) =2^{0.5}+0.4\approx 1.814. \end{aligned}

The average outcome mean at \(Z=0\) is approximately \((0.3071+1.814)/2=1.061\). Simply inserting \(\widehat D=0\) into the nonlinear response gives \(\exp(0)=1\), and it omits the residual-linked error component. The residual-inclusion model instead uses the actual \(D\) in the exponential term and includes the residual with the assumed coefficient \(0.8\). In applied work, \(h\) and the first stage must be estimated; this example shows the identifying structure, not a claim that the coefficient is known.

Why “Add the Residual” Is Not Enough

The conditional-error representation may be nonlinear. If the true function \(h(V,X)\) cannot be represented by the chosen residual term, adding a linear residual can leave conditional dependence in the outcome error. Correctly estimating \(m(Z,X)\) only ensures that the residual has conditional mean zero given \(Z,X\); it does not imply that \(E[\varepsilon\mid D,Z,X]\) is linear in that residual.

Worked Example: A Linear Residual Misses a Nonlinear Error Mean

Consider a hypothetical first stage \(D=Z+V\), with \(Z\) independent of \(V\). Let \(V=-1,0,2\) with probabilities \(1/2,1/4,1/4\), respectively. Then

E[V]=(-1)(1/2)+(0)(1/4)+(2)(1/4)=0, \qquad E[V^2]=(1)(1/2)+(0)(1/4)+(4)(1/4)=1.5.

Thus \(m(Z)=Z\) and the residual is \(V\). Suppose the outcome error conditional mean is \(h(V)=V^2-1.5\), with any remaining error having conditional mean zero. This control function has mean zero:

$$ E[h(V)]=E[V^2]-1.5=0. $$

But its values are \(h(-1)=-0.5\), \(h(0)=-1.5\), and \(h(2)=2.5\). No single coefficient \(\rho\) makes \(h(V)=\rho V\) at all three values: at \(V=0\), every linear residual term is zero while \(h(0)=-1.5\). Therefore, an outcome model that adds only \(\rho V\) fails to represent this conditional error mean. Under the stated setup, a correctly specified inclusion of \(V^2-1.5\) would represent it; whether such a function is appropriate in an application requires substantive and statistical support.

A Short Derivation of the Control-Function Mean

The justification for residual inclusion is a direct application of conditional expectation, not a consequence of the first-stage prediction alone. Assume the structural outcome equation and conditional-error representation stated above. Starting with \(Y=f(D,X;\beta)+\varepsilon\), condition first on \(D,Z,X\):

E[Y\mid D,Z,X] =f(D,X;\beta)+E[\varepsilon\mid D,Z,X] =f(D,X;\beta)+h(V,X).

Because \(V=D-m(Z,X)\) is determined by \(D,Z,X\), and the last expression depends on \(Z\) only through \(V\), conditioning instead on \(D,X,V\) yields

E[Y\mid D,X,V]=f(D,X;\beta)+h(V,X).

This derivation establishes the conditional mean that a residual-inclusion estimator would need to fit. It does not prove that \(\beta\) is identified in every dataset. Identification also requires enough variation and support to distinguish the effect of \(D\) in \(f\) from the included control function, as well as the relevant instrument validity and exclusion assumptions.

Common Mistake and Practical Implications

Wrong reasoning: “The first stage predicts the endogenous regressor, so replacing the regressor by its fitted value removes its correlation with the outcome error; this must work just as it does in 2SLS.” The correction is that a prediction is a conditional mean, not an observed value. In a nonlinear model, evaluating the response at that mean generally differs from averaging responses over the conditional distribution. Moreover, a fitted value does not specify how the outcome error depends on the actual regressor.

A defensible analysis must state the outcome conditional mean, define the first-stage residual, and explain why the conditional error mean is represented by the included function of that residual. It must also respect the nonlinear function of the actual endogenous regressor rather than silently replacing it with a prediction. If a residual is estimated in an earlier step, uncertainty from that estimation must be accounted for in inference; treating the fitted residual as known can understate uncertainty.

These requirements are modeling assumptions, not mechanical diagnostics. Instrument relevance alone does not validate the conditional-error representation, and a good first-stage fit does not establish that a linear residual is sufficient. The central question is not merely whether the first stage predicts \(D\), but whether the chosen control function captures the outcome-error dependence needed for the specified nonlinear conditional mean.

Check Your Understanding

Use the distinction between prediction, curvature, and conditional-error control to answer the following.

Why can \(f(E[D\mid Z,X],X;\beta)\) differ from \(E[f(D,X;\beta)\mid Z,X]\), even when there is no endogeneity in the outcome error?
In a residual-inclusion model, what does the condition \(E[\varepsilon\mid D,Z,X]=h(V,X)\) say about the role of the first-stage residual?
Why does a correct first-stage conditional mean not, by itself, justify adding a linear residual to a nonlinear outcome model?
For the distribution \(V=-1,0,2\) with probabilities \(1/2,1/4,1/4\), verify that \(E[V]=0\) but \(V^2-1.5\) cannot equal \(\rho V\) for every possible \(V\).
In what important respect does the role of a fitted value in linear 2SLS differ from naive substitution in a nonlinear response function?