Generated Regressors and First-Stage Error

Why First-Stage Error Matters

A second-stage estimator may use a regressor that is not directly observed but is constructed using estimates from a first stage. Examples include fitted values, estimated indices, and predictors formed from estimated parameters. Treating such a regressor as if it were known can conceal sampling uncertainty: the first-stage estimation error may contribute to the second-stage estimator at the same order as its own sampling error.

The key questions are separate. Does consistency of the first-stage estimate suffice for second-stage consistency? And, if both estimators converge at rate \(n^{-1/2}\), how does first-stage uncertainty change the second-stage limiting distribution? A stochastic expansion answers both, while making clear that a first-stage error need not matter at first order in every application.

A Two-Stage Setup and Its Expansion

For clarity, consider a scalar second-stage parameter \(\theta_0\) and a scalar generated regressor \(r_i(\gamma)\), where \(\gamma_0\) is an unknown first-stage parameter. The second-stage estimator solves the sample moment equation below. The observations are independent and identically distributed, and both stages use the same sample of size \(n\).

Write \(r_i=r_i(\gamma_0)\), \(D_i=\partial r_i(\gamma_0)/\partial\gamma\), and suppose the population moment satisfies \(E[r_i(Y_i-\theta_0r_i)]=0\). Assume the moment identifies \(\theta_0\), with \(A=E[r_i^2]>0\). The first-stage estimator is assumed consistent and asymptotically linear:

Here \(\phi_i\) is the first-stage influence contribution; it may be statistically dependent on the second-stage observation. Assume the generated regressor is differentiable near \(\gamma_0\), the sample moments and their derivatives obey laws of large numbers, and the relevant second moments are finite. These regularity conditions justify a first-order Taylor expansion and a joint central limit theorem.

Define the second-stage moment at the truth as \(\psi_i=r_i(Y_i-\theta_0r_i)\), and define \(B=E[D_i(Y_i-2\theta_0r_i)]\). The scalar \(B\) is the derivative of the population second-stage moment with respect to \(\gamma\), evaluated at \((\theta_0,\gamma_0)\). A first-order expansion gives:

The \(-A\) term is the derivative with respect to \(\theta\); the \(B\) term is the derivative with respect to the generated regressor's parameter. Rearranging and substituting the first-stage representation yields the second-stage expansion:

This variance includes the covariance between the two contributions. In particular, adding a second-stage variance to a first-stage variance while assuming independence is generally unjustified when the same observations enter both stages. If stages use independent samples, the covariance structure and sample-size scaling must instead reflect that design.

Consistency Is Not the Same as First-Order Negligibility

Consistency usually requires less than a root-\(n\) expansion. If \(\widehat\gamma\) converges in probability to \(\gamma_0\), \(r_i(\gamma)\) is sufficiently continuous in \(\gamma\), the sample moments converge to their population counterparts, and the population second-stage moment has a unique identified root at \(\theta_0\), then the second-stage moment evaluated at \(\widehat\gamma\) converges to the moment evaluated at \(\gamma_0\). With the usual separation and regularity conditions, its root \(\widehat\theta\) is consistent.

That argument does not establish that the first-stage error is negligible at the \(n^{-1/2}\) scale. In the expansion, its first-order effect is governed by \(B\). If \(B\ne 0\) and the first-stage estimator has a root-\(n\) error, that error contributes to the limiting variance. If the first stage converges more slowly than root-\(n\), its contribution can dominate the second-stage sampling error. If \(B=0\), the first-stage error has no first-order contribution under this expansion, though higher-order effects still require their own control.

Deriving the Joint Influence in a Calibration Example

The same calibration setup makes the first-stage contribution explicit. Let the first-stage regression error be \(v_i\), so that \(\widehat\gamma-\gamma_0=(\sum_i W_iv_i)/(\sum_iW_i^2)\). Let \(q=E[W_i^2]>0\). The second-stage coefficient can be written exactly as the outcome slope on \(W_i\), divided by \(\widehat\gamma\):

To obtain its first-order expansion, write the numerator as \(\theta_0\gamma_0+a_n\) and the denominator as \(\gamma_0+b_n\), where \(a_n=\sum_iW_iu_i/\sum_iW_i^2\) and \(b_n=\sum_iW_iv_i/\sum_iW_i^2\). Since both are \(O_p(n^{-1/2})\), the first-order ratio expansion is \((\theta_0\gamma_0+a_n)/(\gamma_0+b_n)=\theta_0+(a_n-\theta_0b_n)/\gamma_0+o_p(n^{-1/2})\). As \(\sum_iW_i^2/n\) converges to \(q\), this gives:

This expression shows that first-stage and second-stage errors combine within one influence contribution. Under \(E[W_i u_i]=E[W_i v_i]=0\), finite second moments, and a central limit theorem, the asymptotic variance is:

This variance formula is valid without imposing zero conditional means. If conditional means are nonzero, the variance includes their contribution through \(\operatorname{Var}\{W_i(u_i-\theta_0v_i)\}\); conditional variances alone do not determine it.

When First-Stage Convergence Is Slower

A nonzero first-stage derivative can also change the rate of the second-stage estimator. In the calibration expansion, the first-stage contribution is proportional to \(\widehat\gamma-\gamma_0\). If that error is larger than order \(n^{-1/2}\), it can dominate the second-stage outcome noise.

Common Pitfall: Using Conditional Variances Without Conditional Means

A tempting shortcut is to calculate a conditional variance and then treat it as the conditional second moment. That is correct only when the conditional mean is zero. In the calibration example, let \(Z_i=u_i-\theta_0v_i\). In general:

Consequently, the simplified variance based on conditional variances alone requires suitable conditional mean-zero restrictions, such as \(E[u_i\mid W_i]=E[v_i\mid W_i]=0\). Without them, use the variance of the full influence contribution, \(\operatorname{Var}\{W_i(u_i-\theta_0v_i)\}/(\gamma_0^2q^2)\), under the required unconditional moment and regularity conditions. Also distinguish conditional covariance from a conditional cross-moment: they are equal only when the relevant conditional means are zero.

For practical inference, estimate the variance of the full influence contribution, or use an equivalent joint sandwich calculation based on both stages. This retains any dependence between stages and avoids silently imposing assumptions that the data do not support.