Second-Order Remainders and Product-Rate Conditions

Why Second-Order Remainders Matter

An estimating equation can depend on both a target parameter and nuisance functions. The target is the quantity of interest; nuisance functions help construct a moment that identifies it. In practice, those functions must be estimated. The key question is whether their estimation errors distort the target equation at a consequential order.

Neyman orthogonality makes the population estimating equation locally insensitive, to first order, to perturbations of the nuisance functions. It does not make the equation insensitive to every possible error. After the first-order effect vanishes, a second-order remainder remains. Product-rate conditions control that remainder by requiring the nuisance estimation errors to be sufficiently small jointly.

More precisely, for any admissible direction \(h\), orthogonality means \( \left.\frac{d}{dt}M(\theta_0,\eta_0+th)\right|_{t=0}=0 \). The target equation still needs to identify \(\theta_0\): changing \(\theta\) should change the moment in a way that permits the parameter to be distinguished. Orthogonality concerns perturbations of \(\eta\), not perturbations of \(\theta\).

From Orthogonality to a Remainder Bound

Write an estimated nuisance function as \(\widehat{\eta}=\eta_0+\Delta\eta\). A Taylor expansion of the population moment around the truth has the form

where \(D_{\eta}M[\Delta\eta]\) is the first directional derivative, \(D_{\eta}^{2}M[\Delta\eta,\Delta\eta]\) is the second derivative applied twice to the perturbation, and \(r\) is the higher-order Taylor remainder. The first term is zero because the true moment is zero; the first derivative is zero by orthogonality. If the second derivative is bounded in a suitable norm and the higher-order remainder is controlled, the remaining moment error is quadratic in the nuisance error. With multiple nuisance components, it often takes the more useful form of a product of errors.

That qualification matters. Orthogonality alone is a statement about a derivative at one point. It does not, by itself, guarantee a particular quadratic bound far from that point. A product-rate argument needs both orthogonality and enough regularity to control the remainder over the estimation errors under consideration.

Product Rates in a Partially Linear Model

A second example shows why a product bound can involve errors from distinct nuisance functions. Suppose \(Y=\theta_0D+g_0(X)+U\), with \(E[U\mid D,X]=0\), and let \(m_0(X)=E[D\mid X]\). Consider the score \(\psi(W;\theta,m,g)=(D-m(X))\{Y-\theta D-g(X)\}\). Assume \(E[(D-m_0(X))^2]>0\), so the residualized regressor has nonzero variation.

At the true parameter, the residual in the second factor is \(U\). The conditional mean restriction implies \(E[U\mid X]=0\) and \(E[(D-m_0(X))U\mid X]=0\). Consequently, the true moment is zero. The derivative of the moment with respect to \(\theta\) at the truth is \(-E[(D-m_0(X))D]=-E[(D-m_0(X))^2]\), which is nonzero by assumption.

What the Rate Condition Requires

Suppose a target estimator is based on an empirical estimating equation and the target moment changes at a nonzero rate as \(\theta\) changes. For the nuisance-induced population remainder not to affect a root-\(n\) limit, a common requirement is \(R(\widehat{\eta})=o_p(n^{-1/2})\). For a product remainder, sufficient conditions take the form

This condition concerns the product, not a universal requirement that each nuisance function converge at the same rate. If both errors are \(O_p(n^{-1/4})\), their product is only \(O_p(n^{-1/2})\); that bound alone does not establish the stricter \(o_p(n^{-1/2})\) condition. If one error is \(O_p(n^{-0.30})\) and the other is \(O_p(n^{-0.23})\), their product is \(O_p(n^{-0.53})\), which does meet it.

For a quadratic remainder bounded by \(C\|\widehat{\eta}-\eta_0\|_2^2\), it is sufficient that \(\|\widehat{\eta}-\eta_0\|_2=o_p(n^{-1/4})\). More generally, with several nuisance errors, the relevant condition follows from the actual remainder bound: one component may need a faster rate, or one error may compensate for another. Always derive or establish the bound for the score at hand rather than assuming a product condition automatically applies.

Common Pitfall: Orthogonality Is Not a Rate Guarantee

A common mistake is to reason, “The score is orthogonal, so nuisance estimation cannot affect the target.” Orthogonality only eliminates the first-order population effect at the true nuisance functions. A second-order remainder remains, and it can still be too large. For example, two errors each of order \(n^{-0.20}\) have a product of order \(n^{-0.40}\), which is not negligible relative to \(n^{-1/2}\).

There is also a distinction between controlling the population remainder and controlling the full empirical estimating equation. The product-rate bound addresses the former. Sampling fluctuation from estimating and evaluating the score, dependence between observations, and the validity of a limiting distribution require their own conditions. In particular, a product-rate condition by itself does not prove asymptotic normality or justify a confidence interval.