One-Step Estimation from an Influence Function

Why Add an Influence-Function Correction?

A plug-in estimator evaluates a target quantity at estimated features of a distribution. Even when those features are estimated flexibly, the plug-in estimate can retain a first-order bias. An influence-function correction adjusts the plug-in estimate using the observed data. Under suitable conditions, the correction removes the leading error while leaving a remainder small enough that the estimator behaves, to first order, like an average of influence-function values.

This construction is useful because it separates the estimator into two parts: an initial estimate, which may use a complicated model, and a correction whose first-order behavior is characterized by an influence function. The correction does not automatically guarantee good performance. Its asymptotic justification depends on the quality of the initial estimate, the remainder in the expansion, and control of the empirical fluctuation of the estimated influence function.

Write \(P_n f=n^{-1}\sum_{i=1}^n f(O_i)\) for the empirical average. If \(Q_n\) is an estimated distribution or estimated nuisance structure, the basic one-step form is

The first term is the initial plug-in estimate. The second is the empirical mean of an influence function evaluated using the estimated structure. Because \(\phi_{Q_n}\) is centered under \(Q_n\), its empirical mean is intended to measure how the observed data depart from what that fitted structure predicts. In implementations that estimate only nuisance functions, the formula is written using those functions rather than an entire distribution.

The Error Expansion and Conditions for Linearity

Let \(P_0\) be the true distribution and \(\psi_0=\psi(P_0)\). Suppose the influence-function expansion around an estimated distribution \(Q_n\) can be written as

where \(R_2(Q_n,P_0)\) is a second-order remainder. This equation fixes the sign convention for the remainder used here. Substitute it into the one-step error and add and subtract \(P_0\phi_{Q_n}\):

This identity explains the method. The remainder captures bias from approximating the target by the fitted structure. The empirical term captures sampling fluctuation. To obtain a first-order representation in terms of the true influence function \(\phi_{P_0}\), one needs the remainder to be negligible at the \(n^{-1/2}\) scale and the empirical fluctuation of the estimated influence function to be replaceable by that of the true one.

For example, under sample splitting, nuisance estimates are trained on data separate from the observations used in the empirical correction. Conditional on the training data, the estimated influence function is fixed on the evaluation sample. If it converges to \(\phi_{P_0}\) in \(L_2(P_0)\), the variance of the difference in empirical averages is controlled by that squared \(L_2\) distance divided by the evaluation sample size. Cross-fitting applies this idea across folds, so each observation’s correction uses nuisance estimates trained without that observation.

If \(\phi_{P_0}\) has finite variance, the central limit theorem then gives

This is asymptotic linearity: the estimator’s leading error is an average of independent, mean-zero influence-function values. If the influence function is the canonical gradient in the model, this representation also attains the semiparametric efficiency bound under the stated regularity conditions. Asymptotic linearity itself, however, does not establish that an estimator is efficient.

Example with Estimated Nuisance Functions

A common application is estimating a population outcome mean when outcomes are missing. Let \(X\) be covariates, \(R\) indicate whether the outcome \(Y\) is observed, \(m(X)=E(Y\mid X)\), and \(\pi(X)=P(R=1\mid X)\). Under missing at random, \(R\) is conditionally independent of \(Y\) given \(X\); positivity requires \(\pi(X)>0\) on the covariate values of interest. The target is \(\psi_0=E\{m(X)\}=E(Y)\). Its influence function is

With estimates \(\hat m\) and \(\hat\pi\), the one-step estimate can be written as the mean of predicted outcomes plus a weighted residual correction. Outcomes \(Y\) in the table below are shown only when \(R=1\).

A Nonlinear Target: The Quantile

One-step estimation is not restricted to means. For a distribution with a continuous density \(f\) that is positive at its \(\tau\)-quantile \(q_\tau\), the quantile influence function is

The estimated correction compares the empirical fraction at or below an initial quantile with the target fraction \(\tau\), then scales that difference by the estimated density. A small density makes the correction large, reflecting the fact that a small probability change can correspond to a substantial change in the quantile.

Common Pitfall and Practical Interpretation

A frequent mistake is to assume that a one-step estimate is reliable merely because the correction is small—or, conversely, that a large correction automatically means the procedure has failed. The correction is only one component of the error expansion. A small correction can coexist with a poor initial estimate, a large second-order remainder, or an unstable estimated influence function. A large correction can arise from finite-sample fluctuation or from a sensitive target, as when the density near a quantile is small.

The appropriate question is whether the estimator’s conditions support the expansion: the target must have a valid influence function in the model, the nuisance estimates must make the remainder negligible, and the estimated influence-function empirical fluctuation must be controlled. Sample splitting or cross-fitting addresses the last issue in many flexible-estimation settings; it does not by itself guarantee fast enough nuisance rates or a small remainder. When these conditions hold, the correction gives a principled first-order adjustment rather than an arbitrary tweak to a plug-in estimate.