Inference After Data-Adaptive Nuisance Estimation

Why Inference Needs More Than an AIPW Estimate

A cross-fitted AIPW estimate can use flexible, data-adaptive models for treatment assignment and outcomes. But a point estimate alone does not show how much it varies across samples. Inference requires estimating that variation while accounting for the fact that nuisance functions were estimated from data. Under suitable conditions, the influence function provides a way to do this: its empirical variation estimates the leading sampling variability of the AIPW estimate.

The resulting confidence interval is not automatically reliable merely because cross-fitting was used. Its coverage depends on causal assumptions, adequate overlap, controlled nuisance-estimation error, finite variability, and an asymptotic approximation that is accurate enough for the sample size. The calculations below show how to construct the interval and what to check before interpreting it.

For a \(K\)-fold cross-fit, each observation belongs to one held-out fold. The nuisance estimates used for that observation are trained without its fold; write them as \(\hat e_{-k(i)}(X_i)\) and \(\hat m_{a,-k(i)}(X_i)\). Define its estimated AIPW score by

The cross-fitted AIPW estimate is \(\hat\theta=n^{-1}\sum_{i=1}^n\hat\psi_i\). The estimated influence value for observation \(i\) is \(\hat\phi_i=\hat\psi_i-\hat\theta\). Centering by \(\hat\theta\) makes the empirical mean of these values zero, as it should be for estimated deviations around the estimated target.

From Estimated Influence Values to a Confidence Interval

If the AIPW estimate is asymptotically linear, its leading sampling error is the average of its influence function. At the true nuisance functions, that influence function is \(\phi_0(O)=\psi(e_0,m_{10},m_{00})(O)-\theta\). Thus, under a central limit theorem and a nonzero finite variance, the estimate is approximately normal around \(\theta\), with variance approximately \(E\{\phi_0(O)^2\}/n\).

Here \(\widehat{\operatorname{Var}}(\hat\phi_i)=\sum_{i=1}^n\hat\phi_i^2/(n-1)\) is the sample variance of the estimated influence values. An approximate two-sided \(100(1-\alpha)\%\) Wald confidence interval is

where \(z_{1-\alpha/2}\) is the corresponding standard-normal quantile. For example, \(z_{0.975}\) is approximately \(1.96\) for a 95% interval. Its usual frequentist interpretation is that, over repeated samples under the stated assumptions, intervals constructed this way cover the target at approximately the stated rate. It is not a statement that the fixed parameter has a 95% probability of lying in this particular interval.

Why Cross-Fitting Can Support Valid Coverage

Using the same observations both to fit complex nuisance functions and to evaluate their residual contributions can create empirical dependence that is difficult to control. Cross-fitting separates these roles: each observation's score is evaluated using nuisance predictions trained outside its fold. This separation helps control the effect of adaptive fitting; it does not itself guarantee that the fitted functions are accurate enough or that the target is identified.

A useful justification comes from the conditional bias of the AIPW score. Treat \(e,m_1,m_0\) as fixed candidate functions for this calculation. Given \(X\), the difference between the conditional expectation of their score and the true conditional treatment effect is

To see the signs and the product structure, put \(\delta_1=m_1-m_{10}\) and \(\delta_0=m_0-m_{00}\). The treated residual has conditional mean \(-\delta_1\) when \(A=1\); the control residual has conditional mean \(-\delta_0\) when \(A=0\), and its term is subtracted in the score. Substitution gives the two contributions \((e-e_0)\delta_1/e\) and \((e-e_0)\delta_0/(1-e)\), which add. In particular, the control-regression error has a plus sign inside the braces. This second-order bias is one expression of the score's orthogonality: small errors in both nuisance components interact rather than contributing separate first-order bias terms.

Under bounded denominators and suitable integrability, a common sufficient rate requirement is that the product of the propensity-score error and the outcome-regression errors is smaller than \(n^{-1/2}\). One convenient version uses \(L_2\) norms over the covariate distribution:

This is a sufficient condition, not a universal necessary one; precise conditions depend on moments and how errors behave in regions with small propensity scores. Cross-fitting, nuisance consistency, and control of empirical variability together help establish the asymptotic linear representation and consistency of the estimated influence-function variance.

Assumptions and Practical Diagnostics

For a causal interpretation of the ATE, consistency links each observed outcome to the potential outcome under received treatment, and there must be no interference between units. Conditional exchangeability, \((Y(1),Y(0))\mathbin{\perp\!\!\!\perp} A\mid X\), rules out unmeasured confounding after conditioning on \(X\). Positivity requires \(0<e_0(X)<1\) in the target population. For stable inference, it is often necessary to impose stronger practical control, such as \(e_0(X)\) bounded away from zero and one, or to impose tail and moment conditions that control the weighted residuals.

A sufficient set of inferential conditions also includes independent sampling, finite and nonzero variance of the influence function, nuisance estimates with suitable consistency and product rates, and consistent estimation of the influence-function variance. Appropriate finite \(2+\delta\) moments for some \(\delta>0\) are a common sufficient condition for controlling tails and invoking a central limit theorem. These requirements are not guaranteed by a good-looking fit or by a large number of folds.

Diagnostics should therefore examine the ingredients that drive the interval, not just its width. Inspect the distribution of estimated propensity scores and the largest inverse weights; look for very small denominators, sparse treated or control regions, and large influence values. Check whether the nuisance predictions are plausible and whether the fitted procedures achieve credible out-of-fold performance. A narrow interval can be misleading if overlap or nuisance-rate assumptions fail. Conversely, extreme estimated influence values often signal unstable variance estimates, even when the algebra of the interval is correct.

What the Interval Does—and Does Not—Account For

The influence-function standard error estimates the leading sampling variance of the AIPW estimate under the asymptotic linearity conditions. It is not a separate variance calculation for each fitted nuisance model, nor does it repair bias from an incorrect causal design or inadequate overlap. If nuisance errors leave a first-order bias, the usual centered, normal interval may miss the target even if its standard error has been computed exactly from the estimated influence values.

This differs from a naive approach that treats fitted propensity and outcome predictions as known constants and reports only a regression-model standard error. That approach generally omits the relevant variation in the AIPW score and is not justified by the AIPW influence-function result. A bootstrap that repeats the entire fitting and cross-fitting procedure is another possible strategy, but its validity depends on the estimators and resampling scheme; it is not automatically a remedy for failed overlap or failed rate conditions. The influence-function interval is attractive because it is direct and computationally economical, while remaining an asymptotic approximation rather than an exact finite-sample guarantee.