Extreme Scores and Finite-Sample AIPW Instability

Why Extreme Scores Can Destabilize AIPW

The augmented inverse probability weighted estimator (AIPW) combines outcome predictions with residual corrections weighted by estimated treatment probabilities. Those corrections help protect the average treatment effect (ATE) against certain nuisance-model errors, but their size depends directly on the propensity score. If an observed treated unit has an estimated treatment probability close to zero, its treated residual is divided by a small number. If an observed control unit has an estimated treatment probability close to one, its control residual is divided by a small number. Either event can make one observation exert a disproportionate influence on the estimate.

This is a finite-sample stability issue, not a guarantee that the estimator is biased in every sample or that the identification assumptions have failed. It is important, however, because a numerically computable AIPW estimate can be highly variable, sensitive to a few observations, or poorly approximated by routine standard-error calculations.

Estimator, Assumptions, and the Source of Amplification

Let \(A_i\) be a binary treatment, \(Y_i\) the observed outcome, and \(X_i\) the measured covariates for unit \(i\). Write \(e(X)=P(A=1\mid X)\) for the propensity score and \(m_a(X)=E[Y\mid A=a,X]\) for the outcome regression under treatment level \(a\). The AIPW estimator of the ATE, using fitted nuisance functions \(\hat e\) and \(\hat m_a\), is

The bracketed quantity is the unit’s AIPW contribution, sometimes called its pseudo-outcome. It contains a prediction contrast and two residual corrections. For a treated unit, only the treated correction is active; for a control unit, only the control correction is active.

The residual corrections are amplified because the denominators are \(\hat e(X_i)\) for treated units and \(1-\hat e(X_i)\) for controls. A small denominator does not guarantee a large correction: the residual might be close to zero. But when a non-negligible residual occurs with a small denominator, the resulting term can be large.

A Variance Calculation Explains the Risk

The following calculation isolates the residual correction when the nuisance functions equal their true values. Conditional on \(X\), define the treated and control outcome variances as \(\sigma_1^2(X)=\operatorname{Var}(Y\mid A=1,X)\) and \(\sigma_0^2(X)=\operatorname{Var}(Y\mid A=0,X)\). Under exchangeability and positivity, the residual correction is

Its conditional mean is zero: for treated units the residual has conditional mean zero, as it does for controls. To calculate its conditional second moment, note that \(A(1-A)=0\), so the treated and control terms cannot both be nonzero for the same unit. Therefore,

Since \(E[R\mid X]=0\), this second moment is also the conditional variance of \(R\). The \(1/e(X)\) and \(1/\{1-e(X)\}\) factors show why residual variability can become large near the score boundaries, even with correctly specified nuisance functions. This calculation concerns the residual component under true nuisance functions; with estimated functions, the exact finite-sample behavior also depends on their errors and on how the scores were estimated.

Worked Contributions Show Finite-Sample Instability

Diagnostics: Find the Influential Terms

A useful diagnostic examines the components of the estimator rather than relying only on a summary of the propensity-score distribution. For each unit, calculate or inspect the treated correction \(A_i\{Y_i-\hat m_1(X_i)\}/\hat e(X_i)\), the control correction \(-(1-A_i)\{Y_i-\hat m_0(X_i)\}/\{1-\hat e(X_i)\}\), and the full contribution \(q_i\). Look for large absolute values, a small number of terms dominating the sum, and sensitivity of the estimate to individual observations.

Also inspect the estimated-score distribution separately among treated and control units. A treated unit with a very small score and a control unit with a very large score are the direct sources of large inverse-probability factors. The score extremes in the other treatment group still signal sparse covariate overlap and can indicate that comparable observations are scarce. Check that the extreme scores and covariate patterns are not caused by coding errors, separation, or unstable model fitting. If using a flexible fit, review how predictions were obtained and whether the fitting procedure produces scores extremely close to zero or one.

Weight summaries can add context. For example, for treated units with weights \(w_i=1/\hat e(X_i)\), the quantity \((\sum_{A_i=1}w_i)^2/\sum_{A_i=1}w_i^2\) summarizes how concentrated those weights are; an analogous calculation applies to controls with weights \(1/\{1-\hat e(X_i)\}\). Such summaries do not replace inspection of residual contributions: a large weight paired with a near-zero residual may have little direct effect on this particular estimate.

A binary outcome does not automatically make the fitted residual bounded by one. If \(Y\) is binary and the fitted mean is constrained to \([0,1]\), then \(|Y-\hat m_a(X)|\leq 1\). But an unconstrained outcome model can predict outside that interval, in which case the residual need not have that bound. Even when the residual is bounded, division by a very small score can still yield a large correction.

What Instability Is—and Is Not

Extreme estimated scores can arise even when the population satisfies positivity: a finite sample may contain few observations in a covariate region, and an estimated model may return predictions near the boundaries. If the true treatment probability is zero or one for a target-population covariate pattern, that is a positivity violation and the causal contrast is not identified there from the observed data without additional assumptions. A large correction alone does not prove such a violation; it signals a need to investigate overlap, model fitting, and the target population.

Double robustness is not protection against this problem. Under its standard conditions, AIPW can be consistent if either the propensity model or the outcome model is correctly specified, but this property does not ensure small finite-sample variance, prevent extreme fitted scores, or guarantee reliable normal approximations. Likewise, an influence-function standard error can be large when contributions are extreme, but a conventional standard error does not repair weak overlap or undue sensitivity to a few observations.

The main alternatives have different tradeoffs. Outcome-regression estimation avoids inverse-score residual corrections but depends more directly on outcome-model extrapolation in sparse regions. Inverse-probability weighting uses inverse scores without the AIPW prediction augmentation, so it can also be sensitive to extreme weights. Matching seeks comparable units but may discard observations or change the effective target population. None creates information where treatment groups scarcely overlap. Any modification of extreme weights or scores changes the procedure and may trade variance for bias; that decision requires explicit consideration rather than treating a numerical cap as a harmless repair.

Common Mistakes

• Assuming double robustness guarantees stability. It addresses consistency under specified nuisance-model conditions, not the size of finite-sample residual corrections.
• Calling every extreme fitted score a positivity violation. A fitted score is an estimate. Investigate the covariate support and fitting behavior before making a claim about the true treatment mechanism.
• Checking scores but not contributions. The residual determines whether an extreme inverse factor actually produces a large term. Inspect both.
• Assuming a binary outcome always gives a residual bounded by one. That bound requires the fitted mean to lie in \([0,1]\). Unconstrained fitted values can fall outside that range.
• Treating a large AIPW estimate as a plausible effect just because the formula produced it. For a binary outcome, an ATE must lie between \(-1\) and \(1\); a finite-sample estimate outside this range can reveal instability.