The Missing-Data Mean Influence Function

Why Missing Outcomes Change the Mean’s Influence Function

When some outcomes are not recorded, the mean among observed cases is not generally the population mean. The difference matters even if the chance of observing an outcome is unrelated to its value after conditioning on observed covariates. In that setting, the covariates help recover both the target mean and the additional uncertainty caused by missingness.

The efficient influence function makes this adjustment explicit. It combines a prediction based on covariates with a weighted residual for people whose outcomes are observed. Its form also reveals why observations with a low response probability can contribute substantial uncertainty.

Model, Target, and Assumptions

Let \(Y\) be an outcome of interest, \(X\) a vector of fully observed covariates, and \(R\) an indicator equal to 1 when \(Y\) is observed and 0 otherwise. The observed data for one unit are \(O=(X,R,RY)\); when \(R=0\), the recorded value \(RY=0\) is only a placeholder, not the outcome. The target is the population mean \(\mu=E[Y]\).

The equality of conditional means is the feature needed to identify this particular target: the observed outcomes reveal \(m(X)\) for covariate values with positive response probability. The full MAR assumption is stronger: it states that the entire conditional distribution of \(Y\), not only its mean, is the same for responders and nonresponders at a given \(X\). A common sufficient positivity condition is that \(\pi(X)\) is bounded below by a positive constant. Pointwise positivity identifies the mean, but may not be enough for a finite efficiency bound.

For the usual nonparametric MAR model, assume independent, identically distributed observations, \(E[Y^2]<\infty\), and a sufficiently regular model for the observed-data distribution. The square-integrability condition needed for a finite bound is more specifically \(E[\operatorname{Var}(Y\mid X)/\pi(X)]<\infty\). These conditions ensure that the influence function below has finite variance.

The Efficient Influence Function

Under the stated nonparametric MAR model, the efficient influence function is

The expression is evaluated only when \(R=1\) for its residual term; when \(R=0\), that term is zero and the unobserved \(Y\) need not be supplied. The first component, \(m(X)-\mu\), records how the covariate-specific predicted mean differs from the population mean. The second component corrects that prediction using the observed outcome residual \(Y-m(X)\), reweighted by the inverse response probability.

Here is a score calculation that both verifies the derivative representation and explains the weighting. In a regular path through the nonparametric MAR model, an observed-data score can be decomposed as

where \(E[S_X(X)]=0\), \(a(X)\) is a response-mechanism score component, and \(E[S_Y(Y,X)\mid X]=0\). The target’s pathwise derivative is the full-data mean derivative

The second equality follows by conditioning on \(X\): the part of \(Y-\mu\) that varies around \(m(X)\) has conditional mean zero, and the conditional score \(S_Y\) has mean zero given \(X\). Now take the inner product of \(\phi\) with the observed-data score. Its pairing with \(S_X\) is \(E[(m(X)-\mu)S_X(X)]\), because the weighted residual has conditional expectation zero. Its pairing with \((R-\pi(X))a(X)\) is zero: the covariate component is orthogonal to this response score, while the residual component has conditional expectation zero given \(X\). Finally, pairing the residual component with \(R S_Y\) gives

because \(R^2=R\) and, under MAR, \(E[R\mid X,Y]=\pi(X)\). Thus \(E[\phi(O)S_O]=\dot{\mu}\) for every such score. Also, \(E[\phi(O)]=0\), since \(E[m(X)]=\mu\) and \(E[R\{Y-m(X)\}/\pi(X)\mid X]=0\). The covariate component is a mean-zero function of \(X\), and the residual component is a conditional-mean-zero function of the observed outcome among responders; both lie in the nonparametric observed-data tangent space. The derivative representation is therefore the canonical gradient, not merely an influence function with an unnecessary extra component.

Worked Examples

Why This Form Matters

The formula separates two sources of sampling variation. Variation in \(m(X)\) reflects differences in predicted outcomes across covariate values. The residual term reflects outcome variation not explained by \(X\), amplified when outcomes are rarely observed. In particular, the variance identity

shows how limited response can reduce precision. If \(\pi(X)\) is small where the conditional outcome variance is substantial, the efficiency bound can be large or even infinite. Positivity therefore has a precision implication as well as an identification role.

In applications, the function \(m(X)\) is a conditional mean prediction and \(\pi(X)\) is a response probability. The influence function specifies the first-order contribution of each observation when these functions are known; when they are estimated, additional conditions are needed for valid inference. The expression alone does not establish that MAR is credible. Whether missingness is plausibly independent of the outcome after conditioning on \(X\) is a substantive assumption that cannot generally be verified from observed outcomes alone.

Common Pitfall

• Wrong reasoning: “The outcomes are missing at random, so averaging the observed outcomes estimates the population mean.”
• Correction: MAR means independence from \(Y\) conditional on \(X\), not necessarily independence without conditioning. If response probabilities vary with \(X\), the observed cases can have a different distribution of \(X\) from the population. The complete-case mean then generally targets a response-weighted mean. The influence function corrects this through covariate-specific prediction and the inverse-probability-weighted residual.
• Another error: Treating positivity as sufficient for a stable estimator. Positive response probabilities identify the mean under MAR, but probabilities close to zero can inflate the efficiency bound; finite variance requires the weighted conditional variance to be integrable.