Doubly Robust Estimation With Missing Outcomes

Why Combine Outcome Regression and Weighting?

When an outcome is missing for some individuals, an analysis of observed outcomes alone can describe a selected subset rather than the target population. Outcome regression addresses this by predicting outcomes for everyone from their covariates. Inverse-probability weighting instead gives more influence to observed outcomes from individuals who were less likely to be observed. Each approach depends on a model: the outcome model or the observation model. An augmented inverse-probability-weighted estimator combines them so that, under suitable conditions, either model can protect consistency.

The target here is a population average treatment effect. Treatment may be observational, so adjustment for measured confounders is required. To isolate robustness to missing outcomes, the treatment propensity model is taken to be known or correctly specified when the observation-model route to consistency is used. In a randomized study, the assignment probabilities are known by design.

Setup, Target, and Assumptions

Definition: Let \(A\in\{0,1\}\) be treatment, \(X\) the measured pre-treatment covariates, \(Y\) the outcome, and \(R\) an indicator equal to 1 when \(Y\) is observed. Let \(Y(a)\) denote the potential outcome under treatment level \(a\). The target is the average treatment effect \(E[Y(1)-Y(0)]\). Define the treatment propensity \(g_a(X)=P(A=a\mid X)\), the observation probability \(\pi_a(X)=P(R=1\mid A=a,X)\), and the observed-outcome regression \(m_a(X)=E[Y\mid A=a,X,R=1]\).

Identification requires consistency, meaning that the observed outcome equals \(Y(A)\) when observed, and conditional exchangeability of treatment: \(Y(a)\perp A\mid X\) for each \(a\). It also requires treatment positivity, \(g_a(X)>0\) in the target population. For missingness, assume outcomes are missing at random conditional on treatment and covariates: \(R\perp Y\mid A,X\). This implies that the mean outcome among observed individuals with given \(A,X\) equals the mean for all individuals with those values. Finally, require observation positivity, \(\pi_a(X)>0\) wherever the target population has positive probability.

Under these assumptions, the target can be written as \(\psi=\mu_1-\mu_0\), where \(\mu_a=E[m_a(X)]\). The expectation averages over the target population's covariate distribution. Missing at random makes \(m_a(X)\) identifiable from observed outcomes; treatment exchangeability and positivity then connect these conditional means to the potential-outcome means.

The Augmented Estimator

For each treatment level, combine a prediction for every individual with a weighted correction based on observed residuals. The weight is large when an individual both had a lower probability of receiving that treatment and a lower probability of having the outcome observed.

\widehat{\mu}_a = \frac{1}{n}\sum_{i=1}^{n} \left[ \widehat m_a(X_i) + \frac{\mathbb{1}(A_i=a)R_i} {\widehat g_a(X_i)\widehat\pi_a(X_i)} \{Y_i-\widehat m_a(X_i)\} \right], \qquad \widehat{\psi}=\widehat{\mu}_1-\widehat{\mu}_0 .

Here \(\mathbb{1}(A_i=a)\) is 1 when individual \(i\) received treatment \(a\), and 0 otherwise. The residual term is used only when both treatment equals \(a\) and the outcome is observed. In practice, \(Y_i\) need not be available when \(R_i=0\); the product with \(R_i\) denotes a contribution of zero in that case. Hats indicate fitted or known nuisance quantities. For a randomized study, \(\widehat g_a\) can be replaced by the known assignment probability.

Worked Example: Calculating the Augmented Contrast

Consider eight hypothetical individuals, with four assigned to each treatment. Covariates are balanced across the two arms. The fitted outcome predictions are 10 for treatment 1 and 8 for treatment 0 when \(X=0\), and 14 and 12, respectively, when \(X=1\). Suppose the known treatment probability is 0.5 for each arm and the fitted observation probability is 0.8 for both arms and covariate values.

Individual	A	X	R	Y if observed	\(\widehat m_1(X)\)	\(\widehat m_0(X)\)
1	1	0	1	12	10	8
2	1	1	1	16	14	12
3	1	0	0	Missing	10	8
4	1	1	1	13	14	12
5	0	0	1	7	10	8
6	0	1	1	11	14	12
7	0	0	0	Missing	10	8
8	0	1	1	13	14	12

The average predictions across all eight individuals are 12 for treatment 1 and 10 for treatment 0. For an observed treated individual, the residual weight is \(1/(0.5\times0.8)=2.5\). The observed treated residuals are \(12-10=2\), \(16-14=2\), and \(13-14=-1\). Thus:

\widehat{\mu}_1 =12+\frac{2.5(2+2-1)}{8} =12.9375.

For an observed control individual, the residuals are \(7-8=-1\), \(11-12=-1\), and \(13-12=1\). The control-arm estimate is:

\widehat{\mu}_0 =10+\frac{2.5(-1-1+1)}{8} =9.6875, \qquad \widehat{\psi}=12.9375-9.6875=3.250.

The estimated contrast is 3.250 outcome units. Its causal interpretation depends on the identification assumptions and the validity conditions for the nuisance models; the arithmetic alone does not establish those conditions.

Why the Estimator Is Doubly Robust

The robustness claim follows by taking the expectation of an arm-specific contribution, treating the fitted functions as fixed limits. For clarity, suppose first that \(g_a\) and \(\pi_a\) denote the functions used in the estimator. Conditional on \(X\), the expected contribution is:

E\left[ m_a(X)+ \frac{\mathbb{1}(A=a)R}{g_a(X)\pi_a(X)} \{Y-m_a(X)\} \middle|X \right] = m_a(X)+ \frac{g_a^*(X)\pi_a^*(X)} {g_a(X)\pi_a(X)} \{m_a^*(X)-m_a(X)\}.

Here \(g_a^*(X)\) is the true treatment probability and \(\pi_a^*(X)\) the true observation probability. The equality uses \(P(A=a,R=1\mid X)=g_a^*(X)\pi_a^*(X)\), and, under missing at random, \(E[Y-m_a(X)\mid A=a,R=1,X]=m_a^*(X)-m_a(X)\), where \(m_a^*(X)\) is the true conditional outcome mean.

If the outcome regression is correct, then \(m_a(X)=m_a^*(X)\). The residual difference is zero, so the conditional expectation reduces to \(m_a^*(X)\), regardless of misspecification of the observation probability. If instead both the treatment and observation probabilities are correct, the ratio of probabilities is 1. The conditional expectation becomes \(m_a(X)+m_a^*(X)-m_a(X)=m_a^*(X)\), even if the outcome regression is wrong. Averaging over \(X\) gives \(\mu_a\) in either case.

Theorem: Under the identification assumptions and positivity, the augmented estimator is consistent for each treatment-arm mean if either (1) the outcome regression \(m_a\) is correct, or (2) both the treatment propensity \(g_a\) and observation probability \(\pi_a\) are correct. Thus, with treatment assignment known or correctly modeled, it is robust to misspecification of either the outcome model or the missingness model. This robustness does not protect against misspecification of both relevant models.

The estimator is called augmented because the weighted residual correction is added to the outcome-regression estimate. If the outcome model predicts conditional means well, residuals tend to be small and the correction may improve efficiency. If the outcome model is wrong but the probabilities are right, the weighted residuals correct its average error. These are consistency statements, not guarantees of low variance or good finite-sample performance.

Worked Example: Correct Outcome Model, Incorrect Observation Model

Suppose that within a covariate stratum and treatment arm, the true mean outcome is 10 units. The outcome model also predicts 10, but the true observation probability is 0.8 while the fitted model says 0.5. Let the treatment probability be correctly known as 0.5. The conditional expectation of the residual is \(10-10=0\), so the expected augmentation is zero even with the incorrect observation model:

10+ \frac{(0.5)(0.8)}{(0.5)(0.5)}(10-10) =10.

The arm mean remains correctly recovered in expectation. This example illustrates the outcome-model branch of robustness; it does not imply that the wrong observation model is harmless for precision or for finite-sample estimates.

Worked Example: Correct Probability Models, Incorrect Outcome Model

Now suppose the true conditional mean is 10, but the outcome model predicts 8. Let the treatment probability be 0.5 and the observation probability be 0.8, both correctly specified. Among observed individuals, the expected residual is \(10-8=2\). The expected weighted correction is:

\frac{(0.5)(0.8)}{(0.5)(0.8)}(10-8)=2.

Adding this correction to the prediction gives \(8+2=10\), the true conditional mean. If the fitted observation probability were instead 0.5, the correction would be \(\{(0.5)(0.8)/[(0.5)(0.5)]\}\times2=3.2\), giving 11.2 rather than 10. This shows why a correct outcome model or correct probability models are needed for the stated robustness result.

Practical Checks and a Common Pitfall

Before interpreting an estimate, check that treatment and observation probabilities are positive and not so close to zero that a few observations dominate the weighted correction. Very large weights can make estimates unstable even when the models are correctly specified. Inspect the fitted probabilities and the distribution of the weights, and report how the nuisance models were chosen. Robustness to one model's misspecification does not remove the need to justify the assumptions, including missing at random and treatment exchangeability.

For uncertainty quantification, standard errors must reflect the sampling variability of the augmented estimate and the fitted nuisance models. When both nuisance models are estimated consistently, the estimator has a standard influence-function-based variance under regularity conditions; resampling procedures can also be used when appropriate. A confidence interval describes uncertainty under the sampling and modeling assumptions, not uncertainty about whether those assumptions are true.

Mistake: “The estimator is doubly robust, so the treatment model can be wrong whenever the observation model is right.”
Correction: In an observational study, the observation-model route requires the treatment propensity to be correct as well. With known randomized assignment, this requirement is satisfied by design. The other route is a correct outcome regression, under the stated identification assumptions.
Mistake: “The correction fills in each missing outcome with its predicted value.”
Correction: The prediction contributes for everyone, but the residual correction uses observed outcomes weighted by their probabilities. It is a population-level estimating adjustment, not an individual recovery of a missing value.

Key takeaway: The augmented inverse-probability-weighted estimator averages outcome predictions and adds weighted observed residuals. With treatment adjustment valid, it consistently estimates a causal contrast if the outcome regression is correct or if the observation model is correct; neither robustness nor weighting replaces the causal and missingness assumptions.

Check Your Understanding

Use the assumptions and estimator to answer the following questions.

Why does the arm-specific estimator include both the treatment propensity and the observation probability in the residual weight?
Under what conditions does a correct outcome regression make the expected residual correction zero?
In an observational study, what additional model must be correct for the observation-model route to consistency?
If the outcome prediction is 8, the true conditional mean is 10, and both probabilities are correct, what is the expected weighted residual correction?
Why does a positive estimated causal contrast not, by itself, establish that treatment caused an increase in the outcome?