A Worked Comparison of Matching, IPW, and AIPW

Why Compare the Estimators on the Same Data?

Matching, inverse-probability weighting (IPW), and augmented inverse-probability weighting (AIPW) are different ways to use observed covariates to estimate a causal effect from observational data. Applying all three to the same data makes their differences visible: matching imputes missing potential outcomes from selected comparison units, IPW reweights observed outcomes, and AIPW combines weighting with outcome regression. Similar estimates are not proof that the assumptions behind causal interpretation hold.

The target here is the average treatment effect (ATE) for the units represented by the observed sample. Let \(T_i\) indicate treatment for unit \(i\), let \(X_i\) denote observed pretreatment covariates, and let \(Y_i(1)\) and \(Y_i(0)\) be that unit’s potential outcomes under treatment and control. The observed outcome is \(Y_i=T_iY_i(1)+(1-T_i)Y_i(0)\). The ATE is the average difference \(Y_i(1)-Y_i(0)\) across the target population; neither potential outcome pair is observed for any one unit.

For this worked comparison, \(X\) is a binary pretreatment characteristic with values 0 and 1. All values and outcomes below are hypothetical. The treatment and control groups both contain units at each value of \(X\), so there is empirical overlap. The sample is deliberately small to keep each calculation inspectable; that also makes every numerical result fragile.

The Hypothetical Data and Its First Diagnostics

The treatment propensity is \(e(x)=P(T=1\mid X=x)\). Its empirical estimate within each covariate stratum is \(\hat e(0)=2/3\) and \(\hat e(1)=2/5\). Both are strictly between zero and one, but the small stratum counts mean these estimates are imprecise. In the raw data, half the treated units have \(X=0\), compared with one quarter of controls. Thus the groups are not balanced on \(X\) before adjustment.

One-Nearest-Neighbor Matching

For clarity, use exact matching on \(X\), which is nearest-neighbor matching here because units in the same stratum have distance zero and units in different strata cannot serve as exact matches. Match with replacement: the same comparison unit may be selected more than once. When several opposite-treatment units are equally near, select the one with the earliest letter in the table. The rule selects exactly one neighbor per unit; it does not average outcomes over all tied neighbors.

To estimate the ATE, predict each unit’s unobserved potential outcome using its selected opposite-treatment neighbor. For a treated unit, subtract its matched control’s outcome from its observed outcome. For a control, subtract its observed outcome from its matched treated unit’s outcome. Average these \(n\) imputed individual contrasts. This is a particular matching estimator, and with replacement and ties its value can depend on the stated tie-breaking rule.

IPW: Reweighting Observed Outcomes

The Horvitz–Thompson IPW estimator for the ATE weights each treated outcome by the inverse of its estimated treatment propensity and each control outcome by the inverse of its estimated probability of control. In this example, treated units at \(X=0\) receive weight \(1/(2/3)=1.5\); treated units at \(X=1\) receive weight \(1/(2/5)=2.5\). Control weights are \(1/(1-2/3)=3\) at \(X=0\) and \(1/(1-2/5)=5/3\) at \(X=1\).

AIPW: Combining Outcome Regression and Weighting

AIPW augments the weighting calculation with outcome regression. Let \(m_t(x)=E[Y\mid T=t,X=x]\), estimated separately for treatment state \(t\). The sample AIPW estimator averages the predicted contrast \(m_1(X_i)-m_0(X_i)\), plus a weighted residual correction for the observed outcome in the treatment state actually received. Under the standard conditions, it is consistent if either the propensity model or the outcome-regression model is correctly specified, provided the other model is adequately controlled for the relevant estimation conditions. This double robustness does not protect against failure of both models or the causal identification assumptions.

Use saturated outcome regressions within the two \(X\) strata. The observed treated means are \(m_1(0)=9\) and \(m_1(1)=13\); the control means are \(m_0(0)=5\) and \(m_0(1)=11\). The predicted contrasts are therefore 4 at \(X=0\) and 2 at \(X=1\). The sample contains three units at \(X=0\) and five at \(X=1\).

Here the stratum-specific residual corrections cancel because each outcome mean was estimated from the same observations in that stratum. The estimate consequently equals the sample average of the two fitted stratum contrasts. That equality is a feature of this small saturated example, not a general identity between AIPW and outcome regression.

Comparing the Results and Their Uncertainty

The matching estimate differs from both weighted estimates because the single-neighbor tie rule selects particular outcomes. All methods target the ATE under their assumptions, but they need not give the same finite-sample answer. The difference is not evidence by itself that one method is correct; it is a reason to examine modeling choices, diagnostics, and uncertainty.

Uncertainty must reflect how each estimate was constructed. For scale, the AIPW unit-level estimating values in this example are \(2.5, 5.5, 4, 4.5, -0.5, 16/3, 2,\) and \(-4/3\). Their sample standard deviation is about 2.584, giving the simple plug-in standard error \(2.584/\sqrt{8}\approx0.913\). A normal-approximation interval would be \(2.750\pm1.96(0.913)\), or about \([0.960,4.540]\) outcome units.

That interval is only an illustration of an influence-value calculation, not a dependable small-sample guarantee: the propensity and outcome models were both estimated from just eight observations, with sparse strata. IPW can be especially variable when estimated propensities are near zero or one. Matching inference must also account for the dependence created when a unit is reused as a neighbor; treating all matched contrasts as independent can give misleading uncertainty. The appropriate variance method depends on the estimator and its design, and routine bootstrap resampling can be problematic for matching.

What the Comparison Can—and Cannot—Establish

The raw imbalance in \(X\) is removed by the IPW weights in this example, and exact matching pairs units only within the same observed stratum. AIPW uses the same covariate in its propensity and outcome models. These facts describe adjustment for the measured covariate; they do not show that all confounders were measured, that the models generalize, or that a causal effect has been identified. In particular, empirical overlap means both treatment states appear in the observed strata; it is not proof of population positivity.

A careful comparison reports the estimator definition, the target population, the propensity and outcome models, the matching rule—including replacement and tie-breaking—and relevant diagnostics. It should also report uncertainty with its method and limitations. If estimates differ materially, investigate which observations, weights, or matches drive the discrepancy rather than selecting the estimate that best fits expectations.

• Pitfall: averaging tied neighbors while calling the procedure one-nearest-neighbor matching. That changes the estimator. Specify a single-neighbor tie rule, or explicitly describe an averaging-over-neighbors procedure.
• Pitfall: treating balance as proof of no confounding. Balance diagnostics address measured covariates; conditional exchangeability is an assumption about potential outcomes that the data alone cannot verify.
• Pitfall: interpreting a small-sample plug-in interval as definitive. Estimated nuisance models, sparse strata, extreme weights, and reused matches can all make nominal uncertainty calculations unreliable.