Why the Matched-Sample Target Needs a Name
Matching can make treated and control units more comparable on measured covariates, but it may also leave some units out of the analysis. The causal effect for the units that remain is not automatically the effect for everyone in the original study population. To interpret a matched analysis, first state whose effect it targets.
Let \(D_i\in\{0,1\}\) indicate whether unit \(i\) receives treatment, and let \(Y_i(1)\) and \(Y_i(0)\) denote that unit’s potential outcomes under treatment and control. The individual treatment effect is \(\tau_i=Y_i(1)-Y_i(0)\). Let \(S_i=1\) indicate that unit \(i\) is retained in the matched analysis. This tutorial focuses on the equal-unit-weight target among retained units; a design that gives units different target weights defines a different target.
The matched-sample target is defined by retention, not by the observed outcomes or by the treatment effect estimates eventually calculated. In practice, the exact target depends on the population and retention rule the analysis regards as relevant. If retention is determined using observed covariates and treatment assignment, the analyst should describe the retained population and the design; the target should not be treated as though it were automatically the original population.
How the Target Can Differ From the Original ATE
The original-population average treatment effect is \(E[Y(1)-Y(0)]\). The matched-sample ATE is instead conditional on \(S=1\). These quantities coincide only under conditions that make the retained units’ average effect equal to the original population’s average effect. For example, they coincide if treatment effects are constant across units. With heterogeneous effects, changing the mix of units can change the average even when every unit-level effect stays fixed.
A useful way to see this is to let \(X\) be a measured baseline covariate and \(\tau(x)=E[Y(1)-Y(0)\mid X=x]\) the conditional average effect. Then the target averages \(\tau(x)\) over the covariate distribution in the population it names. If retention changes that distribution, the average may shift.
Here \(f(x\mid S=1)\) is the covariate distribution among retained units. The equation does not say that matching necessarily changes the effect; it shows why it can. If effects vary with \(X\), changing the distribution used for averaging can change the estimand.
Worked Example: The Effect in the Retained Units
Consider a hypothetical finite study population of six units. Suppose the following potential outcomes are known for teaching purposes. In a real study, only one potential outcome per unit would be observed.
| Unit | Retained? | \(Y_i(0)\) | \(Y_i(1)\) | \(\tau_i\) |
|---|---|---|---|---|
| 1 | Yes | 10 | 13 | 3 |
| 2 | Yes | 12 | 16 | 4 |
| 3 | Yes | 8 | 9 | 1 |
| 4 | Yes | 15 | 17 | 2 |
| 5 | No | 11 | 18 | 7 |
| 6 | No | 9 | 15 | 6 |
There are four retained units, with effects \(3,4,1,\) and \(2\) outcome units. Thus $$ \tau_M=\frac{3+4+1+2}{4}=\frac{10}{4}=2.5 $$ The original six-unit population ATE is $$ \tau_{\mathrm{original}}=\frac{3+4+1+2+7+6}{6}=\frac{23}{6}\approx3.833. $$ The difference is not a calculation error: the two excluded units have larger effects, so the average among retained units is lower. The retained-sample target is 2.5 outcome units, while the original-population target is about 3.833 outcome units.
When a Matched Comparison Identifies This Effect
The target definition alone does not make an observed treated-versus-control contrast causal. Under the potential-outcomes framework, a standard identification argument requires assumptions for the retained population. Write \(X\) for measured pretreatment covariates. The key conditions are:
- Consistency: the observed outcome is \(Y_i=Y_i(D_i)\), so a treated unit’s observed outcome is \(Y_i(1)\) and a control unit’s is \(Y_i(0)\). Treatment versions and interference must be handled so these potential outcomes are well-defined.
- Conditional exchangeability among retained units: \((Y(1),Y(0))\perp D\mid X,S=1\). After conditioning on the measured covariates, treatment assignment among retained units is unrelated to their potential outcomes.
- Positivity in the retained target: for covariate values represented in the target, \(0<P(D=1\mid X=x,S=1)<1\). Both treatment conditions must be possible at those values of \(X\).
These conditions must be credible in context. In particular, matching does not establish exchangeability: important unmeasured confounding can remain. Also, if the selection or matching procedure depends on outcomes, the usual interpretation needs additional justification. The target and assumptions should refer to the same retained population.
Under these conditions, the conditional outcome means identify the conditional potential-outcome means. For treatment, consistency gives \(E[Y\mid D=1,X=x,S=1]=E[Y(1)\mid D=1,X=x,S=1]\). Exchangeability allows the conditioning on \(D=1\) to be removed, and the same reasoning applies to control. Positivity ensures both conditional means can be observed at the relevant \(x\). Therefore,
Averaging both sides over the covariate distribution of the retained target yields
This derivation identifies the matched-sample ATE as an average of covariate-specific treated-control contrasts, weighted by the retained population’s covariate distribution. A raw difference in the overall retained treated and control means equals this target only under further conditions, such as adequate covariate balance for the relevant effect structure. Matching is a design strategy intended to improve comparability; it is not itself a proof that the raw contrast has the desired weighting or is unbiased.
Worked Example: Standardizing Within Retained Covariate Groups
In hypothetical retained data, suppose \(X\) has two categories, A and B. Each category has two treated and two control units. The observed outcome means are:
| Retained covariate group | Treated mean | Control mean | Difference |
|---|---|---|---|
| A | 12 | 10 | 2 |
| B | 20 | 15 | 5 |
Suppose the target gives equal weight to the two retained groups. The standardized contrast is $$ \hat{\tau}_M=\frac{1}{2}(12-10)+\frac{1}{2}(20-15) =1+2.5=3.5 $$ outcome units. Because each group has the same number of treated and control units and the retained group proportions are equal, the overall treated mean is \((12+20)/2=16\) and the overall control mean is \((10+15)/2=12.5\); their difference is also \(16-12.5=3.5\) outcome units.
This calculation is an estimate, not a demonstration that the assumptions hold. Its causal interpretation requires consistency, conditional exchangeability within these groups, and treatment positivity in both groups. The table supports the arithmetic and the balance description, but it cannot rule out unmeasured confounding or prove exchangeability.
Effect Heterogeneity Makes Retention Matter
The next example isolates the role of composition. Suppose \(X\) has two groups with different average treatment effects, and matching retains a different proportion of each group than the original population. The target changes because the averaging weights change, not because matching changes anyone’s potential outcomes.
Worked Example: Reweighting the Average by Retention
Imagine a hypothetical population split equally between groups A and B. Their conditional average effects are 2 and 8 outcome units, respectively. The original ATE is $$ \tau_{\mathrm{original}}=(0.50)(2)+(0.50)(8)=1+4=5 $$ outcome units. Now suppose the retained target is 80% group A and 20% group B. Its matched-sample ATE is $$ \tau_M=(0.80)(2)+(0.20)(8)=1.6+1.6=3.2 $$ outcome units. The target falls by \(5-3.2=1.8\) outcome units. This is an estimand shift caused by the retained target’s composition; it does not by itself indicate that the estimate is biased. Whether 3.2 is the scientifically relevant target depends on the question being asked.
Diagnostics, Alternatives, and Common Pitfalls
A careful analysis should make the retained target visible. Report how many units remain, describe their baseline covariate distribution, and compare it with the original population when the original ATE is also of interest. Examine covariate balance in the retained treated and control groups, and assess whether there are retained covariate regions with little or no representation from one treatment condition. These checks can reveal poor comparability or practical violations of positivity; they cannot verify that all confounders were measured.
Compare the matched-sample target with the main alternatives by asking which units define the average. The ATE averages effects over the original target population; the ATT averages effects for treated units; the ATC averages effects for controls. The matched-sample ATE instead averages across the units retained under the matching design. These targets need not coincide when effects vary or when the relevant populations differ. A matched analysis should not be labelled simply “the ATE” without specifying its population.
- Wrong reasoning: “The matched treated and control groups look similar, so the result estimates the original population ATE.”
- Correction: Similarity after matching concerns comparability within the retained analysis and measured covariates. It does not make retained units representative of excluded units, establish balance on unmeasured factors, or guarantee that the effect is constant across populations.
- Another pitfall: Computing an unweighted retained-group contrast when the target definition implies different unit weights. State the target weights explicitly; an estimator’s weights and the target population must correspond.
The matched-sample ATE is useful when the retained population is itself the population of substantive interest, or when a credible estimate for that population is preferable to an unsupported extrapolation to units without comparable counterparts. Its limitation is equally important: by itself, it does not answer what the effect would have been for excluded units or the original population. If the aim is to generalize beyond the retained target, additional assumptions and an analysis designed for that target are required.
Check Your Understanding
Answer each question, then compare with the brief solutions.
- A matching analysis retains 40 units. State the finite-sample matched-sample ATE using potential outcomes.
- In a two-group population, effects are 1 and 5. The original population is 25% group A and 75% group B; the retained population is 50% of each. Calculate both average effects.
- Why does covariate balance after matching not, by itself, establish that the matched estimate is causal?
- Name the three identification conditions used above, and state what positivity requires.
Solutions: (1) \(\tau_M=\frac{1}{40}\sum_{i:S_i=1}\{Y_i(1)-Y_i(0)\}\). (2) The original effect is \((0.25)(1)+(0.75)(5)=4\); the retained effect is \((0.50)(1)+(0.50)(5)=3\), each in outcome units. (3) Balance describes measured covariates, not unmeasured confounding or representativeness of retained units; causal interpretation also requires the relevant assumptions and a target consistent with the analysis. (4) Consistency, conditional exchangeability among retained units, and positivity. Positivity requires a nonzero probability of both treatment conditions for covariate values represented in the retained target.