Treated-Unit Matching and the ATT Target

Why the Set of Treated Units Matters

Matching treated units to controls is often described as a way to estimate the average treatment effect on the treated. That description needs a qualification: if some treated units cannot be matched and are omitted, the directly represented target is generally the effect among the retained treated units, not among every treated unit in the population. The distinction concerns the estimand—the population quantity the analysis aims to estimate—not merely the numerical details of a matching algorithm.

The earlier material on the average treatment effect on the treated defines the ATT for all treated units. Here the key question is how that target changes when matching retains only a subset. A change in the distribution of measured covariates among retained units can change the target when effects vary across covariates. Selection among units with the same covariates can also matter if retention is related to their effects.

Definition: Let $A_i\in\{0,1\}$ indicate whether unit $i$ receives treatment, and let $Y_i(1)$ and $Y_i(0)$ be that unit’s potential outcomes under treatment and control. The individual treatment effect is $\tau_i=Y_i(1)-Y_i(0)$. The average treatment effect on all treated units is $\mathrm{ATT}=E[\tau_i\mid A_i=1]$. Let $S_i=1$ indicate that a treated unit is retained by the matching procedure. The retained-treated target is $\mathrm{ATT}_{S}=E[\tau_i\mid A_i=1,S_i=1]$.

These are different conditional averages unless their conditioning groups yield the same mean effect. If every treated unit is retained, then $S_i=1$ for all treated units and the two targets coincide. Otherwise, the retained-treated target is the appropriate description of what an equal-weighted treated-unit comparison represents, provided its matched controls adequately represent the retained treated units’ untreated outcomes.

What a Treated-Unit Matched Contrast Represents

For each retained treated unit, matching supplies one or more control units to represent the outcome that unit would have had without treatment. In a simple one-to-one illustration, let $j(i)$ denote the control matched to treated unit $i$. The average matched contrast is

\widehat{\Delta}_{S} = \frac{1}{n_{1S}}\sum_{i:A_i=1,S_i=1} \left(Y_i-Y_{j(i)}\right),

where $n_{1S}$ is the number of retained treated units. The expression gives each retained treated unit one contribution. It does not give omitted treated units a contribution, because no matched contrast for them appears in the sum. With a valid comparison, each difference approximates that treated unit’s effect; averaging those differences therefore aims at the retained-treated target.

This interpretation requires more than the act of pairing. Write $X_i$ for measured pretreatment covariates. A standard causal justification requires consistency (the observed outcome equals $Y_i(A_i)$), no interference between units, and conditional exchangeability of treatment assignment given the relevant measured covariates. It also requires that retained treated units have eligible controls at comparable covariate values. In population notation, a key exchangeability condition is $Y_i(0)\perp A_i\mid X_i$, together with control availability at the covariate values represented among retained treated units. Matching must achieve sufficiently good covariate comparability for the matched control outcomes to stand in for the retained treated units’ untreated outcomes. These assumptions are not guaranteed by a close-looking match alone.

The selection rule matters as well. For example, retention may be determined by measured baseline covariates and the availability of suitable controls. If selection instead depends on unobserved prognostic factors or on treatment effects even after conditioning on $X_i$, the retained group can differ in ways that covariate balance does not reveal. The target $\mathrm{ATT}_{S}$ is still definable, but whether the matched comparison identifies it is a separate question.

Justification: Under consistency, a retained treated unit’s observed outcome is $Y_i(1)$. If its matched control outcome correctly represents its untreated potential outcome $Y_i(0)$, its contrast represents $Y_i(1)-Y_i(0)=\tau_i$. Averaging over the retained treated units gives $$ E[Y_i(1)-Y_i(0)\mid A_i=1,S_i=1] = E[\tau_i\mid A_i=1,S_i=1] = \mathrm{ATT}_{S}. $$ Thus the target follows from which treated units contribute and what their matched controls represent. The equality is a population identification argument; a finite matched sample need not equal this expectation exactly.

Worked Example: Covariate-Based Retention Changes the ATT

Suppose a hypothetical treated population has 10 units. Eight have covariate pattern $X=L$ and treatment effect 2 health-score points each; two have $X=H$ and treatment effect 8 points each. Assume the matching rule retains both $H$ units but no $L$ units because suitable controls are unavailable for $L$. Calculate the ATT for all treated units and the retained-treated target.

For all 10 treated units, the total of their effects is $8(2)+2(8)=16+16=32$ health-score points. Therefore,

\mathrm{ATT}=\frac{32}{10}=3.2\text{ health-score points}.

The retained treated group consists of the two $H$ units, each with effect 8 points, so

\mathrm{ATT}_{S}=\frac{8+8}{2}=8\text{ health-score points}.

The target differs by $8-3.2=4.8$ points. In this example, effects are determined by the measured covariate pattern, and retention is determined by that pattern; there is no further selection within either pattern. The difference arises because the retained treated units have a different covariate distribution from all treated units and effects vary across those patterns. The matched estimate may be useful, but it does not estimate the all-treated ATT in this population.

When Does Retention Leave the ATT Unchanged?

A familiar way to describe effect heterogeneity uses the conditional mean effect among all treated units, $m(x)=E[\tau_i\mid A_i=1,X_i=x]$. It is important not to apply this function to the retained group without a condition: in general, the retained units’ conditional mean effect is $E[\tau_i\mid A_i=1,S_i=1,X_i=x]$, which may differ from $m(x)$.

A sufficient condition for the usual covariate-distribution argument is that retention is conditionally independent of treatment effects among treated units given $X_i$:

S_i\perp \tau_i\mid X_i,A_i=1.

This condition holds, for example, if retention is determined only by $X_i$ and the matching design data, with no additional selection on effects within covariate levels. Under it, the retained conditional mean effect equals $m(x)$ wherever retained units occur. The full and retained targets can then be written as

\mathrm{ATT}=E[m(X_i)\mid A_i=1], \qquad \mathrm{ATT}_{S}=E[m(X_i)\mid A_i=1,S_i=1].

If $m(x)$ is constant across the covariate values of treated units, these two averages are equal. They can also coincide when $m(x)$ varies but the two treated groups have the same average of $m(X_i)$. Neither condition is automatic. Without conditional independence of retention and effects, even a constant $m(x)$ need not make the targets equal.

Worked Example: Constant Conditional Mean Does Not Always Protect Against Selection

Consider two hypothetical treated units with identical measured covariates $X=x_0$. Their treatment effects are 0 and 10 outcome units. For all treated units, the conditional mean effect at $x_0$ is $m(x_0)=(0+10)/2=5$ units, so the ATT is 5 units. Suppose, however, that only the unit with effect 10 is retained. Then the retained-treated target is

\mathrm{ATT}_{S}=10\text{ outcome units}, \qquad \mathrm{ATT}=5\text{ outcome units}.

Here $m(x)$ is constant over the only covariate value, but retention is not independent of effects given $X$: among units with $X=x_0$, retention selects the unit with the larger effect. Therefore $E[\tau_i\mid A_i=1,S_i=1,X_i=x_0]=10$, not $m(x_0)=5$. This selection rule is not a covariate-only matching rule; it illustrates why the conditional-independence condition must be stated rather than assumed. In practice, outcome-related selection, unmeasured eligibility, or other design features can raise related concerns.

Worked Example: Interpreting a Matched Contrast

In a hypothetical analysis, two retained treated units have observed outcomes of 12 and 16 symptom-free days. Their matched controls have 8 and 11 symptom-free days, respectively. Assume for this illustration that each control provides the correct untreated counterfactual for its treated match, and that the retained treated units are the intended target. Calculate and interpret the matched contrast.

The treated-minus-control differences are $12-8=4$ and $16-11=5$ symptom-free days. Their average is

\widehat{\Delta}_{S} = \frac{(12-8)+(16-11)}{2} = \frac{9}{2} = 4.5\text{ symptom-free days}.

Under the stated counterfactual-representation assumption, this estimates an average effect of 4.5 additional symptom-free days among the retained treated units. It does not, by itself, estimate the ATT for all treated units. For example, if a third treated unit was not retained and its effect were 1 day, the all-treated ATT across these three units would be $(4+5+1)/3=3.33$ days, not 4.5. The example’s assumed counterfactual representation is deliberately explicit: observed matched outcomes alone cannot verify each person’s unobserved untreated outcome.

Diagnostics, Alternatives, and Limits

A defensible report should make the target visible as well as describe the matching result. State how many treated units were eligible and how many were retained; compare the retained treated units with all treated units on important baseline characteristics; and assess whether the matched controls are comparable with the retained treated units. Large changes in covariate distributions after retention are a warning that the target population changed. Poor balance after matching or a lack of suitable controls for some retained covariate patterns is a warning that the matched contrast may not identify even the retained-treated effect well.

These checks have limits. Covariate balance can reveal measured differences, but it cannot establish exchangeability or rule out unmeasured effect-related selection. A small matched sample can also make an estimate unstable. Crucially, acceptable balance among retained units says nothing by itself about whether the retained group represents all treated units. That is an estimand question, not a balance statistic.

Approach	Typical target or emphasis	Important qualification
Treated-unit matching with retention	Effect among retained treated units	Units omitted by the rule do not contribute to this target.
Matching that retains all treated units	Potentially the ATT for all treated units	Requires adequate control comparisons for every treated unit and credible identification assumptions.
Outcome regression or weighting for the ATT	Can target the ATT for all treated units	Still requires appropriate modeling or weighting, exchangeability, and treatment support; it does not automatically solve selection or confounding.

The main practical alternative is to use a design or estimator that retains all treated units and represents their untreated outcomes, if the data support it. Regression or weighting can also be used to target the all-treated ATT, but changing the estimator does not remove the need for causal assumptions. When treated units lack comparable controls, broadening the target is not a technical fix: the analyst must state which population is supported by the data and which effect is being reported.

Common Mistakes

Calling every treated-unit match an ATT for all treated units. If some treated units are omitted, an equal-weighted matched contrast generally represents the retained-treated target.
Assuming covariate balance means the target did not change. Balance within the matched sample concerns comparability there; it does not show that retained treated units represent all treated units.
Using $m(x)$ for retained units without an assumption. The retained conditional mean effect can differ from the all-treated conditional mean when retention selects on effects within covariate levels. Use $E[\tau_i\mid A_i=1,S_i=1,X_i=x]$ unless conditional independence justifies substituting $m(x)$.
Treating a matched difference as automatically causal. Matching must be supported by consistency, exchangeability, adequate control availability, and sufficiently good covariate comparability.

Key takeaway: Treated-unit matching with retention generally targets $E[\tau_i\mid A_i=1,S_i=1]$. It equals the ATT for all treated units only under conditions that make the retained and full treated groups have the same mean effect; covariate balance alone does not establish that equality.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

There are 20 treated units. Matching retains 15 and gives each retained treated unit equal weight. Which treated population does the matched contrast directly target?
Suppose retention depends only on $X$, effects are conditionally independent of retention given $X$ among treated units, and $m(X)$ is constant at 3 outcome units. What are the ATT and retained-treated target, assuming the same effect mean applies to all treated covariate values?
Why is it not generally valid to claim that a constant $m(x)$ guarantees equal targets?
A matched analysis has excellent measured-covariate balance among retained treated units, but 40% of all treated units were omitted. What does the balance result establish, and what does it not establish?

Solutions. (1) It directly targets the average effect among the 15 retained treated units, $\mathrm{ATT}_{S}$, subject to valid representation of their untreated outcomes by matched controls. (2) Both targets are 3 outcome units: under the stated conditional independence, both average the same constant $m(X)$. (3) If retention selects on treatment effects within covariate levels, the retained conditional mean effect can differ from $m(x)$. For example, retaining only a unit with effect 10 from units with effects 0 and 10 gives a retained mean of 10 while the full conditional mean is 5. (4) The balance result supports measured comparability within the matched sample. It does not establish that the retained treated units represent all treated units, that unmeasured confounding is absent, or that the matched contrast estimates the all-treated ATT.