Matching Without Replacement and Treated-Unit Targets

Why the Matching Rule Defines the Target

In one-to-one matching without replacement, each retained treated unit is paired with a different control unit. When eligible controls are scarce, some treated units may not receive a match and are left out of the estimate. Even when all treated units can be matched, the allocation of controls can affect match quality. The procedure therefore does more than choose comparison units: it determines which treated units contribute to the estimate and, consequently, which treated-unit average effect the estimate is intended to describe.

The distinction matters when treatment effects vary across units. An estimate based on treated units that found acceptable matches need not target the ATT for every treated unit in the original sample. Its direct target is the average effect for the treated units actually retained by the procedure.

Definition: Let $A_i=1$ indicate that unit $i$ is treated, let $X_i$ denote its measured covariates, and let $Y_i(1)$ and $Y_i(0)$ be its potential outcomes under treatment and control. Let $R$ be the set of treated units retained after matching, and let $m$ be the number of units in $R$. The retained-treated effect is $$ \tau_R=\frac{1}{m}\sum_{i\in R}\left\{Y_i(1)-Y_i(0)\right\}. $$ It is the average treatment effect for the particular treated units retained by the matching procedure. It is not automatically the ATT for all treated units.

In one-to-one matching without replacement, each $i\in R$ is assigned one control unit $j(i)$, and no control is assigned to more than one treated unit. A common estimate is the average of the matched differences:

\hat{\tau}_R=\frac{1}{m}\sum_{i\in R}\left\{Y_i^{\mathrm{obs}}-Y_{j(i)}^{\mathrm{obs}}\right\}.

Here $Y_i^{\mathrm{obs}}=Y_i(1)$ for a treated unit, and $Y_{j(i)}^{\mathrm{obs}}=Y_{j(i)}(0)$ for its control. This estimator gives each retained treated unit one contribution, so it averages over retained treated units rather than over all treated units in the starting sample.

Assumptions and the Link to the Target

The potential-outcomes interpretation requires consistency (the observed outcome equals the potential outcome under the treatment actually received), no interference between units, and a well-defined treatment. To identify causal effects from observational data, a standard condition is conditional exchangeability: $Y(0),Y(1)$ are independent of $A$ given $X$. There must also be overlap for the covariate profiles represented by the retained treated units: comparable controls must be available in those regions of $X$. Matching itself does not guarantee these conditions.

In practice, matching aims to make each assigned control similar to its treated partner on relevant covariates. If matches are exact on $X$, exchangeability implies that, conditional on $X$, control outcomes provide information about the untreated potential outcomes of treated units with the same covariates. With approximate matches, the quality of this proxy depends on the remaining covariate differences and on how untreated outcomes vary with those covariates. Thus, even when the target is clearly $\tau_R$, the matched estimate may not equal it.

Why Matched Differences Target Retained Treated Units

The algebra shows both what the estimator is aiming at and where discrepancy can arise. For a retained treated unit, the observed pair difference can be written as its individual treatment effect plus the difference between its untreated potential outcome and the matched control’s untreated outcome:

Y_i^{\mathrm{obs}}-Y_{j(i)}^{\mathrm{obs}} = \left\{Y_i(1)-Y_i(0)\right\} + \left\{Y_i(0)-Y_{j(i)}(0)\right\}.

Averaging over the $m$ retained treated units gives:

\hat{\tau}_R = \tau_R + \frac{1}{m}\sum_{i\in R}\left\{Y_i(0)-Y_{j(i)}(0)\right\}.

The second term is the average mismatch in untreated potential outcomes between treated units and their assigned controls. It would be zero if every pair had identical untreated potential outcomes. In real data those outcomes are not jointly observed, so the term cannot generally be calculated directly. Covariate balance and close matches help make it plausible that the term is small; they do not prove that it is zero.

Worked Example: A Caliper Leaves One Treated Unit Out

Suppose four treated units have covariate values $X=0,2,8,10$, and three controls have values $X=0.1,2.1,8.1$. Use one-to-one matching without replacement and a caliper of 0.5 on the absolute difference in $X$. A treated unit is retained only if an eligible control is within that distance.

The first three treated units can be paired with the controls at 0.1, 2.1, and 8.1, respectively. Each distance is 0.1. The treated unit at $X=10$ is 1.9 units from the nearest control, so it is discarded. Thus $R$ contains three treated units, not all four.

For illustration, suppose the retained treated units’ treatment effects are 2, 4, and 6 outcome units, while the discarded treated unit’s effect is 20. The retained-treated target is

\tau_R=\frac{2+4+6}{3}=\frac{12}{3}=4 \quad\text{outcome units}.

By contrast, the ATT among all four treated units is

\tau_{\mathrm{ATT}}=\frac{2+4+6+20}{4}=\frac{32}{4}=8 \quad\text{outcome units}.

The two averages differ because the excluded treated unit has a much larger effect. In an actual observational study, individual effects are not known; this hypothetical calculation makes the target shift visible. The matched estimate uses outcomes for the three retained treated units and their three controls, so its direct treated-unit target is the average effect for those three—not the full-sample ATT.

Allocation Can Change Which Treated Units Define the Target

When there are fewer controls than treated units, the rule for allocating controls may determine which treated units are retained. This can matter even if each potential match satisfies the same caliper. In particular, a sequential matching rule can use a control for an early match that would have served another treated unit better. The resulting retained set, and therefore its target, can depend on matching order.

Worked Example: Matching Order Changes the Retained-Treated Target

Suppose two treated units have $X=0$ and $X=0.8$, and one control has $X=0.3$. A caliper of 0.5 makes the control eligible for either treated unit: the distances are $|0-0.3|=0.3$ and $|0.8-0.3|=0.5$. Matching is one-to-one without replacement, so only one treated unit can be matched.

Assume a sequential rule processes the $X=0.8$ treated unit first. It receives the control, and the $X=0$ treated unit is then discarded. If the two treatment effects are 10 and 2 outcome units at $X=0.8$ and $X=0$, respectively, the retained-treated target is 10 outcome units.

If the rule processes the $X=0$ unit first, that unit receives the control and the other is discarded; the target is then 2 outcome units. The available units and caliper have not changed, but the retained set has. A matching algorithm should therefore be specified and reported, and conclusions should not treat its retained set as an incidental detail.

Match Quality, Diagnostics, and Alternatives

Retaining a treated unit does not make its match adequate. After matching, assess covariate balance in the matched sample, inspect the distribution of pairwise covariate distances, and identify treated units near the caliper boundary. A single average balance summary can conceal poor matches for particular treated units. Also check whether the retained group differs meaningfully from the discarded treated group; this helps describe who the estimate represents.

The next example isolates the effect of imperfect matches. It shows why a matched estimate can differ from the retained-treated target even when the retained set is fixed.

Worked Example: Residual Outcome Differences Shift the Estimate

Suppose two retained treated units have untreated potential outcomes 10 and 20 outcome units and treatment effects 3 and 5 outcome units. Their observed treated outcomes are therefore $10+3=13$ and $20+5=25$. The matched controls have observed outcomes 12 and 17 outcome units.

The retained-treated target is

\tau_R=\frac{3+5}{2}=4 \quad\text{outcome units}.

The matched estimate is

\hat{\tau}_R=\frac{(13-12)+(25-17)}{2} =\frac{1+8}{2}=4.5 \quad\text{outcome units}.

The average untreated-outcome mismatch is

\frac{(10-12)+(20-17)}{2} =\frac{-2+3}{2}=0.5 \quad\text{outcome units}.

Accordingly, the decomposition gives $\hat{\tau}_R=\tau_R+0.5=4.5$ outcome units. These potential outcomes are hypothetical and would not all be observed together in a real study. The example demonstrates the source of discrepancy: matching defines a retained-treated target, while imperfect comparability can still shift the estimate away from that target.

The principal target comparison is with the ATT for all treated units. The retained-treated effect equals that ATT if all treated units are retained, or if the excluded and retained treated units happen to have the same average effect. Neither condition follows merely from using a matching algorithm. Regression adjustment can instead use a model to adjust for covariates, but it relies on its own modeling choices and does not automatically target the retained treated group. Matching with replacement relaxes the rule that each control can be used only once; it changes how controls contribute and is a distinct design choice.

No-replacement matching has practical limitations. A limited control pool can leave treated units unmatched; competition for controls can create sensitivity to matching order; and close covariate matches do not eliminate unmeasured confounding. Moreover, the retained-treated effect is a sample-defined target. Extending it to a broader population requires additional justification about how the retained units relate to that population.

Common mistake: “The matched estimate is an ATT, so it describes the average effect for every treated unit.” This is only justified if the matched procedure retains all treated units, or if the effect target for retained units is known to coincide with the full ATT. Otherwise, state that the estimate targets the treated units retained by the matching rule, and report which treated units were excluded and why.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions below.

There are 12 treated units, but a no-replacement matching procedure retains 9. What treated-unit average is directly targeted by the matched estimator?
In one-to-one matching without replacement, can the same control contribute to two matched pairs? What does the rule require instead?
Suppose the retained treated units have effects 1, 4, and 7 outcome units, while two discarded treated units have effects 10 and 12 outcome units. Calculate the retained-treated target and the ATT for all five treated units.
Why does good covariate balance not, by itself, prove that the matched estimate equals the retained-treated target?

Solutions. (1) It directly targets the average treatment effect among the 9 retained treated units; it does not automatically target the ATT for all 12. (2) No. Each control can be assigned to at most one treated unit. (3) The retained target is $(1+4+7)/3=4$ outcome units. The full ATT is $(1+4+7+10+12)/5=34/5=6.8$ outcome units. (4) Balance is evidence about similarity on measured covariates, not proof that matched controls have the same untreated potential outcomes as treated units. Unmeasured differences or residual covariate differences can leave the mismatch term nonzero.