Why Naming the Target Requires More Than Naming the Matching Method
A matched analysis may report one treatment-effect estimate, but that number does not identify its own target. Matching can exclude units, prioritize one treatment group, and assign different weights to matched units or sets. Those choices determine which population the estimate represents. Thus, “we used propensity score matching” is not enough to justify calling the result an ATE, ATT, or ATO.
The practical task is to reconstruct the target from the design and analysis: identify the units that contribute, determine how each unit or set contributes, and state the population whose average effect those contributions represent. This matters especially when treatment effects vary across units. Different target populations can then have different average effects even if they are analyzed using the same observed data.
Common labels refer to particular target distributions. The ATE averages effects over the full population of interest. The ATT averages effects among units who received treatment in that population, and the ATC averages effects among units who received control. The ATO averages effects in an overlap population, with greater emphasis on covariate profiles represented in both treatment groups. A matched analysis may target one of these populations, a restricted version of one, or another weighted population. The label must follow from the actual retained units and weights.
A useful distinction is between an estimate and its estimand. The estimate is calculated from observed outcomes, such as a weighted difference between matched treated and control outcomes. The estimand is the population quantity that this calculation is intended to estimate. Matching alone does not ensure that the estimate identifies a causal estimand: causal interpretation also requires appropriate assumptions. In particular, consistency and no interference give observed outcomes their potential-outcome interpretation, while conditional exchangeability and adequate treatment overlap are needed for covariate adjustment or matching to identify effects. A good match may improve covariate comparability, but does not establish these assumptions by itself.
A Practical Rule for Naming a Matched Estimand
Start with the units that remain after matching, not the name of the algorithm. Then ask which group anchors the contrast. In many treated-anchored matching designs, each retained treated unit is compared with one or more matched controls. If each treated unit contributes equally, the natural empirical target is the average effect among those retained treated units. That is not automatically the ATT among all treated units: treated units discarded by the matching procedure are outside this empirical target.
Next, inspect the aggregation weights. Equal weighting of matched sets gives each set equal influence, even if sets contain different numbers of units. Weighting sets by their size gives larger sets more influence. Weighting individual treated units equally instead gives each retained treated unit equal influence. These procedures generally define different target distributions when set sizes or effects differ.
The control group’s role also needs to be stated accurately. In a treated-anchored design, matched controls commonly provide comparisons for the retained treated units. They do not thereby become the target population. If a design instead weights treated and control units to represent a specified common population, the target must be described using both groups’ resulting weights. When no standard population label is warranted, describe the target directly—for example, “the average effect among retained treated units, weighted equally”—rather than forcing an ATE or ATT label.
Record which treated units and control units were retained, and which units were discarded. State whether the target is tied to treated units, control units, matched sets, or a weighted population represented by both groups.
Specify the weight for each unit or matched set. Check whether the calculation gives equal weight to sets, to individuals, or to another defined group.
Use ATE, ATT, ATC, or ATO only if the units and weights correspond to that population. Otherwise, name the restricted or weighted target explicitly.
State that the contrast estimates a causal effect only under the assumptions required by the design and analysis. Do not treat covariate balance as proof of those assumptions.
Worked Examples: Recovering the Target From the Calculation
Worked Example: One-to-One Matching Anchored on Treated Units
Suppose a hypothetical study retains three treated units and matches one control to each. The observed treated-minus-control outcome differences are 4, 2, and 7 points. The analysis averages the three differences equally. What is the estimate, and what target does this design support naming?
The estimate is
Each retained treated unit contributes one difference with equal weight. Under the causal assumptions needed for the matched comparisons to represent the treated units’ missing counterfactual outcomes, the target is the average effect among these three retained treated units. For this finite matched sample, it can be called the average treatment effect among retained treated units. It should not be called the full-population ATE, since the calculation gives no weight to units outside the retained matched sample. Nor should it be called the ATT among all treated units unless the retained treated units represent that full treated population under a justified design or analysis.
The arithmetic alone establishes a matched outcome contrast of 4.333 points. It does not establish exchangeability or prove that the matched controls supply valid counterfactual outcomes. The causal label therefore depends both on the target implied by the weights and on the assumptions supporting identification.
Worked Example: Equal-Set Versus Size-Weighted Aggregation
Consider two hypothetical matched sets. The first represents two treated units and has an estimated set effect of 3 points. The second represents four treated units and has an estimated set effect of 6 points. Compare equal weighting of sets with weighting by the number of represented treated units.
Giving each set equal weight produces
This calculation targets an average over the two sets, with each set contributing one-half. It does not give every represented treated unit equal weight.
If the set effects are instead weighted by the numbers of represented treated units, the estimate is
This aggregation gives the second set two-thirds of the total weight and the first one-third. If the set sizes represent the numbers of retained treated units and the set estimates summarize effects for those units, the size-weighted calculation targets the average among those represented treated units. The equal-set calculation targets the average across sets instead. Neither is inherently preferable; the relevant question is which population the analysis intends to describe. The calculations have causal interpretations only if the matched comparisons identify the corresponding set effects under appropriate assumptions.
Worked Example: Why a Retained-Treated Effect Is Not Automatically the ATT
Suppose a hypothetical matching procedure begins with four treated units but retains only two. Their matched outcome differences are 5 and 3 points, and the analysis averages them equally. An analyst describes the result as “the ATT.” Is that label justified?
The calculated contrast is
The weights show that the empirical target is the two retained treated units. The estimate can therefore be described as a matched estimate of the average effect among retained treated units, subject to the assumptions that justify the matched comparisons. It is not automatically the ATT among all four treated units, because the two excluded treated units receive zero weight.
For example, suppose the effects for the excluded treated units were 1 and 9 points. The average effect across all four treated units would then be
not 4.000 points. In an actual observational analysis, those excluded units’ effects are not known from their observed outcomes alone. The example demonstrates why the retained-sample estimate cannot be relabeled as the full ATT without further justification, such as a valid approach that transports the retained-sample effect to the excluded treated units.
Deriving the Target From Unit Weights
A compact way to make the target explicit is to write the weighted average of unit effects. Let \(S\) be the retained set of units whose effects define the target, and let \(w_i\) be their nonnegative aggregation weights, not all zero. The finite-sample target is
Define normalized weights \(\tilde{w}_i=w_i/\sum_{j\in S}w_j\). Then
This identity shows why both retention and aggregation matter. A unit outside \(S\) has no contribution. A unit with a larger normalized weight has more influence on the target. If the analysis weights retained treated units equally, \(w_i=1\) for those units, the target is their average effect. If it weights matched sets equally, the implied unit weights depend on how each set’s effect is formed; the target is not generally the same as an individual-level average.
In a matched analysis, the \(\tau_i\) values are not observed, so the calculation uses observed outcomes and matched comparisons to estimate this target. For instance, a treated-anchored estimator may compare each treated unit’s outcome with a weighted mean of its matched controls. To interpret that estimator as estimating \(\tau_{S,w}\), the control comparisons must identify the relevant treated units’ counterfactual outcomes, at least in the weighted aggregate. The formula clarifies the target; it does not, by itself, establish identification.
Diagnostics, Alternatives, and Limits
A naming check should be part of the analysis, not a label added after the estimate is computed. Inspect the retained-unit list, the treatment status of each unit, the number of matched sets, any control reuse, and the final analysis weights. Then verify that the written estimand describes those features. If a control is reused, for example, the analyst should determine whether the calculation gives that control repeated influence and how that affects the comparison distribution; the target cannot be inferred from the phrase “one-to-one matching” alone.
Warning signs include reporting an ATE after discarding units without a stated transport argument; calling an estimate ATT when it is anchored only on a subset of treated units; and describing an equal-set average as an individual-level average when set sizes differ. Another warning sign is naming an ATO solely because matching improved overlap. An ATO is defined by an overlap target and corresponding weighting; it is not a generic name for any analysis conducted where overlap is adequate.
Alternatives are available when the immediate matched target is not the scientific target. One can report the retained-sample effect honestly, choose weights that standardize to a prespecified target population, or use an analysis designed to estimate a broader population effect. These approaches answer different questions and rely on different assumptions. A broader target may require information or modeling beyond what the matched subset supplies; simply changing the label does not transport an effect.
The target is also scale-specific. The examples use an additive difference in outcome points. An average risk difference, risk ratio, or other effect summary can vary across target populations in its own way. Name both the population and the effect scale. Finally, distinguish the finite matched-sample target from a superpopulation estimand: the former averages effects over the realized retained units, while the latter concerns a population distribution that those units are intended to represent.
Check Your Understanding
Answer each question by identifying the units, weights, and target. Solutions follow the questions.
- A matched analysis retains 20 treated units, gives each one equal weight, and matches controls to estimate their counterfactual outcomes. What target is directly represented by the treated-unit weights?
- Why does discarding treated units prevent an automatic claim that the estimate is the ATT for all treated units?
- Two matched sets have effects of 2 and 8 points and sizes of 1 and 3. Calculate the equal-set and size-weighted averages, and state how their targets differ.
- Does good covariate balance alone establish that a matched contrast is a causal effect? Explain briefly.
Solutions. (1) The average effect among the 20 retained treated units, if the matched comparisons identify their counterfactual outcomes; it is not automatically the full treated-population ATT. (2) Excluded treated units receive no weight, so the calculation does not average effects over all treated units unless a further design or transport argument justifies that target. (3) Equal-set average: \((2+8)/2=5\) points. Size-weighted average: \(\{1(2)+3(8)\}/(1+3)=26/4=6.5\) points. The first weights sets equally; the second weights represented units by set size. (4) No. Balance describes observed covariates; causal interpretation additionally requires assumptions such as consistency, no interference, appropriate exchangeability, and adequate overlap.