Reporting the Target Population of a Matched Estimate

Why the Target Population Must Be Reported

A matched estimate is not defined by a treatment contrast alone. It also reflects which units were eligible, which were retained, and how retained observations were weighted. If matching excludes units with no suitable counterpart, the estimate generally does not represent everyone eligible for the study. If some observations receive more weight than others, the estimate represents a weighted population rather than the unweighted matched sample. Reporting these choices makes the estimand interpretable and helps readers judge whether it answers the intended question.

The focus here is the population represented by the estimate, not a general recipe for constructing matches. Eligibility defines who could enter the analysis; matching and subsequent weighting determine which parts of that eligible population contribute, and by how much. A transparent report names each stage instead of using “the matched population” as if it had only one possible meaning.

Let \(A_i=1\) denote treatment and \(A_i=0\) denote control for unit \(i\), and let \(Y_i(1)\) and \(Y_i(0)\) denote that unit’s potential outcomes under treatment and control. Matching may retain different numbers of treated and control units. For an effect defined as a difference in mean outcomes, a typical matched contrast is the weighted mean outcome among treated units minus the weighted mean outcome among controls. The target represented by this contrast depends on the weights as well as on which units remain.

From Eligibility to a Weighted Target

Suppose matching retains treated observations indexed by \(i\) and control observations indexed by \(j\). Let \(w_i^1\) and \(w_j^0\) be their nonnegative analysis weights. When each treatment arm is normalized separately, the estimated contrast can be written as

Within each arm, dividing by the sum of weights makes the weights add to one. Those normalized weights describe the empirical distribution represented by that arm’s weighted mean. For the contrast to represent one common target population in a straightforward way, the two weighted arms should represent the same covariate distribution, or the analysis should otherwise specify how their differing distributions define the contrast. Matching is intended to make the arms comparable, but the mere fact that matching was performed does not establish that their weighted distributions are identical.

A useful way to understand the target is through conditional mean outcomes. Suppose the weighted treated and control groups represent a common distribution \(Q\) of covariates \(X\). Under consistency and conditional exchangeability, the conditional mean difference at \(X=x\) is the conditional average treatment effect at that covariate value. Averaging those conditional differences over \(Q\) gives the effect for the population represented by \(Q\). The short derivation below makes clear why weights and exclusions matter.

The second equality uses consistency, so observed outcomes equal the potential outcome under the treatment received, and conditional exchangeability, so treatment groups’ conditional means identify the corresponding potential-outcome means. The argument also requires relevant treatment and control observations across the covariate values represented by \(Q\), and the same \(Q\) for both arms. In practice, matching weights approximate a target distribution; they do not prove that these causal conditions hold. If the arms have materially different weighted covariate distributions, their raw weighted mean difference need not equal an average causal effect over a single shared target.

Weights, Match Slots, and Aggregation

Matching with replacement can use the same control unit more than once. A repeated match is often best understood as a repeated slot in a matched set, not as a new person. If each slot receives a weight, a control used in multiple slots accumulates those weights. Counting slots, unique people, and total analysis weight as though they were interchangeable leads to incorrect descriptions of how much influence observations have.

The derivation of an arm-specific weighted mean also shows the role of aggregation. If a control appears in several slots, summing its slot-level weights first and then multiplying its outcome by that total gives the same weighted sum as including the outcome once for each slot. It does not make the repeated slots independent observations. The target-population description should therefore report unique retained units as well as match slots, and should identify any reuse and its effect on weights.

Exclusions and Changing the Aggregation Target

A matched estimate can be targeted to a retained group, a weighted mixture of covariate subgroups, or a deliberately chosen standardization population. These are distinct interpretations. Exclusion changes which units can contribute; weighting changes their relative contribution; and aggregation combines subgroup effects according to a particular distribution. State which operation defines the target rather than relying on a label such as “ATT” or “matched effect” by itself.

What a Clear Report Includes

A concise report can still make the target population explicit. Give the initial eligible counts by treatment arm, the numbers retained and excluded, the reasons for exclusion, and the matching structure. State whether matching used replacement, how repeated controls were aggregated, and whether weights were normalized within arms or across the full analysis. Then describe the population represented by each arm’s weights and the population to which the contrast is intended to refer.

The following distinctions are useful when writing results:

Diagnostics should accompany the narrative. Check covariate distributions after matching, especially in regions where eligible units were excluded, and inspect the distribution of weights and reuse counts. Report whether important covariate differences remain. Large imbalance, extreme concentration of weight, or a high exclusion rate can indicate that the practical target is much narrower than the stated eligible population or that the weighted contrast is sensitive to a few units.

Matching is not the only way to define a target. Standardization can explicitly average conditional effects over a chosen population, while weighting approaches can represent a specified covariate distribution through analysis weights. These alternatives do not eliminate the need to define eligibility, support, and assumptions; they may also depend on modeling choices. The relevant comparison is not “matching versus a target population,” but whether the method’s actual weighting and available data support the target the report claims.

A key limitation is that detailed reporting cannot recover effects for units with no comparable observations or guarantee causal identification. Nor does naming a retained population establish that it is the population of scientific interest. The earlier material on naming the estimand after matching is relevant here: the name should agree with the units and weights that actually define the effect, and the report should explain any departure from the originally intended target.

Common Mistake

Wrong reasoning: “We matched the sample, so the estimate represents the full eligible population, and every control person contributes equally.” This overlooks both exclusions and weighting. Matching may discard eligible treated or control units, and a reused control may contribute multiple match slots and consequently receive more total weight than a control used once.

Correction: Trace the analysis from eligibility to retention to the final weights. Count unique units and match slots separately, calculate the total weight in each arm, and describe the target implied by the normalized weights. Claim a broader target only when the design and analysis justify extending beyond the retained weighted population.