Why a Matched Effect May Not Be a Full-Population Effect
Matching can improve comparability between treated and untreated units, but the resulting estimate concerns the units represented in the matched analysis. Those units may differ from the full population in age, disease severity, or other effect-modifying characteristics. If the treatment effect varies with those characteristics, the matched average and the full-population average can differ even when the matched estimate is unbiased for its own target.
Transporting an effect means using information about the matched units to estimate a causal effect for a specified broader population. The key is to distinguish two steps: identifying treatment effects within the matched data, and averaging those effects over the covariate distribution of the full target population. The second step requires assumptions; it does not follow merely from having matched units.
The notation \(E_T\) emphasizes that the target is the full population, not just the matched units. If \(X\) takes discrete values, the target effect is the weighted average of the conditional target effects, with weights equal to the target population’s covariate proportions. For continuous \(X\), the corresponding average is an integral over the target covariate distribution.
Conditions for Transport
A transported effect requires both a credible matched causal estimate and a basis for applying its conditional effects beyond the matched units. These are distinct requirements. Matching may address treatment-group comparability among included units, while transportability concerns whether effects among included units represent effects among the rest of the target population at the same covariate values.
- Consistency and a well-defined treatment contrast: each unit’s observed outcome equals its potential outcome under the treatment received, and the treatment and control strategies are sufficiently specified.
- No interference: one unit’s potential outcomes do not depend on other units’ treatment assignments, unless the estimand and model explicitly account for interference.
- Valid causal identification within matched units: conditional on the covariates used for adjustment, treatment assignment among matched units is exchangeable with respect to the potential outcomes, and both treatment states are represented at the relevant covariate profiles. In observational studies, this includes the substantive assumption that no remaining unmeasured confounding invalidates the matched comparison.
- Conditional effect transportability: at each target covariate profile \(x\), the mean treatment effect among matched units equals the mean treatment effect in the full target population: \(\tau_M(x)=\tau_T(x)\).
- Covariate support: every covariate profile that has positive probability in the target population must be represented among matched units, or the effect there must be supplied by additional assumptions or data. For continuous covariates, target profiles must lie within the support of the matched covariate distribution.
- Target covariate distribution is known or estimable: the distribution of \(X\) in the full target population must be available well enough to perform the required averaging.
Conditional effect transportability is a substantive assumption, not a consequence of matching. It says that at a fixed \(X=x\), inclusion in the matched analysis does not mark a group with a different mean causal effect. It does not require the matched and unmatched units to have the same distribution of \(X\). Indeed, their distributions may differ substantially; standardization accounts for that difference when the conditional effects are transportable.
A common sufficient setup is that all important effect modifiers are included in \(X\), matched conditional effects are identified at those covariate values, and the target has no unsupported profiles. If a relevant effect modifier is omitted, then matched and unmatched units can have different effects even among units appearing similar on the recorded \(X\). Matching balance alone cannot establish transportability.
Deriving the Transported Effect
The derivation begins with the target estimand itself. By averaging the conditional target effect over the target population’s covariate distribution, and then applying conditional effect transportability, we obtain the standardization formula.
The second line is the law of iterated expectation: the inner conditional expectation is defined in the full target population. The last equality uses conditional effect transportability at each target covariate profile. It is valid only where the matched data support those profiles, or where additional justified assumptions provide the missing conditional effects. This derivation does not replace a conditional effect among unmatched units with a matched effect before defining the full target average.
For discrete covariates, the formula is \(\tau_T=\sum_x P_T(X=x)\tau_M(x)\). For continuous covariates with target density \(f_T(x)\), it is \(\tau_T=\int \tau_M(x)f_T(x)\,dx\). If the target distribution is absolutely continuous with respect to the matched covariate distribution, one can also express the average as a weighted matched-population average, using the density ratio \(f_T(x)/f_M(x)\). Large or unstable ratios can make the estimate imprecise and sensitive to modeling choices.
Worked Example: Different Covariate Mix, Different Average Effect
Suppose a hypothetical matched analysis contains two severity groups. The estimated causal effect is a reduction of 2 hospital days for lower-severity patients and a reduction of 8 days for higher-severity patients. In this example, define effects as days saved, so both numbers are positive. The matched group is 50% lower severity and 50% higher severity; the full target population is 80% lower severity and 20% higher severity. Assume the conditional effects are causally identified, transportable across matched and unmatched patients within each severity group, and both groups are represented among matched patients.
The matched average effect is \(0.50(2)+0.50(8)=1+4=5.00\) days saved per patient. The target-population effect instead uses the target proportions:
Thus, the matched average is 5.00 days saved, while the transported full-population average is 3.20 days saved. The difference does not indicate that either calculation is arithmetically wrong: they average the same conditional effects over different covariate distributions. Here the matched sample places more weight on the group with the larger effect.
Transporting Requires Conditional Effects, Not Just a Matched Average
A single overall matched estimate usually does not reveal how effects vary across covariate profiles. If the target and matched populations have different covariate distributions, the matched average alone is generally insufficient to compute the target average. One exception is a constant treatment effect across the relevant profiles: then every population averages the same conditional effect. Another sufficient case is that the target and matched populations have the same covariate distribution. Neither condition should be assumed without justification.
Worked Example: Standardizing Conditional Effects to the Target
Consider hypothetical matched data with conditional effects of 1.5 days saved in a lower-risk group and 4.5 days saved in a higher-risk group. The matched sample is 40% lower risk and 60% higher risk. The target population is 70% lower risk and 30% higher risk. Assume the conditional effects are identified and transportable and that both target groups occur in the matched data.
First calculate the matched average: \(0.40(1.5)+0.60(4.5)=0.60+2.70=3.30\) days saved per patient. To transport, retain the conditional effects but replace the matched proportions with the target proportions:
The target effect is 2.40 days saved, not 3.30 days. The aggregate matched value of 3.30 days does not by itself tell us that the conditional effects are 1.5 and 4.5; that information must come from estimating effects within covariate profiles or from a justified model. Without such information, the target average cannot generally be recovered from the matched average alone.
Support Gaps and What They Prevent
The support condition is essential. If a target covariate profile is absent from the matched data, there is no matched conditional effect for that profile to average. A regression model may produce a number there, but the number is extrapolation, not information supplied by the matched comparisons. Whether that extrapolation is credible depends on additional modeling assumptions.
Worked Example: A Target Group Missing From the Matched Data
Suppose matched units include only covariate groups \(X=0\) and \(X=1\), with estimated effects of 2 and 5 units, respectively. The target population is 40% \(X=0\), 40% \(X=1\), and 20% \(X=2\). Assume the first two conditional effects are valid, but no matched units have \(X=2\), and no additional information identifies its effect.
The known contribution to the target average is \(0.40(2)+0.40(5)=0.80+2.00=2.80\) units. If the effect at \(X=2\) is denoted by \(\tau_T(2)\), then the target effect would be:
Because \(\tau_T(2)\) is unknown, the full target effect is not point-identified by these matched data. For illustration only, if a justified external bound placed that effect between 0 and 10 units, the target effect would lie between \(2.80+0.20(0)=2.80\) and \(2.80+0.20(10)=4.80\) units. Without such a bound or other assumptions, even that interval is unavailable. The missing profile cannot be fixed merely by describing the matched sample as balanced.
Diagnostics, Alternatives, and Limitations
Useful diagnostics focus on whether the proposed transport calculation is supported by the data and assumptions:
- Compare covariate distributions in the matched sample and target population, especially covariates believed to modify treatment effects. A distributional difference is not automatically a failure, but it signals that the aggregate effects may differ.
- Check whether target profiles are represented among matched units. Inspect ranges and profile counts, not only average covariate balance.
- Examine whether conditional effect estimates are based on enough matched treated and control units. Sparse profiles can make estimated effects unstable even when formal support exists.
- Assess whether measured covariates plausibly capture important differences in treatment-effect response between included and excluded units. This is a scientific judgment; a balance statistic cannot verify it.
- When using target-to-matched weighting, inspect the distribution and concentration of weights. A few very large weights can make the transported estimate sensitive to a small number of observations.
Standardization, transport weighting, and direct modeling of the target mean are different ways to implement the same basic task when their assumptions are met. Standardization estimates conditional effects and averages them using target proportions. Weighting reweights matched units so their covariate distribution represents the target. A model-based approach predicts conditional effects and averages predictions over target covariates. These approaches can differ in finite samples and model dependence, but none resolves unsupported target profiles or unmeasured effect modifiers by itself.
A different target may be more defensible when full-population transport is not supported. For example, reporting the effect for the matched population can avoid extrapolation beyond matched support, although it answers a narrower question. Restricting the target to profiles represented in the matched data may also be reasonable, provided the restricted target is clearly defined. The choice should follow the scientific question and available support rather than a desire to report a broader effect.
Common Mistake
Wrong reasoning: “The matched estimate is unbiased, so it estimates the full-population effect.” Unbiasedness for the matched target does not change the population over which the effect is averaged. When conditional effects vary and the covariate distributions differ, the two averages can differ, as the worked examples show.
Correction: First identify the target population and its covariate distribution. Then estimate conditional effects from the matched data, justify transporting them at each target profile, verify support, and average using the target distribution. If only the aggregate matched effect is available, a full-population effect requires further conditions, such as effect homogeneity, or additional conditional-effect information.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- A matched group has conditional effects of 3 and 7 units in two covariate groups. The target proportions are 25% and 75%. What is the transported effect, assuming the required conditions hold?
- Why is an overall matched effect alone generally not enough to calculate an effect for a target with a different covariate distribution?
- What does conditional effect transportability assert, and what does it not assert about covariate distributions?
- A target population includes a covariate profile absent from matched units. Can the full target effect be obtained from the matched data without additional assumptions or information? Explain.
Solutions. (1) \(0.25(3)+0.75(7)=0.75+5.25=6.00\) units. (2) The aggregate matched effect averages conditional effects using matched proportions; it does not generally reveal the separate conditional effects needed for averaging with target proportions. (3) It asserts that at each covariate profile, the matched and target mean causal effects are equal. It does not assert that the matched and target covariate distributions are equal. (4) No. The conditional effect for that unsupported profile is not identified by the matched comparisons; additional data, a justified extrapolation assumption, or a narrower target is needed.