Why Transport Errors Matter
A causal effect estimated in a study does not automatically describe the effect in another population. The populations may differ in their distributions of characteristics that modify treatment effects, and some target-population groups may be absent from the study. Even when a study effect is internally valid, reporting it as the target-population effect can therefore be wrong.
Several common mistakes arise because transport analyses involve two distinct tasks: accounting for how study participants differ from the target population, and accounting for how treatment was assigned within the study. The weights, assumptions, and estimands for these tasks are not interchangeable.
Keep the Two Weighting Problems Separate
A treatment weight addresses treatment assignment: it reweights observed treatment groups to account for differences in the likelihood of receiving treatment. For example, in an observational study, the treatment propensity is \(e(x)=P(A=1\mid X=x,S=1)\). Inverse-probability treatment weights are based on \(1/e(x)\) for treated participants and \(1/[1-e(x)]\) for controls. Their role is to address measured confounding under appropriate assumptions.
A sampling or transport weight addresses study participation: it reweights the study covariate distribution to represent the target population. In a setting where the relevant density ratio exists, a sampling weight is proportional to \(f_T(x)/f_S(x)\), where \(f_T\) and \(f_S\) are the target and study covariate densities. Its role is to adjust population composition, not treatment assignment.
These weights can sometimes be combined, but one cannot substitute for the other. Treatment weighting alone does not generally make a study representative of a target population. Sampling weighting alone does not generally remove confounding between treatment and outcome within the study.
Worked Example: Sampling Weights Do Not Remove Treatment Confounding
Consider a hypothetical observational study with a binary covariate \(X\). In the study, 80% of participants have \(X=0\) and 20% have \(X=1\). The target population is evenly divided: 50% have each value of \(X\). Treatment is more common among those with \(X=1\): \(P(A=1\mid X=0,S=1)=0.25\) and \(P(A=1\mid X=1,S=1)=0.75\). Suppose the conditional mean outcomes are 0 under control and 2 under treatment when \(X=0\), and 10 under control and 12 under treatment when \(X=1\). Thus, the causal effect is 2 outcome units in each covariate group.
The sampling weights that align the study’s covariate proportions with the target are proportional to target proportion divided by study proportion. They are \(0.50/0.80=0.625\) for \(X=0\) and \(0.50/0.20=2.5\) for \(X=1\). After applying these weights, the treated group’s share with \(X=1\) is
The weighted treated mean is therefore \((0.25)(2)+(0.75)(12)=9.5\). In the weighted control group, the share with \(X=1\) is
The weighted control mean is \((0.75)(0)+(0.25)(10)=2.5\), so the weighted difference is \(9.5-2.5=7\) outcome units. That is not the target causal effect: the treatment groups remain imbalanced in \(X\), despite the sampling weights. Standardizing the conditional effects to the target gives \((0.50)(2)+(0.50)(2)=2\) outcome units. The error was using population-composition weights as if they also adjusted treatment assignment.
Target Composition Determines the Marginal Effect
The target average effect averages conditional effects using the target population’s covariate distribution. If effects vary across groups, changing the population being averaged over changes the marginal effect, even when the conditional effects themselves remain the same. A study’s marginal effect is therefore not a universal summary that can be carried unchanged to every population.
and the target average treatment effect is the target-standardized treated mean minus the target-standardized control mean. These assumptions include that the covariates capture relevant differences in potential outcomes between study and target populations. The formula does not make those assumptions true; it states what the target effect is identified as when they hold.
A short justification follows from iterated expectation and conditional transportability:
The second equality uses conditional transportability; the third uses consistency and treatment exchangeability within the study. If either equality is unsupported, the standardization formula need not identify the target mean.
Worked Example: A Different Target Mix Changes the Marginal Effect
Suppose a hypothetical randomized study estimates a treatment effect of 10 outcome units for participants with \(X=0\), and 2 units for participants with \(X=1\). The study is 80% \(X=0\), so its average effect is
outcome units. Now suppose the target population is 20% \(X=0\) and 80% \(X=1\). Under the transport assumptions just stated, the target average effect is
outcome units. The difference is not a contradiction: the same conditional effects are averaged over different populations. Calling 8.4 the target effect would incorrectly carry over the study’s composition instead of using the target’s.
Do Not Extrapolate Silently Beyond Support
Transport requires support: target covariate patterns must be represented in the study well enough to estimate the relevant conditional outcomes. If a target group has no study representation, the data do not reveal that group’s treatment effect. A fitted outcome model may still produce a numerical prediction there, but that prediction relies on extrapolation—additional modeling assumptions rather than direct support in observed study data.
Worked Example: A Target Group Missing From the Study
Suppose a hypothetical study includes only younger adults and estimates their treatment effect as 4 points on an outcome scale. The target population is 70% younger adults and 30% older adults, but no older adults were included in the study. The target effect would be
where \(\tau_{\text{older}}\) is the unobserved treatment effect among older adults. If it were 0 points, the target effect would be \((0.70)(4)+(0.30)(0)=2.8\) points. If it were 12 points, the target effect would be \((0.70)(4)+(0.30)(12)=6.4\) points. Both values agree with the study’s observed younger-adult effect. The study data alone cannot distinguish them. Reporting a target effect requires additional information or a defensible assumption about older adults’ outcomes; a model’s prediction does not eliminate this limitation.
Match the Reported Effect to Its Population
A transport analysis should name both the estimand and the population to which it refers. “The study treatment effect” and “the effect in the target population” answer different questions. The target effect also depends on the effect scale: an average risk difference, for example, averages individual-level differences in risk, while other scales may behave differently under changes in population composition. A reported number without its scale and population is incomplete.
When comparing weighting with standardization, check what each method is intended to accomplish. Standardization predicts conditional outcome means in the study and averages them over target covariates. A weighting method instead uses weights to construct a corresponding target-representative contrast. Agreement between methods can be reassuring, but does not establish that their shared identification assumptions are valid or that unsupported covariate regions have been resolved.
Specify the target population, outcome, treatment contrast, and effect scale before interpreting an estimate.
Ask whether each weight addresses treatment assignment or study-population composition. Do not treat one as a substitute for the other.
Check whether the study represents the target on measured effect modifiers and whether the target has covariate patterns outside study support.
Describe the exchangeability, consistency, and positivity conditions required, and flag any result that depends on extrapolation.
Common Errors and Corrections
- Wrong: “We used target-population sampling weights, so treatment confounding is controlled.” Correction: Sampling weights address population composition; treatment exchangeability and appropriate treatment adjustment are separate requirements.
- Wrong: “The study effect is the target effect because both concern the same treatment.” Correction: The effects average over different populations. Use the target covariate distribution when estimating the target marginal effect.
- Wrong: “A model returned an estimate for a target subgroup absent from the study, so that subgroup is covered by the data.” Correction: The estimate depends on extrapolation and additional modeling assumptions; the observed study data do not identify that subgroup’s effect.
- Wrong: “A large target weight corrects any problem with overlap.” Correction: Extreme weights can signal weak support and unstable estimates. A weight cannot create outcome information for a covariate pattern absent from the study.
Check Your Understanding
Use the distinctions and assumptions in this tutorial to answer the following questions.
- In one sentence, distinguish the purpose of a treatment weight from the purpose of a sampling weight.
- A study and target have different proportions of an effect-modifying covariate. Why might their average treatment effects differ even if conditional effects are unchanged?
- What does a model-based target estimate rely on if some target covariate values are absent from the study?
- Which assumption justifies replacing a target conditional potential-outcome mean with the corresponding study conditional mean?
- A report presents a marginal effect but does not name the population or effect scale. What information is missing for interpretation?