Why Transport Can Fail Even With a Randomized Source
A randomized trial can identify an average treatment effect for its participants without identifying the effect for a different population. The gap matters when treatment effects vary across people and the source and target populations differ in the prevalence of characteristics that modify those effects. If such a characteristic is not measured, the source and target data may be compatible with several different target effects.
The issue is not simply that the target population has a different average outcome. A difference in baseline risk does not, by itself, prevent transport of an effect on every scale. The central problem here is that an unmeasured characteristic changes the treatment effect on the scale of interest, and its distribution in the target is unknown.
For clarity, suppose \(A\) is randomized in the source study, but outcomes are not observed in the target population. The available information may include source treatment and outcome data, plus the target distribution of measured covariates \(Z\). Let \(U\) be an effect modifier not recorded in either population. Randomization can identify source effects, but does not reveal how common each value of \(U\) is in the target. Nor does it establish that the source average effect equals the target average effect.
A Formula Shows What Information Is Missing
For a binary effect modifier \(U\), define the conditional treatment effect in population \(s\) by $$ \tau_s(u)=E[Y(1)-Y(0)\mid U=u,S=s]. $$ If the conditional effect transports from source to target given \(U\), so that \(\tau_0(u)=\tau_1(u)\), then the target effect is an average of those conditional effects using the target distribution of \(U\):
The formula requires both ingredients: source conditional effects and target modifier frequencies. When \(U\) is unmeasured, source treatment and outcome data may reveal only an average over the source distribution of \(U\), while the target data reveal neither the target distribution of \(U\) nor its conditional effects. The source average generally cannot supply the missing target average.
This expression also makes the role of assumptions explicit. Randomization in the source supports causal comparisons there. To use source conditional effects for the target, one additionally needs the relevant conditional treatment effects (or potential-outcome means) to be transportable given the variables being conditioned on. One also needs the target distribution of those variables and sufficient source support for their values. If \(U\) is missing, those requirements cannot be verified or used for standardization merely by having a large randomized trial.
Worked Example: Two Causal Worlds With the Same Observed Data
Consider a hypothetical randomized source study with a binary, unmeasured modifier \(U\). Suppose the potential outcomes are deterministic: for every individual, \(Y(0)=10\); under treatment, \(Y(1)=8\) if \(U=0\) and \(Y(1)=14\) if \(U=1\). In the source, half the individuals have each value of \(U\). Target outcomes are unavailable, and the target data contain no measurement of \(U\).
In the source, the average control outcome is 10. The average treatment outcome is $$ 0.5(8)+0.5(14)=11. $$ Thus the randomized source identifies a source average treatment effect of \(11-10=1\) outcome unit. The conditional effects, however, are different: $$ \tau_1(0)=8-10=-2,\qquad \tau_1(1)=14-10=4. $$
Now consider two possible target populations. In World A, \(P(U=1\mid S=0)=0.2\), so the target effect is $$ 0.8(-2)+0.2(4)=-1.6+0.8=-0.8 $$ outcome units. In World B, \(P(U=1\mid S=0)=0.8\), so it is $$ 0.2(-2)+0.8(4)=-0.4+3.2=2.8 $$ outcome units.
Both worlds have exactly the same source distribution of \(U\), the same source randomized outcomes, and the same recorded target data if \(U\) is not among the target measurements. Yet their target effects differ by 3.6 outcome units. The target effect is therefore not identified from those observed data, even though the source trial is randomized and its source effect is known.
What Can Restore Identification?
The most direct remedy is to measure the relevant modifier in the source and target, then use a defensible transport assumption. In the simple binary example, this means learning the source effects within levels of \(U\) and the target proportion with \(U=1\). More generally, measurements may need to capture a set of effect modifiers sufficient to make potential-outcome means transportable across source and target conditional on those variables.
Measurement alone is not enough. A standardization calculation also relies on source treatment assignment being exchangeable for the potential outcomes within the source (as ensured by valid randomization), consistency, conditional transportability of the relevant potential-outcome means, and positivity: target covariate profiles used in the calculation must be represented in the source. If these conditions fail, measuring \(U\) does not automatically identify the target effect.
Worked Example: Measuring the Modifier and Standardizing
Continue with the hypothetical outcome model above. Suppose \(U\) is now measured in both populations, the source trial is randomized, and the source conditional effects transport to the target within levels of \(U\). Suppose the target data show that \(P(U=1\mid S=0)=0.3\), and both modifier levels occur in the source.
The conditional effects from the source are \(-2\) outcome units for \(U=0\) and \(4\) outcome units for \(U=1\). Standardizing these effects to the target distribution gives $$ \operatorname{ATE}_{T} =0.7(-2)+0.3(4) =-1.4+1.2=-0.2 $$ outcome units.
Equivalently, the target mean under treatment is \(0.7(8)+0.3(14)=9.8\), and the target mean under control is \(0.7(10)+0.3(10)=10\). Their difference is \(9.8-10=-0.2\) outcome units. The calculation is justified here by the stated conditional transportability assumption, the randomized source, measurement of the target modifier distribution, and overlap of both \(U\) levels in the source. It is not justified by randomization alone.
Other information or assumptions can sometimes substitute for direct measurement, but they must be stated rather than silently presumed. For example, external data could establish the target distribution of \(U\). Subject-matter knowledge might justify that the treatment effect is constant over \(U\) on the chosen effect scale; in that case the distribution of \(U\) would not change the average effect. Such a homogeneity assumption is substantive and cannot be inferred just because an interaction was not detected in a limited source sample.
Partial Identification and Sensitivity Analysis
When the target distribution of \(U\) is not known exactly, it may still be possible to report a range of effects under explicit restrictions. This is partial identification: the data and assumptions restrict the target effect to a set of values without selecting one unique value. The resulting range reflects the uncertainty left by missing information; it is not a confidence interval unless sampling uncertainty is also incorporated.
Worked Example: Bounding the Target Effect
Suppose source evidence and a conditional transportability assumption support the same conditional effects as before: \(-2\) for \(U=0\) and \(4\) for \(U=1\). The target prevalence of \(U=1\) is unavailable, but external information supports the bound \(0.25\leq P(U=1\mid S=0)\leq0.60\). Write this prevalence as \(p\).
Under these assumptions, the target effect is $$ \operatorname{ATE}_{T}=(1-p)(-2)+p(4)=-2+6p. $$ Because this expression increases with \(p\), its minimum over the stated range occurs at \(p=0.25\), and its maximum occurs at \(p=0.60\): $$ -2+6(0.25)=-0.5,\qquad -2+6(0.60)=1.6. $$ Thus the target effect is bounded between \(-0.5\) and \(1.6\) outcome units, under the stated assumptions and prevalence restriction. The interval includes both negative and positive effects, so those inputs do not determine even the direction of the target effect.
A Short Identification Argument
The reason a source average does not generally solve the problem can be seen directly. If the source has prevalence \(q=P(U=1\mid S=1)\), its average effect is $$ \operatorname{ATE}_{S}=(1-q)\tau_1(0)+q\tau_1(1). $$ This is one weighted average. The target effect, under conditional transportability, is $$ \operatorname{ATE}_{T}=(1-p)\tau_1(0)+p\tau_1(1), \qquad p=P(U=1\mid S=0). $$ Subtracting the source average from the target average gives $$ \operatorname{ATE}_{T}-\operatorname{ATE}_{S} =(p-q)\{\tau_1(1)-\tau_1(0)\}. $$
This identity isolates the two ingredients that create a transport difference: the modifier distribution changes (\(p\ne q\)) and the conditional effects differ (\(\tau_1(1)\ne\tau_1(0)\)). If \(p\) is unknown and the effects vary, the source average alone cannot determine the target average. If either the distributions are equal or the effect is constant across \(U\), this particular source-to-target difference is zero. That observation is scale-specific: homogeneity on a risk-difference scale does not imply homogeneity on a risk-ratio or odds-ratio scale.
What the Data Do—and Do Not—Establish
An unmeasured effect modifier is not automatically fatal to every target estimand. If its distribution is the same in source and target, or if the treatment effect is homogeneous over it on the selected scale, it need not alter the transported average effect. But those facts must be supported by measurement, external knowledge, or a stated assumption. A source trial’s failure to show effect heterogeneity is not proof of homogeneity, especially when the modifier itself was not recorded.
Target outcome data could provide additional information, but observed outcome differences in a target population do not by themselves identify its causal effect. Their interpretation still depends on how treatment was assigned and on the assumptions needed to address confounding. Likewise, a model that predicts outcomes accurately in the source does not reveal an unobserved target modifier distribution or guarantee that the treatment response transports.
Common Mistake
Wrong reasoning: “Treatment was randomized, so the trial’s average effect is the causal effect for the target population too.”
Correction: Randomization supports causal identification in the source study. It does not make source and target populations interchangeable. If treatment effects vary with a characteristic whose target distribution is unknown, different target effects can be compatible with the same randomized source data. The target effect requires additional measurements, a justified homogeneity or transportability assumption, or an explicitly reported range.
Check Your Understanding
Use the definitions and examples above to answer the following.
- In the two-world example, why does knowing the randomized source average effect of 1 outcome unit not determine the target effect?
- Suppose \(\tau_1(0)=\tau_1(1)\). Under conditional transportability, why does the target prevalence of \(U\) no longer change the target average effect?
- In the standardization example, identify the information about \(U\) needed from the source and from the target.
- For the sensitivity example, what target effect results if \(P(U=1\mid S=0)=0.4\)?
- Why is the range from the sensitivity example not automatically a 95% confidence interval?