Why Population Transportability Requires More Than Replication
A finding can reproduce in a validation cohort and still be a poor guide to decisions in a different population. The validation and intended target may differ in age, disease severity, treatment history, or other characteristics that affect the outcome or modify the effect. Transportability asks whether an estimate from the available source population can be used for the intended target population, and what assumptions are needed to do so.
The target of transport must be stated precisely. It might be a causal effect of an intervention, an association between a molecular measurement and an outcome, or a prediction performance measure. These quantities are not interchangeable. In this tutorial, causal effects are used to show the standardization logic; when the scientific target is associational, the same population-composition issue applies, but the result must be described as an association rather than a causal effect.
Let \(S=1\) indicate membership in the source population and \(S=0\) indicate membership in the target population. Let \(A\) denote an exposure or intervention, \(Y\) the outcome, and \(Z\) measured baseline characteristics that may differ between populations and modify the effect. For a causal question, let \(Y^a\) be the outcome that would be observed under intervention level \(a\). A common target estimand is the target average treatment effect:
This is an average difference on the outcome scale in the target population. If \(Y\) is measured in points, \(\tau_T\) is in points; if \(Y\) is a binary outcome, a risk difference is in percentage points or proportions. The effect scale matters: effects that differ on the risk-difference scale need not differ in the same way on the odds-ratio scale.
Standardization to the Target Population
A direct way to assess transportability is to estimate effects within relevant subgroups of \(Z\), then average those effects using the target population’s distribution of \(Z\). For discrete \(Z\), the standardized effect is a weighted sum. For continuous \(Z\), the corresponding expression is an integral over the target distribution.
This expression uses source-population conditional effects but target-population subgroup proportions. It is justified only when the source data identify the conditional causal effects and those effects are applicable to target individuals with the same \(Z\). In particular, the formula does not make an unsupported source estimate transportable merely by changing weights.
- Consistency: the observed outcome for an individual receiving \(A=a\) equals that individual’s potential outcome \(Y^a\), with the intervention well defined.
- Conditional treatment exchangeability in the source: within levels of measured covariates used for adjustment, treatment assignment in the source is independent of the potential outcomes. This supports estimating source conditional causal effects.
- Conditional transport exchangeability: given \(Z\), the potential outcome distribution relevant to the effect is the same in source and target populations. In mean form, \(E[Y^a\mid Z=z,S=1]=E[Y^a\mid Z=z,S=0]\) for each \(a\) and target-relevant \(z\).
- Positivity: each target subgroup used in the calculation is represented in the source, and each treatment level needed to estimate its effect has positive probability within that subgroup.
- Comparable measurement and outcome definitions: \(A\), \(Y\), and the transport variables must represent the same constructs across populations.
The conditional transport assumption is substantive, not something guaranteed by a statistical adjustment. It can fail if an unmeasured factor both differs between populations and modifies the effect. Similarly, source treatment exchangeability is an assumption about treatment assignment, not about population membership. If the goal is only to transport an association, define that association explicitly and avoid interpreting it as a causal effect.
Worked Example: Different Composition, Same Subgroup Effects
Suppose a hypothetical study estimates risk differences for an intervention in two age groups. The source population is 80% younger and 20% older; the target population is 30% younger and 70% older. The estimated subgroup risk differences are 0.02 among younger participants and 0.10 among older participants. Assume, for this calculation, that these subgroup effects apply in the target and that the measured outcome and intervention are comparable.
The source-population average, if calculated using source proportions, is
The target-standardized risk difference is instead
Thus the source-composition average is 3.6 percentage points, while the target-standardized estimate is 7.6 percentage points. The difference arises because the larger effect occurs in the older group, which makes up a larger share of the target. This calculation does not establish that the subgroup effects truly transport; it shows what the target average would be under that assumption.
A Short Justification of the Standardization Formula
The derivation follows by partitioning the target expectation across values of \(Z\). For each intervention level \(a\), the law of total expectation gives
Under conditional transport exchangeability, replace each target conditional mean by its source counterpart. Under consistency and conditional treatment exchangeability in the source, that counterpart can be identified from the observed source data among people with \(A=a\) and \(Z=z\). Subtracting the expression for \(a=0\) from the expression for \(a=1\) yields
The equality relies on all the stated assumptions. If a target subgroup has no source observations, its conditional means cannot be recovered from the source data without additional modeling or external information.
Worked Example: Calculating an Estimate and Its Uncertainty
In a second hypothetical analysis, the outcome is a symptom score in points, and the effect is intervention minus control, so negative values favor the intervention. Source estimates are \(-2\) points for younger participants, with standard error 1 point, and \(-5\) points for older participants, with standard error 2 points. The target population is 25% younger and 75% older. Assume independent subgroup effect estimates, fixed target proportions, and that the conditional effects transport.
The target-standardized estimate is
Under the stated independence assumption, and treating the target proportions as fixed, the estimated variance is the sum of the squared weights times the subgroup variances:
Therefore \(SE(\hat{\tau}_T)=\sqrt{2.3125}=1.521\) points. A large-sample 95% confidence interval is
Under the model and assumptions used for this interval, the data are compatible with target average effects from a 7.23-point reduction to a 1.27-point reduction. The interval accounts for uncertainty in the subgroup estimates, but not uncertainty in the target proportions or uncertainty about whether the transport assumptions hold.
Weighting, Diagnostics, and Failure Modes
An equivalent approach reweights source observations so their measured \(Z\) distribution resembles the target distribution. For discrete \(Z\), the transport weight for a source individual in subgroup \(z\) is proportional to
Weighted estimates can be useful when there are many covariates, but the weights must be estimated and inspected. A large weight means that a source subgroup is scarce relative to its importance in the target; a few observations can then dominate the estimate.
Worked Example: Checking Overlap and Weight Stability
Suppose a hypothetical source cohort contains 80 younger and 20 older participants, whereas the target contains 30 younger and 70 older participants. The weights are
The older source participants each receive a weight 3.50, reflecting their relative scarcity in the source and prevalence in the target. The weighted source shares are proportional to \(80(0.375)=30\) younger and \(20(3.50)=70\) older, reproducing the target’s 30% and 70% composition after normalization. This arithmetic confirms the composition match in this two-group example; it does not confirm effect transportability.
If only one older source participant had been observed, the same target proportion would require that observation to represent many target participants. Its contribution could be highly influential, and subgroup effects would be imprecise. If no older source participants existed, the older-group effect would be unidentified from these data. No finite reweighting can create the missing subgroup evidence.
Useful diagnostics include comparing the distributions of prespecified effect modifiers and prognostic variables, checking whether target values fall within source support, inspecting the range and concentration of weights, and examining subgroup estimates with their uncertainty. A weighted effective sample size can also reveal concentration: for weights \(w_i\), one common diagnostic is \((\sum_i w_i)^2/\sum_i w_i^2\). This is not a replacement for the original sample size, but a small value relative to the cohort size indicates that the weighted estimate relies heavily on relatively few observations.
Diagnostics can reveal problems but cannot prove the assumptions. Similar measured characteristics do not rule out unmeasured effect modifiers. Conversely, observed differences in the overall cohort averages do not alone imply nontransportability: if conditional effects are stable and the relevant covariates are measured with adequate overlap, standardization can address composition differences.
Choosing an Approach and Interpreting the Result
| Approach | What it answers | Main limitation |
|---|---|---|
| Compare crude cohort estimates | Whether the observed overall estimates differ | Differences may reflect composition, effect modification, sampling error, or other design differences. |
| Standardize subgroup estimates | What the source conditional effects imply under the target covariate distribution | Requires reliable subgroup estimates, measured effect modifiers, and target support in the source. |
| Transport weighting | What a source estimate looks like after reweighting measured characteristics toward the target | Can be unstable under poor overlap and depends on correctly specified weights and transport assumptions. |
| Meta-analysis | A pooled summary across contributing cohorts under a specified synthesis model | A pooled estimate is not automatically the effect in a particular target population. |
Direct comparison remains informative, but it is a descriptive starting point rather than a transport calculation. Meta-analysis summarizes cohort estimates; target standardization instead specifies a population distribution and asks what effect is implied for that population. The methods can complement each other, but neither an aligned p-value nor a pooled estimate establishes target relevance by itself.
Common Pitfall and Practical Limits
Transport estimates are limited by the choice and measurement of \(Z\), model misspecification, sparse subgroup data, and uncertainty in the target population’s characteristics. More covariates can reduce residual differences only if they are measured comparably and supported by adequate data; adding many variables can instead create sparse strata and unstable weights. Report the target population and estimand, the variables used to adjust or reweight, the overlap diagnostics, and the uncertainty that is and is not included. State conclusions conditionally: the estimate supports transport under the specified assumptions, rather than proving universal generalizability.
Check Your Understanding
Answer each question, then compare with the solutions below.
- A source cohort is 60% group A and 40% group B. The estimated effects are 1 unit in group A and 5 units in group B. The target is 20% group A and 80% group B. What is the target-standardized effect?
- Why does a target subgroup with no source participants create a positivity problem?
- Does matching the measured covariate distributions prove that an effect transports? Explain.
- A weighted estimate has a much smaller effective sample size than the source cohort size. What does this suggest?
Solutions. (1) The target-standardized effect is \((0.20)(1)+(0.80)(5)=0.20+4.00=4.20\) units. (2) The subgroup’s source conditional effect cannot be estimated from absent source observations, so the target average requires unsupported extrapolation. (3) No. Similar measured distributions do not rule out unmeasured effect modification, and transport also requires the other identification and measurement assumptions. (4) The weighted estimate is concentrated in relatively few observations, suggesting potentially unstable estimates and a need to inspect overlap, influential weights, and uncertainty.