Why Multiple Sites Require a Target-Specific Analysis
A treatment effect estimated at one hospital need not describe the effect at another. Sites can differ in patient characteristics, care practices, baseline outcome risks, and in how treatment effects vary across patients. Consequently, an overall effect from a collection of sites is not automatically the effect for any particular site. Transport across multiple sites requires stating which site is the target, modeling relevant effect modification, and using the target site's covariate distribution.
The central task is to combine information about outcomes under treatment and control from one or more source sites with information about the target site's covariates. The target site may contribute covariate data without contributing outcome data. The resulting estimate is justified only if the measured covariates capture the ways treatment effects differ across sites and the source data provide information for the target's covariate profiles.
Conditional Effects and the Target-Site Distribution
Within a site, define the conditional treatment effect at covariate value \(x\) as \(\tau_s(x)=E[Y(1)-Y(0)\mid X=x,S=s]\). If these conditional effects are the same across all source sites and the target site, write the common effect as \(\tau(x)\). The target average effect is then the average of \(\tau(x)\) over the target site's covariate distribution:
Here \(F_t\) is the distribution of \(X\) in the target site. For discrete covariates, the integral is a sum over covariate profiles. For continuous covariates, it denotes averaging over the target distribution. In either case, the weights come from the target site, not from the relative sample sizes of the source sites.
This formula follows directly by conditioning the target effect on \(X\). First, apply iterated expectation to \(Y(1)-Y(0)\) within the target site. Then use the cross-site invariance assumption to replace the target's conditional effect with the common conditional effect learned from the sources:
The first equality is a probability identity. The second is a substantive transport assumption: conditional on the measured effect modifiers, the treatment effect in the target site equals the effect in the source sites. It does not require the sites to have the same covariate distribution, nor does it require their untreated outcome levels to be identical.
The exchangeability condition concerns treatment assignment within a site. The invariance condition concerns whether treatment effects can be carried across sites. They are different claims: randomization within every source site can identify those sites' causal effects, but it does not by itself show that a target site's conditional effects are the same.
Modeling Several Sites
A practical analysis can first estimate conditional treatment effects separately by site, or fit a model that uses data from several sites. A model can allow each site its own baseline outcome pattern while imposing a shared treatment-effect pattern conditional on \(X\). For an additive outcome model, one possible form is:
In this expression, \(m_s(X)\) allows baseline outcome levels to differ by site, while \(\tau(X)\) represents the shared conditional treatment effect. Including \(A\)-by-\(X\) terms allows effects to vary with measured covariates. The restriction that \(\tau(X)\) is shared across sites is the model's expression of conditional effect invariance; it should be justified substantively and examined against site-specific estimates where the data allow.
This is one model form, not a guarantee that invariance holds. A model with site-by-treatment interactions instead estimates site-specific conditional effects and can reveal variation that a shared-effect model would conceal. With many sites or sparse data, estimates can be noisy, so apparent differences may reflect sampling error; conversely, a failure to detect differences does not prove invariance. The key is to report the assumption and its limits rather than treat a pooled model as proof.
For a binary outcome, the effect scale matters. If the target is a risk difference, estimate conditional risk differences and average them over the target covariate distribution. If a model estimates conditional risks, predict both treatment-arm risks for target profiles and average each arm's predictions over that distribution before taking their difference. Do not assume that a coefficient on a model's link scale is already the target site's marginal effect.
Worked Example: Standardizing Conditional Effects to a Target Site
Suppose two hypothetical randomized source-site studies examine a treatment's effect on systolic blood pressure. Let \(X\) indicate whether a patient is in a higher-risk covariate group. The conditional mean outcomes, in mmHg, are as follows:
| Site | Group | Control mean | Treatment mean | Conditional difference |
|---|---|---|---|---|
| Source 1 | Lower-risk | 140 | 138 | -2 |
| Source 1 | Higher-risk | 150 | 142 | -8 |
| Source 2 | Lower-risk | 142 | 140 | -2 |
| Source 2 | Higher-risk | 153 | 145 | -8 |
The conditional treatment differences agree across the two source sites: \(-2\) mmHg in the lower-risk group and \(-8\) mmHg in the higher-risk group. Assume, for this example, randomized treatment within both sites, adequate treatment and covariate overlap, and conditional effect invariance for the target site. The target site has 40% lower-risk and 60% higher-risk patients. Its standardized effect is:
Under the stated assumptions, treatment is estimated to lower mean systolic blood pressure by 5.60 mmHg in the target site's population. The source sites have different covariate compositions: Source 1 has 20% higher-risk patients and Source 2 has 70%. Their marginal effects would therefore be \(-3.20\) mmHg and \(-6.20\) mmHg, respectively. Neither is the target effect. The target's 40% and 60% shares determine the standardization weights.
Why Averaging Site-Level Effects Can Mislead
Averaging marginal effects across sites gives each site's already-aggregated effect a weight, but those effects were computed using different covariate distributions. That average need not correspond to the covariate distribution of the target site. This problem can arise even when conditional treatment effects are stable across sites. It is especially important to specify the effect scale: for a risk difference, conditional differences can vary across covariate groups even when a relative effect is constant.
Worked Example: The Target Risk Difference Is Not the Average of Two Site Effects
Consider a hypothetical binary outcome and assume the conditional risks under each treatment are the same across sites. In the lower-risk group, the control risk is 0.10 and the treatment risk is 0.08, giving a risk difference of \(-0.02\). In the higher-risk group, the control risk is 0.30 and the treatment risk is 0.24, giving a risk difference of \(-0.06\). Suppose the higher-risk share is 0.80 in Source A, 0.20 in Source B, and 0.25 in the target site.
In Source A, the standardized control risk is \(0.80(0.30)+0.20(0.10)=0.26\), and the treatment risk is \(0.80(0.24)+0.20(0.08)=0.208\). Its marginal risk difference is \(0.208-0.26=-0.052\). In Source B, those risks are \(0.20(0.30)+0.80(0.10)=0.14\) and \(0.20(0.24)+0.80(0.08)=0.112\), so the marginal risk difference is \(0.112-0.14=-0.028\).
The target's control risk is \(0.25(0.30)+0.75(0.10)=0.15\), and its treatment risk is \(0.25(0.24)+0.75(0.08)=0.12\). Thus the target risk difference is \(0.12-0.15=-0.03\), or a reduction of 3 percentage points. By contrast, the unweighted average of the two source marginal risk differences is \((-0.052-0.028)/2=-0.040\), a reduction of 4 percentage points. It answers neither the target standardization nor necessarily any policy-relevant target question. The discrepancy arises from using source-site weights rather than the target's covariate distribution.
Using Site Variation Carefully
Multiple sites provide an opportunity to learn about conditional effects across a wider range of patients and to examine whether estimated effects differ by site. They do not make cross-site invariance automatic. For example, a site-specific treatment effect may vary because a treatment is delivered differently, because an important effect modifier was not measured, or because the target population differs in a way not captured by \(X\). If such differences remain after conditioning on the measured covariates, then a common \(\tau(x)\) is not justified.
Worked Example: Combining Estimates While Checking Site Variation
Suppose two hypothetical randomized source sites estimate treatment effects in two covariate groups, in days of symptom duration. The estimates are \(-2\) days and \(-8\) days for the lower- and higher-risk groups at Site A, and \(-3\) days and \(-7\) days for those groups at Site B. Assume, for illustration, that estimates from the two sites have equal precision within each group. A simple equal-precision pooled conditional estimate is the average of the two site estimates in each group:
If the target site is 40% lower-risk and 60% higher-risk, the standardized estimate is:
This calculation is an estimate under a common conditional effect assumption, not proof that the site-specific effects are identical. The differences of 1 day in each group could be sampling variation, but their uncertainty must be considered. If real site-specific differences remain, the pooled conditional effects may not transport to the target. A more elaborate model can use precision-weighted estimates or site-by-treatment terms, but it cannot repair a false invariance assumption without additional information.
Support, Interpretation, and Common Mistakes
Transport requires overlap in the covariates that modify the effect. If a target site contains a covariate profile absent from every source site, source data alone cannot estimate the effect for that profile without extrapolation. If only some target profiles lack support, the effect may be identified for the supported part of the target population but not for the full target without extra assumptions. Restricting the target population changes the estimand and should be reported as such.
The site itself may also matter for effect modification. A model that includes site as a predictor of baseline outcomes but omits site-by-treatment differences assumes that, after conditioning on \(X\), the treatment effect does not vary by site. A site indicator alone does not make an effect transportable. Likewise, a pooled estimate over all source participants gives more influence to sites with more observations; that may improve precision for the pooled model but does not substitute for standardization to \(F_t\).
- Mistake: average the source sites' marginal effects. This generally uses source composition rather than the target distribution. Estimate conditional effects and standardize them to the target.
- Mistake: treat randomized source sites as proof of transportability. Randomization addresses treatment confounding within a source; it does not establish cross-site effect invariance.
- Mistake: assume a good-fitting pooled model validates its assumptions. Model fit cannot establish that all relevant effect modifiers were measured or that target effects match source effects.
- Mistake: ignore the effect scale. State whether the target is a risk difference, risk ratio, mean difference, or another contrast, and standardize quantities appropriate to that scale.
Name the target site, outcome, treatment contrast, and effect scale.
Use source outcomes to estimate effects by measured effect modifiers, allowing site differences where the data support them.
Assess treatment assignment assumptions within sites, covariate overlap, and the plausibility of conditional effect invariance.
Average the estimated conditional effects using the target covariate distribution, and report uncertainty and limitations.
Check Your Understanding
Answer each question by identifying the estimand and the assumption that supports the calculation.
- Two source sites have different proportions of an effect-modifying covariate. Which site's covariate distribution should determine a transported effect for a third target site?
- Why does random treatment assignment within source sites not, by itself, establish that their conditional treatment effects apply to a target site?
- A target site includes a covariate profile absent from every source site. What limitation does this create for transport from the available data?
- Under what condition can a shared conditional effect model have different site-specific baseline outcome models?
- In the risk example, why does the equal average of the source-site marginal risk differences fail to equal the target-site risk difference?