Exchangeability Assumptions for Extrapolation

Why Exchangeability Matters for Extrapolation

A trial may provide a precise estimate of efficacy or safety in its participants, yet the decision concerns a different population: for example, patients in another region, people with a different disease severity distribution, or patients treated in routine practice. The central question is not simply whether the trial result is credible. It is whether the outcome evidence from the trial population can stand in for evidence in the population of interest.

The earlier material on selection diagrams for population transport gives a way to represent how trial inclusion and population differences relate to outcomes. Here the focus is the assumption behind extrapolation: after conditioning on relevant measured characteristics, would people in the source and target populations have the same potential outcomes under the treatment being considered? The answer is not established by randomization alone.

Definition: Let $S=1$ denote membership in the source population (such as a clinical trial) and $S=0$ membership in the target population. Let $A$ denote treatment, $Y^a$ the outcome that would be observed under treatment $a$, and $X$ a set of measured population characteristics. Conditional exchangeability for the potential outcome under $a$ means $$ Y^a \perp S \mid X, $$ or, for a mean-outcome estimand, $E[Y^a\mid X,S=1]=E[Y^a\mid X,S=0]$ for relevant values of $X$. It says that, within levels of $X$, source-versus-target membership carries no further information about the potential outcome under $a$.

The required form of exchangeability depends on the estimand. If the goal is a mean outcome, equality of conditional means is sufficient for the exchangeability step. If the goal is the entire outcome distribution, the stronger condition $P(Y^a\leq y\mid X,S=1)=P(Y^a\leq y\mid X,S=0)$ must hold for relevant $y$. For a treatment contrast, exchangeability of both potential outcomes supports transport of the contrast on its stated scale. A weaker, contrast-specific assumption may sometimes be defensible: the conditional treatment effect is the same across populations, even if their baseline outcomes differ. That weaker assumption does not imply that either potential outcome is individually exchangeable.

Conditions Behind the Transport Argument

Exchangeability is one part of a causal argument, not a stand-alone guarantee. A transport claim also needs a well-defined treatment and outcome across populations, and a way to learn the relevant potential outcomes from the source data. In particular, consistency means that a person who receives treatment $a$ has observed outcome $Y=Y^a$. The treatment versions, outcome definitions, follow-up period, and handling of intercurrent events must therefore correspond to the estimand in both populations.

Treatment assignment and population membership are different questions. In a randomized source trial, randomization supports comparing treatment arms among trial participants. It does not show that trial participants and target-population members have exchangeable potential outcomes. In a nonrandomized source, treatment assignment must also be adequately controlled to identify the source potential outcomes; this is a separate condition from transportability.

For a mean outcome, the key implication of conditional exchangeability follows by partitioning the target population according to $X$. For discrete $X$, the law of total expectation gives

E[Y^a\mid S=0] =\sum_x E[Y^a\mid X=x,S=0]P(X=x\mid S=0) =\sum_x E[Y^a\mid X=x,S=1]P(X=x\mid S=0).

The second equality uses conditional exchangeability. It replaces each target conditional mean with the corresponding source conditional mean, while retaining the target distribution of $X$. For continuous $X$, the sum is replaced by an integral. To connect this identity to trial observations, the source conditional potential outcome means must themselves be identified from the treatment data—for example, by randomization, consistency, and treatment representation in the relevant strata. The identity explains why exchangeability matters; it does not establish that the assumption is true.

Important distinction: Randomization can support $Y^a \perp A\mid X,S=1$, a statement about treatment assignment within the source. Transport requires a statement about $S$, such as $Y^a \perp S\mid X$. The first does not entail the second.

Worked Implications and Plausibility Checks

Worked Example: Measured Severity and a Mean Efficacy Outcome

Hypothetical data. Suppose a source trial measures baseline disease severity as mild or severe. The outcome is improvement on a scale where higher values are better. Source-trial conditional means are 8 and 2 points under treatment and control, respectively, for mild patients; the corresponding means for severe patients are 5 and 1 points. The target population is estimated to be 60% mild and 40% severe.

If conditional mean exchangeability holds given severity, the target conditional means under each treatment are the same as the source means. The implied target mean under treatment is

0.60(8)+0.40(5)=4.8+2.0=6.8\ \text{points}.

Under control, the corresponding mean is

0.60(2)+0.40(1)=1.2+0.4=1.6\ \text{points}.

The implied target mean difference, treatment minus control, is $6.8-1.6=5.2$ points. Equivalently, the conditional differences are 6 points for mild patients and 4 points for severe patients, so $0.60(6)+0.40(4)=5.2$ points. These calculations are implications of the stated assumption, not evidence that it holds. Its plausibility would depend on whether severity captures the important source–target differences in potential improvement, whether severity and outcomes were measured comparably, and whether other factors could alter the conditional means.

Worked Example: An Unmeasured Difference in Safety Risk

Hypothetical data. Consider a binary safety outcome and a risk-difference estimand, treatment minus control. Let $U$ denote an unmeasured renal impairment characteristic. Suppose the treatment-minus-control risk difference is 15 percentage points for patients with $U=1$, and 1 percentage point for those with $U=0$. Assume $U$ occurs in 10% of the source population but 40% of the target population. All other measured characteristics are held fixed for this illustration.

The source-population risk difference implied by these conditional effects is

$$ 0.10(0.15)+0.90(0.01)=0.015+0.009=0.024, $$

or a 2.4-percentage-point increase in risk. The corresponding target-population difference is

$$ 0.40(0.15)+0.60(0.01)=0.060+0.006=0.066, $$

or a 6.6-percentage-point increase. The source contrast does not represent the target contrast because the prevalence of $U$ differs and the treatment effect varies with $U$. If $U$ is not measured or otherwise accounted for, conditioning on the available $X$ does not make the populations exchangeable for this effect. This example illustrates a substantive threat, not a test that can be ruled out by a randomized source trial.

Worked Example: Transportability Depends on the Effect Scale

Hypothetical data. In the source population, suppose the control risk of an adverse event is 20% and the treated risk is 10%. In the target population, suppose the corresponding risks are 50% and 25%. The relative risk is $0.10/0.20=0.50$ in the source and $0.25/0.50=0.50$ in the target. Thus, the relative risk reduction is 50% in both populations.

The absolute risk reductions are different. In the source population, $0.20-0.10=0.10$, or 10 percentage points. In the target population, $0.50-0.25=0.25$, or 25 percentage points. Therefore, equality of the relative effect does not establish exchangeability of the potential outcome risks, nor does it transport the absolute risk difference. It may support a scale-specific claim about the relative effect if that claim is justified, but it cannot be silently substituted for a claim about absolute benefit or harm.

Assessing Plausibility, Diagnostics, and Alternatives

Conditional exchangeability is generally not empirically testable from source-trial data alone: the trial does not reveal target-population potential outcomes. Assessment therefore combines design knowledge, subject-matter evidence, available data from both populations, and explicit sensitivity analysis. Useful questions include:

Are the source and target populations defined using comparable eligibility, disease-stage, and care-context information?
Could a difference not included in $X$—such as organ function, concomitant treatment, adherence, or standards of supportive care—change outcomes under either treatment?
Were the candidate characteristics measured with comparable definitions, timing, and quality in the source and target data?
Do external studies, pharmacology, disease biology, or prior experience suggest that conditional outcomes could differ across settings?
Would plausible departures from the assumption materially change the decision, especially for safety outcomes with serious consequences?

Observed conditional outcome patterns can be informative, but they do not prove exchangeability: source and target outcomes may be available for only selected groups, and residual unmeasured differences can remain. Large unexplained differences after adjustment are a warning. Conversely, a lack of a statistically detectable difference is not proof of equality; limited sample size can make important differences difficult to detect. Sensitivity analyses can state how large an unmeasured conditional difference would need to be to change the conclusion, and can show the decision’s dependence on that difference.

A useful comparison is between three strategies. Unconditional exchangeability assumes that source and target potential outcomes agree without conditioning; it is simple but often implausible when populations differ in outcome-relevant characteristics. Conditional exchangeability allows those characteristics to have different distributions, provided that conditioning on measured $X$ removes the remaining population differences relevant to the potential outcomes. Local evidence or a mechanistic model can reduce reliance on extrapolation from the source alone, but introduces its own requirements—for example, comparable conduct and endpoints for a local study, or a credible and adequately evaluated mechanistic model.

No single diagnostic verifies the assumption, and no adjustment can repair an omitted, poorly measured, or incorrectly defined outcome-relevant characteristic by itself. The conclusion should therefore specify the outcome, treatment contrast, population, and effect scale to which exchangeability is being applied, and distinguish supporting evidence from remaining uncertainty.

Common Mistake

Wrong reasoning: “The source study was randomized, so its treatment effect is exchangeable with the target population and can be transported directly.” Randomization addresses treatment comparisons within the trial, not the way trial participation relates to potential outcomes in the target population.

Correction: State the treatment-assignment condition and the population-exchangeability condition separately. Then ask whether the measured $X$ is sufficient to account for outcome-relevant differences between source and target populations, and whether the intended treatment contrast is on a scale for which the assumed invariance is defensible.

Key takeaway: Extrapolation relies on a claim about potential outcomes across populations, not merely on the internal validity of the source trial. Conditional exchangeability is estimand-specific, depends on what has been measured, and must be supported by scientific reasoning and sensitivity analysis.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions below.

A target population has a different distribution of a measured severity variable from a source trial. What conditional statement would allow the source conditional mean outcomes to inform target mean outcomes?
Why does random treatment assignment in a trial not, by itself, establish exchangeability between trial participants and a target population?
In a hypothetical source population, control risk is 20% and treatment risk is 12%. If the target control risk is 40% and the relative risk is assumed to remain constant, what target treatment risk and absolute risk reduction follow?
Suppose an unmeasured characteristic is more common in the target population and changes the treatment-minus-control risk difference. What does this imply for transport based only on measured covariates?

Solutions. (1) For a mean estimand, require $E[Y^a\mid X=x,S=1]=E[Y^a\mid X=x,S=0]$ for each relevant treatment $a$ and target-relevant $x$. The populations may have different severity distributions; the conditional means must agree. (2) Randomization concerns $A$ within $S=1$, whereas population exchangeability concerns $S$ conditional on $X$. One does not imply the other. (3) The source relative risk is $0.12/0.20=0.60$. Applying that assumed relative risk to the target control risk gives $0.60(0.40)=0.24$, or a 24% treatment risk. The target absolute risk reduction is $0.40-0.24=0.16$, or 16 percentage points. (4) The measured covariates do not suffice for exchangeability of that risk difference if the omitted characteristic changes the conditional contrast and differs in prevalence. Additional evidence, measurement, or sensitivity analysis is needed; the source contrast cannot be assumed to equal the target contrast.