Exchangeability Assumptions for Historical Controls

Why Exchangeability Is the Key Question

A nonconcurrent control patient enrolled earlier in a platform trial was not enrolled at the same time as a patient entering a newly opened treatment arm. The historical patient’s outcome under control is observed, but the later patient’s outcome under control is not: that patient receives the new treatment. The central question is therefore whether earlier control outcomes can represent the control outcomes the later patients would have had.

Similarity in eligibility criteria is not enough. Patient characteristics, routine care, diagnostic methods, outcome measurement, and follow-up practices can change over calendar time. Some differences may be measured and accounted for; others may not be recorded or even recognized. Exchangeability is the assumption that, after conditioning on the relevant measured characteristics, the control outcomes do not differ systematically between enrollment periods.

Definition: Let $T=0$ indicate an earlier enrollment period and $T=1$ the later period targeted by the new arm. Let $Y(0)$ be the outcome a patient would have under control, and let $X$ denote measured characteristics determined before treatment. Conditional exchangeability of control outcomes across periods means $$ Y(0) \perp T \mid X $$ among patients eligible for the target comparison. Equivalently, for relevant values of $X$, the distribution of $Y(0)$ given $X$ is the same in the earlier and later periods.

This condition does not require the earlier and later cohorts to have the same distribution of $X$. Their measured characteristics may differ, and those differences may be related to outcomes. Exchangeability instead requires that, within levels of the characteristics being conditioned on, the control outcome distribution is stable across periods. If $X$ contains no useful prognostic variables, the condition reduces to the stronger requirement that control outcomes are exchangeable across periods without conditioning.

Conditions for Using Historical Controls

Exchangeability is one part of a set of assumptions. Each addresses a different reason the earlier control observations might fail to represent the later target cohort.

Comparable eligibility and target population. Patients in the historical data and the later target cohort must be defined according to the intended comparison. Changes in eligibility can alter who is represented, even if the written criteria appear similar.
Conditional exchangeability. After conditioning on $X$, there must be no remaining period-related difference in the control potential outcome distribution. This rules out both measured differences left unaccounted for and relevant unmeasured temporal changes.
Consistency of the control condition and outcome. The observed control outcome must correspond to the potential outcome denoted by $Y(0)$. The meaning of “control,” the outcome definition, and its measurement must be sufficiently comparable across periods.
Availability of historical controls across the target covariate profiles. For each $X$-profile represented in the later target cohort, there must be historical control information to support a control-outcome comparison. This is a support requirement, distinct from exchangeability.
No interference relevant to the comparison. One patient’s control outcome must not depend on another patient’s treatment assignment in a way that changes the control outcome being represented.

The assumptions concern the scientific comparison, not just the statistical model. For example, adding calendar period or baseline covariates to an analysis does not establish that all important temporal changes have been measured. A model can adjust only for differences represented in its data and assumptions.

What the Assumption Identifies

Suppose the target is the mean control outcome for patients in the later period. If the outcome is binary, this mean is a risk; for a continuous outcome, it is the corresponding mean on that outcome’s scale. Under consistency, conditional exchangeability, and historical control availability for the target covariate profiles, the target mean can be expressed using earlier controls:

E\{Y(0)\mid T=1\} = \int E\{Y\mid A=0,T=0,X=x\}\,dF(x\mid T=1)

Here $A=0$ denotes observed control treatment, and $F(x\mid T=1)$ is the distribution of baseline characteristics in the later target cohort. The equation says to take the earlier control outcome at each covariate profile and average those outcomes over the later cohort’s covariate distribution. It does not say that the crude earlier control average must equal the later cohort’s mean control outcome.

A Short Justification

The identifying expression follows by averaging within covariate profiles. First, the law of total expectation gives

E\{Y(0)\mid T=1\} = \int E\{Y(0)\mid T=1,X=x\}\,dF(x\mid T=1).

Conditional exchangeability replaces the unobserved later-period control expectation inside the integral with the earlier-period expectation:

E\{Y(0)\mid T=1,X=x\} = E\{Y(0)\mid T=0,X=x\}.

Among earlier-period patients receiving control, consistency makes the observed outcome equal to their control potential outcome, so

E\{Y(0)\mid T=0,X=x\} = E\{Y\mid A=0,T=0,X=x\}.

Substituting the last two equalities into the first yields the identifying expression. The support condition matters because the earlier-period conditional expectation must be informed by historical controls at the target covariate profiles. If that information is absent, the equation cannot be evaluated from observed data without additional assumptions.

This result identifies a mean control outcome, not automatically a treatment effect. To interpret a comparison with patients receiving the new treatment as a treatment effect, the treatment outcome must also be defined consistently and the treatment-group outcome must be appropriate for the same target population. The exchangeability condition here specifically concerns the missing control outcomes for later patients.

Worked Example: Different Patient Mix, Stable Outcomes Within Groups

Hypothetical data. Consider a binary 30-day event and two measured baseline risk groups, low and high. Among 200 earlier control patients, 8 of 100 low-risk patients and 24 of 100 high-risk patients have the event. The later target cohort is 40% low-risk and 60% high-risk. Assume, for this calculation, that the event risk under control is stable within each risk group across the two periods.

The earlier control risks are

\hat p_{0,\text{low}}=\frac{8}{100}=0.08, \qquad \hat p_{0,\text{high}}=\frac{24}{100}=0.24.

Applying these risks to the later cohort’s mix gives an estimated later-period control risk of

$$ (0.40)(0.08)+(0.60)(0.24) =0.032+0.144 =0.176. $$

Thus the estimated mean control outcome for the later cohort is a 17.6% event risk. The earlier cohort’s crude risk is $32/200=16.0\%$, but that crude figure averages over a different mix: half low-risk and half high-risk. The 17.6% estimate uses the later cohort’s measured risk distribution. Its interpretation depends on the assumed stability of control risks within each group; the arithmetic does not establish that assumption.

How Time Changes Can Break Exchangeability

A shift in the distribution of a measured prognostic factor is not, by itself, a violation of conditional exchangeability. If that factor is recorded, defined comparably, and the control outcome distribution is stable within its levels (or within the relevant joint profiles of measured factors), the shift can be reflected in the target average. By contrast, if the outcome changes within those profiles, conditioning on those factors does not make the periods exchangeable.

Worked Example: A Measured Time Change That Is Not Captured by Risk Group

Hypothetical scenario. Suppose the same two measured risk groups are used, and each period’s cohort is 50% low-risk and 50% high-risk. In the earlier period, the control event risks are 10% and 20% in the two groups. A change in routine supportive care occurs before the later period; under control, the corresponding risks are now 15% and 25%.

The earlier control data would predict a later control risk of

$$ (0.50)(0.10)+(0.50)(0.20)=0.15. $$

But the later-period control risk under the stated scenario is

$$ (0.50)(0.15)+(0.50)(0.25)=0.20. $$

The five-percentage-point discrepancy remains even though the measured risk-group composition is identical. The control outcome distribution changed within each measured group, so conditional exchangeability given that group fails. If a later treatment group had a 12% event risk, subtracting the historical prediction would give $0.12-0.15=-0.03$, or a three-percentage-point lower risk than the historical estimate. Relative to the later control risk in this hypothetical scenario, the contrast is $0.12-0.20=-0.08$, an eight-percentage-point lower risk. These are different comparisons because the historical estimate does not represent the later control outcome here.

Time-related changes can be observed or unobserved. Examples include a change in routine care, changes to diagnostic thresholds, different sites entering the platform, shifts in disease severity not captured by recorded variables, or altered outcome ascertainment. A recorded change can still threaten exchangeability if it is not incorporated into the comparison or if it changes outcomes within the covariate profiles being used. An unrecorded change threatens the assumption because its effect cannot be removed by conditioning on the available $X$.

Worked Example: An Unmeasured Shift in Severity

Hypothetical scenario. Suppose a factor $U$, indicating a severe presentation, was not recorded in the analysis. In both periods, control event risk is 10% for patients without severe presentation and 30% for those with it. The severe-presentation prevalence is 25% in the earlier period and 60% in the later period. Assume the recorded covariates $X$ do not distinguish these groups.

The earlier control risk is

$$ (0.75)(0.10)+(0.25)(0.30) =0.075+0.075 =0.150. $$

The later cohort’s control risk is instead

$$ (0.40)(0.10)+(0.60)(0.30) =0.040+0.180 =0.220. $$

A historical-control estimate of 15.0% would therefore understate the later control risk of 22.0% by 7.0 percentage points. If the later treatment group’s event risk were 18.0%, comparison with the historical risk would give $0.180-0.150=0.030$, a three-percentage-point higher risk. Comparison with the later control risk would give $0.180-0.220=-0.040$, a four-percentage-point lower risk. The reversal illustrates how an unmeasured change in patient mix can distort the treatment comparison even when risks within levels of the unmeasured factor are stable.

Assessing the Assumption and Its Limits

Exchangeability cannot be confirmed from the observed outcomes alone: later patients’ control potential outcomes are not observed when they receive the new treatment. Assessment therefore combines design information and empirical evidence. Review whether eligibility, control treatment, sites, outcome definitions, measurement procedures, and follow-up were comparable; identify changes in routine care and patient characteristics; and consider whether recorded covariates capture the changes likely to affect control outcomes. These checks can reveal reasons for concern, but a lack of an obvious discrepancy does not prove exchangeability.

A common mistake is to treat “no statistically significant difference between historical and current patients” as evidence that the periods are exchangeable. That reasoning is inadequate: measured summaries cannot rule out unmeasured differences, and a non-significant result may simply be imprecise. The correction is to treat exchangeability as a substantive assumption, justify it using clinical and operational knowledge, and describe remaining uncertainty. A second mistake is to adjust for measured baseline variables and assume that this removes every calendar-time effect. Adjustment addresses only the differences represented and correctly handled by those variables; it does not remove an unmeasured change in care or outcome measurement.

Concurrent controls are generally a more direct reference for the later period because they experience the same calendar-time environment as the new treatment group. Historical controls may add information when concurrent control data are limited, but their usefulness depends on the assumptions above and on the availability of relevant historical observations. Neither a large historical sample nor a sophisticated model guarantees exchangeability. The main limitation is that temporal differences not captured in the data cannot be distinguished reliably from differences in potential control outcomes using those data alone.

Key takeaway: Historical controls can represent later patients’ control outcomes only if, after conditioning on relevant measured baseline characteristics, the control outcome distribution is stable across periods, the control condition and outcome are comparable, and historical data support the later cohort’s covariate profiles. Changes in routine care, measurement, or patient severity can undermine these conditions, whether measured or unmeasured.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

In one sentence, state what conditional exchangeability of control outcomes across periods means.
Earlier controls have an 8% event risk in a low-risk group and a 24% risk in a high-risk group. A later target cohort is 25% low-risk and 75% high-risk. Under stable within-group control risks, calculate the later cohort’s estimated control risk.
Why does an unchanged distribution of measured baseline covariates fail to establish exchangeability if routine care changes outcomes within those covariate groups?
In the unmeasured-severity example, state the historical and later control risks and their difference.
Name one reason a historical-control analysis could fail even if the measured covariate distributions are similar.

Solutions. (1) Conditional exchangeability means that, within levels of the measured baseline characteristics $X$, the control potential outcome distribution is the same in the earlier and later periods. (2) The estimated risk is $(0.25)(0.08)+(0.75)(0.24)=0.02+0.18=0.20$, or 20.0%. (3) Similar measured covariate distributions address patient mix, not whether control outcomes remain stable within those groups; the routine-care change makes the conditional outcome distributions differ. (4) The historical risk is 15.0%, the later risk is 22.0%, and the later risk exceeds the historical risk by 7.0 percentage points. (5) Examples include an unmeasured shift in disease severity, a change in routine care, or a change in outcome definition or measurement.