Tutorials › Causal Inference › Transport Across Time and Changing Care Settings

Transportability and Generalizability · Tutorial 943 of 972

Transport Across Time and Changing Care Settings

Learn to standardize effects across periods and settings, and to recognize when apparent changes instead reflect different treatments, measurements, or effect relationships.

Advanced 10 min read

What You'll Learn

  • State the target-period effect and the assumptions that support transporting a source-period effect.
  • Calculate how a changed distribution of effect modifiers changes a target average effect.
  • Distinguish population-composition shifts from changes in treatment versions or outcome measurement.
  • Recognize when conditional treatment effects differ across periods or settings.
  • Explain what standardization can and cannot resolve when transport assumptions fail.

Why Time and Setting Matter

A treatment effect estimated in one hospital or calendar period does not automatically describe the effect in another. The target population may have a different mix of patients, and the care environment may have changed: the treatment may be delivered differently, the outcome may be recorded differently, or the treatment’s effect for comparable patients may genuinely differ. These possibilities can produce similar-looking changes in reported effects, but they call for different interpretations.

The key is to ask what has changed. A shift in the distribution of known effect modifiers can be addressed by standardizing source-period conditional effects to the target-period distribution. Standardization does not, by itself, make different treatment versions equivalent, correct a changed outcome definition, or guarantee that conditional effects stayed stable.

Definition: Let \(S=0\) denote a source period or setting and \(S=1\) a target period or setting. Let \(A\) be treatment, \(Y\) the outcome, \(X\) measured baseline covariates, and \(Y^a\) the potential outcome under treatment level \(a\). The target average treatment effect is \(E[Y^1-Y^0\mid S=1]\). A source conditional treatment effect is \(E[Y^1-Y^0\mid X=x,S=0]\).

Here, \(S\) marks the population or environment in which a person is observed; it is not itself a treatment. The contrast \(Y^1-Y^0\) must refer to clearly specified treatment options and a clearly specified outcome. For example, “medication plus scheduled monitoring versus usual care” is not automatically the same treatment contrast as “medication alone versus usual care.”

When Standardization Across Periods Is Appropriate

Suppose the treatment and control strategies have the same meaning in both periods, the outcome is defined and measured comparably, and the conditional causal effect given \(X\) is stable across periods. If the source-period conditional effects are identified, and target-period covariate values are available, the target effect can then be obtained by averaging those effects over the target distribution of \(X\):

$$ E[Y^1-Y^0\mid S=1] = \sum_x E[Y^1-Y^0\mid X=x,S=0]P(X=x\mid S=1). $$

For continuous covariates, the sum is replaced by an integral over the target distribution. This formula is useful when \(X\) captures the relevant effect modifiers and the source contains information about each covariate pattern represented in the target. It does not require the source and target to have the same proportions of those patterns.

Transport assumption: For the covariates used in standardization, the conditional causal effect is the same in source and target: \(E[Y^1-Y^0\mid X=x,S=1]=E[Y^1-Y^0\mid X=x,S=0]\). A credible application also requires a common definition of treatment and outcome, adequate source information for target covariate patterns, and valid identification of source conditional effects.

Identifying source conditional effects requires its own causal assumptions. In an observational source, treatment must be exchangeable with the potential outcomes conditional on the adjustment covariates, and both treatment options must be possible across the relevant covariate patterns. In a randomized source, randomization supports treatment exchangeability, subject to the study design and implementation. These treatment-assignment conditions are distinct from the assumption that conditional effects carry across settings.

Deriving the Target Standardization Formula

The derivation makes clear what standardization changes and what it leaves untouched. First, average the individual causal contrast across the target population, grouping people by \(X\). Then use conditional effect stability to replace each target conditional effect with its source counterpart:

$$ \begin{aligned} E[Y^1-Y^0\mid S=1] &= \sum_x E[Y^1-Y^0\mid X=x,S=1]P(X=x\mid S=1)\\ &= \sum_x E[Y^1-Y^0\mid X=x,S=0]P(X=x\mid S=1). \end{aligned} $$

The first equality is the law of total expectation. The second uses the transport assumption for each \(x\). Thus, the target covariate distribution supplies the weights, while the source supplies the conditional effects. If a conditional effect changes across periods, that substitution is not justified; changing the weights cannot repair it.

Worked Example: A New Period Has More High-Severity Patients

Suppose a hypothetical study compares a fixed treatment with the same control strategy in two periods. The outcome is recovery time in days, and a negative treatment effect means fewer days to recovery. Severity at baseline is classified as mild or high. Assume the treatment versions and outcome measurement are unchanged, the source conditional effects are valid, and conditional effects are stable across periods.

In the source period, the conditional effects are \(-4\) days for mild patients and \(-10\) days for high-severity patients. The source population is 80% mild and 20% high severity. The source average effect is:

$$ (-4)(0.80)+(-10)(0.20)=-3.2-2.0=-5.2\text{ days}. $$

In the target period, suppose 40% are mild and 60% have high severity. Standardize the same conditional effects to this target mix:

$$ (-4)(0.40)+(-10)(0.60)=-1.6-6.0=-7.6\text{ days}. $$

Under the stated assumptions, the target average effect is \(-7.6\) days, compared with \(-5.2\) days in the source population. The difference arises from the changed severity mix, not from a changed effect within either severity group. Interpreted in context, the treatment is estimated to shorten recovery time by an average of 7.6 days in the target population.

Separate Composition Changes from Changes in the Care Process

Calendar time and care setting often bring operational changes along with changes in patient mix. A new dose, delivery protocol, co-intervention, or comparator can change what the treatment contrast means. A shift from paper records to active follow-up can change how outcomes are detected. These are not simply different weights over \(X\).

A treatment version is a particular way of implementing a treatment level. If the source estimates the effect of medication alone but the target uses medication plus monitoring, those are different interventions unless the versions can reasonably be treated as equivalent for the question at hand. The source effect for the first intervention is not automatically the effect of the second. A treatment indicator coded “treated” in both periods does not establish that the interventions are the same.

Outcome comparability matters for the same reason. If “readmission within 30 days” is captured by linked hospital records in one period but by a less complete system in another, the recorded outcome may differ even if the underlying clinical outcomes and effects do not. Before comparing estimates, check the outcome definition, follow-up window, data source, and measurement process.

Worked Example: A Changed Treatment Version

Consider a hypothetical randomized source-period comparison of medication alone with usual care. Suppose 90 of every 1,000 patients assigned medication alone experience an event, compared with 100 of every 1,000 assigned usual care. The source risk difference is:

$$ \frac{90}{1000}-\frac{100}{1000} =0.090-0.100=-0.010, $$

or \(-10\) events per 1,000 patients. In a later period, suppose the active treatment is medication plus scheduled monitoring. In that hypothetical comparison, 80 of every 1,000 patients assigned the package experience the event, while 100 of every 1,000 assigned usual care do so. The risk difference is:

$$ \frac{80}{1000}-\frac{100}{1000} =0.080-0.100=-0.020, $$

or \(-20\) events per 1,000 patients. The numerical difference between these estimates is not evidence, by itself, that the same treatment effect changed over time: the active intervention changed from medication alone to a package. Standardizing for patient mix would not make those treatment versions identical. To answer a question about the package, one needs evidence about that package or additional assumptions linking its effect to the effect of medication alone.

Measurement Changes Can Mimic Effect Changes

A measured effect is tied to the outcome actually recorded. The following example uses a binary clinical outcome \(Y\) and a recorded outcome \(Y^*\). Suppose measurement sensitivity is the probability that \(Y^*=1\) when \(Y=1\), and specificity is the probability that \(Y^*=0\) when \(Y=0\). If sensitivity and specificity are the same in both treatment groups within a period, the recorded risk is:

$$ P(Y^*=1\mid A=a) = \text{sensitivity}\,P(Y=1\mid A=a) + (1-\text{specificity})\{1-P(Y=1\mid A=a)\}. $$

Worked Example: Outcome Detection Changes Across Periods

Suppose, hypothetically, that the true event risks are 0.20 under treatment and 0.30 under control in both periods. The true risk difference is therefore \(0.20-0.30=-0.10\), or \(-10\) events per 100 patients. Assume specificity is 0.98 in both periods and measurement is equally accurate in the treatment and control groups within each period. In the source period, sensitivity is 0.90.

The recorded treatment risk is \(0.90(0.20)+0.02(0.80)=0.196\), and the recorded control risk is \(0.90(0.30)+0.02(0.70)=0.284\). The recorded risk difference is \(0.196-0.284=-0.088\), or \(-8.8\) events per 100 patients.

Now suppose target-period sensitivity falls to 0.65, while specificity and the true risks stay the same. The recorded treatment risk is \(0.65(0.20)+0.02(0.80)=0.146\). The recorded control risk is \(0.65(0.30)+0.02(0.70)=0.209\). The recorded risk difference is \(0.146-0.209=-0.063\), or \(-6.3\) events per 100 patients.

The recorded contrast is closer to zero in the target period, although the hypothetical true risks and true risk difference have not changed. This illustrates why comparability of outcome ascertainment is part of transport assessment; a change in recorded effect need not be a change in clinical effect.

When Conditional Effects Change

Even with unchanged treatment and outcome definitions, the effect among people with the same measured \(X\) can differ across periods or settings. Possible reasons include a changed care environment, new co-interventions, or effect modifiers that were not measured or included in \(X\). In that situation, the source conditional effects cannot be substituted for target conditional effects merely by using target-period covariate proportions.

Worked Example: Different Effects Within Severity Groups

Suppose two hypothetical randomized studies use the same treatment versions and outcome measurement. Each period has equal proportions of mild and high-severity patients. In the source period, treatment reduces the event risk by 8 percentage points for mild patients and 12 percentage points for high-severity patients. Standardizing those effects to the target mix would give:

$$ (-0.08)(0.50)+(-0.12)(0.50)=-0.10. $$

That source-based transported estimate is a 10-percentage-point reduction in target risk. But suppose the target-period randomized data show reductions of 4 percentage points for mild patients and 12 percentage points for high-severity patients. The target average is then:

$$ (-0.04)(0.50)+(-0.12)(0.50)=-0.08. $$

The target effect is an 8-percentage-point reduction, not 10 percentage points. Since the covariate mix is identical, composition does not explain the difference; the conditional effect for mild patients changed. The source standardization fails because conditional effect stability is not supported. Target outcome data may reveal such a discrepancy, but when those data are unavailable, the stability assumption cannot be confirmed simply from the source data.

A Practical Assessment

A careful transport assessment separates the questions rather than treating a difference between two averages as a single problem. The order below helps identify which assumptions are relevant before applying a formula.

1
Define the estimand.
Specify the target period or setting, target population, treatment contrast, outcome, follow-up time, and effect scale.
2
Check treatment and outcome comparability.
Ask whether both periods implement the same treatment versions and record the same outcome in a comparable way.
3
Identify relevant effect modifiers.
Determine which measured baseline characteristics could modify the treatment effect and whether the source covers the target covariate patterns.
4
Assess conditional effect stability.
Consider whether the causal effect given those characteristics is plausibly stable across periods or settings, and use target outcome information when available.
5
Standardize only when justified.
Average source conditional effects using the target distribution of effect modifiers, while stating the assumptions that make this calculation relevant to the target.

Common Mistake

A common mistake is to see a different average effect in a later period and immediately attribute it to a different patient mix, then reweight the source estimate and treat the result as definitive. That reasoning silently assumes that the treatment, outcome, and conditional causal effects are stable. A corrected analysis first checks those features, then asks whether covariate composition accounts for the difference. Standardization addresses the distribution of measured covariates; it does not harmonize interventions, repair outcome measurement, or demonstrate that conditional effects are unchanged.

Key takeaway: A change in the target distribution of effect modifiers can change the target average effect even when conditional effects are stable. But transporting across time or settings also requires attention to treatment versions, outcome measurement, and conditional effect stability. These are separate questions and should not be collapsed into a single adjustment step.

Check Your Understanding

Use the definitions and distinctions above to answer the following questions.

  1. Why can the target average treatment effect differ from the source average even if every conditional effect is stable?
  2. A later-period protocol adds scheduled monitoring to a treatment used in the source period. What problem does this create for interpreting a transported effect?
  3. In the outcome-detection example, did the underlying treatment effect change when the recorded risk difference moved closer to zero? Explain.
  4. If the target and source have the same distribution of measured effect modifiers but different conditional effects in one stratum, which transport assumption is at issue?
  5. What information does target-period covariate data provide for standardization, and what does it not establish?
Need help applying this method to your own data?

Our team supports biostatistics, statistical programming and pharmacometric analyses. Send us a request and we will reply by email.

Request an analysis
Clinical Biostats