Trial Conduct Differences as Sources of Nontransportability

Why Trial Conduct Matters for Extrapolation

A trial effect is not determined by the drug and population alone. It is also shaped by how consistently participants take treatment, what other therapies they receive, when follow-up ends, whether rescue medication is available, and how outcomes are measured. If these features differ between a source trial and the setting to which its result will be applied, the trial’s estimate may not represent the effect expected under the target setting’s care.

This does not mean that every conduct difference invalidates a trial. Randomization can support a valid comparison of assigned treatments within a trial, even when adherence is imperfect or rescue medication is used. The question for extrapolation is more specific: does the effect estimated under the source trial’s conduct correspond to the treatment strategies and outcome definition relevant to the target setting?

The introductory material on endpoint harmonization establishes why outcomes must have comparable definitions before their effects are compared. Here the emphasis is on additional conduct differences and on the adjustments or qualifications needed when the trials do not implement the same strategies. The treatment effect should always be tied to a clearly stated outcome, time horizon, population, and treatment strategy.

Define the Effect Before Adjusting for Conduct

Let \(Y=1\) indicate an unfavorable binary outcome by a specified time, and let \(A=1\) denote assignment to treatment and \(A=0\) assignment to control. Define the treatment benefit as a risk reduction: the control risk minus the treatment risk. A positive value therefore favors treatment. For a particular population and conduct regime \(g\), write the effect as:

This is an effect under the specified conduct regime, not an abstract effect independent of implementation. In a randomized trial, an intention-to-treat comparison generally estimates the effect of assignment under the trial’s actual adherence, rescue, and follow-up practices, subject to the trial’s outcome definition and missing-data approach. It does not automatically estimate the effect of perfect adherence or of a different rescue policy.

For a simple binary outcome, a target-setting effect can be represented by averaging conditional effects across target covariate strata \(x\), provided those strata capture the relevant differences and the conditions below hold:

Here \(P_T(X=x)\) is the target population’s stratum proportion, and \(g_T\) is the target conduct regime. This expression is useful only if the conditional risks for that population and regime are identified or defensibly predicted. Substituting source-trial risks is justified only when the relevant conditional effects (and any needed outcome risks) are transportable to the target regime.

How Conduct Differences Change the Effect

Adherence can reduce the separation in treatment actually received between randomized arms. If treatment benefit depends on receiving treatment, a setting with lower adherence may have a smaller benefit under assignment, even if the effect among people who take the drug as intended is unchanged. However, observed adherence is post-randomization: participants who adhere may differ in prognosis, health behavior, or tolerability. Comparing adherent participants with nonadherent participants, or simply adjusting for observed adherence, can therefore destroy the protection provided by randomization.

Concomitant therapy can also alter the treatment effect. A background therapy might enhance the study drug’s effect, reduce its incremental benefit, or mark a clinical subgroup with a different response. A difference in the prevalence of such therapy can change the average effect. If concomitant therapy is initiated after randomization, however, it may be influenced by treatment or evolving health status. Its role must be analyzed with an appropriate causal estimand and method rather than treated automatically as a baseline confounder.

Rescue medication and follow-up rules define what happens after inadequate response or worsening. If rescue use is counted as treatment failure, the endpoint measures a composite strategy outcome. If outcomes after rescue are instead recorded, the endpoint reflects symptoms under the observed rescue policy. Neither is inherently wrong, but their effects answer different questions. Similarly, follow-up duration affects the opportunity to observe events, and ascertainment procedures can change which events are recorded or classified.

Adjusting for Differences in the Mix of Background Therapy

When a relevant modifier is measured before treatment assignment, conditional effects can be averaged using the target population’s distribution. This is standardization: the conditional effect in each stratum is weighted by the proportion of the target population in that stratum. The interpretation depends on the conditional effects being applicable to the target setting and on having sufficient evidence in every target stratum. For a post-randomization co-intervention, this same calculation is not automatically valid.

Rescue, Follow-Up, and Endpoint Ascertainment

A conduct adjustment must preserve the scientific question. Consider two common strategies for rescue medication: count rescue use as failure, or measure symptoms at the scheduled endpoint after rescue is allowed. The first estimates the effect on a composite involving both worsening and rescue use. The second estimates symptoms under the rescue practices that occurred. These are not interchangeable endpoints, even if both are described informally as “response.”

Ascertainment differences can change measured event risks even when the underlying clinical state is the same. A simple correction is possible only under explicit measurement assumptions. For example, if a binary endpoint has known sensitivity \(Se\) and specificity \(Sp\) that apply equally in both arms, then the observed positive fraction \(p_{\text{obs}}\) and true event risk \(p\) satisfy \(p_{\text{obs}}=Se\,p+(1-Sp)(1-p)\). Solving gives \(p=(p_{\text{obs}}+Sp-1)/(Se+Sp-1)\), provided \(Se+Sp>1\).

Diagnostics and Practical Choices

Before transporting an estimate, describe conduct differences concretely rather than labeling a trial simply “well conducted” or “poorly conducted.” Compare adherence over time, treatment discontinuations and crossovers, permitted and actual concomitant therapies, rescue eligibility and use, visit timing, length of follow-up, missing outcome data, and endpoint definitions and adjudication. Check whether these differences are associated with observed treatment effects or with the patient characteristics that differ between settings.

For an adjustment to be credible, verify that each target-relevant stratum has source data, that the definition of the stratum is consistent, and that outcome measurement is comparable. Large weights or strata with few observations signal that the extrapolation relies heavily on limited information. If treatment effects are estimated only among a selected subset, investigate how selection occurred; an adherent-only analysis can be biased even when adherence is measured accurately.

A common mistake is to say, “The treatment effect is smaller in the target region because adherence is lower, so we will adjust for adherence by comparing only adherent participants.” This reasoning is not justified: adherence occurs after randomization and can depend on prognosis, tolerability, or early response. The correction is to define the intended estimand first, then use an analysis appropriate to that estimand and the available longitudinal data; if its assumptions cannot be supported, report the randomized assignment effect under trial conduct and qualify its applicability.

Conduct differences matter because they can change either the treatment received, the co-interventions, the time or circumstances in which outcomes arise, or the probability that outcomes are observed and classified. Some differences can be addressed by harmonization or adjustment. Others require evidence that is not available from the source trial alone. A transparent extrapolation states which conduct regime the estimate represents, what differs in the target setting, what assumptions bridge that gap, and where uncertainty remains.