Why Treatment Versions Matter
An adaptive trial may keep the same arm label while changing what participants actually receive. A dose might be prepared in a new formulation, administered by a different route, or accompanied by revised supportive care after an interim decision. These changes matter for an estimand when they change the intervention whose effect is being described. If “treatment” does not identify a sufficiently well-defined condition, the potential outcome attributed to that treatment may be ambiguous.
The earlier material on treatment conditions and dynamic treatment regimes gives a way to describe interventions as strategies, including rules that depend on trial history. This tutorial focuses on a specific requirement for interpreting those conditions: the consistency assumption. The central question is whether a participant who follows the specified treatment condition has the outcome assigned to that condition, rather than to an unrecorded or changing version of it.
Consistency is not the claim that everyone assigned to an arm receives treatment identically in every operational detail. Nor does it require every participant to have the same outcome. It requires the treatment condition used to define the potential outcome to correspond to what was received. Ordinary variation that is part of the condition as specified—such as dose timing within a defined window—need not create distinct versions. A change that may alter outcomes, however, cannot be silently treated as irrelevant.
Consistency is also distinct from randomization. Randomization supports comparability between assigned groups; it does not make two different formulations or care packages into one treatment condition. Conversely, a treatment condition can be precisely defined even when other assumptions needed to identify its causal effect are not met. These assumptions answer different questions.
When Is a Change a Distinct Version?
A treatment version is a particular implementation of a broadly named treatment condition. Versions may differ in formulation, route, device, administration schedule, or supportive-care package. The relevant question is not whether two implementations have different labels, but whether the estimand’s treatment condition distinguishes them and whether treating them as interchangeable is defensible for the outcome and population of interest.
For example, “receive the study drug” is often too vague if the trial uses both an oral formulation and an infusion with different exposure or burden. By contrast, a new supplier or a revised package insert does not automatically create a distinct version if the product and administration remain the same in outcome-relevant respects. The protocol should specify the intervention features that define the condition, rather than relying on an arm name to carry that meaning.
“Equivalent” should not be asserted just because the trial has not detected a difference. It is a scientific and design judgment, informed by pharmacology, clinical practice, and the outcome being studied. A formulation change might be immaterial to one outcome but alter tolerability or treatment burden relevant to another. Supportive care can also change the treatment condition: if one stage provides a specified prophylactic medicine and another does not, the observed contrast may compare different care packages.
A defined treatment strategy may itself include a rule for selecting versions. For example, a protocol could specify that the formulation depends on a participant’s baseline characteristic or that the site uses one of two approved delivery methods. Then the rule and its conditions—not merely the drug name—define the intervention. This is different from leaving version choice unspecified and later assuming the options are interchangeable.
How Version Changes Affect Stage Comparisons
Suppose the active condition has two versions, \(v=1\) and \(v=2\), and the control condition is unchanged. If stage \(s\) uses version \(v\) for a fraction \(q_s\) of active participants, its average active outcome can be expressed as a mixture of the version-specific means, provided the mixture is defined for the population and assignment process being considered. Let \(\mu_{1v}\) be the mean potential outcome under active version \(v\), and \(\mu_0\) the mean under control.
This expression shows why an arm label alone may not support a stable active condition across stages. If the version proportions \(q_s\) change and the version-specific means differ, the stage-specific contrast can change even when the control condition and target population are otherwise held fixed. A pooled active mean then describes the mixture represented by the pooled data; it does not automatically estimate the effect of either particular version or a fixed intervention strategy.
Worked Example: A Changed Formulation Across Stages
Consider a hypothetical trial measuring a symptom score at 12 weeks, where lower scores are better. Assume, for illustration, that randomization is valid within each stage, the target population is the same, and the control condition is unchanged. Stage 1 uses formulation A for all active participants; Stage 2 uses formulation B for all active participants. The observed stage means are:
Stage 1: A, 18 points (\(n=100\)); control, 22 points (\(n=100\)). Stage 2: B, 16 points (\(n=100\)); control, 22 points (\(n=100\)).
The active-minus-control contrast in Stage 1 is \(18-22=-4\) points. In Stage 2 it is \(16-22=-6\) points. The negative contrasts favor active treatment because lower scores are better. These arithmetic contrasts are clear, but they correspond to different active versions.
If the stages are pooled with equal sample sizes, the pooled active mean is \[ (18+16)/2=17\text{ points}, \] and the pooled control mean is \[ (22+22)/2=22\text{ points}. \] The pooled contrast is \(17-22=-5\) points. It describes a 50:50 mixture of formulations A and B in these data, not necessarily the effect of A, B, or a treatment condition that will use either formulation alone.
If formulation is plausibly outcome-relevant, consistency for one unchanged active condition across both stages is not justified merely by pooling the arm labels. The protocol and estimand should either distinguish A and B or define a treatment strategy that includes the formulation transition. The example’s equal means and sample sizes make the pooled arithmetic transparent; in a real analysis, uncertainty and the estimand’s target population also matter.
A Short Justification for the Mixture Result
The mixture calculation follows by conditioning on version. Let \(V\) denote the version received among active participants in stage \(s\). If the version proportion is \(q_s=P(V=1\mid A=1,s)\), then the law of total expectation gives:
The final equality uses the assumption that the version-specific means represented by \(\mu_{11}\) and \(\mu_{12}\) apply to the stage-specific participants being averaged. Randomization alone does not guarantee this equality across stages if the population or version assignment differs. The formula is a description of a mixture, not a proof that the mixture is a single well-defined intervention. If \(q_s\) changes, the mixture changes unless the version means are equal or the mixture is deliberately part of the treatment strategy.
Worked Example: Supportive Care as Part of the Condition
In a hypothetical randomized trial, the active drug and control are unchanged across two stages, but Stage 2 adds a preventive medicine to the active arm. The outcome is the average number of nausea episodes over four weeks, where lower is better. Suppose Stage 1 has a mean of 8 episodes in the active arm and 10 in control; Stage 2 has 5 in the active-plus-prevention arm and 10 in control. Each stage has 80 participants per arm.
The Stage 1 contrast is \(8-10=-2\) episodes per participant over four weeks. The Stage 2 contrast is \(5-10=-5\) episodes per participant over four weeks. Pooling the active arms gives \[ (8+5)/2=6.5\text{ episodes}, \] while the pooled control mean remains 10 episodes. The pooled contrast is \(6.5-10=-3.5\) episodes per participant over four weeks.
The pooled result is a contrast for a mixture of drug alone and drug plus preventive medicine, not automatically the effect of the drug alone. Because the added supportive care could directly affect nausea, the two active conditions should be considered distinct unless there is a compelling reason, relevant to this outcome, to regard the change as immaterial. A defensible description would name the care package or define the stage-dependent strategy explicitly.
The equal sample sizes and unchanged control condition allow the simple mean calculation shown. They do not establish causal equivalence between the active versions. The outcome difference may reflect the added preventive medicine, other stage differences, or sampling variability; the arithmetic alone does not separate these explanations.
Assessing Consistency in an Adaptive Trial
For each treatment arm and stage, record the intervention actually intended and delivered. Then assess whether the planned changes are part of one defined strategy or create distinct conditions. A practical review can follow these steps:
Describe drug, dose, formulation, route, schedule, and any supportive-care components that are part of the intervention. State which variations are allowed by the condition.
Compare protocol versions and implementation records. Identify changes in product, administration, concomitant care, and rules determining which participants receive each option.
Ask whether each change could alter the outcome, treatment burden, or response process. Do not infer equivalence solely from a non-significant comparison or from common arm terminology.
If versions are distinct, name them separately or define a strategy that includes their selection and timing. Ensure that the contrast being reported refers to that specified condition.
Useful diagnostics include protocol-version tables, pharmacy and administration records, supportive-care logs, and stage-specific summaries of who received each version. Check that the recorded version is available for the participants whose outcomes are analyzed. If version records are missing or assigned inconsistently, the intended distinction may not be recoverable from arm labels alone. A large stage difference in outcomes can flag a question for investigation, but it does not by itself prove a version effect: chance, population changes, or other operational differences could also explain it.
When versions are intentionally assigned by a prespecified rule, document the rule and its inputs. This supports treating the rule as part of a treatment strategy rather than treating the version as an unexplained departure. When the version is chosen informally by site or participant, the observed groups may also differ in ways related to outcomes. A simple comparison of their means then does not isolate the causal effect of the version.
Worked Example: Defining a Stage-Dependent Strategy
Suppose a hypothetical trial gives one of two delivery devices. Under a prespecified protocol, participants enrolled before a planned manufacturing transition receive Device 1, and participants enrolled afterward receive Device 2. For a 30-day functional score, assume the transition was chosen for operational reasons and the study intends to evaluate the entire stage-dependent policy.
If the condition is written only as “receive the active therapy,” it conceals that device assignment depends on enrollment stage. A more precise condition is: “receive the active therapy using Device 1 before the specified transition and Device 2 afterward.” That statement defines a strategy, including its rule for selecting versions.
For illustration, if the mean score is 70 under Device 1 and 74 under Device 2, the difference is \(74-70=4\) score points. This is a descriptive contrast between version-specific means, not proof that the device caused the difference. The strategy’s overall mean depends on the proportion assigned under each period: if 60% receive Device 1 and 40% Device 2, the mixture mean is \[ 0.60(70)+0.40(74)=42+29.6=71.6\text{ score points}. \]
The calculation identifies what the specified version mix would average under those means. It does not make the two devices equivalent, and another trial with a different stage mix could have a different average. If the target is the effect of the stage-dependent policy, that policy must be the treatment condition; if the target is a particular device, the mixture does not answer that question.
Alternatives, Limitations, and Common Pitfalls
There are three common ways to respond when treatment versions differ. First, if the intervention is intended to be a rule-based policy, define the rule and treat the full policy as the condition. Second, if versions are scientifically distinct, define separate treatment conditions and interpret their contrasts separately. Third, if there is a sound scientific basis for treating versions as equivalent for the outcome, retain a common condition while documenting the allowed variation. These are different estimand choices, not interchangeable statistical adjustments.
A version-specific analysis can clarify which condition is being described, but it may be imprecise or confounded if version assignment was not randomized. A pooled analysis can improve precision for the observed mixture, but its interpretation depends on the mixture and may not transport to a different version distribution. Restricting analysis to one version may change the participants represented and does not by itself repair selection into that version. No calculation can make an ambiguously defined intervention precise after the fact.
Consistency is an assumption about linking an observed outcome to a specified potential outcome. It cannot generally be tested from observed data alone, because each participant receives only one treatment condition. Protocol review, subject-matter knowledge, and careful intervention documentation support its plausibility; data may reveal inconsistencies in implementation, but cannot establish that unobserved outcomes under alternate versions would have been identical.
- Pitfall: “The arm name did not change, so the treatment condition stayed the same.” The correction is to compare the delivered formulation, administration, and care package, not just the label.
- Pitfall: “The version comparison was not statistically significant, so the versions are equivalent.” Failure to detect a difference is not evidence that outcomes are identical. Equivalence requires a justified margin and suitable evidence, and may be outcome-specific.
- Pitfall: “Randomization guarantees consistency.” Randomization addresses assignment comparability; it does not remove ambiguity about what the intervention means.
- Pitfall: “The pooled estimate is the effect of the active drug.” If active participants received distinct versions, the pooled estimate may instead describe a particular mixture of drug and implementation conditions.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- A trial changes an active drug from a capsule to an infusion after an interim review. Name two questions to ask before treating both stages as one active condition.
- In a hypothetical trial, Version A has a mean outcome of 12 and Version B a mean of 18. If the pooled active group is 25% A and 75% B, calculate its mean and state what it represents.
- True or false: randomization to an arm labelled “active treatment” establishes that all participants in that arm received the same treatment condition. Explain.
- A protocol explicitly assigns Device 1 before a planned date and Device 2 afterward, defining the target as the effect of that policy. What must the treatment-condition description include?
Solutions. (1) Ask whether route or formulation could affect the outcome or treatment burden, and whether the estimand defines them as one condition or as distinct versions. (2) The pooled mean is \(0.25(12)+0.75(18)=3+13.5=16.5\). It represents the specified 25:75 mixture, not either version alone. (3) False. Randomization concerns assignment; the delivered intervention must still be described, and a shared label does not make different versions identical. (4) Include both devices and the date-based rule selecting between them, so the policy itself is defined as the treatment condition.