Interim Impressions Can Change a Trial Without Changing Its Design
A trial can continue under the same protocol, allocation, and analysis plan while the people in it respond to emerging impressions about treatment. A participant who believes an intervention is helping may take it more consistently. Someone who thinks the trial is futile may stop attending visits. A participant who suspects toxicity may pay closer attention to symptoms and report them more often. These responses can begin before a committee recommends any formal design change.
The distinction matters. A formal adaptation is a planned change to trial conduct or design, made under specified rules and governance. A behavior change prompted by an interim impression is not necessarily an adaptation: the protocol may be unchanged, while adherence, outcome availability, or reporting changes. The observed data can therefore shift even though the design documents do not.
The pathway can be summarized as impression, behavior, and then data: an interim trend is noticed or inferred; that impression changes what someone does; and the changed behavior affects outcomes, measurements, or whether measurements are available. This is one way operational bias can arise. It does not require deliberate misconduct, and it does not mean every observed difference is biased. The size and direction of any effect depend on what changed and how that change relates to the trial outcomes.
From Impression to Observed Evidence
Let \(A\) denote randomized assignment, \(S\) a perceived signal, \(M\) a post-randomization behavior, and \(Y\) an outcome. A useful conceptual pathway is \(S \rightarrow M \rightarrow Y\). If behavior also affects whether \(Y\) is recorded, the pathway can instead lead to outcome availability. Randomization helps balance characteristics measured before assignment, but it does not ensure that post-randomization behavior remains the same across groups when people respond differently to emerging impressions.
The distinction between a real treatment effect and a behavior-related contribution can be made explicit with a simple illustrative model. Suppose the outcome is measured in units where higher values are better, and, for demonstration, assume \(Y = \tau A + \beta M + \varepsilon\). Here \(\tau\) is the treatment contrast when behavior is held fixed, \(\beta\) is the outcome change per unit of behavior, and \(\varepsilon\) collects other influences. Assume the model is additive, the same \(\beta\) applies in both groups, and the mean of \(\varepsilon\) is equal across randomized groups. Then:
This identity is a justification within the stated model: subtract the group mean under \(A=0\) from the group mean under \(A=1\); the treatment term contributes \(\tau\), the behavior term contributes \(\beta\) times the mean behavior difference, and equal mean errors cancel. It shows why an observed contrast can include both a treatment contribution and a behavior-related contribution. It does not establish that a particular trial follows this model, nor that the behavior difference was caused by an interim impression. In practice, treatment and behavior may interact, behavior may have several dimensions, and other post-randomization events may also matter.
Behavior can also change which outcomes are observed. If attendance or continued participation depends on a person’s response to an interim impression and on their likely outcome, the observed participants may no longer represent all randomized participants. The direction of this selection effect is not determined just by knowing that follow-up decreased: it depends on whose outcomes are missing and what those outcomes would have been.
Worked Example: Perceived Efficacy and Adherence
Hypothetical example. In a randomized trial, higher scores mean greater improvement. Suppose the treatment’s mean effect at the same adherence level is assumed to be 2.00 improvement points. For illustration, suppose taking assigned treatment consistently rather than inconsistently is associated with 1.50 additional points of improvement, with the same additive effect in each group. After a perceived favorable interim trend, the proportion taking treatment consistently is 0.90 in the treatment group and 0.80 in the control group.
Model and assumptions. Use the additive model above, treating \(M=1\) for consistent use and \(M=0\) otherwise. This calculation assumes the stated treatment and adherence effects, equal mean errors between groups, and no treatment-by-adherence interaction. These are simplifying assumptions for the example, not facts established by the observed adherence rates.
The mean adherence difference is \(0.90-0.80=0.10\). The behavior-related contribution is:
The model-implied observed contrast is therefore:
Under these assumptions, 0.150 points of the contrast arise from the difference in consistent use, in addition to the 2.00-point treatment contrast at a common adherence level. The calculation does not show that the perceived trend caused the adherence difference, or that the model is appropriate for real trial data. It does show how behavior can contribute to an observed group difference before any formal adaptation occurs.
When Perceived Futility Changes Outcome Availability
A perceived lack of benefit can affect whether participants continue visits or complete outcome assessments. This matters even when the assigned treatment and formal follow-up schedule are unchanged. An analysis of only participants with recorded outcomes can then compare selected subsets rather than all randomized participants. The observed complete-case contrast is not automatically an estimate of the contrast that would have been seen if outcomes were available for everyone.
Worked Example: Perceived Futility and Missing Follow-Up
Hypothetical example. Suppose 100 participants were randomized to each group. After an interim impression of futility, fewer participants in the treatment group complete follow-up. For a transparent illustration, imagine the outcomes that would have been measured for all 100 people in each group are known: the treatment group has 60 observed outcomes averaging 5 points and 40 unobserved outcomes averaging 1 point; the control group has 90 observed outcomes averaging 4 points and 10 unobserved outcomes averaging 2 points. Higher values are better. In an actual trial, the averages among those with missing outcomes would not generally be known.
Among participants with observed outcomes, the treatment-minus-control contrast is \(5-4=1.00\) point. If all outcomes in this hypothetical example were included, the treatment-group mean would be:
The corresponding control-group mean would be:
The all-participant contrast in this constructed data is \(3.40-3.80=-0.40\) points, whereas the complete-case contrast is \(1.00\) point. The reversal occurs because the assumed outcomes among those missing differ from the observed outcomes. In a real trial, those missing values cannot simply be filled in with the example’s values; the example demonstrates why the amount and pattern of missingness, and the assumptions used to address it, matter.
A change in follow-up is not by itself proof of bias or proof that the interim impression caused the change. It is a warning to examine whether missingness differs by group, time, or participant characteristics, and whether reasons for missingness are related to outcomes. The analysis plan and the estimand—the treatment effect the trial aims to estimate—determine how such missing outcomes should be handled.
When Perceived Toxicity Changes Reporting
A perceived toxicity trend may change how carefully participants monitor symptoms or whether they report a symptom when asked. That response can alter measured event rates even if the underlying occurrence of symptoms is unchanged. Conversely, changes in conduct can also affect actual health outcomes. It is important to distinguish a change in the underlying event from a change in detection or reporting; a count of reported events alone may not separate them.
Worked Example: Perceived Toxicity and Symptom Reports
Hypothetical example. Assume 50 participants in each group have the same underlying symptom incidence: 10 participants per group experience a symptom. After a perceived toxicity signal, suppose 8 of those 10 affected participants in the treatment group report it, compared with 5 of the 10 affected participants in the control group. Assume for this illustration that no unaffected participant reports the symptom.
The reported event proportions are \(8/50=0.160\), or 16.0%, in the treatment group and \(5/50=0.100\), or 10.0%, in the control group. The observed risk difference is:
That is a difference of 6.00 percentage points in reported events. Yet, under the example’s assumptions, the underlying symptom incidence is \(10/50=0.200\), or 20.0%, in each group. The observed difference comes from unequal reporting of symptoms that occurred, not from different underlying incidence. In real data, equal underlying incidence and no false reports would need justification; the reported counts alone cannot establish those conditions.
Recognizing and Managing Behavioral Signals
A useful review asks not only whether an interim trend was available, but also whether trial behavior changed after it became known or suspected. Timing is informative: a change that starts after an impression becomes plausible deserves attention, though timing alone does not prove causation. Compare conduct and outcome availability across randomized groups and over time, and document whether an authorized formal change occurred. Keep the distinction clear between a prespecified decision and informal responses that may happen without one.
The main alternatives are not interchangeable. A formal, prespecified adaptation uses defined rules and governance; it is not the same as a behavior change that arises informally during an unchanged trial. Preventing or limiting access to interim information can reduce opportunities for some responses, but it cannot guarantee that participants or staff will not infer a trend from other information. Statistical adjustment for a measured post-randomization behavior may also change the question being answered: if behavior is part of the pathway from treatment or information to outcome, adjusting for it does not automatically recover the treatment effect of interest. The appropriate analysis depends on the estimand, design, and assumptions about missing data and behavior.
These checks can help identify when trial conduct may be changing:
- Compare timing of changes in adherence, visit completion, or reporting with the timing of possible exposure to an interim impression.
- Examine whether changes differ by randomized group, while remembering that group differences do not by themselves identify the cause.
- Track denominators and outcome availability, not only the number or average of observed outcomes.
- Record reasons for missed visits or changed conduct where feasible, and avoid treating undocumented reasons as known.
- Interpret reported events alongside how they were elicited and recorded; observed reports are not always identical to underlying event incidence.
A common mistake is to reason, “Randomization balances the groups, so any later difference must be caused by treatment.” Randomization balances groups in expectation at assignment; it does not prevent behavior after assignment from changing in response to emerging impressions, nor does it ensure that missing outcomes are comparable. Another mistake is to call every change in conduct an adaptation. The correction is to identify whether a prespecified design rule was formally invoked, then separately investigate informal behavior and its possible effects on the data. Neither an observed difference nor its timing alone proves that an interim impression caused the change.
Check Your Understanding
For each question, first identify the relevant behavior or data pathway, then compare your reasoning with the solutions.
- A perceived efficacy trend is followed by consistent-use rates of 0.85 in one group and 0.75 in the other. Under the worked example’s assumed adherence effect of 1.50 outcome points per unit of \(M\), what behavior-related contribution does the additive model imply?
- Why does randomization not guarantee that a complete-case comparison remains free from selection concerns after follow-up changes?
- Two groups have the same underlying symptom incidence, but participants in one group report symptoms more often. What can and cannot be concluded from a higher reported event proportion alone?
- A trial’s allocation and analysis plan are unchanged, but participants alter adherence after hearing a claim that treatment is effective. Is this necessarily a formal adaptation? Explain.
Solutions. (1) The adherence difference is \(0.85-0.75=0.10\). The implied contribution is \(1.50 \times 0.10=0.150\) outcome points, under the additive model and its assumptions. (2) Randomization balances groups at assignment, but it does not ensure that outcome availability remains unrelated to participants’ potential outcomes after post-randomization behavior changes. A complete-case contrast may compare selected subsets. (3) The observed report proportion is higher; without further information, it does not establish that underlying incidence is higher, because detection or reporting may differ. (4) No. With no prespecified design change invoked, the protocol may remain formally unchanged while behavior responds to an impression. That response can still affect trial data and should be examined separately from formal adaptation.