Why Adherence Changes Require a Clear Target
In a trial, participants may take less study treatment than planned, stop it, or start another therapy. An adaptive trial may also change how adherence is supported—for example, by introducing additional reminders after an interim review. These changes affect the data and may affect outcomes. They do not, by themselves, require changing the estimand. The key is to state what is being compared and to distinguish that target from the procedures used to carry out the trial.
A treatment-policy estimand is useful when the scientific question concerns the effect of assigning treatment strategies in the presence of the adherence and discontinuation patterns that follow. Outcomes are included regardless of whether participants adhere to, stop, or otherwise depart from their assigned treatment. This is an assignment-based question, not a comparison of participants who happened to adhere.
For a two-arm randomized trial, let \(Z=1\) denote assignment to the experimental treatment strategy and \(Z=0\) assignment to the control strategy. Let \(Y\) be a specified outcome at a fixed assessment time; for example, a symptom score in points, for which lower values are better. A mean-difference treatment-policy estimand is
where \(Y^{z}\) is the outcome that would be observed under assignment to strategy \(z\), allowing adherence, discontinuation, rescue treatment, and other specified intercurrent events to occur as they would under that assignment. The superscript denotes assignment to a strategy, not guaranteed receipt of every planned dose. A negative value favors the experimental strategy when lower scores are better.
A complete estimand also specifies the target population, the treatment conditions, the outcome variable, how relevant intercurrent events are handled, and the population-level summary. Those features must remain clear when adherence procedures change. In particular, “regardless of adherence” does not mean “regardless of what treatment was assigned,” and it does not turn a missing outcome into an observed one.
What Is Preserved—and What May Change
Suppose the assigned treatment conditions, target population, assessment time, outcome, and treatment-policy strategy are held fixed. A change in reminder frequency or adherence counselling can be an operational change rather than a change to the estimand’s definition. The target remains the contrast between outcomes under assignment to the two specified strategies, with post-assignment outcomes included.
However, preservation of the definition does not guarantee preservation of the estimand’s numerical value. If more intensive adherence support changes adherence or outcomes, then the distribution of outcomes under assignment may change. If the support affects the arms differently, the treatment-policy contrast may also change. Thus, a trial must not claim that the effect is numerically unchanged merely because it continues to use the same label for its estimand.
Write \(M\) for the adherence-management policy in place during a trial stage. If that policy can affect outcomes, potential outcomes can be represented as \(Y^{z,m}\), and the assignment contrast under a particular policy is
Changing \(m\) need not change the target definition if management is not part of the specified treatment conditions. But it can change \(\Delta_{\mathrm{TP}}(m)\). To interpret results across stages as estimating the same numerical contrast, the trial needs a defensible basis—for example, management has no relevant effect on the contrast, its effects are common and additive on the outcome scale, or the target explicitly specifies a common management policy or a prespecified distribution of policies. Without such a basis, report which management context applies to each estimate and avoid treating them as automatically interchangeable.
The distinction is especially important if the change in adherence management effectively changes what is assigned. If one arm receives a new adherence intervention while the other does not, and that intervention is part of the treatment condition, the trial may now be comparing different strategies. In that case, the treatment conditions—not just adherence rates—may have changed.
A Marginal View of Adherence
A treatment-policy mean includes the outcomes from all adherence patterns under the assigned strategy. For a simplified illustration, let \(R\) indicate whether a participant adheres to a prespecified pattern, and let \(Y\) be observed for everyone. In arm \(z\), the law of total expectation gives
This identity shows why a change in adherence management can change an arm mean: it can change the proportions in the adherence groups, their outcome means, or both. The treatment-policy contrast subtracts these marginal arm means. It does not hold the adherence distribution fixed, and it does not compare only those with \(R=1\).
Worked Examples
Worked Example: Including Outcomes After Discontinuation
In a hypothetical randomized trial, 100 participants are assigned to each of two treatment strategies. At 12 weeks, the mean symptom score is 8.4 points in the experimental arm and 10.1 points in the control arm. The score is lower when symptoms are less severe. Some participants discontinued assigned treatment, but their 12-week outcomes were collected and included in these means. Calculate and interpret the treatment-policy contrast.
The treatment-policy mean difference is
The estimated mean 12-week symptom score under assignment to the experimental strategy is 1.7 points lower than under assignment to the control strategy, including outcomes after discontinuation. The estimate describes the effect of assignment in this hypothetical trial context; it is not an estimate of the effect among participants who completed treatment. The calculation assumes the stated means include the relevant randomized participants’ outcomes. If some outcomes were not observed, the simple difference would require a missing-data analysis before it could support the same interpretation.
Worked Example: Averaging Over Different Adherence Patterns
In another hypothetical trial, suppose the outcome is observed for everyone. In the experimental arm, 60% adhere to the planned pattern and have a mean score of 7.0 points; the remaining 40% have a mean of 11.0 points. In the control arm, 80% adhere and have a mean of 9.0 points; the remaining 20% have a mean of 12.0 points. Calculate the arm means and treatment-policy contrast.
For the experimental arm, the marginal mean averages both adherence groups:
For the control arm,
Therefore,
The experimental assignment is associated with a 1.0-point lower marginal mean score in this hypothetical trial. The adherence groups’ conditional means are used here only to reconstruct the arm means. They do not identify a treatment effect among adherers: adherence occurs after randomization, and the participants who adhere in one arm need not be comparable to those who adhere in the other.
Worked Example: A Change in Support Across Trial Stages
Consider a hypothetical trial that adds common adherence reminders for both arms in a later stage. The outcome is a 12-week score in points, measured for all participants. In the earlier stage, the experimental and control means are 8.0 and 9.2 points. In the later stage, they are 6.0 and 7.2 points. Calculate the stage-specific contrasts and state what the numbers do—and do not—show.
The earlier-stage contrast is
The later-stage contrast is
In this hypothetical example, the estimated assignment contrast is the same in both stages. The later-stage arm means are each 2.0 points lower, which is consistent with a common additive shift in both arms. The arithmetic does not prove that reminders caused this shift or that future changes in support will leave the contrast unchanged. The comparison supports treating the two estimates as numerically aligned only if the trial’s design and assumptions justify that interpretation—for example, if the support change affects both arm means equally on this outcome scale and does not otherwise alter the treatment contrast.
Assumptions, Diagnostics, and Failure Modes
The treatment-policy strategy determines which outcomes belong in the target; it does not, by itself, guarantee that a particular estimator is valid. A direct difference in randomized-arm means is interpretable as an estimate of the assignment contrast when the treatment conditions are well defined, randomization is implemented as planned, outcomes correspond to the specified variable and assessment time, and the observed data adequately represent the outcomes needed for the target. The randomized comparison also relies on consistency and no interference between participants. When outcomes are missing, the analysis must state and defend additional assumptions or use suitable sensitivity analyses.
Useful checks include confirming that follow-up continues after discontinuation, comparing outcome-completion rates across arms and stages, and documenting adherence and management procedures by stage. Review whether the treatment conditions and assessment schedule remained as specified, whether adherence support differed by arm, and whether stage-specific arm contrasts are compatible with the intended interpretation. Such checks can reveal problems; they cannot establish the assumptions simply by showing similar summary statistics.
A particularly important failure mode is to collect outcomes only while participants are taking treatment. That practice leaves the treatment-policy target unobserved after discontinuation. Another is to exclude participants with missing outcomes without justification: a treatment-policy strategy includes outcomes regardless of intercurrent events, but it does not automatically recover outcomes that were never measured. Missingness needs its own analysis.
| Approach | Question answered | Key distinction |
|---|---|---|
| Treatment-policy strategy | What is the outcome contrast under assignment when outcomes after the event are included? | Adherence and discontinuation do not remove outcomes from the target. |
| Hypothetical strategy | What would the contrast be if a specified event had not occurred? | Requires a counterfactual scenario beyond the outcomes observed after the event. |
| While-on-treatment analysis | What outcomes are observed during treatment? | Restricting to that period is not the same as including post-discontinuation outcomes. |
| Adherer-only comparison | How do observed adherers’ outcomes compare? | Conditioning on post-randomization adherence generally loses the original randomized comparison. |
Common Mistake
Wrong reasoning: “The treatment-policy estimand includes everyone regardless of adherence, so an adherence-management change cannot affect the estimand.”
Correction: Including all relevant outcomes preserves the strategy for handling adherence-related events; it does not make outcomes immune to changes in adherence. A management change can alter the outcome distributions and hence the numerical treatment-policy contrast. State whether the target definition is unchanged, explain why estimates from different management contexts can be combined or compared, and report uncertainty about that explanation. Also avoid treating an adherer-only analysis as a substitute: adherence is post-randomization, so the comparison may be confounded by factors that influence both adherence and outcome.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- A hypothetical trial reports mean outcomes only for participants who remained on assigned treatment. Can those means directly estimate a treatment-policy contrast? Why or why not?
- In a hypothetical trial, both arms’ mean outcomes improve after a common adherence-support change. Does that alone show the treatment-policy contrast is unchanged?
- Explain why the observed mean among adherers in one randomized arm is not generally a causal estimate of the treatment effect among adherers.
- Outcomes are collected after discontinuation, but some are missing. Does the treatment-policy strategy itself resolve the missing outcomes?
Solutions. (1) No. Outcomes after discontinuation are excluded, so the reported means do not directly represent the required treatment-policy means. (2) No. The change may affect the two arms by different amounts; calculate and compare the arm contrasts and justify any cross-stage interpretation. (3) Adherence is post-randomization and can depend on prognostic factors, so conditioning on it can break the comparability created by randomization. (4) No. The strategy specifies which outcomes belong in the target, not how unobserved outcomes are recovered; missing-data assumptions and analyses are still required.