Why Discontinuation Changes the Question
Participants may stop assigned treatment because of side effects, lack of perceived benefit, burden, or other reasons. Others may take less than the assigned dose or interrupt treatment without formally discontinuing. These behaviors matter clinically, but they do not determine the estimand by themselves. The estimand must state what treatment effect is of interest when discontinuation or nonadherence occurs.
A treatment-policy estimand asks about the effect of assignment to each treatment condition, allowing discontinuation and subsequent behavior to occur as they would in the trial. A hypothetical estimand instead asks what the contrast would be under a specified hypothetical condition, such as all participants continuing their assigned treatment through the outcome time. These answer different questions: one describes an assignment strategy in a setting where treatment may be stopped; the other describes a counterfactual strategy that prevents or otherwise specifies what happens after discontinuation.
For example, let \(Y(a)\) represent the endpoint that would be observed under assignment to \(a\), with discontinuation and subsequent care allowed to unfold. Let \(Y(a,g)\) represent the endpoint under assignment to \(a\) and a specified adherence strategy \(g\). If the target summary is a population mean difference, the corresponding contrasts are \(E[Y(1)]-E[Y(0)]\) for treatment policy and \(E[Y(1,g)]-E[Y(0,g)]\) for the hypothetical strategy. The population, treatment conditions, endpoint, and summary must be held clear in both statements.
Adherence is not always a single event. It can involve dose reductions, missed doses, temporary interruptions, and permanent discontinuation. A hypothetical question such as “what if nobody discontinued?” does not by itself specify what happens to missed doses or dose reductions. The intervention defining \(g\) should say which deviations are prevented or permitted and how treatment is managed. A contrast that conditions on observed adherence is different again: it compares a selected post-randomization subset, rather than necessarily estimating either the treatment-policy or hypothetical effect.
Treatment-Policy Estimands Include Outcomes After Stopping
For a treatment-policy estimand, discontinuation does not remove a participant from the target population or change their assigned treatment condition. The endpoint is collected at the planned time whether the participant is still taking treatment or has stopped. For a mean contrast, if \(D\) indicates discontinuation before the assessment, the arm-specific mean can be written as a weighted average:
This identity follows by partitioning the population into those who discontinue and those who do not. It makes explicit why the mean among participants still taking treatment is not, by itself, the treatment-policy mean. The outcomes after stopping are part of the treatment-policy endpoint, not an optional addition.
Worked Example: Calculating a Treatment-Policy Contrast
In a hypothetical trial, 100 participants are assigned to each of two treatments. At week 12, the endpoint is a symptom-improvement score in points, with higher scores better. In the experimental arm, 70 participants have not discontinued and have a mean score of 8.5 points; 30 have discontinued and have a mean week-12 score of 5.0 points. In the comparator arm, 90 have not discontinued and average 6.5 points; 10 have discontinued and average 4.5 points. Assume the week-12 score is collected for everyone.
The treatment-policy mean for the experimental assignment is
For comparator assignment, it is
Thus the estimated treatment-policy mean difference is \(7.45-6.30=1.15\) points in favor of experimental assignment. This calculation uses each participant’s outcome at week 12, including outcomes after discontinuation. It describes the contrast between the two assigned treatment strategies as implemented in this hypothetical trial; it does not isolate the pharmacologic effect of uninterrupted exposure.
This contrast can be useful when the practical decision concerns which treatment to assign in a setting where stopping is possible. It does not mean discontinuation is irrelevant: discontinuation patterns can influence the observed outcome distributions and are important to report. Rather, the treatment-policy question deliberately includes the consequences of those patterns.
Hypothetical Effects Require Counterfactual Outcomes
A hypothetical estimand asks about outcomes in a situation that may not have occurred for some participants. For a no-discontinuation question, the outcome of a participant who actually stopped is not observed under the hypothetical condition of continuing treatment. It must be inferred from a model or other identifying assumptions. The hypothetical contrast is not obtained simply by deleting discontinuers: the remaining participants may differ systematically from those who stopped.
A causal interpretation requires, at minimum, a well-defined hypothetical strategy, consistency between that strategy and observed treatment histories when they coincide, adequate representation of the relevant adherence histories (positivity), and control of common causes of adherence and outcome. In a longitudinal setting, this includes time-varying factors that both predict later adherence and are affected by prior treatment. Randomization protects the original assignment comparison, but it does not randomize who discontinues.
Worked Example: Why an Adherer-Only Contrast Can Mislead
Consider hypothetical groups of 100 participants each, split at baseline into 50 with favorable prognosis and 50 with unfavorable prognosis. Suppose, for illustration, the week-12 score under either treatment is 8 points for favorable-prognosis participants and 4 points for unfavorable-prognosis participants; thus there is no treatment effect within either prognosis category. In the experimental group, 40 favorable-prognosis and 20 unfavorable-prognosis participants remain adherent. The other 10 favorable-prognosis and 30 unfavorable-prognosis participants discontinue. In the comparator group, 25 of each prognosis category remain adherent, and the other 25 of each category discontinue.
Among experimental-arm adherers, the mean is
Among comparator adherers, the mean is
The adherer-only difference is about \(0.67\) points even though the stipulated treatment effect is zero in each prognosis category. The reason is composition: 40 of 60 experimental adherers have favorable prognosis, compared with 25 of 50 comparator adherers. The adherer comparison therefore mixes treatment and prognosis differences. It is neither the treatment-policy contrast nor, without additional assumptions, the hypothetical effect under universal adherence.
This example also illustrates why “per protocol” is not a complete estimand definition. It can refer to different populations, adherence rules, and analyses. If the intended question is the effect under sustained treatment, define the sustained-treatment strategy and estimate its counterfactual outcomes; do not assume that selecting observed adherers recreates the population that would have adhered under both assignments.
Standardizing a Hypothetical Contrast
One approach to a hypothetical mean contrast is to predict outcomes under the specified sustained-treatment strategy for participants in each treatment arm, then average those predictions over the target population. The calculation below is illustrative; the predicted means are not identified merely by the arithmetic. They require a credible model and assumptions that connect observed data to the unobserved hypothetical outcomes.
Worked Example: A Hypothetical No-Discontinuation Contrast
Suppose a hypothetical trial has 100 participants assigned to each treatment, with 50 in each baseline prognosis category per arm. A prespecified model predicts week-12 symptom-improvement means under continued assigned treatment of 9 points for favorable-prognosis participants and 5 points for unfavorable-prognosis participants in the experimental arm. For comparator assignment, the corresponding predicted means are 7.5 and 4.5 points. The target is the mean contrast under a strategy that maintains assigned treatment through week 12 for everyone.
Standardize the predicted outcomes to the stated target population, which has equal numbers in the two prognosis categories. The experimental-arm hypothetical mean is
The comparator hypothetical mean is
The hypothetical mean difference is \(7.00-6.00=1.00\) point in favor of experimental treatment under the specified continued-treatment strategy. It is not an observed adherer contrast. It is a model-based contrast for a defined counterfactual situation, and its credibility depends on whether the model and identifying assumptions adequately represent what would happen under that strategy.
A Bayesian analysis can represent uncertainty about model parameters and the resulting estimand, but a posterior distribution cannot repair a poorly defined hypothetical strategy or unsupported identification assumptions. The target question and assumptions must be stated before interpreting model output.
Choosing Between the Questions
| Feature | Treatment-policy strategy | Hypothetical strategy |
|---|---|---|
| Clinical question | What is the effect of assignment when discontinuation and subsequent behavior can occur? | What would the effect be under a specified situation, such as continued assigned treatment? |
| Outcome after discontinuation | Included in the endpoint at the planned time. | Not the target outcome if it differs from the specified hypothetical condition; it informs estimation only under modeling assumptions. |
| Key interpretive feature | Reflects assignment strategies in the trial context, including discontinuation patterns. | Reflects a counterfactual strategy that may require extrapolation beyond observed treatment histories. |
| Common failure | Missing post-discontinuation outcomes can prevent direct estimation of the intended contrast. | Unmeasured predictors, poor overlap, or an underspecified intervention can undermine identification. |
A practical choice begins with the decision the result should inform. If clinicians or patients want to know the consequences of starting one treatment rather than another while real-world stopping remains possible, treatment policy may align with that question. If the decision concerns benefit under continued exposure, a hypothetical estimand may be relevant, but it needs a clear intervention and stronger assumptions. Neither strategy is universally preferable, and they need not produce the same numerical contrast.
Diagnostics, Limitations, and a Common Pitfall
For a treatment-policy analysis, check whether endpoint follow-up continued after discontinuation and whether missing assessments differ by treatment arm or by reasons for stopping. If outcomes are unavailable after stopping, the intended treatment-policy estimand remains conceptually defined, but the observed data do not directly supply all its outcomes. Missing-data assumptions and methods then matter; missingness should not be silently equated with discontinuation.
For a hypothetical analysis, inspect adherence histories over time, the number of participants following relevant histories, and overlap in prognostic factors between those who continue and those who stop. Examine whether the model captures reasons for discontinuation and predictors of the endpoint, including time-varying predictors when relevant. Sensitivity analyses should assess how conclusions change under plausible alternative assumptions about unobserved outcomes. Sparse support for sustained treatment or strong model dependence is a warning that the counterfactual contrast may be weakly identified.
- Common pitfall: “Compare only those who adhered, because that is the effect of taking treatment.” This conditions on a post-randomization behavior that may be related to prognosis and treatment response, and can destroy the comparability created by randomization.
- Correction: State whether the target is the assignment effect including discontinuation or a counterfactual effect under a defined adherence strategy. For the latter, account for measured causes of adherence and outcome, justify the assumptions, and do not treat observed adherers as automatically representing the hypothetical population.
- Another pitfall: calling any outcome after a participant stops “nonadherent” and excluding it. Under a treatment-policy strategy, that outcome remains part of the endpoint by design.
The key limitation is that a hypothetical effect is not directly observed for participants whose treatment histories depart from the specified strategy. Its interpretation can be clinically valuable, but it is conditional on a well-defined intervention and assumptions that data alone may not verify. Conversely, a treatment-policy effect is tied to the discontinuation and follow-up context in which it is estimated; changing those patterns can change the contrast’s practical relevance.
Check Your Understanding
Answer each question, then compare with the solutions below.
- A trial measures the week-12 outcome for everyone, including participants who stopped assigned treatment. Which estimand strategy naturally includes these outcomes, and what question does it answer?
- In a hypothetical no-discontinuation analysis, why is the mean among observed adherers not automatically the target mean?
- A hypothetical trial has 80 participants per arm. Under a specified continued-treatment strategy, predicted means are 10 points for 40 favorable-prognosis and 6 points for 40 unfavorable-prognosis participants in the experimental arm. The corresponding comparator predictions are 8 and 5 points. Calculate the hypothetical arm means and their difference.
- Name two assumptions or data features that should be examined when estimating a hypothetical contrast.
Solutions. (1) Treatment policy includes outcomes after stopping and answers the effect of assignment while discontinuation and subsequent behavior occur as in the trial. (2) Adherence is post-randomization and may select participants by prognosis or other outcome predictors; the selected group need not represent the target population under universal continuation. (3) The experimental mean is \([40(10)+40(6)]/80=8.00\) points. The comparator mean is \([40(8)+40(5)]/80=6.50\) points. The difference is \(8.00-6.50=1.50\) points in favor of experimental treatment under the specified strategy. (4) Appropriate considerations include a precisely defined hypothetical intervention, measured common causes of adherence and outcome, sufficient overlap in adherence histories (positivity), and adequate modeling of outcome predictors.