Why Follow-Up Must Not Automatically End With Treatment
A participant may stop assigned treatment before the planned endpoint because of adverse effects, lack of benefit, inconvenience, or a decision to start another therapy. That event changes the participant’s treatment exposure; it does not, by itself, answer the endpoint question. If the trial’s target is a treatment-policy estimand, outcomes after discontinuation remain relevant: the target concerns the effect of assignment to each treatment strategy, including the consequences of treatment discontinuation and subsequent care as they occur in the trial context.
The practical implication is important. A trial can permit treatment discontinuation while continuing to collect the planned endpoint. For example, a participant who stops a study drug at week 12 may still provide a week-24 symptom score. Ending study-drug exposure and ending outcome follow-up are separate decisions. If follow-up also stops, the endpoint is missing for that participant, and the data available for estimating the target contrast change.
Let \(A\) denote randomized assignment, with \(A=1\) for the experimental strategy and \(A=0\) for the comparator. Let \(Y^a\) denote the endpoint that would be observed under assignment to strategy \(a\), with discontinuation and subsequent care handled as specified by the treatment-policy question. For a continuous endpoint, one possible target is \(E[Y^1]-E[Y^0]\). Let \(R=1\) indicate that the endpoint is observed and \(R=0\) that it is missing. Continuing follow-up aims to make \(R=1\) for more participants, including those who discontinue. It does not redefine \(A\), and it does not turn the observed outcome into an outcome under uninterrupted treatment.
What Continued Collection Preserves
The target population and endpoint must be stated before considering collection procedures. Suppose the endpoint is change in pain score at week 24, measured in points, and smaller values are better. Under a treatment-policy question, a participant assigned to the experimental strategy who stops the study drug at week 10 can still have a week-24 pain score. That value reflects what happened under the assigned strategy in the trial, including discontinuation and any subsequent care that occurred. It is part of the outcome relevant to that strategy.
This is different from asking what the week-24 outcome would have been if every participant had continued assigned treatment. That is a different question: it excludes the consequences of discontinuation in a different way. The distinction is not a technical detail about data handling; it is a distinction between target questions. The earlier material on estimands and intercurrent events establishes why the strategy for handling discontinuation must be explicit.
Continued collection also protects against a common but invalid shortcut: analyzing only participants who remain on treatment. Staying on treatment is not generally a randomized condition. Participants who discontinue may differ in prognosis, tolerability, or response from those who continue. Consequently, the observed outcomes among treatment adherers need not represent the outcomes for everyone assigned to that strategy.
Worked Example: Including Outcomes From Participants Who Discontinue
In a hypothetical randomized trial, 50 participants are assigned to each of two strategies. The continuous week-24 symptom score is measured in points, with lower scores better. In the experimental arm, 8 participants discontinue assigned treatment; in the comparator arm, 5 discontinue. Follow-up obtains the week-24 score for every participant. The total score is 320 points in the experimental arm and 355 points in the comparator arm. Calculate the treatment-policy contrast in mean scores.
Because all 50 participants in each arm have an endpoint, the arm means include participants who discontinued:
The estimated contrast, experimental minus comparator, is
In this hypothetical sample, the experimental-strategy mean is 0.70 points lower at week 24. The calculation describes the randomized strategies in this trial context, including outcomes after discontinuation; it does not show what the scores would have been had everyone continued assigned treatment. The assumption relevant to this calculation is that week-24 scores were ascertained for all randomized participants and measured comparably in both arms. Randomization supports the arm comparison, but does not make the sample contrast a certainty about the population effect.
How Stopping Follow-Up Changes the Available Data
If outcome collection ends when treatment ends, then some participants have \(R=0\). A complete-case mean uses only those with observed outcomes. Even if assignment was randomized, conditioning on \(R=1\) can undo the balance created by randomization when the probability of being observed is related to prognosis or the outcome itself.
A short identity shows what is lost. Within an assigned arm, write \(q=P(R=1)\), and suppose \(Y\) is defined for everyone in the target group. The population mean separates into observed and unobserved parts:
This follows by partitioning the expectation over the two mutually exclusive events \(R=1\) and \(R=0\): \(E[Y]=E[YI(R=1)]+E[YI(R=0)]\), where \(I(\cdot)\) is an indicator. Each term equals its conditional mean multiplied by the probability of its event. A complete-case mean estimates \(E[Y\mid R=1]\), not automatically \(E[Y]\). They coincide if \(q=1\), or under conditions that make the observed and unobserved conditional means equal. When outcomes are missing, that equality cannot be established merely by looking at the observed outcomes.
Worked Example: Complete Cases Can Change the Observed Contrast
Consider a hypothetical binary endpoint: whether a participant has improved by week 24. In the experimental arm, 45 of 80 participants with observed outcomes improved; 20 of the 100 randomized participants have no week-24 outcome. In the comparator arm, 40 of 90 participants with observed outcomes improved; 10 of the 100 randomized participants have no week-24 outcome. Calculate the complete-case contrast and the range of possible full-arm risk differences without assumptions about the missing outcomes.
The complete-case risks are \(45/80=0.5625\) and \(40/90=0.4444\). Thus the observed complete-case risk difference, experimental minus comparator, is
or about 11.8 percentage points. This is a contrast among participants with observed outcomes, not necessarily the treatment-policy contrast among all randomized participants.
For the full experimental arm, between 45 and 65 participants could have improved: 45 observed improvements, plus anywhere from 0 to 20 improvements among those missing. Its risk is therefore between \(45/100=0.45\) and \(65/100=0.65\). For the comparator arm, the corresponding range is \(40/100=0.40\) to \(50/100=0.50\). The full-arm risk difference can therefore range from \(0.45-0.50=-0.05\) to \(0.65-0.40=0.25\), or from -5 to 25 percentage points. These are worst-case bounds, not a probability interval. They show that the observed data alone do not determine the full-arm contrast in this example.
The example illustrates why a large observed complete-case contrast is not enough to establish the target result. Continued collection would replace some unknown outcomes with observed outcomes and could narrow the range of uncertainty about the arm-level risks. It would not guarantee a particular treatment effect, and it would not eliminate every possible source of missing data.
Collection Plans and Practical Checks
A collection plan should specify how the trial will seek the endpoint after assigned treatment stops. The plan may use scheduled visits, remote questionnaires, record review, or other prespecified methods, depending on the endpoint and consent. Collection should preserve the endpoint definition and timing as far as possible. If the endpoint is a week-24 laboratory value, a week-16 value is not automatically a substitute for the week-24 value.
Record whether treatment was stopped and whether the planned endpoint was obtained. Discontinuation is not itself a missing endpoint.
Use the prespecified collection method and time point even when assigned treatment has ended, subject to participant consent and safety requirements.
Distinguish withdrawal from treatment, withdrawal from follow-up, missed visits, administrative closure, and other reasons. These patterns may have different implications for interpreting the data.
Report how many participants stopped treatment, how many still provided the endpoint, and how many had missing endpoint data. This makes clear whether stopping treatment commonly coincided with loss of follow-up.
Useful diagnostics include comparing endpoint completion rates across randomized arms and across participants who did and did not discontinue; checking whether missingness clusters by time or reason; and confirming that the endpoint was measured at the intended time with the intended instrument. Such checks describe the data-collection process. Similar completion rates do not prove that missing outcomes are harmless, and different completion rates do not by themselves determine the direction of any bias.
How This Differs From Other Strategies
Continued collection supports a treatment-policy question, but it is not the only way to formulate a treatment effect. The alternative depends on the scientific question, not simply on which analysis is easiest. The table gives a concise contrast.
| Approach | Question represented | Role of post-discontinuation outcomes |
|---|---|---|
| Treatment-policy strategy | What is the effect of assignment to each strategy in the trial context? | Collected outcomes contribute, including outcomes after discontinuation. |
| Hypothetical strategy | What would the effect be under a specified scenario in which an intercurrent event did not occur? | Observed post-discontinuation outcomes may not directly represent that scenario. |
| On-treatment summary | What outcomes are observed while participants remain on assigned treatment? | Outcomes after stopping may fall outside the summary, so the target differs. |
These approaches are not interchangeable. Stopping follow-up does not, by itself, turn a treatment-policy question into a valid hypothetical or on-treatment question. A new question would need to be defined, and the data and assumptions would need to support it. Likewise, continued collection does not force one particular statistical model or guarantee that all endpoint values will be observed.
Limitations and Common Pitfalls
Continued outcome collection reduces missingness due to treatment discontinuation, but cannot prevent every missing endpoint. Participants may withdraw consent, become too ill to complete an assessment, or be unreachable. Some endpoints may require in-person measurement. Data collection after discontinuation must also respect safety, privacy, and participant burden. The goal is to preserve relevant outcome information, not to pressure participants to remain in a study.
- Mistake: “Once treatment stops, the participant no longer belongs in the treatment-policy analysis.” Treatment discontinuation changes exposure, not randomized assignment. If the endpoint is obtained, it remains relevant to the treatment-policy question.
- Mistake: “An observed-only analysis is automatically valid because treatment was randomized.” Randomization applies to assignment, but analyzing only \(R=1\) conditions on outcome observation. The observed participants may not represent everyone randomized.
- Mistake: “Collecting outcomes after discontinuation estimates the effect of continuous treatment.” It does not. The collected outcomes reflect the assigned strategy and the trial context after discontinuation.
- Mistake: “Equal missing-data percentages mean no concern.” Equal percentages do not establish that missing outcomes resemble observed ones, or that complete-case comparisons target the intended estimand.
Check Your Understanding
Answer each question, then compare with the solutions below.
- A participant assigned to an experimental treatment stops it at week 8 but completes the planned week-20 endpoint. Should the endpoint be excluded solely because treatment stopped? Explain.
- In one randomized arm, 75% of participants have observed outcomes. The mean among observed participants is 10 units and the mean among participants with missing outcomes is 14 units. Find the full-arm mean.
- Why does randomization alone not guarantee that the complete-case contrast is the treatment-policy contrast?
- A trial continues collecting outcomes after discontinuation, but some participants still miss the endpoint. What has the collection plan achieved, and what has it not established?
Solutions. (1) No. Under a treatment-policy question, the endpoint after discontinuation remains relevant to the randomized assignment. (2) Using the partition identity, \(0.75(10)+0.25(14)=7.5+3.5=11\) units. (3) Complete-case analysis conditions on having an observed outcome; if observation is related to prognosis or outcome, the observed participants may differ from the full randomized arm. (4) The plan has retained outcomes for participants who discontinue and may have reduced missingness. It has not made every endpoint observed or proved that remaining missing outcomes are ignorable.