What the Treatment-Policy Estimand Answers
A participant may stop assigned treatment before the outcome visit, for example because of adverse effects, lack of benefit, or a personal decision. The clinical question need not stop with treatment: a trial may still ask how outcomes differ between the strategies of assigning patients to the two randomized groups, with outcomes measured whether or not assigned treatment was discontinued. This is a treatment-policy estimand.
The estimand is about a treatment strategy, not the biological effect of taking every dose. Outcomes after discontinuation remain part of the outcome for the participant’s randomized group. Any subsequent care that occurs in the trial setting is part of the treatment-policy context; the result does not isolate the effect of the assigned drug from all subsequent care.
The estimand must be distinguished from the available data. Discontinuation is an intercurrent event: it occurs after treatment assignment and may affect subsequent outcomes. It does not, by itself, make the outcome undefined. If a participant stops treatment but their scheduled outcome is collected, that outcome contributes directly to the treatment-policy analysis. If the outcome is not collected, there is missing data. The estimand still includes that participant, but inference about their unobserved outcome requires assumptions or additional modeling.
Randomization supports a comparison by assigned group, provided the trial is analyzed in a way that preserves the randomized comparison and the outcome data are handled appropriately. It does not guarantee complete follow-up, correct outcome modeling, or that the result is an effect of adherence. In particular, comparing only participants who continued treatment can break the balance created by randomization: continuation may depend on prognosis, response, or adverse effects.
A Bayesian Model for the Treatment-Policy Contrast
A simple model makes the calculation transparent. Suppose \(Y_{ai}\) is the fixed-time outcome for participant \(i\) assigned to group \(a\), where \(a=0\) denotes control and \(a=1\) denotes the experimental strategy. Suppose outcomes are available for all randomized participants, including those who discontinued assigned treatment, and use the model
For this illustrative conjugate model, assume outcomes within each group are conditionally independent, the specified variance \(\sigma^2\) is known and common to both groups, and the normal model is adequate for the outcome. The priors for the two group means are independent. These assumptions are modeling choices, not consequences of randomization. A different outcome type, appreciable variance differences, or strong covariate structure may call for a different likelihood.
For group \(a\), let \(n_a\) be the number with observed outcomes and \(\bar{Y}_a\) their sample mean. The posterior variance and mean are
The treatment-policy contrast is \(\Delta=\mu_1-\mu_0\). Under the independent normal posteriors, its posterior distribution is normal with mean \(m_1-m_0\) and variance \(v_1+v_0\). Thus, a posterior probability such as \(P(\Delta>0\mid\text{data})\) describes uncertainty about whether the population mean outcome is higher under assignment to strategy 1, given the model and prior.
Why the Posterior Formula Holds
For one group, the likelihood for \(\mu_a\), omitting factors that do not depend on \(\mu_a\), is proportional to \(\exp\{-n_a(\bar{Y}_a-\mu_a)^2/(2\sigma^2)\}\). Multiplying this likelihood by the normal prior gives a kernel whose quadratic terms in \(\mu_a\) are
Completing the square gives a normal density with precision \(1/v_{0a}+n_a/\sigma^2\), hence variance \(v_a\), and mean \(m_a\) as stated above. Since independent normal random variables have a normal difference, subtracting the two posterior means gives the posterior mean of \(\Delta\), while adding their variances gives its posterior variance. This derivation also shows why observed post-discontinuation outcomes enter the likelihood just like other outcomes in their randomized groups.
Worked Example: Outcomes Collected After Discontinuation
In a hypothetical randomized trial, the outcome is a six-month symptom-improvement score in points, with larger values indicating greater improvement. There are 30 participants in each group. Eight participants assigned to the experimental strategy and five assigned to control discontinued assigned treatment, but all 60 six-month outcomes were collected. The group means, including every observed outcome regardless of discontinuation, are 6.0 points and 2.0 points, respectively. For illustration, take the known within-group standard deviation to be 10 points and use independent priors \(\mu_a\sim N(0,100)\), so the prior mean is 0 and prior variance is 100 points squared in each group.
The posterior variance in either group is
The experimental-group posterior mean is \(m_1=(100/31)(30\times6.0/100)=5.806\) points. The control-group posterior mean is \(m_0=(100/31)(30\times2.0/100)=1.935\) points. Therefore the posterior contrast has mean \(5.806-1.935=3.871\) points and variance \(100/31+100/31=6.452\) points squared, giving posterior standard deviation \(\sqrt{6.452}=2.540\) points.
A central 95% posterior credible interval is \(3.871\pm1.96(2.540)\), or approximately \((-1.107,8.849)\) points. Under this model and prior, the posterior probability that assignment to the experimental strategy yields a higher population mean improvement is about 93.6%. The interval expresses posterior uncertainty about the mean contrast; it does not imply that each participant benefits, nor that the probability of benefit for an individual is 93.6%.
Assignment Groups, Not Treatment Continuers
A treatment-policy analysis uses the randomized assignment to define the groups and retains outcomes after discontinuation. This is often implemented as an analysis by randomized group, but the estimand is not secured merely by labeling an analysis “intention-to-treat.” Incomplete outcome collection, exclusions, or a model that conditions on post-randomization events can change what is estimated.
Worked Example: How Excluding Discontinuers Changes the Contrast
In a second hypothetical trial, 20 participants are assigned to each group. In the experimental group, 14 participants continue assigned treatment and have a mean six-month score of 8 points; 6 discontinue and have a mean score of 1 point, with those outcomes collected. In control, 18 continue and have a mean score of 3 points; 2 discontinue and have a mean score of -1 point. All scores are in points, and larger values are better.
The treatment-policy mean for the experimental assignment is the weighted mean over all 20 participants: \((14\times8+6\times1)/20=118/20=5.9\) points. For control it is \((18\times3+2\times(-1))/20=52/20=2.6\) points. The treatment-policy contrast is \(5.9-2.6=3.3\) points.
If the analysis instead excludes everyone who discontinued, the contrast is \(8-3=5\) points. That is not the treatment-policy contrast: it compares observed outcomes among selected treatment continuers. The two estimates differ because outcomes among discontinuers are part of the specified treatment-policy outcome and because continuation is a post-randomization event.
Missing Outcomes Require Additional Assumptions
When outcomes are missing, the treatment-policy estimand does not tell us what the unobserved values would have been. A Bayesian model can represent uncertainty about missing values, but the data alone may not identify the relationship between missingness and those values. The assumption used to connect observed and unobserved outcomes should be stated and examined.
Worked Example: Posterior Inference With Incomplete Follow-Up
In a third hypothetical trial, 20 participants are randomized to each group. At six months, outcomes are observed for 18 experimental-group participants, with mean 5.0 points, and 16 control participants, with mean 2.0 points. Some missing outcomes follow discontinuation. As a deliberately simple illustration, assume outcomes are missing completely at random within each randomized group, use known variance \(\sigma^2=100\) points squared, and retain the independent \(N(0,100)\) prior for each mean. Under this assumption, the observed outcomes provide a normal likelihood for each group mean; the missing outcomes do not add observed-data information.
For the experimental group, \(v_1=(1/100+18/100)^{-1}=100/19=5.263\), and \(m_1=(100/19)(18\times5.0/100)=4.737\) points. For control, \(v_0=(1/100+16/100)^{-1}=100/17=5.882\), and \(m_0=(100/17)(16\times2.0/100)=1.882\) points. Thus the posterior contrast mean is \(4.737-1.882=2.855\) points, with variance \(5.263+5.882=11.145\) points squared and standard deviation \(3.338\) points.
This calculation targets the treatment-policy contrast only under the stated missing-completely-at-random assumption and the normal model. If missingness depends on unobserved outcome values, or on discontinuation in a way not accounted for, the observed outcomes may not represent the missing ones. A different missing-data model could therefore produce a different posterior. The assumed random-missingness condition is not established by the fact that treatment was randomized.
Diagnostics, Alternatives, and Limitations
A credible analysis checks both the clinical data process and the statistical model. Report discontinuation numbers and reasons by randomized group, and show how many outcomes were collected after discontinuation at each relevant visit. Compare follow-up completeness across groups and examine whether missingness is associated with observed baseline characteristics, earlier outcomes, or discontinuation. Such patterns cannot prove that missingness is ignorable, but they can reveal implausible assumptions and motivate sensitivity analyses.
For the outcome model, inspect whether the assumed normal distribution and common variance are reasonable for the measurement scale. Strong skewness, outliers, floor or ceiling effects, and markedly different spreads can make a normal common-variance model inadequate. Evaluate prior sensitivity by repeating the analysis with clinically defensible alternative priors; a posterior dominated by the prior should be described accordingly. In a fuller analysis, covariates or repeated measurements may be modeled, but their inclusion should preserve the treatment-policy question and account appropriately for the design.
| Approach | Question it addresses | Key distinction |
|---|---|---|
| Treatment-policy analysis | What is the outcome contrast between randomized strategies, including outcomes after discontinuation? | Uses post-discontinuation outcomes in the assigned groups. |
| Analysis restricted to continuers | How do observed outcomes compare among those who continued? | Conditions on a post-randomization event and does not generally retain the randomized comparison. |
| While-on-treatment summary | What outcomes were observed during exposure to assigned treatment? | Uses a different outcome window and may not answer the fixed-time treatment-policy question. |
The principal advantage of a treatment-policy estimand is clinical relevance when decisions concern assignment to strategies in which treatment changes or stops may occur. Its interpretation is specific to the trial’s discontinuation and subsequent-care context; it should not automatically be transported to a setting with different practices. Its main practical limitation is that complete post-discontinuation follow-up can be difficult, and missing outcomes can make inference depend substantially on untestable assumptions. No posterior calculation can remove that limitation.
- Mistake: “Participants who stopped treatment are no longer part of the treatment effect, so I will omit them.” Correction: Under a treatment-policy estimand, their outcomes remain relevant to the randomized strategy they were assigned to; omitting them changes the target comparison.
- Mistake: “Because the study is randomized, missing outcomes can be ignored.” Correction: Randomization balances assignment at baseline, but does not ensure that follow-up is complete or that missingness is unrelated to unobserved outcomes.
- Mistake: “A high posterior probability of a positive contrast means most individuals benefit.” Correction: It is a probability statement about the population mean contrast under the model and prior, not about the proportion of individuals who benefit.
Check Your Understanding
Answer each question, then compare your reasoning with the brief solutions.
- A participant stops assigned treatment but completes the six-month outcome assessment. Should that value be included in a treatment-policy analysis, and in which group?
- In the first worked example, why is the posterior variance of the treatment contrast the sum of the two group posterior variances?
- Why does analyzing only treatment continuers not generally estimate the treatment-policy contrast?
- What additional assumption supported the calculation with incomplete follow-up, and what would be a warning sign that it may be inadequate?
Solutions. (1) Yes. Include the value in the group defined by the participant’s randomized assignment, regardless of discontinuation. (2) The model gives independent posterior distributions for the two group means, and the variance of the difference of independent random variables is the sum of their variances. (3) Continuation occurs after randomization and may be related to prognosis or response, so restricting to continuers selects participants and changes the comparison. (4) The example assumed outcomes were missing completely at random within groups. Different missingness rates by group, or missingness related to earlier outcomes or discontinuation, would prompt concern and sensitivity analyses, though no observed pattern alone can establish the missing-data mechanism.