Worked Estimand Analysis for a Chronic-Disease Trial

From Trial Details to a Defined Estimand

A randomized trial can produce a precise posterior distribution and still answer the wrong clinical question if its estimand is unclear. This is especially important in chronic disease: participants may stop assigned treatment because of lack of benefit or adverse effects, and may start rescue medication before the endpoint is measured. The analysis must say what those events mean for the treatment effect, not merely describe how missing values will be handled.

Consider a hypothetical 24-week trial of an investigational therapy versus control for adults with a chronic inflammatory disease. The outcome is change from baseline in a symptom score, measured in points, with lower values indicating fewer symptoms. A negative treatment contrast therefore favors the investigational therapy. The primary analysis below targets the average effect in the protocol-defined eligible population and uses a treatment-policy strategy for both treatment discontinuation and rescue medication: outcomes at week 24 are included regardless of whether either event occurred.

Definition: An estimand is a precisely specified treatment-effect quantity for a defined population, endpoint, and treatment contrast, together with a strategy for handling each relevant intercurrent event. An intercurrent event occurs after treatment begins and affects either the interpretation or existence of the outcome, such as stopping assigned treatment or starting rescue medication. Missing outcome data are a separate issue: they concern whether the outcome was observed, not what the estimand means.

The treatment-policy strategy asks about the effect of assignment to each randomized treatment in the presence of the subsequent care that actually occurs under that assignment. It does not ask what the treatment effect would have been if everyone had adhered or if rescue medication had been unavailable. In a trial with rescue treatment, the result may reflect both the assigned therapy and the downstream use of rescue medication, which is part of the treatment-policy question.

Specify the Question Before the Model

For this example, the protocol defines the target population as adults aged 40–75 years with moderate-to-severe disease meeting the stated symptom-score and background-therapy criteria at randomization. The population is the eligible trial population; the estimand is not automatically a claim about patients who would fail those criteria. The endpoint is each participant’s change from baseline to week 24, in symptom-score points. The contrast is investigational therapy minus control.

\delta_T = E_T(Y_{24}-Y_0\mid A=1) - E_T(Y_{24}-Y_0\mid A=0).

Here \(Y_0\) and \(Y_{24}\) denote baseline and week-24 scores, \(A=1\) denotes assignment to investigational therapy, \(A=0\) denotes assignment to control, and \(E_T\) denotes an expectation for the prespecified target population under the corresponding randomized assignment. The definition includes outcomes after discontinuation and after rescue medication. The subscript \(T\) emphasizes that the population is part of the question, not a detail that can be changed after seeing the results.

Randomization supports a causal comparison of assignment in the trial population when allocation is implemented as planned, outcome collection is sufficiently complete or missingness is handled under defensible assumptions, and the analysis corresponds to the stated assignment strategies. It does not by itself ensure that the trial population represents every patient with the disease. Nor does it turn an association between rescue use and outcome into a causal effect of rescue use.

Worked Example: Writing the Event Strategies Explicitly

Suppose participants can discontinue assigned treatment and can begin rescue medication. The protocol collects the week-24 symptom score even after either event. State the primary estimand’s handling of both events and explain what its contrast means.

Population and outcome. Use all randomized participants meeting the prespecified eligibility criteria, and measure change in symptom score from baseline to week 24, in points.

Discontinuation strategy. Use a treatment-policy strategy: retain the participant in the assigned group and use the week-24 outcome whether or not assigned treatment was discontinued.

Rescue strategy. Also use a treatment-policy strategy: retain the participant in the assigned group and use the outcome whether or not rescue medication was started.

Interpretation. The contrast is the average difference in week-24 change associated with assignment to investigational therapy rather than control, under the subsequent discontinuation and rescue use that arise in the trial. It is not the effect of continuous use of assigned treatment in the absence of rescue.

The strategy is usable only if the relevant outcomes are sought after these events. If the protocol stops collecting scores when treatment stops, a treatment-policy estimand remains a possible clinical question, but its data are not directly observed for those participants; the missing-outcome problem then requires explicit assumptions and sensitivity analysis.

A Bayesian Analysis for the Continuous Contrast

A Bayesian estimand is represented by a posterior distribution for the defined effect, conditional on the observed data, model, and prior. For a transparent calculation, suppose the trial’s endpoint model gives a sampling distribution for the estimated treatment contrast that is approximately normal with known standard error. Assign a normal prior to the contrast. This calculation assumes the model’s treatment contrast matches the target-population estimand, the likelihood is appropriate for the continuous outcome, and the observed outcomes are available or the missing-data assumptions used by the model are adequate.

\hat{\delta}\mid\delta_T \sim N(\delta_T,SE^2), \qquad \delta_T\sim N(m_0,v_0).

For this normal likelihood and normal prior, the posterior is normal. Its variance is the reciprocal of the sum of the likelihood and prior precisions; its mean is a precision-weighted average of the observed estimate and prior mean:

v_1=\left(\frac{1}{SE^2}+\frac{1}{v_0}\right)^{-1}, \qquad m_1=v_1\left(\frac{\hat{\delta}}{SE^2}+\frac{m_0}{v_0}\right).

The precision-weighting follows by multiplying the normal likelihood kernel by the normal prior kernel. The terms involving \(\delta_T\) in the exponent are \(-\tfrac12[(\hat{\delta}-\delta_T)^2/SE^2+(\delta_T-m_0)^2/v_0]\). Collecting the coefficient of \(\delta_T^2\) gives posterior precision \(1/SE^2+1/v_0\); collecting the linear terms gives posterior mean \(v_1(\hat{\delta}/SE^2+m_0/v_0)\). Completing the square yields the stated normal posterior. This is a calculation under the specified model and prior, not a guarantee that either adequately describes the trial.

Worked Example: Posterior Effect and Credible Interval

In hypothetical data, 50 participants per group have week-24 outcomes, including measurements after discontinuation and rescue. The adjusted mean changes are \(-8.173\) points for investigational therapy and \(-5.500\) points for control. Suppose the common within-group outcome standard deviation is treated as known at 7.035 points, and the normal sampling approximation is used. Use a prior \(\delta_T\sim N(0,1000^2)\), where the prior standard deviation is 1000 points.

The observed contrast is \(\hat{\delta}=-8.173-(-5.500)=-2.673\) points. The standard error under the stated independent-groups model is \(SE=7.035\sqrt{1/50+1/50}=1.407\) points (to four significant figures). A negative contrast favors investigational therapy.

The posterior variance and standard deviation are \(v_1=(1/1.407^2+1/1000^2)^{-1}\) points squared and \(\sqrt{v_1}\approx1.407\) points. The posterior mean is \(v_1[-2.673/1.407^2+0/1000^2]\approx-2.673\) points. The very broad prior has little influence at the reported precision.

The posterior is therefore approximately \(N(-2.673,1.407^2)\) points. A 95% equal-tailed credible interval is \(-2.673\pm1.96(1.407)\), or approximately \((-5.431,\ 0.085)\) points. Conditional on this model, prior, and data, the posterior probability that the estimand lies in this interval is 0.95. The interval is compatible with an average benefit as large as about 5.4 points and with a small average difference in the opposite direction; it does not establish that every patient benefits.

If a 2-point reduction is prespecified as the minimum clinically important average benefit, the relevant probability is \(P(\delta_T<-2\mid\text{data})\). Standardizing gives \(z=(-2-(-2.673))/1.407\approx0.478\), so this probability is approximately \(\Phi(0.478)=0.684\). This is the posterior probability of exceeding that benefit threshold under the model, not the probability that a null hypothesis is true.

Missing Outcomes, Diagnostics, and Sensitivity

A treatment-policy strategy does not solve missing data. If a participant discontinues or starts rescue but the week-24 score is not recorded, the outcome remains part of the estimand but is unobserved. A model that uses only observed outcomes must therefore rely on assumptions about the unobserved values. Under a missing-at-random assumption, missingness may depend on observed information included in the model but not on the missing outcome after conditioning on that information. This assumption cannot generally be confirmed from the observed data alone.

Useful checks include comparing follow-up and missingness rates by randomized group and by discontinuation or rescue status; examining whether reasons for missingness differ between groups; checking outcome distributions, outliers, and model residuals; and verifying that outcomes after intercurrent events were collected as specified. In a Bayesian analysis, also inspect whether the posterior is overly sensitive to prior choices or influential observations. These checks can reveal conflicts between the protocol’s estimand, the collected data, and the analysis model, but reassuring diagnostics cannot prove that missingness is ignorable.

Worked Example: A Missing-Data Sensitivity Shift

Use the posterior result above as a missing-at-random primary analysis. Suppose a prespecified sensitivity analysis assumes that unobserved outcomes in the investigational group are, on average, 1.5 points worse than the primary model predicts, while the control-group predictions are unchanged. For illustration, suppose the sensitivity adjustment shifts the contrast estimate by 1.5 points and leaves its standard error at 1.407 points.

The adjusted contrast is \(-2.673+1.5=-1.173\) points. With the same broad prior, the sensitivity posterior is approximately \(N(-1.173,1.407^2)\) points. Its 95% credible interval is \(-1.173\pm1.96(1.407)=(-3.931,\ 1.585)\) points, rounded to three decimals. The posterior probability that the contrast is below \(-2\) points is approximately \(\Phi[(-2+1.173)/1.407]=\Phi(-0.588)\approx0.278\).

This calculation shows how a specific departure from the primary missing-data assumption can change the clinical conclusion: the primary analysis gave about 0.684 posterior probability of a benefit exceeding 2 points, while this sensitivity scenario gives about 0.278. The shift is illustrative, not evidence that the assumed missing outcomes are correct. A real analysis should justify the sensitivity range clinically and evaluate relevant departures in both treatment groups where appropriate.

For a credible sensitivity analysis, state which unobserved outcomes are shifted, in which direction, and by how much, then recompute the estimand’s posterior under that assumption. Also compare the target-population definition with the enrolled population: if the intended population differs, the analysis may require standardization to prespecified population characteristics, with uncertainty and adequate data support. Changing the target population or event strategy changes the estimand; it is not merely a different model for the same estimand.

Alternatives and Limits of Interpretation

The treatment-policy strategy is not the only legitimate way to address discontinuation or rescue. A hypothetical strategy targets the effect under a specified scenario in which an event did not occur, and requires assumptions or modeling to estimate outcomes in that scenario. A composite strategy can incorporate an event into the endpoint, for example by assigning a poor outcome after rescue, but changes the outcome’s meaning and scale. A while-on-treatment analysis focuses on outcomes before discontinuation and may answer a narrower question while being vulnerable to informative stopping. Each approach needs a clinically meaningful question and a clear definition; choosing one because it gives a more favorable result is not a sound basis for inference.

The continuous normal model above is deliberately simple. In an actual trial, the standard deviation may be estimated, baseline score and randomization strata may need to be modeled, repeated measurements may be used, and the posterior may be non-normal. A model may also fail to align with the estimand, for example by excluding post-discontinuation outcomes while claiming a treatment-policy effect. These are not minor technical differences: the estimand defines the target, while the model and assumptions determine how the data inform it.

Check the direction and units. With lower scores better, a negative investigational-minus-control contrast is favorable, and the contrast is measured in symptom-score points.
Separate event strategy from missingness assumptions. Keeping participants in their randomized groups does not impute an unobserved week-24 outcome.
Do not overstate the target. A result for the eligible trial population need not generalize to patients outside its eligibility criteria.
Interpret posterior probabilities conditionally. They depend on the likelihood, prior, estimand, and assumptions, including those used to address missing outcomes.

Key takeaway: A defensible Bayesian trial conclusion starts with a prespecified estimand. Name the target population, endpoint and treatment contrast, specify the strategy for discontinuation and rescue, and then use a model that matches that question. Report posterior uncertainty and examine how conclusions change under clinically plausible assumptions about unobserved outcomes.

Check Your Understanding

Answer each question, then compare with the brief solutions below.

If rescue medication is handled with a treatment-policy strategy, should an observed week-24 score after rescue be excluded from the assigned-treatment group?
For a score where lower values indicate fewer symptoms, what does a contrast of \(-1.8\) points mean when defined as investigational therapy minus control?
Why does a treatment-policy strategy not, by itself, resolve a missing week-24 outcome after discontinuation?
In the worked posterior, how should the probability \(P(\delta_T<-2\mid\text{data})\) be interpreted?
Name one way a hypothetical strategy differs from the treatment-policy strategy in the example.

Solutions. (1) No. The observed outcome is included under the treatment-policy strategy, with the participant retained in the randomized group. (2) The investigational group has an estimated average reduction 1.8 points greater than the control group; the contrast favors investigational therapy, without implying that every participant benefits. (3) The event strategy specifies how outcomes after discontinuation count, but a score that was never observed still requires assumptions or a sensitivity analysis. (4) It is the posterior probability, conditional on the stated model, prior, and data, that the average contrast exceeds the 2-point benefit threshold. (5) A hypothetical strategy targets outcomes under a specified scenario where an event did not occur; treatment policy includes outcomes under the event’s actual occurrence and subsequent care.