Translating Protocol Language Into Estimands

From Protocol Language to a Statistical Question

A protocol usually describes actions: who is enrolled, what treatment is assigned, when outcomes are measured, and what happens if a participant stops treatment or receives rescue therapy. An estimand turns those descriptions into a precise question about a population-level treatment effect. The translation matters because phrases such as “the treatment effect at week 12” do not, by themselves, say what outcome is targeted after an important event.

The task here is to make the target explicit, not to choose an estimator or fill in missing outcomes. In particular, a participant’s outcome after treatment discontinuation may still be part of the target if follow-up continues. The introductory material on the estimand framework establishes the role of the estimand; the practical task in this tutorial is to derive its components from protocol wording.

Definition: An estimand is a precise description of the treatment effect a trial seeks to quantify. Its core attributes are the target population, the treatment conditions being compared, the outcome variable, the strategy for each relevant intercurrent event, and the population-level summary used to compare treatments. An intercurrent event is an event after treatment initiation that affects either the interpretation or existence of measurements for the outcome of interest.

A useful discipline is to separate the protocol’s facts from the decisions needed to define the target. “Visits occur through week 12” is a schedule. “Include the week-12 outcome regardless of whether rescue medication was used” states how an intercurrent event is handled. “Compare mean change from baseline” specifies a population summary. These are different pieces of the definition.

A Translation Sequence

Read a protocol description by asking five questions. The order is helpful because later choices depend on earlier ones: the outcome must be defined before deciding what its post-event values mean, and the summary must be named before the treatment contrast can be interpreted.

Who is the target population?
State the population to which the treatment effect refers, using relevant eligibility characteristics. Do not substitute the set of participants with observed outcomes for the target population.

Which treatment conditions are compared?
Specify the treatments or treatment policies, including assignment and any protocol rules that define the comparison. A condition may include more than the name of a drug.

What outcome is measured, and when?
Name the variable, scale, time point, and any transformation such as change from baseline. “Clinical response” is not sufficiently precise unless response is defined.

How is each intercurrent event handled?
Choose the target meaning of outcomes after events such as rescue medication or treatment discontinuation. Continuing to collect a measurement does not, on its own, decide how that measurement enters the estimand.

What population-level summary is compared?
Specify, for example, a difference in means or a ratio of response probabilities. State the direction of subtraction and what its sign means.

Potential outcomes provide compact notation for the result. Let \(a=1\) denote assignment to the experimental treatment and \(a=0\) assignment to control. Let \(g_a\) denote the event strategy attached to treatment condition \(a\), and let \(Y^{a,g_a}\) be the outcome that would be defined for a participant under that treatment condition and strategy. For a mean difference, the estimand can be written

\theta=E_{\mathcal P}\!\left(Y^{1,g_1}\right)-E_{\mathcal P}\!\left(Y^{0,g_0}\right),

where \(\mathcal P\) denotes the target population. If \(Y\) is change from baseline on a scale where lower values are better, a negative \(\theta\) favors the experimental treatment. The superscript \(g_a\) is important: it makes clear that the outcome is defined under a stated approach to intercurrent events, rather than leaving that approach hidden in an analysis description.

Worked Example: A Week-12 Pain Trial With Rescue Medication

Hypothetical protocol. Adults with moderate-to-severe chronic pain are randomly assigned to an experimental oral treatment or placebo for 12 weeks. Pain is rated from 0 (no pain) to 10 (worst pain). The protocol records pain at baseline and week 12, and continues week-12 follow-up even if a participant uses rescue medication. The intended question is the effect of assignment to the experimental treatment rather than placebo, allowing rescue medication as it occurs in the trial.

Population. Adults meeting the stated moderate-to-severe chronic pain eligibility criteria and randomized in the trial. The target is not only people who complete treatment or avoid rescue medication.

Treatment conditions and event strategy. Compare assignment to experimental treatment with assignment to placebo. For rescue medication, use a treatment-policy strategy: the week-12 pain outcome is included regardless of whether rescue medication was used. Thus, the target concerns the effect of the assigned treatment conditions in a setting where rescue use can occur; it is not the effect that would have occurred if nobody had used rescue medication.

Variable and summary. Let \(Y\) be week-12 pain score minus baseline pain score, in points. Compare mean change in the target population, experimental treatment minus placebo. Suppose, solely to illustrate interpretation, the corresponding hypothetical population means under this estimand are \(-2.1\) points and \(-1.3\) points. Then

\theta=(-2.1)-(-1.3)=-0.8\text{ points}.

Under these illustrative values, mean pain reduction is 0.8 points greater under assignment to the experimental treatment than under assignment to placebo, with rescue medication allowed as specified. These are hypothetical target means, not results from a real study. They illustrate the estimand’s interpretation; they do not establish how to estimate it from incomplete trial data.

Why Post-Event Follow-Up Must Be Stated

An intercurrent event strategy defines what the treatment effect is about. Under a treatment-policy strategy, outcomes are used regardless of the event, so the target includes the consequences of the event and subsequent care as they occur under the treatment conditions. Under a hypothetical strategy, the target is defined for a specified scenario in which the event did not occur. A composite strategy incorporates the event into the outcome definition itself. A while-on-treatment strategy targets outcomes up to the event rather than outcomes at a later fixed time point. A principal-stratum strategy targets a subgroup defined by potential event status under treatment conditions.

These choices are not interchangeable analysis options for one unchanged question. They can define different outcomes and different target populations for the comparison. For a treatment-policy estimand, continuing follow-up after discontinuation or rescue may be necessary to observe the outcome the estimand calls for. But collecting that follow-up does not mean the event is ignored: it means the post-event measurement is included under the treatment-policy definition.

Also distinguish an intercurrent event from missing data. If a participant uses rescue medication and a week-12 score is recorded, rescue is an event and the score is observed. If the participant misses the week-12 visit, the score is missing. The event strategy defines the target outcome; missing-data methods concern how to learn about that target when some measurements are unavailable. Treating “missing after discontinuation” as though discontinuation itself automatically specified the estimand confuses these two questions.

Worked Example: A Hypothetical Question About Rescue Use

Hypothetical protocol question. In the same pain setting, investigators instead want to know how assignment would affect week-12 pain if participants had not used rescue medication. This wording calls for a hypothetical strategy for rescue use. The population, assignment comparison, and pain-change variable can remain as before, but the outcome is now defined under the no-rescue scenario.

Suppose the hypothetical mean changes under this estimand are \(-2.4\) points for experimental treatment and \(-1.7\) points for placebo. The mean-difference estimand is

\theta_{\text{no rescue}}=(-2.4)-(-1.7)=-0.7\text{ points}.

The negative difference favors the experimental treatment: under the specified no-rescue scenario, mean improvement is 0.7 points greater than under placebo. The arithmetic is direct, but the target is not the same as in the first example. In particular, observed pain scores collected after rescue use do not automatically equal the outcomes that would have occurred without rescue. The hypothetical strategy therefore requires assumptions and an analysis capable of addressing that unobserved scenario; it cannot be justified merely by relabeling observed post-rescue scores.

Reading a Protocol for Ambiguity

Protocol language often leaves a consequential choice unstated. Consider “the primary analysis will compare week-12 pain between groups, with participants followed after discontinuation.” This tells us that follow-up is planned, but not whether week-12 scores after discontinuation are included in the target outcome. Nor does it define the score being compared: raw week-12 pain, change from baseline, or a response category could all fit the phrase.

Worked Example: Clarifying Follow-Up After Discontinuation

Hypothetical protocol wording. A 12-week trial compares an experimental treatment with control in eligible adults with a chronic condition. Participants are asked to complete a week-12 symptom questionnaire even if they stop their assigned treatment. The protocol says only, “The groups will be compared on week-12 symptoms.”

The target population and treatment conditions can be stated from the protocol, but the outcome and discontinuation strategy need clarification. Suppose investigators confirm that the variable is the week-12 symptom score, measured in points, and that scores after discontinuation are included. That defines a treatment-policy strategy for discontinuation. It does not define what to do about rescue treatment, which must be considered separately if it can occur.

For an illustrative numerical interpretation, suppose the hypothetical population mean week-12 scores under the two treatment policies are 4.6 points for experimental treatment and 5.2 points for control, with lower scores better. The mean difference is

\theta=4.6-5.2=-0.6\text{ points}.

The target contrast is 0.6 points lower mean symptom score under assignment to experimental treatment, with week-12 scores after discontinuation included. If the protocol had instead specified a hypothetical strategy for discontinuation, these means would need to refer to the outcomes under a defined no-discontinuation scenario; the same arithmetic would not make the estimands equivalent.

Justifying the Link Between Assignment and the Estimand

The estimand is a target quantity, not automatically the quantity a simple comparison of observed means estimates. A short justification shows what would be needed for a direct randomized comparison. Suppose treatment is randomized in the target population, treatment assignment is implemented as specified, and the outcome for each participant is observed under the event strategy in the estimand. Let \(A\) be randomized assignment. Randomization makes assignment independent of the potential outcomes, while consistency connects a participant’s observed outcome to the potential outcome for the treatment condition actually assigned. Therefore, for either arm \(a\),

E(Y\mid A=a)=E(Y^{a,g_a}).

Subtracting the two arm means gives the mean-difference estimand:

E(Y\mid A=1)-E(Y\mid A=0) = E(Y^{1,g_1})-E(Y^{0,g_0}) = \theta.

This argument requires the observed outcome to correspond to the event strategy being targeted. It also assumes the target population is the randomized population, or that any generalization to another population is justified. If outcomes are missing, the equality involving observed arm means does not follow just from randomization; additional assumptions or methods may be required to recover the target means. Thus, specification comes first: an analysis cannot repair an unclear target.

Diagnostics, Alternatives, and Limitations

Before finalizing a translation, check whether each component has a clear answer and whether the components fit together. A protocol description is at risk of incoherence when it names a fixed-time endpoint but says outcomes are only collected until treatment stops; when it says to include post-rescue outcomes but does not define the outcome scale; or when the stated treatment-policy question is paired with an analysis that discards all post-event outcomes without explaining how it still targets the same quantity.

Check the endpoint. Confirm the measure, unit or scale, time point, and whether the target uses a level, change, or response.
Check each event separately. Rescue medication and treatment discontinuation may need different strategies. A strategy for one does not automatically settle the other.
Check follow-up against the target. If a fixed-time outcome after an event is part of the estimand, the protocol should make a reasonable effort to collect it.
Check the contrast. Write the subtraction order and verify that the sign has the intended clinical interpretation.
Check population wording. Eligibility criteria describe who enters the target population; completion status generally does not redefine that population after randomization.

The main alternative strategies answer different clinical questions. Treatment-policy estimands include the effects of subsequent care as it occurs; hypothetical estimands ask about a specified scenario; composite estimands treat an event as part of the outcome; while-on-treatment estimands target a restricted treatment period; and principal-stratum estimands concern a subgroup defined using potential event behavior. None is universally preferable. The protocol question and clinical interpretation should determine the choice.

A fully specified estimand also has limits. It does not guarantee that the needed data will be collected, that its assumptions are plausible, or that the effect can be estimated precisely. A hypothetical strategy may require strong assumptions about outcomes in a scenario that was not observed. A treatment-policy strategy may reflect patterns of rescue use or discontinuation that depend on the trial setting. These are reasons to state the target explicitly and examine its assumptions, not reasons to leave the target implicit.

Common pitfall: “We followed participants after discontinuation, so discontinuation has been handled.” Follow-up is a data-collection plan, not an event strategy. The correction is to state whether post-discontinuation outcomes are included under a treatment-policy strategy, replaced by a hypothetical outcome under a specified scenario, or treated in another explicitly defined way. Then address missing measurements separately.

Check Your Understanding

For each item, identify the relevant estimand choice. Solutions follow the questions.

A protocol compares mean change in a score at week 16, regardless of rescue medication, in adults meeting specified eligibility criteria. What are the endpoint, summary, and rescue strategy?
Why does collecting outcomes after treatment discontinuation not by itself specify how discontinuation is handled?
For a mean-difference estimand, suppose mean change is \(-1.8\) points in the experimental arm and \(-1.1\) points in control, with lower values better. Calculate and interpret the contrast when experimental minus control is the convention.
A participant uses rescue medication and completes the week-16 questionnaire. Is the outcome missing? Explain the distinction.

Solutions. (1) The endpoint is change from baseline in the specified score at week 16; the population summary is the difference in population mean changes. The rescue strategy is treatment policy because outcomes are included regardless of rescue use. (2) Follow-up indicates that a measurement may be collected. The estimand still must say whether that post-discontinuation measurement is included, excluded, or defined under another strategy. (3) \(\theta=(-1.8)-(-1.1)=-0.7\) points. The experimental arm has 0.7 points greater mean reduction under this convention; the negative contrast favors experimental treatment. (4) No. The score is observed. Rescue medication is an intercurrent event; missingness concerns an outcome that was not obtained.