Preserving the Estimand Across the Phase Boundary

Why the Phase Boundary Can Change the Question

A seamless phase 2/3 trial may move directly from a learning stage to a confirmatory stage. The protocol may change at that boundary: eligibility may narrow, the selected regimen may differ from the one studied earlier, or the handling of treatment discontinuation may become different. Such changes are not merely operational if they alter what treatment effect is being estimated. A precise estimand makes the clinical question explicit before considering how evidence from the stages will be interpreted.

The aim is not to force every stage to have identical procedures. It is to determine whether the stages still address the same treatment question, or whether the change creates a new estimand that must be described and interpreted separately. The introductory material on seamless phase 2/3 design distinguishes the purposes of the stages; here the focus is the meaning of the treatment effect across their boundary.

Definition: An estimand is a precise description of the treatment effect that a trial seeks to estimate. It specifies the target population, the treatment conditions being compared, the outcome variable and its assessment time, how intercurrent events are handled, and the population-level summary used to compare treatments. An intercurrent event is an event occurring after treatment initiation that affects either the interpretation or existence of the outcome, such as treatment discontinuation, rescue medication, or starting another therapy.

For a simple continuous outcome, one possible population-level summary is a difference in means. Let \(Y\) denote the outcome at the specified assessment time, and let \(a\) and \(c\) denote the treatment and comparator conditions. Under a stated strategy for intercurrent events, the estimand might be written as \(\theta=E(Y^a)-E(Y^c)\), where the expectations refer to the target population. The superscripts identify treatment conditions, not observed treatment received. The expression is meaningful only after the population, treatment conditions, outcome, and event strategy have been specified.

A phase label is not itself an estimand attribute. A change from phase 2 to phase 3 can leave the estimand intact if the clinical question stays the same, even when trial procedures change. Conversely, stages can have similar procedures but answer different questions if, for example, the target population or treatment condition changes.

Four Checks for Preserving the Estimand

At the transition, compare the intended question across four connected components. The population-level summary must also remain clear: a mean difference, risk difference, or another summary can define different treatment effects even when the other components match.

Treatment conditions. Specify the intervention and comparator sufficiently to distinguish the regimens: dose, schedule, duration, and any relevant co-interventions. “Assigned to the study drug” may be too vague if dose or schedule changes between stages.
Target population. Describe the people to whom the effect applies, using clinically meaningful eligibility characteristics. A narrower phase 3 population does not automatically represent the broader population enrolled in phase 2.
Outcome. State the variable, how it is measured, and when it is assessed. The same label for an outcome does not ensure that its measurement or assessment time is equivalent.
Intercurrent-event strategy. State what effect is of interest when an event occurs. The strategy determines which outcomes contribute to the clinical interpretation.

Key distinction: An intercurrent event is not the same as missing outcome data. Discontinuation may be an intercurrent event even when the outcome is later measured. Missingness describes an unavailable measurement; it requires additional assumptions and methods for analysis, but does not by itself define the clinical question.

Common strategies include a treatment-policy strategy, which uses outcomes regardless of whether the event occurred; a hypothetical strategy, which targets outcomes under a specified scenario in which the event did not occur; and a composite strategy, which incorporates the event into the outcome definition. A while-on-treatment strategy concerns outcomes only while participants remain on assigned treatment. A principal-stratum strategy targets a subgroup defined by potential event status under treatment conditions and requires its own precise definition. These strategies are not interchangeable labels for the same effect.

Worked Example: A Population Change at the Transition

Suppose a hypothetical treatment lowers a disease-severity score, so a negative treatment-minus-control difference favors treatment. In two biomarker subgroups, the subgroup-specific mean differences are \(-4\) points for marker-positive patients and \(-1\) point for marker-negative patients. Assume these effects are applicable to the intended target population. The target population is 40% marker-positive and 60% marker-negative.

For this target population, the standardized mean difference is the weighted average of the subgroup effects:

\theta_{\text{target}} =0.40(-4)+0.60(-1) =-1.60-0.60 =-2.20\text{ points}.

Now suppose phase 2 enrolled 70% marker-positive and 30% marker-negative patients. Its corresponding weighted contrast would be

\theta_{\text{phase 2}} =0.70(-4)+0.30(-1) =-2.80-0.30 =-3.10\text{ points}.

The \(-3.10\)-point contrast pertains to the phase 2 population mix; the \(-2.20\)-point contrast pertains to the stated target population. They differ even though the subgroup effects, treatment comparison, and outcome definition were held fixed. If phase 3 enrolls only marker-positive patients, its direct population is narrower still. A claim about all eligible patients then requires an explicit target population and a defensible basis for transporting the subgroup effects; it cannot be justified merely by calling the stages seamless.

Justifying a Population-Level Summary

The preceding calculation follows from a mixture identity. Suppose the target population consists of mutually exclusive subgroups indexed by \(j\), with target proportions \(w_j\), where \(\sum_j w_j=1\). Let \(\tau_j=E(Y^a\mid j)-E(Y^c\mid j)\) be the treatment contrast within subgroup \(j\). Under the stated subgroup definitions and target weights, the population contrast is:

E(Y^a)-E(Y^c) =\sum_j w_j E(Y^a\mid j)-\sum_j w_j E(Y^c\mid j) =\sum_j w_j\tau_j.

The first equality uses the law of total expectation separately for each treatment condition. The second follows by subtracting the subgroup-specific expectations with the same target weights. This justification depends on the subgroup effects being relevant to the target population and on the outcome and treatment conditions being comparable across subgroups. If those conditions fail, reweighting alone does not establish transportability.

This identity also explains why enrollment proportions matter when a marginal effect is the target. A phase-specific marginal average reflects that phase’s population mix. If the target mix differs, the phase-specific marginal effect need not estimate the target-population effect, even when subgroup-specific effects are unchanged.

Worked Example: A Regimen Change Is a Treatment-Condition Change

Consider a hypothetical 12-week severity-score outcome, measured in points, with lower values better. In phase 2, the treatment condition is 20 mg once daily and the comparator is placebo. The mean score is 8 points on treatment and 10 on placebo, giving:

\widehat{\theta}_{20\text{ mg}} =8-10=-2\text{ points}.

In phase 3, suppose the treatment condition is changed to 40 mg once daily, while the placebo mean is 10 and the treatment mean is 7. The phase 3 contrast is:

\widehat{\theta}_{40\text{ mg}} =7-10=-3\text{ points}.

The arithmetic is straightforward, but the treatment conditions differ. The phase 2 result describes 20 mg versus placebo; the phase 3 result describes 40 mg versus placebo. The two estimates do not, without further justification, estimate the effect of one unchanged treatment condition. One could define the 40 mg regimen as the target condition for both stages, but phase 2 data from 20 mg do not thereby become observations under 40 mg. The regimen change therefore needs to be explicit in the estimand interpretation.

Intercurrent Events Must Have the Same Meaning

The event strategy is part of the estimand because it determines what outcome under each treatment condition is being summarized. Under a treatment-policy strategy for discontinuation, participants’ outcomes are included regardless of whether they discontinue assigned treatment. Under a hypothetical strategy, the target is instead the outcome that would have been observed under a stated scenario in which discontinuation did not occur. The latter is not obtained simply by discarding observed outcomes after discontinuation; it generally requires assumptions and an analysis that supports the counterfactual scenario.

Worked Example: Discontinuation Strategy Changes the Contrast

In a hypothetical trial, the outcome is a 12-week symptom score in points, with lower values better. Ten participants are assigned to each group. Under a treatment-policy strategy, all 12-week outcomes are included, including those measured after discontinuation. In the treatment group, eight participants who remain on treatment have a total score of 48, and two participants who discontinue have scores totaling 16. In the control group, nine participants who remain on assigned treatment have a total of 63, and one participant who discontinues has a score of 7.

The treatment-policy means are \( (48+16)/10=6.4 \) for treatment and \( (63+7)/10=7.0 \) for control. Thus:

\widehat{\theta}_{\text{treatment policy}} =6.4-7.0=-0.60\text{ points}.

Now consider a hypothetical strategy targeting outcomes if discontinuation had not occurred. Suppose a specified, assumed prediction model gives counterfactual scores of 5 and 5 for the two treatment-group discontinuers, and 7 for the control-group discontinuer. The hypothetical means are then \( (48+5+5)/10=5.8 \) and \( (63+7)/10=7.0 \), respectively, giving:

\widehat{\theta}_{\text{hypothetical}} =5.8-7.0=-1.20\text{ points}.

These are different estimands, not two computations of the same estimand. The hypothetical result depends on the model’s assumptions about unobserved outcomes had discontinuation not occurred; those counterfactual values are not directly observed. If one stage uses a treatment-policy strategy and the other a hypothetical strategy, a single pooled interpretation would obscure which clinical question is being answered.

Diagnostics, Alternatives, and Limitations

A practical diagnostic is to write the estimand for each stage in parallel, filling in the same attributes explicitly. Differences are easier to detect when the description says, for example, “20 mg once daily versus placebo in all eligible adults, 12-week score, treatment-policy strategy, mean difference in points” rather than relying on broad labels such as “efficacy.”

Write the clinical question for each stage.
Name the target population, treatment and comparator conditions, outcome and assessment time, event strategy, and summary measure.

Compare each attribute across the boundary.
Look for changed eligibility, dose or schedule, measurement definitions, or event handling. A difference may change the estimand even when other attributes match.

State the consequence.
If the attributes match, describe the common target question. If they do not, specify which stage-specific estimands differ and what additional assumptions would be needed for a broader interpretation.

There are three defensible responses to a mismatch. The design may preserve the original target question by keeping the relevant attributes aligned; it may define a revised estimand and explain how earlier-stage evidence relates to it; or it may report stage-specific effects as answers to distinct questions. A statistical combination method cannot, by itself, make distinct treatment conditions, populations, or event strategies into one clinically identical estimand.

Warning signs include a protocol that says “same endpoint” but omits assessment timing, a phase 3 eligibility restriction that is not reflected in the stated target population, or a discontinuation rule that changes without an explicit strategy. Another warning is to describe treatment assignment as the treatment condition while leaving the actual dose or schedule unspecified. These issues do not prove that results are unusable; they show that the intended interpretation needs clarification.

Preserving an estimand also has limits. Matching its written attributes does not guarantee that measurements are implemented comparably or that effects transport between populations. Standardization requires suitable subgroup information and assumptions; hypothetical strategies require assumptions about outcomes that are not observed under the specified scenario. Finally, this tutorial concerns the clinical meaning of the target effect, not the inferential procedures needed to control error or combine stagewise evidence.

Common pitfall: “Both phases used the same symptom score, so they estimate the same treatment effect.” This reasoning overlooks population, regimen, assessment time, event strategy, and summary measure. The correction is to compare all estimand attributes explicitly; matching one outcome label is not enough.

Check Your Understanding

For each item, first identify the issue, then compare your reasoning with the solutions.

Phase 2 studies 10 mg daily and phase 3 studies 20 mg daily, both against placebo. Are the treatment conditions identical?
Within two target-population subgroups, mean differences are \(-3\) and \(1\) points. The target population is 25% in the first subgroup and 75% in the second. Calculate the marginal contrast.
One stage measures outcomes after treatment discontinuation; the other targets outcomes under a hypothetical no-discontinuation scenario. Is the event strategy preserved?
Explain why standardizing subgroup effects to target-population proportions does not, by itself, prove that those effects apply to the target population.

Solutions. (1) No. The treatment condition differs by dose, so the stages describe different dose-versus-placebo contrasts. (2) The target contrast is \(0.25(-3)+0.75(1)=-0.75+0.75=0\) points. (3) No. The first is consistent with a treatment-policy strategy if outcomes are included regardless of discontinuation; the second is a hypothetical strategy and targets a different outcome scenario. (4) Standardization uses subgroup effects as inputs, but transport requires those effects to be relevant to the target population and the treatment and outcome definitions to be comparable. Reweighting does not establish those assumptions.