Interpreting Changes to an Estimand During a Trial

When a Protocol Change Changes the Question

A protocol amendment can affect more than how a trial is conducted. If it changes who the treatment effect is meant to describe, what outcome is measured, or how an event after treatment initiation is handled, it may change the estimand—the clinical quantity the trial is intended to estimate. A posterior distribution for the amended estimand answers a different question from a posterior distribution for the original one, even when both analyses use data from the same trial.

The key interpretive task is therefore not simply to ask whether the updated posterior favors treatment. First identify the target of that posterior and how it differs from the original target. Then assess whether the available data and the model’s assumptions support inference about the new target. A protocol amendment does not, by itself, make the original analysis wrong or make the amended analysis valid.

For either version of the estimand, reporting should make clear the treatment conditions being compared, the population, the outcome variable and time point, and the strategy for intercurrent events. Also identify which protocol version defined the analysis, when the amendment was made relative to data review, and which observations support the reported posterior. These details are essential to interpreting a Bayesian probability such as \(P(\Delta > c \mid D)\): its meaning depends on the definition of \(\Delta\), the model, the prior, and the data \(D\).

Population Changes and Marginal Effects

Changing eligibility criteria or the target population can alter a marginal treatment effect even if the treatment effect within each subgroup remains the same. A marginal effect averages over the subgroup composition of the specified target population. Consequently, a result for a population with mostly severe disease need not apply unchanged to a broader population that includes participants with moderate disease. The target population and any subgroup weights used in the analysis must be stated.

This example illustrates a general calculation. For fixed subgroup weights \(w_j\) that sum to one, a marginal effect can be written as \(\Delta_{\mathrm{target}}=\sum_j w_j\Delta_j\). Its posterior mean is the weighted sum of the subgroup posterior means. Its posterior variance is

This follows by applying the variance-of-a-linear-combination rule to the weighted sum. Omitting covariance is justified only when independence is a reasonable assumption or when the covariance terms are known to be negligible. A changed target population may also require data from subgroups that were not enrolled or were sparsely represented. In that case, the amended estimand may rely on extrapolation or transport assumptions rather than direct information from the trial.

Endpoint Changes and Intercurrent-Event Strategies

An endpoint amendment can change the outcome itself, its measurement scale, or its assessment time. For example, a symptom score at week 12 and the same score at week 24 are distinct outcomes, even when measured in the same units. A posterior from one endpoint cannot be described as if it were the posterior for the other. If the endpoint changes after data collection has begun, the analyst should check whether the amended outcome was collected consistently and whether its observation schedule, measurement procedures, and missingness are compatible with the target question.

An intercurrent event, such as treatment discontinuation or use of rescue medication, occurs after treatment initiation and affects how the outcome relates to the treatment condition. A treatment-policy strategy targets the effect of assignment while incorporating outcomes regardless of whether the event occurred. A hypothetical strategy targets an effect under a specified scenario in which the event did not occur. These strategies do not generally define the same clinical question. In particular, a hypothetical result depends on assumptions about outcomes under a scenario that may not be directly observed.

Deriving and Reporting the Amended Result

A useful way to assess an amendment is to trace how it affects the target quantity, the data needed to estimate it, and the interpretation of its posterior. This sequence helps distinguish a genuine change in the clinical question from a technical change in estimation.

A change to an estimator or model does not automatically change the estimand, but the model must still target the specified clinical quantity. Conversely, applying the same model to a new population, endpoint, or event strategy does not guarantee that the original posterior summary remains relevant. When the protocol has been amended, report the original and amended versions, the timing and rationale for the amendment, and whether the analysis used data available before or after the change.

Diagnostics, Alternatives, and Limitations

Several warning signs merit particular attention. A broadened population may contain few or no participants from a newly included subgroup. A changed endpoint may not have been collected at the amended time point. A hypothetical intercurrent-event strategy may depend heavily on untestable assumptions. In each case, a precise posterior can still be produced, but numerical precision does not resolve weak identification or limited data support.

Check whether the analysis population and observed data match the amended target; whether subgroup weights reflect the intended population; and whether the model allows the treatment effects and their uncertainty to vary appropriately across subgroups. For an endpoint change, check measurement timing and missing outcome patterns. For an intercurrent-event strategy, state which data inform the scenario and which parts depend on assumptions. Sensitivity analyses can show how posterior conclusions change under alternative plausible assumptions, but they do not make distinct estimands interchangeable.

The main alternatives are to retain the original estimand as primary, formally amend the primary estimand, or report both the original and amended estimands with a clear distinction between their roles. Retaining the original preserves the planned clinical question but may not address a newly relevant question. Amending the primary estimand makes the new question explicit, but requires careful documentation and evidence that the trial data can address it. Reporting both can clarify the impact of the change, provided neither is presented as a substitute for the other without explanation.

A common mistake is to say, “The posterior probability of benefit increased, so the treatment effect became more convincing.” If the amendment changed the target population, endpoint, or event strategy, the two probabilities may concern different clinical quantities. The correction is to name each estimand, compare its posterior only in that context, and explain what assumptions and data support each result. If a decision threshold is used, state its scale and clinical meaning for each estimand.