Abrupt Protocol Changes and Discontinuities in Outcomes

Why a Protocol Change Needs Its Own Time Term

During a trial, a protocol revision or change in standard care can coincide with a sudden shift in outcomes. This may reflect a real change in participants’ prognosis or treatment context, but it can also arise because eligibility, measurement, follow-up, or outcome ascertainment changed. Meanwhile, outcomes may drift gradually over calendar time. These patterns are distinct: a gradual trend does not, by itself, imply a jump at the change date.

A model that includes only a single linear calendar-time trend forces the fitted mean to change smoothly. It cannot represent an abrupt level shift. Conversely, comparing average outcomes before and after a change without accounting for time trend can mistake gradual drift for a discontinuity. A segmented model makes these components explicit.

Definition: Let $t_i$ be participant $i$'s enrollment time on a numerical scale, and let $c$ be a prespecified protocol-change time. Define $D_i=I(t_i\ge c)$, an indicator for enrollment at or after the change, and $h_i=\max(t_i-c,0)$, the post-change hinge term. A segmented linear mean model is $$ E(Y_i\mid t_i)=\beta_0+\beta_1(t_i-c)+\beta_2D_i+\beta_3h_i. $$ Here $\beta_2$ is the level discontinuity at $c$, while $\beta_1$ is the pre-change slope and $\beta_1+\beta_3$ is the post-change slope.

The time origin is centered at $c$, so the intercept refers to the fitted mean immediately before the change. By convention, participants enrolled exactly at $c$ are classified as post-change; the protocol record must make that classification meaningful. If exposure to the revised protocol depends on a different date, such as randomization or treatment initiation, use and justify the relevant time definition rather than assuming enrollment date is adequate.

What the Segmented Terms Mean

For $t_i<c$, $D_i=0$ and $h_i=0$, so the fitted mean is $\beta_0+\beta_1(t_i-c)$. For $t_i\ge c$, it is $\beta_0+\beta_1(t_i-c)+\beta_2+\beta_3(t_i-c)$. Therefore the fitted value at the boundary on the pre-change side is $\beta_0$, and on the post-change side it is $\beta_0+\beta_2$. Their difference is $\beta_2$. The change in slope is $\beta_3$.

Interpretation: A nonzero $\beta_2$ represents a fitted abrupt change in the outcome mean at the specified boundary, conditional on the model terms. A nonzero $\beta_3$ represents a change in the rate of linear time drift after that boundary. Neither coefficient alone establishes that the protocol revision caused the change.

The model assumes that a straight-line trend is an adequate description on each side of the change, with a possible jump and a possible slope change. For a continuous outcome, a typical analysis also assumes that the conditional mean is correctly specified and uses an appropriate uncertainty estimate for the outcome distribution and dependence structure. Repeated measurements, site clustering, or other dependence require methods that account for that dependence. The change time should be determined from the protocol history, not selected merely because it produces a striking outcome pattern.

Worked Example: Separating a Jump From a Smooth Trend

Suppose a hypothetical trial records mean outcome values at six successive monthly enrollment times. A protocol change takes effect at month 4. The reported monthly means are 10, 11, 12, 16, 17, and 18 outcome units at months 1 through 6, respectively. For this illustration, treat these values as the means to be fitted by the segmented model.

Set $c=4$. The centered times $t-c$ are $-3,-2,-1,0,1,2$. The first three values increase by 1 unit per month, so the pre-change line has slope $\beta_1=1$. Extrapolating that line to month 4 gives $12+1=13$, hence $\beta_0=13$. At month 4 the observed mean is 16, so the jump is $16-13=3$, giving $\beta_2=3$. The post-change values then increase by 1 unit per month, the same as before, so $\beta_3=0$.

\widehat{E}(Y\mid t)=13+1(t-4)+3I(t\ge4)+0\max(t-4,0).

For example, the fitted mean at month 2 is $13+1(-2)=11$, and at month 5 it is $13+1(1)+3=17$, matching the listed values. In this constructed pattern, the model describes a 3-unit abrupt increase at month 4 and no change in the linear slope. In real data, uncertainty around the fitted coefficients would also matter; the six values alone do not establish a precise or causal protocol effect.

Adding Treatment and Allocation Patterns

In a randomized trial, the outcome pattern should generally be examined by treatment arm as well as overall. A protocol or standard-care change may shift both arms similarly, or it may alter the treatment contrast. Let $A_i=1$ denote assignment to the treatment of interest and $A_i=0$ the comparator. One model that permits both possibilities is

E(Y_i\mid A_i,t_i)=\beta_0+\beta_AA_i+\beta_1(t_i-c)+\beta_2D_i+\beta_3h_i+\beta_{AD}A_iD_i.

Under this parameterization, the treatment contrast before the change is $\beta_A$, while the contrast after the change is $\beta_A+\beta_{AD}$. The common discontinuity for the comparator is $\beta_2$; for the treatment arm it is $\beta_2+\beta_{AD}$. This model assumes a common time slope across arms within each period. If arm-specific slopes are scientifically justified and supported by the data, treatment-by-time terms can be added, but they should not be introduced mechanically.

Worked Example: A Common Shift and a Changing Treatment Contrast

In a hypothetical trial, adjusted outcome means in units are 20 for the comparator and 16 for treatment before a protocol change, and 24 for the comparator and 19 for treatment after it. For this cell-mean illustration, suppose the time trend has already been accounted for, so the values are comparable at the change boundary.

Before the change, the comparator mean gives $\beta_0=20$, and the treatment contrast is $16-20=-4$, so $\beta_A=-4$. The comparator’s level shift is $24-20=4$, so $\beta_2=4$. After the change, the model predicts the treatment mean as $20-4+4+\beta_{AD}$. Equating this to the observed value 19 gives

19=20-4+4+\beta_{AD},\qquad \beta_{AD}=-1.

Thus the modeled treatment contrast changes from $-4$ outcome units before the change to $-4+(-1)=-5$ units after it. The comparator has a 4-unit upward discontinuity, while the treatment arm has a $4-1=3$-unit upward discontinuity. These descriptive model contrasts do not show that the protocol change caused either shift. In particular, if the change also altered who enrolled or how outcomes were measured, those changes affect interpretation.

Allocation patterns should be examined alongside outcomes because allocation probabilities may change over enrollment, including when the randomization is response-adaptive. Let $\pi_i=P(A_i=1\mid H_i)$ denote participant $i$'s known assignment probability given the history $H_i$ available immediately before that assignment. A protocol change may coincide with a shift in these probabilities, an update to the allocation rule, or a change in the distribution of participants. Summaries of observed treatment counts and recorded probabilities can describe what happened, but their interpretation depends on how the probabilities were generated.

Worked Example: Recorded Adaptive Probabilities Are Not Expected Counts

Consider a hypothetical two-participant adaptive randomization. Participant 1 has treatment probability $0.5$. Participant 2 has treatment probability $0.1$ if participant 1 received treatment, and $0.9$ otherwise. Let $A_i=1$ indicate treatment assignment.

The unconditional expected number assigned to treatment is calculated sequentially. Participant 1 contributes $E(A_1)=0.5$. Participant 2’s unconditional treatment probability is $0.5(0.1)+0.5(0.9)=0.5$, so

$$ E(A_1+A_2)=0.5+0.5=1. $$

But the realized sum of recorded probabilities is $0.5+0.1=0.6$ if participant 1 received treatment, and $0.5+0.9=1.4$ otherwise. These sums are not the expected treatment counts conditional on the realized probability sequence: that sequence reveals participant 1’s assignment. Given the first sequence, the conditional expected count is $1+0.1=1.1$; given the second, it is $0+0.9=0.9$.

The general unconditional identity is $E[\sum_i A_i]=E[\sum_i\pi_i]$, because $E(A_i\mid H_i)=\pi_i$ for each participant. When probabilities are fixed in advance, their sum is the expected count. With adaptive probabilities, the realized sum is a sum of recorded probabilities; it should not be described as the conditional expected count given that realized sequence.

Assumptions, Checks, and Alternatives

A segmented model is most interpretable when the change time and the nature of the protocol revision are documented, outcome definitions are comparable across the boundary, and there are observations on both sides with adequate representation of the treatment arms. The model also requires a plausible within-segment trend. If outcomes curve substantially within either segment, a straight-line slope may be misleading; if the apparent jump is driven by a measurement change, it may describe the measurement process rather than a clinical shift.

Useful checks include plotting outcomes over enrollment time, marking the protocol boundary, and displaying treatment arms separately. Examine whether the observed pattern supports a jump, a slope change, both, or neither, and inspect residual patterns for remaining curvature or an unmodeled shift. Check protocol logs and outcome definitions to establish whether the boundary and measurement process are correctly represented. These checks diagnose model mismatch; they do not turn a nonrandomized protocol change into a randomized intervention.

The main alternatives answer different questions. A single linear time term describes smooth drift but cannot represent a jump. Time-stratified summaries can show local contrasts without imposing one linear trend, but may be noisy and depend on the chosen time groups. Flexible smooth trends can represent curvature, but a smooth function alone does not encode a prespecified discontinuity. The segmented model is useful when a specific boundary is substantively meaningful and a piecewise linear description is reasonable. Its limitation is that a single boundary and simple slopes may not capture several changes or complex evolution.

A before-and-after difference is not automatically an effect of the protocol revision. For a causal interpretation, one needs a defensible account of what would have happened without the change, including the absence of other coincident causes of the discontinuity and appropriate control of changes in participant composition and outcome measurement. Randomization of treatment can support treatment comparisons within periods, but it does not randomize calendar time or the protocol revision. If treatment allocation also changes sharply, interpretation further relies on valid within-period comparisons and adequate representation of both arms.

Common Mistakes

Calling any before-after difference a jump. A difference between period averages can arise from smooth drift. Fit and interpret the time trend as well as the level indicator.
Using only a slope-change term. A hinge term changes the post-change slope but is zero at the boundary. Include $D_i$ when an abrupt level shift is possible.
Calling the fitted jump a causal protocol effect. The coefficient is a model-based discontinuity. Concurrent changes in enrollment, care, measurement, or follow-up can also explain it.
Treating adaptive probability sums as expected counts conditional on their realized sequence. For adaptive allocation, distinguish the realized sum of recorded probabilities from the unconditional expected count $E[\sum_i\pi_i]$.
Assuming an overall outcome trend describes both arms. Display and model arm-specific patterns when the treatment contrast may change over time.

Key takeaway: At a prespecified change time, use a level indicator to represent a discontinuity and a post-change hinge to represent a slope change. Interpret both in context, check arm-specific and allocation patterns, and avoid attributing an observed jump to the protocol without additional causal assumptions.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions below.

In the segmented model, what does the coefficient of the post-change indicator represent?
Before a change, the fitted mean is 30 at the boundary and 28 one time unit earlier. What is the pre-change slope?
If the comparator’s fitted level shift is 2 units and the treatment-by-post-change coefficient is $-0.5$, what is the treatment arm’s fitted level shift?
Why can the difference between the average outcome before and after a protocol change overstate or understate the discontinuity?
With adaptive assignment probabilities, what is the correct unconditional expectation identity for the total number assigned to treatment?

Solutions. (1) It is the fitted difference between the post-change and pre-change levels at the boundary, conditional on the other model terms. (2) The slope is $(30-28)/1=2$ outcome units per time unit. (3) The treatment arm’s shift is $2+(-0.5)=1.5$ units. (4) The period averages can include gradual drift and differences in participant composition, not just an abrupt boundary change. (5) The identity is $E[\sum_i A_i]=E[\sum_i\pi_i]$; with adaptive probabilities, a realized probability sum is not generally the conditional expected count given that realized sequence.