Separating Prognostic Drift From Treatment-Effect Drift

Two different kinds of change over enrollment

As a response-adaptive trial proceeds, outcomes may change with enrollment time. Later participants might, for example, have better outcomes in either arm because care improves or the enrolled population changes. That is a change in prognosis over calendar time. Separately, the contrast between treatment and control may itself grow, shrink, or reverse over enrollment. That is treatment-effect drift. The distinction matters: a trend in outcomes is not, by itself, evidence that the treatment effect is changing.

The key is to examine the trends in each arm and compare them. A similar trend in both arms points to a common change in outcome level. A difference between the arm-specific trends points to a changing treatment contrast, provided the outcome scale and model are stated. The introductory material on treatment effects that change over calendar time motivates the possibility of such changes; here the focus is how to distinguish them from a shared prognostic trend.

Definition: Prognostic drift is a change over enrollment time in expected outcomes that would be seen in the control condition, under a specified population, outcome definition, and time scale. Treatment-effect drift is a change over enrollment time in the treatment-versus-control contrast, on a specified outcome scale. A common trend in both arms can represent prognostic drift without treatment-effect drift.

Arm-specific trends and the interaction

Let \(A=0\) denote control and \(A=1\) denote treatment, and let \(t\) denote enrollment time. For a continuous outcome, a simple linear mean model is

E(Y\mid A,t)=\beta_0+\beta_A A+\beta_t t+\beta_{At}At.

Here, \(\beta_0\) is the mean for control at \(t=0\); \(\beta_t\) is the control-arm trend per unit of time; \(\beta_A\) is the treatment-control difference at \(t=0\); and \(\beta_{At}\) is the difference between the two arm-specific trends. The model implies the following arm-specific means and treatment contrast:

\begin{aligned} E(Y\mid A=0,t)&=\beta_0+\beta_t t,\\ E(Y\mid A=1,t)&=(\beta_0+\beta_A)+(\beta_t+\beta_{At})t,\\ E(Y\mid A=1,t)-E(Y\mid A=0,t)&=\beta_A+\beta_{At}t. \end{aligned}

Thus, \(\beta_t\) describes the control trend, while \(\beta_t+\beta_{At}\) describes the treatment-arm trend. If \(\beta_{At}=0\), the trends are parallel and the treatment contrast is constant in this model. If \(\beta_{At}\ne 0\), the contrast changes with time. Whether that change is beneficial depends on the outcome’s direction: for an outcome where lower values are better, a more negative contrast is more favorable to treatment.

Interpretation: In the linear model, the treatment-by-time coefficient \(\beta_{At}\) is the change in the treatment-control contrast per unit of enrollment time. It is not the common time trend. Its value, and the treatment contrast it describes, are specific to the chosen outcome scale and time coding.

Why comparing the trends separates the effects

The separation follows from subtracting the control trend from the treatment trend. Over a time increase of one unit, the model predicts a control-arm change of \(\beta_t\) and a treatment-arm change of \(\beta_t+\beta_{At}\). Their difference is

(\beta_t+\beta_{At})-\beta_t=\beta_{At}.

Equivalently, compare the treatment-control contrast at two times, \(t_1\) and \(t_2\). The difference between those contrasts is

\begin{aligned} &\{E(Y\mid A=1,t_2)-E(Y\mid A=0,t_2)\}\\ &\quad-\{E(Y\mid A=1,t_1)-E(Y\mid A=0,t_1)\}\\ &=\beta_{At}(t_2-t_1). \end{aligned}

This is a difference-in-differences: a shared time trend cancels, leaving the change in the treatment contrast. The calculation describes a model-based contrast; it does not, on its own, establish that the model fits the data or that a causal interpretation is warranted.

The time origin also affects interpretation. If time is recentered by defining \(t'=t-c\), then the treatment contrast at the new origin changes from \(\beta_A\) to \(\beta_A+\beta_{At}c\), while the interaction coefficient stays the same. Consequently, report the time origin when interpreting a treatment main effect in an interaction model. The interaction has units of outcome per unit of time; the treatment contrast has units of outcome.

Conditions for a meaningful distinction

The model is a useful description only with assumptions suited to the question. For a causal interpretation of arm contrasts, a response-adaptive design must randomize each participant according to its specified assignment mechanism, conditional on the information available at that assignment. The assigned treatment must be well defined, outcomes must be ascertained comparably across arms and time, and one participant’s assignment must not change another participant’s outcome. There must also be a meaningful chance of observing both arms across the time range being compared. If an arm is absent from part of enrollment, its trend there cannot be learned directly from concurrent comparisons.

The linear model additionally assumes that the conditional mean in each arm is adequately represented by a straight line over the time range considered. A fitted interaction is not a substitute for checking whether that shape is plausible. When these assumptions are doubtful, the coefficient may summarize a poor approximation rather than a stable feature of the trial.

Interpretation also depends on what “time” and “prognosis” mean. A calendar-time trend can reflect changes in clinical practice, eligibility, recruitment, or measurement; it need not represent improvement in the same individuals. If the mix of participants changes, arm-specific means may change even when no individual’s underlying condition changes. Randomization supports treatment comparisons within the design, but it does not make changing cohorts identical or make every observed time trend causal.

Worked Example: Shared Prognostic Drift, Stable Treatment Contrast

In a hypothetical trial, the outcome is symptom burden in points, with lower values better. Enrollment is divided into an early period (\(t=0\)) and a later period (\(t=1\)). Suppose the arm means are 10 points for control and 7 for treatment early, then 14 and 11 points later. Assume both arms have participants in both periods and the means are measured in the same way.

The control trend is \(14-10=4\) points per period. The treatment trend is \(11-7=4\) points per period. The early treatment-control contrast is \(7-10=-3\) points, and the later contrast is \(11-14=-3\) points.

The difference-in-differences is

(11-14)-(7-10)=(-3)-(-3)=0\text{ points per period}.

The observed pattern is consistent with a common increase of 4 points per period in both arms and a stable 3-point lower mean in the treatment arm. In the two-period model, \(\beta_t=4\) points per period and \(\beta_{At}=0\). The arm means alone do not show that treatment-effect drift occurred. This descriptive calculation does not quantify sampling uncertainty, so it is not by itself a test of whether the interaction differs from zero.

Worked Example: A Treatment Contrast That Changes

Consider a second hypothetical trial with the same outcome and time coding. The early means are 12 points in control and 8 in treatment; the later means are 16 and 10 points, respectively. Assume the outcome definition and measurement procedures are unchanged, and both arms are represented in both periods.

The control trend is \(16-12=4\) points per period. The treatment trend is \(10-8=2\) points per period. The early treatment-control contrast is \(8-12=-4\) points; the later contrast is \(10-16=-6\) points. Therefore,

(10-16)-(8-12)=(-6)-(-4)=-2\text{ points per period}.

The model’s time coefficient is \(\beta_t=4\), and its interaction coefficient is \(\beta_{At}=2-4=-2\), both in points per period. The common prognostic trend in control is an increase of 4 points per period, while the treatment contrast becomes 2 points more favorable per period on this outcome scale. The values describe the hypothetical means; deciding whether the data support a nonzero interaction requires an uncertainty analysis that respects the trial design.

Worked Example: The Outcome Scale Changes the Interaction

Suppose a hypothetical binary-outcome trial reports response proportions. The control response proportion rises from 0.50 early to 0.70 late, while the treatment proportion rises from 0.60 to 0.90. Both arms are represented at both times, and response is defined consistently.

On the risk-difference scale, the early treatment contrast is \(0.60-0.50=0.10\), or 10 percentage points. The late contrast is \(0.90-0.70=0.20\), or 20 percentage points. The difference-in-differences is \(0.20-0.10=0.10\), an increase of 10 percentage points in the treatment contrast from early to late. The control response also rises by 20 percentage points, which is a prognostic trend on this scale.

This conclusion is scale-specific. An interaction on the risk-difference scale need not match an interaction on an odds-ratio or log-odds scale, because those scales define treatment contrasts differently. The arm-specific proportions and the contrast scale should therefore be stated before describing treatment-effect drift.

Comparisons, diagnostics, and limits

A pooled time trend without an arm-by-time term can obscure the distinction: it forces the two arms to share a slope, even if their trends differ. Conversely, inspecting separate arm trends without comparing their difference can lead to the mistaken conclusion that any change in either arm is treatment-effect drift. The interaction directly represents the difference in slopes in the specified model. Period-specific treatment contrasts can make the pattern easier to communicate, but estimates from sparse periods may be imprecise.

Observed pattern	What it suggests on the stated scale	What it does not establish by itself
Both arms change similarly over time	Common prognostic drift; little evidence of a changing contrast in this pattern	That the interaction is exactly zero or the change has a particular cause
Arm trends differ	A changing treatment contrast may be present	That the difference is causal, statistically precise, or linear throughout enrollment
Only one arm has observations in a time region	Arm trends cannot be compared there without extrapolation	That an interaction model can recover the missing concurrent contrast reliably

Useful checks include plotting or tabulating outcomes by arm over enrollment time, verifying that both arms contribute observations across the comparison range, and checking that the time scale, outcome definition, and measurement process are consistent. Look for curvature or abrupt changes that a straight-line trend would conceal. Also examine whether a few observations or a narrow time region dominate the fitted interaction. These checks diagnose whether the proposed distinction is supported by the observed data; they do not prove that a trend is biological or that no unmeasured changes occurred.

A major limitation is that a changing arm contrast can arise from changing participant composition as well as from a time-varying biological treatment effect. The interaction describes how the average contrast changes across enrolled participants and time under the model. A more specific explanation requires additional evidence and assumptions. Likewise, a nonsignificant interaction is not proof of a constant treatment effect: limited sample size, little temporal overlap, or a misspecified trend can make changes difficult to detect.

Common mistake

Wrong reasoning: “The control mean improved over enrollment, so the treatment effect must have changed.” This confuses a change in one arm’s outcomes with a change in the difference between arms. Correction: calculate or estimate the treatment-control contrast at relevant times, then compare those contrasts. If both arms move by the same amount, the common trend cancels; if their trends differ, the contrast changes on the chosen scale. Neither pattern alone identifies the cause of the drift.

Key takeaway: The control-arm time trend describes prognostic drift in the stated model; the treatment-by-time interaction describes how the treatment contrast changes over time. Interpret both with the time origin, outcome scale, support across arms, and model assumptions in view.

Check Your Understanding

Use the stated outcome scale and show the contrast calculations. Each question concerns hypothetical data.

Control means are 5 and 8 at early and late enrollment; treatment means are 3 and 6. What are the control trend, treatment trend, and difference-in-differences? What pattern do they indicate?
Control means are 20 and 24; treatment means are 17 and 22. Calculate the early and late treatment-control contrasts and their difference. Interpret the direction of change if lower outcomes are better.
In a linear model with \(t=0\) at the start, \(\beta_t=1.5\) points per month and \(\beta_{At}=-0.4\) points per month. What is the treatment-arm trend, and how does the treatment contrast change per month?
Why is a treatment-by-time interaction difficult to interpret if one arm has no observations late in enrollment?

Solutions. (1) The control trend is \(8-5=3\) units and the treatment trend is \(6-3=3\) units. The contrasts are \(3-5=-2\) early and \(6-8=-2\) late, so the difference-in-differences is \((-2)-(-2)=0\). The pattern is a common increase with a stable observed contrast. (2) The early contrast is \(17-20=-3\) units and the late contrast is \(22-24=-2\) units. The difference is \((-2)-(-3)=1\) unit, so the treatment contrast becomes 1 unit less favorable per early-to-late interval when lower outcomes are better. (3) The treatment-arm trend is \(\beta_t+\beta_{At}=1.5-0.4=1.1\) points per month. The treatment-control contrast decreases by 0.4 points per month; with lower outcomes better, this is a more favorable contrast over time. (4) Without observations from both arms in the late period, the late contrast cannot be estimated directly from concurrent data. A model may extrapolate the missing arm’s trend, but that comparison then depends on the model’s assumptions rather than observed overlap.