Outcome Residual Plots for Temporal Drift

Why Plot Outcome Residuals Against Enrollment Time?

In a response-adaptive trial, the sequence of enrollment matters: allocation probabilities may change as outcome information accumulates, and outcomes may also vary over calendar time. An outcome residual plot provides a visual check for temporal structure that remains after accounting for a chosen reference model. It can reveal patterns that a single overall mean would conceal, including gradual change, curvature, repeated cycles, or an abrupt shift.

The plot is a diagnostic, not a stand-alone test of temporal drift. In particular, a pattern in pooled outcomes does not by itself show why outcomes vary over time. Changes in participant prognosis, the mix of assigned treatments, treatment effects that vary over time, and other changes in trial conduct can all contribute. The earlier material on calendar-time drift and treatment-effect drift motivates this caution; here the focus is how to construct and read the plot.

A treatment-blind model excludes treatment assignment as a predictor. It may use a prespecified reference mean, or baseline predictors known before assignment. For an initial diagnostic, it is often useful not to include a flexible function of enrollment time: otherwise, the model can absorb the very pattern the plot is meant to reveal. The model is a benchmark, not a claim that outcomes truly follow that benchmark.

The endpoint and its scale must be defined consistently. For a continuous outcome, residuals have the same units as \(Y_i\). For a binary outcome coded \(0\) or \(1\), a model of the outcome mean gives residuals in probability units. In either case, larger positive residuals mean observed outcomes are above the model’s fitted means; whether that is clinically favorable depends on how the endpoint is defined.

How to Construct and Read the Plot

First define enrollment time. A participant’s sequence number is useful for detecting order-related patterns, while calendar dates or elapsed days may be more suitable for identifying calendar cycles or external events. Unequal gaps between enrollments matter: points equally spaced on the horizontal axis should represent equal intervals only when the time scale supports that interpretation.

Next specify the treatment-blind reference model, calculate its fitted means, and subtract each fitted mean from the corresponding outcome. Plot the residuals at their enrollment times. A zero line marks the model’s prediction; a smooth visual guide may help summarize the pattern, but it should not replace inspection of the individual observations. If a trend is apparent, check whether it is driven by one or two unusual outcomes, a cluster of similar observations, or a sustained sequence.

A Short Justification for the Zero Line and Linear Summary

For an intercept-only reference model, each fitted value is \(\bar{x}\), so \(e_i=Y_i-\bar{x}\). The residuals sum to zero:

Thus, the residuals balance around zero for this model; zero is not an estimated temporal boundary. A simple numerical summary of a possible linear pattern is the least-squares slope from regressing these residuals on enrollment time. With an intercept, that slope is

The denominator is positive when the enrollment times are not all identical, so the sign of the slope is the sign of the numerator: positive means residuals tend to be higher at later times, and negative means they tend to be lower. This summary does not detect every pattern. A curved, cyclic, or step-like sequence can have a slope near zero.

Assumptions, Diagnostics, and Failure Modes

A residual plot does not require outcomes to be normally distributed or observations to be independent merely to be drawn. Those assumptions may matter for formal inference, but visual diagnosis alone does not supply a valid test or confidence interval. The practical requirements are more basic: outcomes must be defined on a consistent scale, enrollment times must be recorded correctly, fitted values must correspond to the same participants as the observed outcomes, and the reference model must be described clearly.

Outcome availability also matters. If later enrollees have had less time for outcomes to mature, plotting only observed outcomes can create a misleading temporal pattern. Restricting analysis to outcomes that are observed under a comparable follow-up definition, or otherwise accounting for incomplete outcome information, is necessary before interpreting the plot. A changing endpoint definition or measurement procedure can also produce a visible shift that is not a change in underlying prognosis.

Use these checks when reading the figure:

• Look for shape, not only slope. A line-like rise or fall, a U-shape, repeated cycles, and a step are distinct visual patterns. A near-zero linear slope does not rule out the latter three.
• Check sensitivity to the reference model. Adding baseline predictors can change residuals because the model now accounts for some differences in participant composition. Adding a flexible time trend can erase temporal structure by construction, so it should not be the only diagnostic view.
• Inspect the time scale. Sequence order can reveal ordering patterns but cannot, by itself, establish a calendar-period cycle. Calendar dates can reveal timing but may have irregularly spaced observations.
• Inspect the points behind a smooth guide. A single extreme outcome can pull a fitted line or smooth curve. A pattern supported by many consecutive residuals is visually different from one driven by one point.
• Check outcome and enrollment records. Missing or delayed outcomes, changes in measurement, or unusual enrollment gaps can create apparent structure.

The plot is treatment-blind only in the sense that treatment assignment was not used to fit the reference model. In an adaptive trial, the proportions assigned to different treatments can change over enrollment time, and treatments can have different outcome distributions. Consequently, a pooled residual pattern can reflect the changing treatment mix as well as changes in prognosis or other time-related processes. This is why the plot is a signal to investigate, not a diagnosis of a particular cause.

How This Diagnostic Compares With Alternatives

A plot of raw outcomes against enrollment time is more direct, but a baseline-adjusted residual plot can make departures from a chosen reference easier to see when measured baseline predictors account for substantial outcome variation. Conversely, a residual plot depends on the fitted reference model, while the raw plot does not. Viewing both can help show whether the apparent temporal structure is robust to that choice.

An allocation-probability timeline or an arm-by-time enrollment table describes how assignments were distributed over time; it does not show the outcome pattern. The residual plot provides a complementary view of pooled outcomes, but it does not establish that an allocation change caused an outcome pattern. Formal model-based analyses can quantify specified temporal associations, but their conclusions depend on their model and assumptions. The plot is useful before such analyses because it can reveal shapes a single linear summary would miss.

Common Mistake