From Timeline Displays to a Defensible Contrast
In a response-adaptive trial, treatment allocation can change as participants accrue. If prognosis or outcome measurement also changes over enrollment, an overall difference between treatment arms can mix treatment differences with calendar-time differences. The analysis task is not simply to add a time variable to a regression. It is to follow a prespecified workflow: inspect how allocation and outcomes evolve, choose a time adjustment that the design can support, check overlap, examine sensitivity, and state precisely which treatment contrast the analysis estimates.
The workflow below concerns a treatment contrast adjusted for recorded enrollment time. It does not make a time-adjusted analysis automatically valid: conclusions still depend on the trial design, the recorded time variable being adequate for the relevant drift, the model being suitable for the stated contrast, and the availability of both treatments across the times being compared. The earlier material on outcome regression and weighting for temporal balance, positivity, and drift diagnostics supplies the underlying methods; the aim here is to connect them into an auditable sequence of decisions.
Prespecify the Decisions Before Examining Treatment Results
A usable workflow begins in the analysis plan, not after an adjusted estimate looks inconvenient. Prespecify the primary outcome scale, the time index and its resolution, the primary time-adjustment form, the treatment contrast, and the approach to uncertainty. Also specify what diagnostics will trigger caution or prevent a trial-wide adjusted interpretation. Time blocks should have a rationale, such as the enrollment schedule or a known operational epoch, rather than being cut at a point that maximizes the apparent treatment difference.
Before looking at outcome differences by treatment, review the allocation timeline. At minimum, tabulate the number assigned to each arm in each planned time block and, if recorded, the treatment assignment probabilities used during enrollment. Plot or tabulate outcomes over time as well, displaying treatment arms separately where possible. These displays help identify gradual drift, abrupt changes, sparse cells, and possible treatment-by-time patterns. They are diagnostics, not an invitation to choose whichever model produces the preferred estimate.
The primary model should reflect the planned contrast. For a continuous outcome and a common additive treatment effect across time, one possible model is:
Here \(Y_i\) is the outcome, \(\beta_0\) is an intercept, \(\tau\) is the treatment-minus-control contrast on the outcome scale, and \(f(T_i)\) represents the prespecified time adjustment. This model assumes that, conditional on the included time terms, the treatment contrast is common over time. It also assumes that the chosen time terms adequately represent the relevant time pattern for the intended interpretation. If the contrast varies over time, a single common \(\tau\) may not answer the scientific question; the target would need to specify how time-specific contrasts are summarized.
The form of \(f(T_i)\) is a design decision informed by the time scale and known trial events. A linear term may be a reasonable planned approximation for gradual drift. Prespecified time indicators can represent planned blocks or distinct operational epochs. A more flexible smooth can represent nonlinear gradual change if the amount of data supports it. A sudden, documented change should not automatically be forced into a smooth trend. Conversely, dividing time into many small blocks can create sparse treatment-by-time cells and unstable estimates.
A Short Justification for Checking the Crude Contrast
The overall, unadjusted treatment difference is a useful descriptive check, but it need not estimate the same comparison as a time-standardized contrast. For a binary time indicator \(L\), let \(\bar{Y}_{a,l}\) be the mean outcome among participants assigned to arm \(a\) in time group \(l\). The overall mean in arm \(a\) averages those cell means using that arm’s own distribution of enrollment times:
Therefore, subtracting the overall means compares averages that can use different time weights. A standardized comparison instead applies a common set of time weights \(w_l\) to both arms:
This is a direct justification for checking allocation patterns and stating the time target: when arm-specific enrollment-time distributions differ, the crude difference and standardized contrast can differ even if the within-time contrasts are stable.
Worked Example: A Crude Difference Reverses After Time Adjustment
Suppose a hypothetical trial has an early and a late enrollment period. The outcome is a continuous score, with larger values indicating worse status. The cell counts and mean scores are as follows.
| Enrollment period | Control: n | Control: mean | Treatment: n | Treatment: mean |
|---|---|---|---|---|
| Early | 18 | 10 | 2 | 8 |
| Late | 2 | 20 | 18 | 18 |
The crude treatment mean is \((2\times8+18\times18)/20=340/20=17\) score points. The crude control mean is \((18\times10+2\times20)/20=220/20=11\) points. Thus, the unadjusted treatment-minus-control difference is \(17-11=6\) points, appearing worse under treatment.
Within the early period, the difference is \(8-10=-2\) points; within the late period it is \(18-20=-2\) points. A model with a late-period indicator and a common additive treatment effect therefore gives a treatment contrast of \(-2\) points: at either period, the observed treatment mean is 2 points lower. This matches the standardized contrast for any common set of early and late weights:
The crude and adjusted results differ because 18 of 20 treatment participants, but only 2 of 20 control participants, enrolled late, when scores were higher in both arms. The adjusted estimate is not proof that the model is correct: each treatment-by-period cell must be inspected, and the cells with only two participants imply substantial uncertainty. The example shows why a crude contrast alone is not an adequate summary when allocation and outcome levels both change over time.
The Workflow: Screen, Model, Check, Interpret
State the outcome scale, target treatment contrast, time index, primary adjustment form, and uncertainty method before comparing treatment results. If the intended target is a trial-wide average, specify the time distribution used for that average.
Tabulate counts by arm and time block; inspect assignment probabilities if available. Look for empty or very small cells, abrupt allocation changes, and intervals in which one arm is rarely assigned.
Display outcomes by time and arm, and document known changes in enrollment, care, measurement, or protocol. Use these displays to diagnose whether the prespecified time representation is plausible, not to search across models for the smallest p-value.
Check whether each time region contributing to the target contrast contains information from both arms. A fitted regression does not create treatment observations in a region where an arm was never assigned.
Compare with a small set of scientifically justified alternatives. Report the primary estimate, its uncertainty, the time target, diagnostics, and any material dependence on the model or on extrapolation.
Choosing and Checking the Time Adjustment
The time model should be judged by a combination of design knowledge and diagnostic adequacy. A linear trend is concise but can miss curvature or a jump. Block indicators are transparent but can use many parameters and become unstable when cells are sparse. A smooth trend can accommodate gradual nonlinearity, but excessive flexibility can fit noise and amplify instability. The important distinction is between a planned primary analysis and sensitivity analyses: alternatives assess robustness; they do not replace the primary model merely because they produce a more appealing result.
Check overlap directly in the time scale used for analysis. In each relevant block, report the number assigned to both arms and, where informative, the assignment proportions. For continuous time, use time plots and summaries that reveal regions with little or no representation of one arm. The formal requirement is that, throughout the time support included in the target, both treatments have a positive chance of assignment. Empirical zeros or near-zeros are warnings that the data may not support the desired contrast, even if the assignment mechanism allowed both treatments in theory.
Also examine whether a common-effect model is plausible. If treatment contrasts visibly differ across time, the fitted common effect is a model-dependent summary, not a universal effect at every time. A treatment-by-time analysis can help describe that variation, but any summarized contrast still requires a defined time target and adequate overlap. Do not interpret the presence or absence of a significant interaction test as proof that effects are identical or different.
Worked Example: An Abrupt Operational Change Favors a Prespecified Epoch Term
Consider a hypothetical trial with four enrollment quarters. Before enrollment begins, the protocol records that a clinical-care procedure will change at the start of quarter 3. The outcome remains measured on the same scale throughout. A planned analysis uses one indicator for the post-change epoch rather than assuming the entire outcome trend is linear.
| Quarter | Control mean score | Treatment mean score | Treatment allocation proportion |
|---|---|---|---|
| 1 | 10 | 8 | 0.30 |
| 2 | 11 | 9 | 0.40 |
| 3 | 17 | 15 | 0.60 |
| 4 | 18 | 16 | 0.70 |
The within-quarter treatment-minus-control differences are \(8-10=-2\), \(9-11=-2\), \(15-17=-2\), and \(16-18=-2\) points. The means rise by 1 point from quarter 1 to 2, jump by 6 points from quarter 2 to 3, and rise by 1 point from quarter 3 to 4. That pattern is compatible with a change in epoch plus a gradual within-epoch trend; a single straight line across all four quarters would smooth over the documented jump.
The allocation proportions also rise across enrollment, so the analysis must check arm counts in each quarter rather than assuming the observed means alone guarantee overlap. If the prespecified model uses an epoch indicator and the data contain both arms in each quarter, the estimated common treatment contrast can be interpreted conditionally on that adjustment, subject to the common-effect and model assumptions. The documented operational change justifies the primary time representation; the endpoint pattern is a diagnostic, not the reason to select the model after seeing which treatment estimate it yields.
Sensitivity, Failure Modes, and Alternatives
A useful sensitivity analysis varies a plausible modeling choice while preserving the target contrast as much as possible. For example, compare a prespecified linear time term with planned epoch indicators when both are scientifically defensible, or compare the primary outcome-regression estimate with a planned weighting analysis. Report whether estimates and uncertainty change materially and identify why. Agreement across analyses is reassuring only to the extent that they share the relevant assumptions; it is not a guarantee of validity.
When overlap is weak, model sensitivity is especially important. A flexible model can generate a numerical estimate in an empty cell by extrapolating from other times, but the estimate is then driven by its functional-form assumptions rather than direct within-time comparisons. Weighting can also become unstable when some participants have very small assignment probabilities. Earlier material on positivity and temporal weighting describes these issues in detail. The workflow consequence is straightforward: diagnose the problem, state the supported target, and avoid presenting unsupported extrapolation as a data-driven trial-wide comparison.
The main alternatives answer related but not identical questions. A crude overall difference uses each arm’s observed time distribution and may be distorted by different enrollment patterns. Outcome regression adjusts through a specified outcome model. Weighting adjusts through assignment or time-balance weights and can be sensitive to extreme weights. A block-based analysis compares within planned periods but may be imprecise in sparse cells. The primary choice should follow the prespecified estimand and available data, while alternatives serve as checks rather than post hoc selection tools.
Worked Example: Sparse Overlap Limits the Trial-Wide Target
Suppose another hypothetical trial has three planned enrollment periods with these counts: early, 30 treatment and 0 control; middle, 12 treatment and 12 control; late, 0 treatment and 30 control. The allocation proportions are \(30/30=1.00\) early, \(12/24=0.50\) middle, and \(0/30=0.00\) late.
The middle period has observed representation in both arms, but the early and late periods have none in one arm. Suppose the observed middle-period mean is 14 score points under treatment and 16 under control. Its directly observed treatment-minus-control contrast is \(14-16=-2\) points. That supports a middle-period comparison, subject to the relevant design and outcome assumptions.
It does not, by itself, identify a trial-wide contrast standardized across all three periods. There are no observed early control outcomes and no observed late treatment outcomes from which to form the corresponding within-period differences. A regression could still return a number for that broader target, but it would require extrapolating the missing arm-time outcomes. The defensible conclusion is to report the supported middle-period contrast or clearly label any broader estimate as model-dependent; do not describe the trial-wide contrast as established by the observed overlap.
Interpretation and Common Mistake
Report the adjusted estimate as a contrast on the outcome scale and in context. For example: “After adjustment for the prespecified enrollment-period indicator, the estimated treatment-minus-control difference was \(-2.0\) score points, with lower scores indicating better status.” State the uncertainty interval and the method used to obtain it. If effects may differ over time, say whether the reported quantity is a common-effect model coefficient or a standardized average over a specified time distribution. Do not imply that adjustment removes all time-related bias or that the coefficient applies at every enrollment time without the assumptions required for that interpretation.
Common mistake: “The linear-time model gives the smallest p-value, so it is the best adjustment.” This uses the observed treatment result to choose the analysis and treats significance as a model-selection criterion. The correction is to define the primary model and target before comparing treatment estimates; use outcome and allocation timelines to assess adequacy, and use scientifically justified alternatives as sensitivity analyses. A model that gives a favorable result is not thereby better supported.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- Why can treatment-arm-specific averages over enrollment time differ from a standardized treatment contrast?
- A protocol documents a measurement change before enrollment begins. What role should that information play in choosing the primary time adjustment?
- In one time block, no participants receive control. What does that imply for a within-block treatment contrast?
- A primary adjusted estimate is \(-1.5\) points, while a planned alternative is \(-1.4\) points. What can and cannot be concluded from this agreement?
- Why is selecting the time model with the smallest treatment p-value a poor workflow?
Solutions. (1) Each arm’s overall mean weights its time-specific means by that arm’s own enrollment distribution. A standardized contrast uses the same time weights for both arms, so the quantities need not agree when allocation patterns differ. (2) The documented change is design information that can justify a prespecified epoch term, provided the outcome remains interpretable on a comparable scale. Outcome patterns can diagnose the model but should not be used to search for a favorable treatment result. (3) The observed data cannot form a within-block treatment-minus-control comparison because the control mean for that block is unavailable. A broader model-based estimate would rely on extrapolation. (4) The similar estimates suggest that this particular modeling choice does not greatly change the point estimate. They do not prove the models are correct, establish adequate overlap, or remove assumptions shared by both analyses; uncertainty and the target still need to be reported. (5) It makes model choice depend on the observed treatment result and encourages selective reporting. Prespecify the primary model and use justified alternatives to assess sensitivity, not to choose the most favorable p-value.