Why Report Calendar Time?
In an adaptive trial, the probability of assigning a participant to a treatment can change as enrollment proceeds. Meanwhile, prognosis, eligibility, clinical practice, or outcome measurement may also change over calendar time. A pooled table of treatment totals cannot show whether the arms enrolled during comparable periods, and a pooled outcome contrast can combine treatment differences with time trends. Reporting calendar-time patterns makes those features visible and allows readers to judge what a time-adjusted estimate does and does not establish.
A useful report is not just a plot of outcomes against dates. It gives enough information to reconstruct the timing of enrollment, describe the assignment mechanism as it operated, show arm-specific enrollment and outcomes over time, and identify the target and assumptions of any time-adjusted estimate. Calendar periods should be defined in a way that is clinically interpretable and, ideally, specified before outcome patterns are examined.
A Reproducible Reporting Set
Report the time variable used, its precision, and how it was obtained. For example, distinguish the date of consent, randomization, treatment initiation, and outcome assessment when they differ. State the time zone if timestamps are used, how dates were handled for participants with incomplete records, and whether enrollment time is measured as calendar date, elapsed days from trial launch, or enrollment epoch. If the trial had pauses, site openings, protocol amendments, or allocation-rule changes, give their dates and describe their relevance to the time summaries.
For each calendar period, report the period boundaries, number enrolled, number assigned to each arm, and a summary of assignment probabilities. If the probability changed for individual assignments within a period, state how the summary was calculated, such as the minimum and maximum and the mean of the \(p_i\) values. Make clear whether the displayed probability is the probability of treatment 1 or treatment 0. Include the algorithm version or rule in force and identify deviations, overrides, or missing probability records. Readers need the probabilities that were actually used, not only the intended allocation ratio.
For outcomes, show the observed number with outcome data, the summary used, and its units for each arm and period. A plot can show enrollment by date, period-specific arm counts or allocation probabilities, and arm-specific outcome summaries over time. Mark protocol changes and other known events that may affect prognosis or measurement. Report missing outcome counts by arm and period; a trend among observed outcomes can be misleading if missingness also changes over time.
Finally, give any time-adjusted estimate with its target population or time distribution, model or weighting approach, uncertainty method, and assumptions. A contrast standardized to equal weights across periods answers a different question from one standardized to the trial’s enrollment distribution. Neither should be reported simply as “adjusted for time” without specifying what that means.
State the enrollment-time definition, period boundaries, date precision, and any important changes during conduct.
Report arm counts and the assignment-probability distribution by period, with enough detail to identify extreme probabilities and exceptions.
Give arm-specific outcome summaries, sample sizes, units, and missingness for the same periods or clearly explain any different time grouping.
State its estimand, method, uncertainty procedure, temporal overlap, and principal limitations.
Worked Examples of Calendar-Time Reporting
Worked Example: Period Counts and Allocation Probabilities
Hypothetical data. Suppose 90 participants enroll in three consecutive periods of 30 participants each. The allocation rule assigns each participant in the first period to treatment 1 with probability 0.90, in the second period with probability 0.50, and in the third period with probability 0.10. The realized counts are as follows.
| Period | Probability of treatment 1 | Treatment 1 count | Treatment 0 count | Total |
|---|---|---|---|---|
| Early | 0.90 | 27 | 3 | 30 |
| Middle | 0.50 | 15 | 15 | 30 |
| Late | 0.10 | 3 | 27 | 30 |
| Total | — | 45 | 45 | 90 |
The arm totals are obtained directly from the period counts:
Thus, the overall realized allocation is 45:45, or 50:50. Reporting only that overall ratio would hide the substantial shift across periods: treatment 1 accounts for 27 of the 30 early participants but only 3 of the 30 late participants. The table reports realized counts, not expected counts. The probabilities describe the assignment rule, while the counts describe its realized operation.
A full report would also say whether these probabilities were constant for every participant in each period or are period summaries of changing individual \(p_i\) values. In the latter case, it should provide the individual probabilities or additional summaries sufficient to reveal their variation.
Worked Example: A Pooled Contrast Can Hide a Time Pattern
Hypothetical data. Use the counts above and suppose the outcome is a symptom score in points, with lower values indicating fewer symptoms. The observed period-specific means are: early, 10 points for treatment 1 and 12 for treatment 0; middle, 9 and 10; late, 8 and 8.5. Assume for this descriptive calculation that all assigned participants have an observed outcome. The data do not establish that the groups are comparable within each period; the calculation only shows what these summaries yield.
The pooled treatment 1 mean is the count-weighted mean of its three period means:
The pooled treatment 0 mean is:
The pooled difference, treatment 1 minus treatment 0, is \(9.533-9.233=0.300\) points. In contrast, the three within-period differences are \(10-12=-2\), \(9-10=-1\), and \(8-8.5=-0.5\) points. For a descriptive comparison that gives each period equal weight, the standardized difference is:
The two numbers differ because treatment 1 participants are concentrated early, while treatment 0 participants are concentrated late, and the outcome means decline over time in both arms. The pooled contrast describes the observed enrolled groups with their different time distributions. The equal-period contrast describes an average of the three period-specific differences, assigning one third of the weight to each period. It is not automatically a causal effect: its causal interpretation requires an appropriate randomized comparison within periods, sufficient overlap, well-defined treatment and outcome, and a defensible treatment of missing outcomes.
Worked Example: Reporting Weak Temporal Overlap
Hypothetical data. Suppose a 20-participant period has treatment 1 assignment probability 0.95, with 19 assigned to treatment 1 and 1 to treatment 0. A later period also has 20 participants, probability 0.05, with 1 assigned to treatment 1 and 19 to treatment 0. Across the two periods, each arm has 20 participants, but the within-period comparisons are based on only one participant in the less frequently assigned arm.
For the first period, the ratio of the largest to smallest assignment probability is \(0.95/0.05=19\), where 0.05 is the probability of treatment 0. For the later period, the treatment 1 probability is 0.05 and the treatment 0 probability is 0.95, giving the same ratio. Under an inverse-probability weighting approach, an observation assigned to an arm with probability 0.05 receives a factor \(1/0.05=20\), while one assigned with probability 0.95 receives \(1/0.95=1.053\). This contrast in weights makes clear why the nominal total of 20 participants per arm does not guarantee stable time-adjusted comparisons.
A responsible report would display both periods, their probability values, and their arm counts; state that overlap is weak; and explain that period-specific estimates and weighted estimates may be unstable. It would not present the balanced overall total as evidence of adequate temporal overlap.
Justifying a Time-Standardized Summary
A common descriptive target is the average of period-specific treatment contrasts under chosen period weights. Let \(\bar{x}_{1j}\) and \(\bar{x}_{0j}\) denote the observed means in period \(j\), and let \(w_j\) be prespecified weights with \(w_j\geq 0\) and \(\sum_j w_j=1\). Then the standardized contrast is:
This expression follows by first calculating the arm contrast within each period and then averaging those contrasts using the stated target weights. Equal-period weights answer an equal-period descriptive question; weights proportional to enrollment represent a different target distribution. If one arm has no participants in a period, its mean and hence that period’s contrast cannot be estimated from the observed data without additional modeling or extrapolation. Very small counts can make the contrast highly uncertain even when both arms are represented.
For a causal interpretation, the report must connect the target contrast to the trial design and analysis. In particular, assignment probabilities must be correctly recorded, treatment assignment must follow the stated randomization mechanism, and the target periods must have adequate assignment support for both arms. The outcome and treatment definitions must be consistent, and missing outcomes require an appropriate assumption or method. If treatment effects vary over time, the weights determine which average effect is being summarized. A fitted time trend cannot create observed overlap where the design supplied little or none; it can only extrapolate under its model.
Choosing and Checking the Presentation
Different summaries answer different questions and should not be treated as interchangeable.
| Approach | What it shows | Main caution |
|---|---|---|
| Pooled arm summaries | Overall realized outcomes and counts in each arm | Can mix treatment differences with different enrollment-time distributions |
| Period-stratified summaries | Arm comparisons within specified time periods | May be imprecise or unavailable in periods with sparse or absent overlap |
| Outcome regression with time terms | A model-based contrast adjusted for the included time representation | Depends on model form and can extrapolate beyond observed overlap |
| Inverse-probability weighting | A probability-weighted comparison using the recorded assignment probabilities | Extreme weights can make estimates unstable; diagnostics and uncertainty methods matter |
Use the earlier material on inverse-probability weighting, outcome regression, positivity, and uncertainty after calendar-time adjustment when presenting those analyses. This tutorial’s reporting point is to show the inputs and limitations, not to imply that a method removes all time-related bias. Where appropriate, show more than one summary and explain why their targets or assumptions differ.
Diagnostics begin with the data display itself. Check for periods with zero or very few assignments to one arm, sharp probability changes, unexplained gaps in enrollment, mismatches between the recorded probabilities and the algorithm in force, and changing outcome missingness. Compare the reported counts with the participant-level data: period totals must add to the overall sample size, and arm counts must sum correctly. If a probability summary is an average, say so; the mean alone can conceal values close to zero or one. Examine outcome summaries on the same time scale as allocation, and identify whether apparent trends may reflect changing participant composition, measurement, or care.
Limitations and Common Mistakes
Calendar-time adjustment cannot establish that every relevant source of drift was measured, nor can it repair absent overlap without assumptions. Broad periods may conceal rapid changes; narrow periods may leave too few participants for useful summaries. Smooth regression terms can miss abrupt protocol changes, while overly flexible terms can be unstable. Outcome trends are descriptive and may reflect changes in case mix rather than changes in prognosis for a fixed population. State these limitations rather than implying that a time-adjusted estimate is free of bias.
- Mistake: report only the overall arm ratio. Equal totals can coexist with severe separation across time. Give period-specific counts and assignment probabilities.
- Mistake: call a period average the probability used for every participant. If \(p_i\) varied within a period, describe its distribution or provide the participant-level probabilities.
- Mistake: interpret a changing outcome mean as treatment effect. Show arm-specific trends and distinguish descriptive patterns from adjusted comparisons.
- Mistake: choose periods after inspecting which grouping gives the preferred estimate. Specify clinically justified periods in advance where possible; if exploratory boundaries are used, label them and report sensitivity to reasonable alternatives.
- Mistake: claim adjustment solves poor overlap. Report sparse cells and extreme probabilities or weights; model-based estimates in unsupported regions depend on extrapolation.
Check Your Understanding
Answer each question, then compare your reasoning with the brief solutions.
- In a period with 24 participants, 18 are assigned to treatment 1 and 6 to treatment 0. What should be reported in addition to the counts if the assignment probability changed within that period?
- Why can equal overall arm totals fail to demonstrate temporal overlap?
- Using period contrasts of \(-2\) and \(1\) points and equal weights, calculate the standardized contrast.
- What does an inverse-probability factor of 10 indicate for an observation assigned to a particular arm?
- Name two limitations that should accompany a time-adjusted estimate when overlap is weak.
Solutions. (1) Report the individual assignment probabilities or their distribution within the period, including a clear summary such as minimum, maximum, and mean, and state how it was calculated. (2) The arms may be concentrated in different periods even if their totals match; period-specific counts and probabilities reveal this separation. (3) The equal-weight contrast is \(\{(-2)+1\}/2=-0.5\) points. (4) It corresponds to an assignment probability of \(1/10=0.10\) for that arm, so the observation receives substantial relative weight; report the probability mechanism and assess instability. (5) Estimates may be imprecise or depend strongly on modeling or weighting assumptions, and model-based results may extrapolate into periods with little or no support. State the specific limitation in the trial’s context.