Why Calendar Time Matters
An external control can satisfy the same eligibility criteria as a trial participant and still come from a different treatment environment. If routine care, diagnostic practice, or underlying risk changes over calendar time, outcomes observed in an older cohort may not represent the outcomes that trial participants would have experienced under the control strategy during the trial period. The resulting difference can look like a treatment effect even when it is partly—or entirely—a time-period difference.
Calendar time is the date on which a patient enters the study at the defined time zero. It is not the duration of that patient’s subsequent follow-up. Aligning calendar time means selecting and assessing dates of entry so that trial participants and external controls are drawn from sufficiently comparable periods. The aim is not necessarily to require identical enrollment dates, which may leave too few controls, but to justify why the selected periods support the intended comparison.
Calendar time can matter through several pathways. A new standard treatment may lower event risk in later years. Diagnostic thresholds, test availability, or coding systems may change how often an outcome is detected or recorded. Population health, referral patterns, and access to care may alter background risk. Changes in eligibility, treatment availability, or clinical practice can also change which patients enter a cohort. These mechanisms can operate together, and a calendar date is often a marker for them rather than a complete explanation.
Specify the Windows Before Comparing Outcomes
Begin by stating the trial’s entry-date range and the candidate external-control range. Define the date that assigns a patient to a window consistently with the study’s time zero; for example, the date of treatment initiation or the date the eligibility criteria are first met, depending on the target trial definition. Do not substitute a record’s upload date or the date of a later database query unless that date genuinely represents cohort entry.
Then identify changes that could make outcomes non-comparable. Useful evidence includes clinical guidelines, dates of new treatment availability, changes in diagnostic equipment or criteria, coding-system transitions, and documented changes in care pathways. Examine how many eligible patients enter in each calendar period and whether key baseline characteristics or source-data practices shift across periods. These checks describe the data and the context; they do not, by themselves, prove that calendar time caused an outcome difference.
A defensible window balances temporal relevance against data availability. A narrow, recent window may better reflect the trial’s care environment but yield too few external controls. A broad window may improve precision while mixing substantially different eras. The choice should be made using clinical and data-quality considerations, not by trying several boundaries and selecting the one that produces the most favorable treatment contrast.
State the trial and external-control windows using the same clearly specified time-zero concept.
Document when standard care, diagnostic practice, coding, or background conditions changed, and why each change could affect the outcome or cohort composition.
Compare entry-date distributions and examine whether clinically important conditions are reasonably stable within the chosen periods.
Justify the primary window, then repeat the comparison under plausible alternative windows or calendar-time groupings.
How Time Composition Can Create an Apparent Effect
The basic problem can be seen even when trial and external-control patients are comparable within each period. Let \(r_k\) be the outcome risk in calendar period \(k\), and let \(w_k\) be the proportion of a cohort entering in that period. Its overall risk is a weighted average:
If the trial and external cohorts have different \(w_k\) values, their overall risks can differ even if the period-specific risks are the same for both cohorts. This is a composition difference, not evidence of a treatment effect. Conversely, matching the overall calendar distribution does not guarantee comparability if an important care or measurement change occurs within a broad period.
Worked Example: A Change in Standard Care
Suppose a hypothetical trial enrolls in 2024–2025. Investigators find that usual care changed in 2024, and that the risk of a binary outcome under usual care is 18% in 2024–2025, compared with 30% in 2022–2023. Assume, for illustration, that patient mix and outcome measurement are otherwise comparable. The trial treatment group has an observed outcome risk of 20%. A proposed external-control sample contains 80% of patients from 2022–2023 and 20% from 2024–2025.
The external sample’s crude risk is the period-weighted average:
The crude risk difference, trial treatment minus external control, is \(0.20-0.276=-0.076\), or \(-7.6\) percentage points. Taken at face value, that contrast suggests a lower risk in the trial treatment group. But the relevant contemporary usual-care risk in this hypothetical example is 18%, so the treatment-versus-contemporary-control contrast is \(0.20-0.18=0.02\), or 2 percentage points higher in the treatment group. The apparent advantage arose because most external controls came from the earlier, higher-risk era.
Restricting external entry dates to 2024–2025 would address this particular time-composition problem if the assumed within-period comparability is credible and enough controls remain. It would not establish comparability on other factors or prove that the treatment caused any difference.
Diagnostic Practice and Background Risk
Even when clinical care is unchanged, outcome measurement may not be. A change in screening intensity, test sensitivity, diagnostic thresholds, or coding can alter the proportion of events recorded. In that situation, calendar alignment is not merely a way to balance prognosis: it is also a way to avoid comparing differently measured outcomes. Harmonized definitions are necessary, but identical definitions on paper do not guarantee equal ascertainment in practice.
Worked Example: A More Sensitive Diagnostic Test
Consider a hypothetical outcome with a true prevalence of 10% in both an older and a recent period. Assume specificity is 98% in both periods, but the sensitivity of the diagnostic process rises from 60% in the older period to 90% in the recent period. The observed positive proportion is the true-positive proportion plus the false-positive proportion.
In the older period, the observed proportion is
In the recent period, it is
The recorded outcome proportion therefore rises from 7.8% to 10.8%, a difference of 3.0 percentage points, although the assumed true prevalence is 10% in both periods. The arithmetic illustrates why a temporal difference in recorded outcomes cannot automatically be interpreted as a change in underlying disease risk. If the diagnostic change coincides with the trial period, an older external cohort may not be a suitable comparator unless measurement can be made comparable or the limitation can be credibly addressed.
Background risk can also change without a discrete policy or technology change—for example, because of shifts in population health or referral patterns. A gradual trend is easy to miss when years are grouped into a single broad “pre-trial” period. Plotting entry counts and outcome summaries by month, quarter, or year can reveal abrupt discontinuities or gradual drift. Such displays are diagnostics, not a substitute for a justified design: observed outcome trends may themselves reflect changes in patient composition or measurement.
Choosing Among Restriction, Stratification, and Modeling
There are three common ways to address calendar-time differences. Restriction excludes external patients outside a chosen window. It is transparent and may improve comparability when older periods are substantively different, but it reduces sample size and may change which patients the comparison represents. Stratification compares or summarizes patients within calendar periods, then combines those period-specific comparisons using explicit weights. It can make period composition visible, but requires enough information in each period and can fail if important changes occur within the strata. Modeling includes calendar time, often flexibly, as a predictor of outcome or as an effect modifier. This can use a wider date range, but depends on an adequate model and cannot recover information absent from periods with little or no overlap.
| Approach | Useful when | Main limitation |
|---|---|---|
| Restrict to a recent window | Older periods have clearly different care or measurement | Fewer controls; findings apply to the retained population and period |
| Stratify by calendar period | Several periods have adequate data and within-period comparisons are meaningful | Sparse strata or broad groupings can obscure important differences |
| Model calendar time | A plausible time trend can be represented and there is adequate temporal overlap | Results can depend on model form; modeling does not create data where overlap is absent |
These approaches are not interchangeable fixes for every temporal problem. Restriction cannot correct changes in outcome measurement within the retained period. Stratification or modeling cannot make non-overlapping eras equivalent simply by including a date variable. When no credible common period exists, the appropriate conclusion may be that the external controls do not support the intended comparison.
Calendar-time alignment is particularly relevant when external-control outcomes contribute information to a Bayesian analysis. For example, a power prior can weight an external likelihood by a discount \(a\), with \(0\leq a\leq1\). That discount represents reduced borrowing; it does not make a historical cohort exchangeable with the trial cohort or remove a systematic difference caused by changed care or measurement. Calendar-time compatibility is a design and data-assessment question that should be addressed before interpreting borrowed information.
Here \(D_T\) and \(D_E\) denote trial and external data, \(L_T\) and \(L_E\) their likelihood contributions for the modeled parameter \(\theta\), and \(\pi_0(\theta)\) the initial prior. If time-related differences make the external likelihood systematically misleading for the trial-period control experience, reducing its weight may reduce its influence, but it is not a substitute for identifying and addressing the source of incompatibility.
A Practical Justification and Its Limits
A clear analysis plan should record the entry-date definition, primary trial and external windows, the clinical or data evidence supporting the boundaries, and the expected consequences of restricting the external cohort. It should also specify sensitivity analyses, such as a narrower contemporary window, alternative period groupings, or a time trend where justified. Report the number of eligible external patients retained and excluded by each boundary, as well as changes in their date distribution. If a result changes substantially across reasonable windows, that instability is important evidence about the dependence of the comparison on historical data.
A useful justification is specific: “We restricted external entry to dates after the new diagnostic protocol was introduced because the earlier protocol had lower sensitivity for the primary outcome.” “We used all available historical controls because they increase precision” is not, on its own, a comparability argument. Likewise, selecting a window because it produces a preferred estimate risks making the design outcome-dependent.
Common Mistakes and Diagnostics
- Confusing calendar time with follow-up time. Patients can have the same follow-up duration but enter in different clinical eras. Compare their entry dates as well as their elapsed time after entry.
- Assuming matching eligibility is enough. Identical inclusion rules do not undo changes in standard care, diagnostic practice, or background risk.
- Using a discount as a repair for bias. Lower borrowing can reduce the influence of external data, but does not make a systematically different historical outcome unbiased.
- Choosing a broad window for precision without checking stability. Plot dates and examine care and measurement changes before pooling periods.
- Choosing boundaries after seeing the preferred effect estimate. Prespecify a clinically justified primary window and show sensitivity to reasonable alternatives.
Warning signs include an abrupt change in outcome frequency near a guideline or coding transition, a marked shift in the distribution of entry dates, sparse or absent overlap with the trial period, and materially different results under plausible window definitions. Each signal calls for investigation; none alone identifies the cause. A calendar-time analysis also has limits: even a well-chosen window cannot remove unmeasured differences between trial participants and external controls or correct poor outcome measurement by itself.
Worked Example: Standardizing to the Trial’s Calendar Mix
Suppose the trial population enters in two calendar periods: 70% in a recent period and 30% in an earlier period. In a hypothetical external cohort, the outcome risks are 12% in the recent period and 20% in the earlier period. However, only 20% of external patients enter recently, while 80% enter earlier.
The crude external risk is
Thus the crude external risk is 18.4%. To show what the external period-specific risks imply under the trial’s calendar mix, use the trial weights:
The standardized value is 14.4%, which is 4.0 percentage points lower than the crude external value. This calculation demonstrates the effect of different calendar composition under the stated period-specific risks. It is not automatically a valid adjusted control estimate: it requires that the risks within periods are meaningfully comparable and that each period has adequate data. If the trial has few or no participants in one period, or if care changes within a period, the calculation may not represent a credible comparison.
Check Your Understanding
Answer each question, then compare with the solutions.
- What is the difference between calendar time and follow-up time in an external-control analysis?
- A historical cohort has a 25% outcome risk in an earlier era and a 15% risk in a recent era. An external sample is 60% earlier-era and 40% recent-era. Calculate its crude risk.
- Why might a rise in recorded outcomes after a new diagnostic test is introduced fail to show that underlying risk increased?
- Name one advantage and one limitation of restricting external controls to a recent calendar window.
- Why does downweighting historical likelihood information not, by itself, correct calendar-time bias?
Solutions. (1) Calendar time is when a patient enters the study; follow-up time is elapsed time since that entry. (2) The crude risk is \((0.60)(0.25)+(0.40)(0.15)=0.15+0.06=0.21\), or 21%. (3) The test may detect a greater proportion of existing cases, changing recorded outcome frequency even if underlying risk is unchanged. (4) Restriction can exclude a clinically different era and improve temporal relevance; it also reduces sample size and may change the population represented. (5) Downweighting reduces the historical data’s contribution but does not remove systematic differences in care, measurement, or risk between historical and trial periods.