Calendar-Time Changes in Historical Controls

Why Calendar Time Matters

Historical controls can improve precision when current control data are limited, but their outcomes may reflect a different clinical environment. A treatment that became standard after an older trial, a changed eligibility policy, or a new way of measuring outcomes can all make an older control group a poor guide to outcomes under current care. Calendar time is therefore not just a label on a study; it can indicate changes in the process that generated its outcomes.

The key question is not whether an older study is old in years. It is whether, after considering relevant differences, its control outcomes can reasonably inform the control outcome distribution in the current trial. Calendar date can help organize that assessment, but it is not itself a measure of comparability. Two studies conducted in different years may be similar in clinically important respects, while studies close in time may differ because of a change in standard care or trial procedures.

Definition: Calendar-time comparability is the degree to which historical control outcomes remain informative about the control outcomes that would be expected under the current trial’s care, population, outcome measurement, and conduct. A calendar-time change is a relevant difference across periods that may alter those outcomes or their interpretation.

Assessment begins with the clinical and operational history, not with a statistical rule based only on dates. Consider whether standard care changed, including background medications and supportive care; whether eligibility or recruitment changed the patient mix; and whether trial conduct changed, including follow-up schedules, adherence support, or outcome assessment. These changes can affect the underlying outcome, its variability, or the chance that it is observed. Endpoint harmonization is necessary, but matching endpoint labels alone does not establish comparability.

Relating Historical Data to the Current Control Parameter

For borrowing to be directly relevant, the historical and current control observations must inform the same target parameter, or the analysis must explicitly account for their differences. For example, a historical mean outcome can be treated as evidence about the current control mean only under a model that makes that link reasonable. If a new standard therapy has changed expected outcomes, simply adding a discount to the old data does not turn the old mean into a current-care mean.

A fixed power-prior weight \(a\), with \(0 \leq a \leq 1\), tempers the historical likelihood. In the normal example below, the historical contribution to precision is multiplied by \(a\). This can reduce the influence of older data when comparability is uncertain. It does not estimate or remove a time trend, correct an unmeasured case-mix difference, or account for a known shift in the outcome’s meaning. The choice of \(a\) is a modeling decision that needs clinical justification and sensitivity analysis.

Model and assumptions: Suppose historical control outcomes \(Y_i\) are independent with \(Y_i \sim N(\mu,\sigma^2)\), where \(\sigma^2\) is known, and the current control mean of interest is \(\mu\). Let the prior be \(\mu \sim N(m_0,v_0)\). A historical sample of size \(n_H\), with mean \(\bar{y}_H\), contributes its likelihood raised to a fixed power \(a\). This update assumes the historical outcomes are adequately represented by the same normal model and that using them as information about the current \(\mu\) is clinically defensible. A discount weakens that information; it does not establish the assumptions.

The effect of a fixed discount follows directly from the normal likelihood. Up to a constant that does not depend on \(\mu\), the powered historical likelihood is proportional to \(\exp\{-a n_H(\bar{y}_H-\mu)^2/(2\sigma^2)\}\). Combining it with the normal prior gives a normal posterior:

\mu \mid \bar{y}_H \sim N(m_1,v_1), \qquad v_1^{-1}=v_0^{-1}+\frac{a n_H}{\sigma^2}, \qquad m_1=v_1\left(\frac{m_0}{v_0}+\frac{a n_H\bar{y}_H}{\sigma^2}\right).

To see the update, the exponent in the prior times the powered likelihood contains the terms \((\mu-m_0)^2/v_0 + a n_H(\bar{y}_H-\mu)^2/\sigma^2\). The coefficient of \(\mu^2\) is \(v_0^{-1}+a n_H/\sigma^2\), so it is the posterior precision. The coefficient determining the posterior mean is \(m_0/v_0+a n_H\bar{y}_H/\sigma^2\). Completing the square gives the displayed normal distribution. Thus, reducing \(a\) reduces the historical precision contribution, but it does not change \(\bar{y}_H\) or make it more representative of current care.

Worked Example: Discounting an Older Control Mean

Suppose an older trial reported a mean symptom duration of 14 days among 40 controls. A current-control model uses a known standard deviation of 4 days and a prior \(\mu \sim N(10,4)\), where the second argument is the variance. Because standard care changed after the older trial, the analysis team considers a fixed discount \(a=0.25\). Calculate the posterior for \(\mu\), conditional on the model assumptions.

The historical mean is \(\bar{y}_H=14\) days, \(n_H=40\), \(\sigma^2=16\) days squared, \(m_0=10\) days, and \(v_0=4\) days squared. The posterior precision is \(1/4 + (0.25)(40)/16 = 0.25+0.625=0.875\) per day squared. Therefore \(v_1=1/0.875=1.143\) days squared.

The posterior mean is \((10/4 + (0.25)(40)(14)/16)/0.875 = (2.5+8.75)/0.875 = 12.86\) days. The resulting posterior is approximately \(N(12.86,1.143)\), with posterior standard deviation \(\sqrt{1.143}=1.069\) days.

This calculation shows how the chosen weight tempers the old data; it does not show that 14 days is an unbiased estimate of current control outcomes. If the changed standard of care substantially shortened symptom duration, this posterior may still be centered too high for current care. The model-based result should therefore be presented as conditional on the comparability assumption and examined against analyses with less or no borrowing.

Assessing Differences Across Eras

A practical assessment should connect differences in calendar time to specific mechanisms. A useful starting point is a study-by-study timeline containing enrollment dates, standard-care changes, eligibility rules, key conduct features, and outcome definitions. Then consider whether these features could affect the control outcome relevant to the current trial. When individual records and suitable covariates are available, comparing the composition of historical and current participants can help identify measured differences. Such comparisons cannot rule out unmeasured differences.

Observed outcomes can also be summarized by era, but apparent differences are not automatically caused by calendar time. They may reflect random variation, different patient composition, or different trial conduct. Conversely, a lack of a conspicuous difference in a small sample does not establish that the cohorts are exchangeable. Clinical importance, uncertainty, and the plausibility of the data-generating process all matter.

Worked Example: A Change in Standard Care Across Two Eras

Imagine two hypothetical historical control cohorts measuring 30-day mortality. In an older era, 20 of 100 participants died; in a more recent era, 12 of 100 died. A new supportive-care practice was introduced between the eras. Calculate the observed rates and their difference, and explain what the contrast does and does not establish.

The older-era mortality proportion is \(20/100=0.20\), or 20%. The more recent proportion is \(12/100=0.12\), or 12%. The observed risk difference, older minus recent, is \(0.20-0.12=0.08\), an 8 percentage-point higher rate in the older cohort.

For a descriptive measure of sampling uncertainty under independent binomial sampling, the estimated standard error of the difference is \(\sqrt{0.20(0.80)/100+0.12(0.88)/100}=\sqrt{0.002656}=0.05154\), or 5.154 percentage points. The observed contrast is therefore about \(0.08/0.05154=1.552\) standard errors. This calculation describes uncertainty under the simple sampling model; it does not isolate the effect of the new supportive-care practice.

The difference is clinically notable enough to prompt investigation of the change in care and other differences between cohorts. It is not proof that the practice caused an 8 percentage-point reduction: case mix, conduct, and chance may also contribute. If the current trial uses the newer practice, treating the older and recent cohorts as one homogeneous control sample may be misleading. A fixed discount can reduce the older cohort’s influence, but a discount alone does not identify the current-care mortality rate.

Choosing Among Borrowing Strategies

Once differences are understood, the analysis must make a defensible choice about which data to use. Options include restricting borrowing to cohorts that closely resemble current care, using a fixed discount for less comparable but still informative data, or not borrowing when differences make the historical outcomes inappropriate for the target. If a randomized concurrent control group is feasible, it directly represents the current trial setting and provides protection against calendar-time differences that historical controls cannot supply. These approaches address different problems; none makes every historical cohort suitable.

Approach	What it can address	Main limitation
Use only clinically similar historical cohorts	Avoids direct borrowing from clearly outdated care or conduct	May leave little historical information and does not eliminate all differences
Apply a fixed power-prior discount	Reduces the precision contribution of selected historical data	Does not correct a systematic shift, explain why it occurred, or guarantee adequate protection
Use concurrent randomized controls	Provides a comparison group recruited under current trial conditions	May require more current-trial participants and does not make every historical question irrelevant
Model measured differences	Can account for specified covariates or known design features when the model is appropriate	Depends on adequate measurement, overlap, and correct modeling; unmeasured differences remain

The time period alone should not determine a weight. A nearby cohort may still be unsuitable if its standard care or outcome assessment differs materially. A more distant cohort may be informative if relevant practices and populations remained stable, but that case requires evidence rather than an assumption based on the date. Avoid interpreting a particular discount as a literal percentage of patients or as a probability that a cohort is exchangeable. It is a likelihood power that sets the cohort’s contribution within a specified model.

Diagnostics, Failure Modes, and Interpretation

A defensible assessment combines clinical review with checks of the data and model. Plot control outcomes against enrollment time where the data structure permits; mark dates of major changes in care or conduct rather than relying only on a fitted trend. Compare relevant baseline characteristics and protocol features across eras. Investigate whether apparent shifts are driven by one study, a small subgroup, or an assessment change. These checks can reveal concerns, but they cannot prove that all relevant differences have been measured.

A useful sensitivity analysis compares conclusions under a range of prespecified fixed discounts, including no historical contribution where appropriate, and under a restriction to the most comparable historical data. If the current-trial conclusion changes materially, that dependence is important to report. If all choices lead to similar results, that is reassuring about sensitivity to the examined borrowing assumptions, not proof of comparability. The weight should not be chosen only because it produces a preferred conclusion.

Pooling eras without checking care changes: a pooled average can combine outcomes generated under different standard care and may not represent current control outcomes.
Treating a non-significant era contrast as evidence of equivalence: limited data can obscure important differences. Assess clinical relevance and uncertainty, not just a threshold for statistical significance.
Assuming discounting repairs bias: a power weight weakens a likelihood contribution but does not shift a biased historical mean to the current value.
Using date as a substitute for clinical review: time is a clue to investigate, not a sufficient criterion for inclusion or exclusion.
Reporting a posterior without its assumptions: state which eras were borrowed, what changes were considered, and how conclusions varied under alternative borrowing choices.

Key takeaway: Calendar time matters when it signals changes in the processes producing control outcomes. Select or discount historical data based on the clinical and methodological relevance of those data to the current trial. A fixed discount limits influence; it does not establish comparability or correct a systematic time-related shift.

Worked Example: Excluding an Older Cohort With Different Trial Conduct

Consider a hypothetical continuous outcome measured in days. A recent historical control cohort used enhanced adherence support and reported \(\bar{y}_H=8.0\) days for \(n_H=25\) participants. An older cohort of 50 participants used materially different adherence procedures and outcome assessment. Suppose the current analysis uses a known \(\sigma=2\) days, a prior \(\mu \sim N(9,9)\), and, after clinical review, excludes the older cohort and assigns the recent cohort \(a=0.5\). Find the posterior and compare it with the prior.

Here \(v_0=9\) days squared and \(\sigma^2=4\) days squared. The posterior precision is \(1/9+(0.5)(25)/4=0.1111+3.125=3.2361\) per day squared. Thus \(v_1=1/3.2361=0.3090\) days squared.

The posterior mean is \(((9/9)+(0.5)(25)(8.0)/4)/3.2361=(1+25)/3.2361=8.034\) days. The posterior standard deviation is \(\sqrt{0.3090}=0.556\) days. With no borrowing, the posterior would remain the prior \(N(9,9)\). This example illustrates a transparent choice: the older cohort is not included because its conduct differs, while the more comparable recent cohort contributes partially. The calculation does not prove the remaining cohort is exchangeable with the current trial; that remains a substantive assumption to review and test through sensitivity analyses.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

A historical cohort is only two years old, but a new background therapy was introduced before the current trial. Does its recency alone justify full borrowing?
In a normal power-prior update, what does reducing a fixed historical weight \(a\) change, and what does it leave unchanged?
Two historical eras have observed control rates of 18% and 10%. Why is this contrast not, by itself, proof that calendar time caused the difference?
A team reports that a discount of \(a=0.3\) means there is a 30% probability that the old cohort is exchangeable. Is that interpretation correct?

Solutions. (1) No. A recent date does not overcome a potentially important change in standard care; assess whether the historical outcomes inform current care. (2) It reduces the historical likelihood’s contribution to posterior precision and its corresponding influence on the posterior mean. It does not change the observed historical outcomes or correct a systematic shift. (3) The eras may differ in case mix, conduct, or chance as well as calendar time; the contrast is descriptive unless a suitable design and assumptions support a causal interpretation. (4) No. \(a\) is a likelihood power, not a probability of exchangeability. Its choice needs clinical justification and sensitivity analysis.