Why Calendar Time Matters
An external control may come from patients treated before the trial began. Even if those patients resemble trial participants on measured baseline characteristics, their outcomes may have been observed under different conditions. Background risk can change as supportive care improves, diagnostic practice shifts, or the population served by a data source changes. Outcome recording can also change when definitions, surveillance, or data systems are revised. A rate that changes over calendar time is therefore a warning that historical outcomes may not represent the risk that trial participants would have faced under current care.
The aim of this tutorial is to diagnose temporal change in control outcomes, not to determine its cause or prove that an external control is unusable. A visible trend can reflect changing underlying risk, changing patient composition, changing ascertainment, or chance variation. The evidence needs to be examined alongside the way the cohort and outcome were constructed.
For a binary endpoint measured over a fixed window, a natural starting point is to plot the number of events divided by the number of eligible patients in successive calendar periods. The denominator and endpoint window must be defined consistently. For example, a 30-day event rate should not be compared across periods if some periods include only patients with complete 30-day follow-up and others include patients with shorter observation.
Plot Before Choosing a Model
Choose periods fine enough to reveal plausible changes, but not so fine that every rate is dominated by random variation. Plot each period's event rate against its calendar date and display its denominator. Uncertainty intervals help distinguish a stable-looking pattern from noisy fluctuations. When periods have different sizes, the same amount of visual movement does not imply the same precision: a rate based on 20 patients is less informative than one based on 2,000.
A useful plot also marks important dates, such as a change in treatment guidelines, data-system migration, or outcome-definition revision, when these are known. Such dates can suggest explanations to investigate, but their alignment with a rate change does not establish causation. Inspect whether the pattern is gradual, concentrated around a boundary, nonlinear, or driven by a single small period. A smoothed curve can help reveal a pattern, but it should not replace the observed rates and denominators.
Worked Example: Reading a Time-Stratified Plot
Suppose an external-control cohort has one binary outcome recorded within the same fixed follow-up window for each patient. The hypothetical counts are:
| Calendar year | Events | Patients | Observed rate |
|---|---|---|---|
| 2019 | 20 | 100 | 20% |
| 2020 | 21 | 100 | 21% |
| 2021 | 29 | 100 | 29% |
| 2022 | 34 | 100 | 34% |
| 2023 | 36 | 100 | 36% |
The first and last rates differ by \(0.36-0.20=0.16\), or 16 percentage points. Their ratio is \(0.36/0.20=1.80\), so the observed rate in 2023 is 1.8 times the observed rate in 2019. The intermediate values also increase, rather than showing only a difference between two endpoints. A plot would therefore flag a possible upward temporal pattern for investigation.
These calculations describe the observed cohort; they do not show that calendar time caused the rise. For example, the later patients might have greater baseline risk, or outcome recording might have become more complete. The equal denominators make the period rates equally sized samples here, but do not eliminate sampling uncertainty or explain the pattern.
Modeling a Gradual Trend
For patient \(i\) observed in calendar period \(t_i\), let \(Y_i=1\) indicate that the binary outcome occurred and \(Y_i=0\) otherwise. A logistic regression represents the period-specific probability \(p_i=P(Y_i=1)\) as a function of calendar time. Center time at a meaningful reference date so that the intercept has an interpretable meaning.
Here, \(\operatorname{logit}(p)=\log\{p/(1-p)\}\). The coefficient \(\beta\) is the change in log odds per unit of calendar time; \(\exp(\beta)\) is the multiplicative change in odds for each one-unit increase in time. This model assumes the binary outcomes are conditionally independent given the modeled predictors, the event definition is consistent, and the log odds change linearly over the time range. If there are repeated observations or clustering by site, the independence assumption needs an appropriate extension rather than being taken for granted.
A Bayesian analysis assigns prior distributions to the coefficients and updates them with the binomial likelihood. For example, a prior \(\beta\sim N(0,0.2^2)\), when time is in years, places most prior mass on modest annual log-odds changes while allowing the data to move the estimate. The prior should be checked against clinically plausible changes and the units of time. A coefficient prior specified per year is not the same prior as one specified per month.
Worked Example: Quantifying an Annual Trend
Suppose a hypothetical binomial logistic analysis of the external-control data yields an approximately normal likelihood summary for the annual trend: \(\hat{\beta}=0.18\) with standard error \(0.07\). For this calculation, treat the likelihood for \(\beta\) as \(N(0.18,0.07^2)\), and use the prior \(\beta\sim N(0,0.2^2)\). This normal likelihood approximation is an explicit simplifying assumption for the example.
For a normal likelihood and normal prior, the approximate posterior variance is the reciprocal of the sum of the precisions, and the posterior mean is the precision-weighted average:
The posterior standard deviation is \(\sqrt{0.004365}=0.0661\). The approximate 95% credible interval is \(0.1603\pm1.96(0.0661)\), or \(0.031\) to \(0.290\) log-odds units per year. Its posterior probability of being above zero is about 0.992 under this normal posterior approximation. Exponentiating the mean gives an annual odds ratio of \(\exp(0.1603)=1.174\); exponentiating the interval endpoints gives approximately 1.031 to 1.336. Under the model, the odds of the outcome increase by about 17% per calendar year on average, with the stated posterior uncertainty.
This is evidence for an increasing trend under the assumed linear model, likelihood approximation, and prior. It does not tell us whether risk, case mix, care, or recording changed. Nor does it guarantee that the same linear trend applies outside the years represented in the data.
Allowing for Abrupt or Nonlinear Change
A single straight-line time effect can hide important patterns. If a change date is specified in advance for a substantive reason, a segmented model can represent a level shift at that date, a change in slope, or both. For a simple level shift, the model can include an indicator \(I(t\geq c)\), equal to one after the chosen date \(c\) and zero before it. The indicator's coefficient represents the change in log odds at the boundary, conditional on other model terms.
A smooth nonlinear time effect, such as a spline, can represent curvature when a linear trend is implausible. More flexible models need enough information across time to estimate their extra parameters and should be constrained or regularized appropriately. If the change point is selected after looking for the largest observed jump, treating it as if it had been prespecified understates uncertainty about the change. Plotting the rates first helps reveal when a simple trend may be inadequate; it does not license choosing the most dramatic break and reporting an unadjusted comparison.
Worked Example: Comparing Periods Around a Prespecified Change
Suppose a hypothetical registry introduced a new care protocol at a date set before examining these outcome counts. The same binary endpoint and follow-up window were used in all six quarters. The first three quarters had 10, 11, and 12 events among 100 patients each; the next three had 24, 25, and 26 events among 100 patients each. For a simple Bayesian comparison, assume independent binomial sampling within the two groups and independent \(\operatorname{Beta}(1,1)\) priors for the pre- and post-change event probabilities.
The pre-change total is 33 events among 300 patients, and the post-change total is 75 among 300. Under a beta-binomial model, a \(\operatorname{Beta}(a,b)\) prior and \(y\) events among \(n\) patients give a \(\operatorname{Beta}(a+y,b+n-y)\) posterior. Thus the posteriors are \(\operatorname{Beta}(34,268)\) before and \(\operatorname{Beta}(76,226)\) after the date. Their posterior means are \(34/302=0.1126\) and \(76/302=0.2517\), respectively. The posterior mean difference, post minus pre, is \(0.2517-0.1126=0.1391\), or about 13.9 percentage points.
For an approximate interval, the variance of a \(\operatorname{Beta}(a,b)\) distribution is \(ab/\{(a+b)^2(a+b+1)\}\). The pre- and post-change posterior variances are approximately 0.000330 and 0.000622. Because the posteriors are independent, the variance of their difference is their sum, giving a standard deviation of \(\sqrt{0.000330+0.000622}=0.0309\). A normal approximation gives a 95% credible interval for the difference of \(0.1391\pm1.96(0.0309)\), or approximately 0.079 to 0.200. The interval suggests a higher post-change event probability in this example. It does not establish that the protocol caused the difference: other changes at the same time could also explain it.
Diagnostics, Assumptions, and Common Mistakes
A trend estimate is only as useful as the data and model behind it. Check that each period uses the same eligibility rules, outcome definition, and follow-up window. Record whether denominators count eligible patients or only patients with observed outcomes; changing missingness can create an apparent change in rates. Examine the number of patients and events by period, and look for sparse intervals, abrupt jumps, or patterns inconsistent with a linear trend. Compare the model's fitted rates with the plotted observed rates across time. Large, systematic discrepancies suggest that the chosen time structure may be inadequate.
If baseline risk varies across periods, crude rates can drift because the mix of patients changes even when rates within comparable covariate groups do not. Examine time trends after applying an appropriate outcome model or standardizing to a common patient population. This addresses measured case-mix differences, not unmeasured differences. The earlier material on outcome regression and standardizing effects provides the relevant adjustment concepts. Negative-control outcomes and exposures can provide additional clues about changing data processes or background conditions, but they do not by themselves identify the cause of drift.
- Mistake: treating a time trend as proof of a particular cause. A rising rate may be consistent with changing disease severity, care, or measurement, among other explanations. The observed trend alone cannot distinguish them.
- Mistake: comparing only two endpoints in time. A first-to-last difference can be driven by random variation in either period. Inspect intermediate periods and their uncertainty, and use a model suited to the observed pattern.
- Mistake: assuming adjustment removes all temporal change. Adjustment can address measured differences represented by the model. It cannot automatically correct changing outcome definitions, unmeasured case mix, or a misspecified time trend.
- Mistake: calling a credible interval a probability that the fixed parameter lies inside it. In Bayesian analysis, a 95% credible interval contains 95% posterior probability under the model and prior. That statement is conditional on the analysis assumptions.
A useful comparison is with a single pooled rate, which is simpler but conceals temporal structure; a time-stratified plot, which is transparent but may be noisy; and a regression model, which summarizes a pattern and can adjust for measured covariates but depends on its functional form and assumptions. No one summary replaces checking the data construction, clinical context, and plausible mechanisms.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- A control cohort's event rates rise from 8% to 15% across four years. What additional information should be examined before concluding that background risk increased?
- A fitted annual log-odds trend is \(\beta=0.10\). What annual odds ratio does this represent, and why is it not a 10-percentage-point increase in risk?
- Why can a linear time trend be misleading when rates are stable and then jump at a known date?
- A 95% posterior credible interval for an annual trend includes zero. State a correct interpretation and one incorrect interpretation.
Solutions. (1) Examine period denominators and uncertainty, outcome definitions, follow-up completeness, patient case mix, and dates of changes in care or data collection. The observed rise alone cannot distinguish these explanations. (2) The odds ratio is \(\exp(0.10)=1.105\), or about 1.11. It is a multiplicative change in odds, not an absolute change in probability; the corresponding risk change depends on the starting risk. (3) One slope averages over distinct regimes and may fit neither the stable periods nor the jump well. A prespecified segmented model can represent a level shift. (4) Correct: under the specified model and prior, the posterior interval includes both negative and positive annual trends. Incorrect: there is a 95% frequentist probability that the fixed trend lies in this particular interval.