Calibrating Event Rates With Historical Data

Why Historical Calibration Requires More Than a Published Rate

A projection of when a trial will reach an event target depends on assumptions about how quickly eligible participants experience the event. Historical studies can inform those assumptions, but a rate from another study is not automatically a suitable planning rate. Differences in eligibility, endpoint definitions, follow-up, treatment, or censoring can make a numerically precise estimate poorly matched to the trial being planned.

The task here is to estimate an event rate from relevant historical data, assess whether that estimate transports to the planned trial, and show how its statistical uncertainty affects a timing forecast. A rate is an input to a forecast, not a guarantee about the future trial. The forecast remains conditional on assumptions about participants, event occurrence, and the way follow-up is accumulated.

Estimating a Rate From Historical Follow-Up

A common planning model assumes that event times follow a constant hazard \(\lambda\) over the relevant follow-up period. For \(D\) observed events and total time at risk \(Y\), the rate estimate is \(\hat{\lambda}=D/Y\). The model treats the event count as approximately Poisson with mean \(\lambda Y\). This is a useful working model when the hazard is reasonably stable and the recorded person-time represents time participants were genuinely at risk.

The interpretation of \(Y\) matters. Participants contribute time from the point at which they become at risk until the event, censoring, or another defined end of observation. The simple rate calculation assumes censoring is noninformative for the event process, conditional on the factors accounted for: participants who leave observation should not systematically have different unobserved event prospects in a way that distorts the rate. If participants can experience competing events that prevent the event of interest, a cause-specific rate alone does not give the cumulative probability of the event of interest; the competing-risk process must also be considered.

For a constant hazard, the survival probability through time \(t\) is \(S(t)=\exp(-\lambda t)\). Therefore, the event probability by \(t\) is \(1-S(t)=1-\exp(-\lambda t)\). This conversion is important: an incidence rate of 0.04 events per person-year does not mean a 4% event probability at every follow-up duration. At one year the probability is about 3.92%; at two years it is about 7.69%, under the constant-hazard model.

Assessing Whether the Historical Study Is Comparable

Comparability is a scientific judgment supported by specific checks, not a consequence of a large historical sample. The rate is most useful when the historical study and planned trial define the event and the population similarly and observe participants over follow-up that represents the trial’s planning horizon.

• Eligibility and baseline risk: Compare inclusion criteria, disease severity, prior treatment, age, and other important prognostic factors. A broader historical population may have a different event risk from the selected trial population.
• Endpoint definition and ascertainment: Check that events are defined, adjudicated, and detected in comparable ways. Different thresholds or assessment schedules can change the observed rate.
• Follow-up and censoring: Determine how follow-up starts and ends, whether time at risk is counted consistently, and whether losses to follow-up could be related to event risk.
• Treatment and setting: Compare background care, concomitant treatment, calendar period, and clinical setting. Changes in care can alter baseline event occurrence.
• Time pattern: Examine whether the event hazard appears approximately stable over the relevant time span. A single average rate may obscure an early high-risk period followed by a lower rate.

When important prognostic factors differ, a crude pooled rate can misrepresent the trial population. If reliable stratum-specific estimates are available, one approach is to calculate the event probability within each stratum and average those probabilities using the planned trial’s population weights. This targets the population of interest more directly than averaging rates without considering who will be enrolled.

Carrying Rate Uncertainty Into a Timing Forecast

A point estimate hides uncertainty. To see its effect on a forecast, consider a deliberately simple setting: \(N\) participants are all enrolled at the same time, each has the same constant hazard \(\lambda\), and the target is \(d\) events. The expected event count by follow-up time \(t\) is \(N[1-\exp(-\lambda t)]\). Setting this expectation equal to \(d\) and solving gives the forecast time

This calculation forecasts when the expected count reaches the target in this simplified setting; it is not an exact prediction of the realized date. It also isolates uncertainty due to \(\lambda\). In a trial with ongoing recruitment, participants have different follow-up durations at a given calendar date, so the event-count expectation must account for enrollment times as well. The formula below is not a substitute for that more detailed calculation.

A direct sensitivity calculation uses the lower and upper confidence limits for \(\lambda\) in the timing formula. Since \(t_d(\lambda)\) decreases as \(\lambda\) increases, the lower rate limit produces the later forecast and the upper rate limit produces the earlier forecast. This transforms uncertainty in the estimated rate into a range of conditional timing forecasts. The range is not, by itself, a prediction interval for the actual event date: it omits random variation in event counts and any uncertainty in other planning assumptions.

Justification and Practical Diagnostics

The rate-to-time conversion follows directly from the constant-hazard model. If \(S(t)\) is the probability of remaining event-free through time \(t\), a constant hazard \(\lambda\) implies \(S(t)=\exp(-\lambda t)\). Therefore the event probability is \(1-S(t)\). For \(N\) comparable participants, linearity of expectation gives expected event count \(N[1-\exp(-\lambda t)]\). Solving the equation \(N[1-\exp(-\lambda t)]=d\) yields the timing formula. This derivation depends on the same-hazard assumption for the participants in the simplified cohort; it does not require their event indicators to be identical outcomes.

Before using a calibrated rate, check whether the data support the assumptions that give it meaning. Plot event occurrence against follow-up time or examine rates over clinically meaningful time intervals. Marked changes over time, sparse risk sets late in follow-up, or a rate driven by a few events suggest that a constant-rate summary may be unstable or misleading. Also verify the denominator: mixing time before eligibility, time after an event, or time after a participant is no longer at risk will distort the estimate.

A useful workflow is to state the source and target populations, compare eligibility and endpoint definitions, estimate the rate with an uncertainty interval, and recalculate the timing forecast across that interval. If there are clear differences in baseline risk, use relevant strata or restrict the historical data to a comparable population when defensible. If hazards vary substantially over time, use a time-varying survival or cumulative-incidence description rather than forcing one constant rate to represent the whole horizon.

Alternatives, Limitations, and Common Mistakes

A simple pooled rate is transparent and easy to communicate, but it may conceal population or time differences. Alternatives include restricting the historical sample to participants meeting trial eligibility, estimating stratum-specific rates and standardizing to the planned population, or using time-to-event estimates that preserve the shape of follow-up. When competing events matter, cumulative incidence methods are needed for event probabilities. Each approach still depends on having data that are relevant to the planned trial; more elaborate modeling cannot repair an incompatible source population without additional assumptions.

Historical data can also be used as one of several planning scenarios rather than as a single definitive rate. That is useful when estimates from relevant studies differ or transportability is uncertain. The purpose is not to select whichever estimate gives the most convenient date, but to expose how evidence and assumptions influence the forecast. This tutorial’s interval for the rate covers sampling uncertainty under a specified model; it does not quantify all uncertainty about transportability, future care, or model form.

• Mistake: treating events per person-year as a fixed-time risk. The correction is to convert using a model, such as \(1-\exp(-\lambda t)\) under a constant hazard, or use an appropriate time-to-event estimate.
• Mistake: borrowing a precise rate because its confidence interval is narrow. A narrow interval does not establish that eligibility, endpoint ascertainment, follow-up, or care is comparable. Assess relevance separately.
• Mistake: interpreting a transformed rate interval as a 95% prediction interval for the trial’s event date. It reflects uncertainty in the historical rate under the model; event-count variability and other forecast uncertainties remain.
• Mistake: pooling strata before checking the planned population mix. When risk differs by stratum, calculate stratum-specific probabilities and weight them to the target population when the necessary estimates are credible.