Using Historical Data for Design Inputs

Why Historical Inputs Need Careful Extraction

Historical data can help specify the event rates, competing-event burden, censoring, and follow-up expected in a new competing-risks study. The useful input is not simply a percentage of participants with the target event. It is the probability of that event by a prespecified time, in a population relevant to the planned study, while accounting for competing events and incomplete observation.

For design, choose the time horizon that corresponds to the planned analysis or clinical question. A target-event incidence at 12 months, for example, is not interchangeable with an incidence at 24 months. Nor is an estimate from a historical population automatically appropriate for a new one: eligibility, treatment, calendar period, event definitions, and follow-up procedures may all differ.

Definition: Let $T$ be the time to the first event of any cause, and let $J$ identify its cause. For target cause $k$, the cumulative incidence function (CIF) is $F_k(t)=P(T\leq t,J=k)$: the probability of experiencing the target event by time $t$, before any competing event. The event-free survival probability is $S(t)=P(T>t)$. In a study with a target cause and one competing cause, $F_1(t)+F_2(t)+S(t)=1$.

Extracting Incidence From Historical Risk Sets

For a historical cohort, define a common time origin, the target and competing causes, and the design horizon before calculating estimates. Under complete event ascertainment and an appropriate risk-set construction, the Aalen–Johansen estimator for cause $k$ accumulates cause-specific event increments while accounting for survival free of every event up to each event time.

Definition: At distinct event time $t_j$, let $Y_j$ be the number at risk just before $t_j$, $d_{kj}$ the number of cause-$k$ events, and $d_j$ the number of events of all causes. The pooled Aalen–Johansen estimate is $$ \widehat{F}_k(t)=\sum_{t_j\leq t}\widehat{S}(t_j-)\frac{d_{kj}}{Y_j}, \qquad \widehat{S}(t_j)=\widehat{S}(t_j-)\left(1-\frac{d_j}{Y_j}\right), \qquad \widehat{S}(0)=1. $$

This pooled estimator requires marginal independent censoring in the source population: the censoring time must be independent of the event time and cause, so that those still observed represent those who would otherwise remain under observation. If censoring is independent only conditional on covariates, that condition alone does not justify using the unadjusted pooled estimator. When event risks and censoring vary with those covariates, the composition of the pooled risk set can change over time and distort the estimate. Estimate conditional incidences and standardize them to a chosen covariate distribution, or use an appropriate method such as inverse-probability-of-censoring weighting. Also check that event causes are recorded reliably and that any delayed entry is handled in the risk sets.

Worked Example: Calculating Two Incidences at Six Months

Suppose a hypothetical historical cohort begins with 100 participants. At month 3, while all 100 are at risk, 8 experience the target event and 4 experience a competing event. Between months 3 and 6, 18 more participants are censored. Thus 70 are at risk just before month 6, when 7 target events and 7 competing events occur. Assume, for this calculation, that censoring is marginally independent of event time and cause and that causes are completely ascertained.

At month 3, the event-free estimate falls from 1 to $1-(8+4)/100=0.88$. The target-event contribution at that time is $1(8/100)=0.08$; the competing-event contribution is $1(4/100)=0.04$.

At month 6, the conditional event-free fraction among those at risk is $1-(7+7)/70=0.80$. The event-free estimate is therefore $0.88(0.80)=0.704$. The month-6 contributions are $0.88(7/70)=0.088$ for the target cause and $0.88(7/70)=0.088$ for the competing cause.

Consequently, the estimated target-event incidence by month 6 is $0.08+0.088=0.168$, or 16.8%; the competing-event incidence is $0.04+0.088=0.128$, or 12.8%. The remaining event-free probability is 70.4%. These estimates sum to 100%, as required: $0.168+0.128+0.704=1.000$. The 18 censored participants contribute no event increment; their presence reduces later risk sets.

The recurrence behind the estimator is direct. At an event time, each participant’s estimated probability of remaining event-free is multiplied by $1-d_j/Y_j$. The probability mass assigned to cause $k$ at that time is the mass event-free immediately before the time, $\widehat{S}(t_j-)$, multiplied by the observed conditional fraction $d_{kj}/Y_j$. Summing those increments over times gives $\widehat{F}_k(t)$. Since the cause-specific fractions sum to $d_j/Y_j$, the updated event-free mass plus all cause increments equals the pre-event mass. This establishes the stated probability accounting at each event time.

Extracting Censoring and Follow-Up Inputs

Censoring and follow-up affect how much information a proposed study is likely to obtain. Record the number and timing of withdrawals or losses to follow-up, the administrative study-end date, and the distribution of observed follow-up. A simple proportion censored before the design horizon is useful descriptively, but it is not by itself a censoring distribution: it ignores when censoring occurred and who remained under observation at each time.

To estimate a censoring-time distribution, one can use a Kaplan–Meier calculation for censoring, treating observed censoring as the event and clinical events as right-censoring for this specific calculation. This requires an appropriate independent-censoring structure for estimating the censoring distribution. It is distinct from the Aalen–Johansen estimate of clinical event incidence. Also report the amount at risk at the planned horizon; a long nominal follow-up period does not guarantee many participants remain observable that long.

Worked Example: Summarizing Early Censoring

In a hypothetical cohort of 100 participants, by month 3 there are 12 clinical events of either cause and 8 withdrawals. At month 6, 80 remain under observation; 10 experience a clinical event and 6 withdraw. To estimate the probability of remaining uncensored beyond month 6, treat the clinical events as right-censoring observations in the censoring-time Kaplan–Meier calculation.

At month 3, the estimated probability of remaining uncensored is multiplied by $1-8/100=0.92$. At month 6, the multiplier is $1-6/80=0.925$. Thus the estimate is $$ \widehat{P}(C>6)=0.92(0.925)=0.851. $$

The estimated probability of remaining uncensored beyond month 6 is 85.1% under the assumptions of this censoring-distribution calculation. This is not the probability of remaining free of a clinical event: clinical events were treated as censoring only to estimate the censoring-time distribution. The estimate should be accompanied by the numbers at risk and the censoring definition.

Assessing Whether Estimates Transport

Transportability here means whether a historical estimate is a reasonable input for the population and procedures of a planned study. Start by comparing the populations and study conduct, not merely their overall event percentages. Important comparisons include eligibility criteria, baseline prognostic factors, treatment and care standards, calendar period, event definitions and adjudication, competing-event ascertainment, time origin, and visit or follow-up schedules.

If target-population composition differs, a marginal historical CIF may be misleading even when stratum-specific risks are relevant. One option is to estimate CIFs within sufficiently well-defined covariate strata and average them using the planned study’s covariate distribution. This requires that the relevant strata are represented in the historical data and that conditional risks are reasonably stable across the source and target populations. With continuous covariates, standardization averages predicted conditional incidences over the target covariate distribution. If key effect modifiers or prognostic factors are unmeasured, standardization cannot remove the resulting concern.

Worked Example: Standardizing to a New Risk Mix

A hypothetical historical cohort has two prespecified risk strata. At a fixed design horizon, the estimated target-event CIF is 10% in the lower-risk stratum and 30% in the higher-risk stratum. The source cohort is 80% lower-risk and 20% higher-risk, whereas the planned study population is expected to be 40% lower-risk and 60% higher-risk. Assume these stratum-specific CIFs are estimated appropriately and are transportable to the planned study within each stratum.

The source-population marginal CIF is $0.80(0.10)+0.20(0.30)=0.14$, or 14.0%. The standardized estimate for the planned population is $$ \widehat{F}_{\mathrm{target}}(t) =0.40(0.10)+0.60(0.30) =0.22. $$

The standardized target-event incidence is 22.0%, not 14.0%. The difference arises from the planned population’s greater share of the higher-risk stratum, not from a change in either stratum-specific estimate. This calculation is defensible only if the risk strata are defined comparably, the historical data support estimates in both strata, and the within-stratum incidences are relevant to the planned setting.

Diagnostics, Alternatives, and Limitations

Before using historical inputs, inspect the cause-specific event counts, numbers at risk, censoring counts, and uncertainty at the horizon. Sparse late risk sets can make an apparently precise percentage unstable. Compare censoring patterns across important covariates and study sites; strong differences are a warning against relying on a pooled estimator without adjustment. Check whether the historical and planned studies use the same event definitions and time origin, and whether follow-up is long enough to estimate incidence at the intended horizon.

The main alternatives answer different questions. A crude proportion with the target event is appropriate only when event status is fully observed for everyone at a common fixed horizon; with incomplete follow-up it generally mishandles both censoring and competing events. A Kaplan–Meier estimate that treats competing events as ordinary censoring estimates a hypothetical “net” quantity under additional assumptions, not the observed-world CIF, and typically overstates the probability of the target event when a competing cause occurs. The Aalen–Johansen estimator directly estimates observed-world incidence under its censoring condition. Conditional estimation followed by standardization or a suitable weighting approach is preferable when covariate-dependent censoring or a changed covariate mix makes the pooled estimate unsuitable.

Historical estimates also have sampling uncertainty and may be affected by selection, missing causes, changing clinical practice, or differences in treatment. Report confidence intervals or other uncertainty summaries, and consider plausible ranges rather than treating one estimate as known. Sensitivity analysis can examine how conclusions change under alternative incidence, competing-event, censoring, and follow-up inputs. These data support design assumptions; they do not establish that a treatment effect observed elsewhere will occur in the new study.

Key takeaway: Extract target and competing CIFs at a clinically relevant horizon, describe censoring and follow-up separately, and assess population comparability. Use a pooled Aalen–Johansen estimate only when marginal independent censoring is justified; conditional independence alone calls for an analysis that accounts for covariates.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

Why should a competing event not be counted as censoring when estimating the observed-world target-event CIF?
A pooled Aalen–Johansen estimate is calculated in a source population where censoring depends on a covariate that also predicts event risk. Is conditional independence of censoring given that covariate sufficient to justify the pooled estimate?
At a fixed horizon, a target-event CIF is 0.12 in one stratum and 0.32 in another. If the planned population is 25% in the first stratum and 75% in the second, calculate the standardized CIF.
Name two reasons, other than a changed covariate mix, why a historical CIF may not transport to a planned study.

Solutions. (1) A competing event prevents the target event from occurring first and is part of the outcome process, not loss of observation. Censoring it as though the participant could later experience the target event changes the estimand. (2) No. The pooled risk set can have a changing covariate mix, so use conditional incidence estimates with standardization or an appropriate adjustment such as inverse-probability-of-censoring weighting. (3) The standardized CIF is $0.25(0.12)+0.75(0.32)=0.27$, or 27.0%. (4) Examples include changed event definitions, different treatment or care standards, different time origins, altered follow-up schedules, and incomplete or inconsistent cause ascertainment.