Time-Dependent ROC Curves For Survival Predictions

Why Evaluate Discrimination at a Time Horizon?

A survival prediction is about both an event and its timing. A score can rank people well overall yet be less useful for distinguishing who will experience an event by a clinically meaningful horizon, such as one year. A time-dependent receiver operating characteristic (ROC) curve makes that horizon explicit: it compares a chosen risk score among people defined as cases and controls for that time.

Let \(T\) be the event time, \(C\) the censoring time, \(U=\min(T,C)\) the observed follow-up time, and \(\Delta=I(T\leq C)\) the event indicator. Let \(R\) be a prediction score, with larger values interpreted as higher event risk. The score may come from a multi-omics survival model. For a threshold \(c\), a person is classified as high risk when \(R>c\). The ROC curve varies \(c\) and plots sensitivity against one minus specificity.

The cumulative definition asks whether the score distinguishes people who have experienced the event by \(t\) from those who remain event-free beyond \(t\). The incident definition asks how well it distinguishes people experiencing the event around \(t\) from those still event-free at \(t\). Both are time-specific; their case groups are different, so their sensitivities and curves need not agree.

Definitions and Censoring Assumptions

For either definition, the population sensitivity and specificity at threshold \(c\) are conditional probabilities in the stated case and control groups. A curve is obtained by evaluating these probabilities across thresholds. We use \(R>c\) for a positive classification and \(R\leq c\) for a negative one; this convention assigns tied scores to the negative group.

With right censoring, an event by \(t\) is known only for people whose event is observed by \(t\), and a control at \(t\) is known only for someone observed past \(t\). Simply calculating proportions from those observed groups can be biased if censoring changes the score distribution. Inverse-probability-of-censoring weighting (IPCW) compensates for the chance that a person with particular characteristics remains observable.

The conditional independence condition is important: independence of \(C\) and \(T\) given \(Z\) alone is not enough if censoring still depends on \(R\) after conditioning on \(Z\). The ROC calculation selects people by their scores, so that remaining dependence can distort the selected case and control score distributions. If censoring is independent of both event time and score without conditioning, an unconditional censoring model may be appropriate; otherwise the relevant variables must be included in \(Z\).

Under these assumptions, define \(w_i^{\mathrm{case}}=1/G_-(U_i\mid Z_i)\) for observed events and \(w_i^{\mathrm{control}}=1/G_+(t\mid Z_i)\) for people observed beyond \(t\). The cumulative IPCW estimates are

Here \(\widehat G_-\) and \(\widehat G_+\) estimate the corresponding conditional censoring probabilities. A control contributes only if \(U_i>t\), which entails both \(T_i>t\) and \(C_i>t\). Therefore its weight uses \(G_+(t\mid Z_i)\), not \(P(C\geq t\mid Z_i)\). If censoring has no atom at \(t\), the two probabilities coincide, but the strict-boundary formula remains valid without that additional assumption.

Why the Weights Work

The essential justification is that weighting by the inverse chance of observation cancels the selection caused by censoring. For an event at time \(s\), the event is observed when \(C\geq s\). Conditional independence gives

Consequently, for any score-defined event subset, including \(T\leq t\) and \(R>c\), its expected weighted observed-event contribution equals its population contribution. For controls, \(U>t\) requires \(T>t\) and \(C>t\); likewise,

Applying these identities to both the numerator and denominator of each conditional proportion yields the target sensitivity or specificity. The argument also shows why censoring weights that omit a variable related to both censoring and score can fail: the conditional observation probability in the numerator may then differ from the one used in the weight.

Incident Curves and Local Estimation

For continuous event times, the probability \(P(T=t)\) is zero, so incident sensitivity is understood as a conditional distribution at the event time, estimated from events in a neighborhood of \(t\). A common estimator uses a nonnegative kernel \(K\) and bandwidth \(h\), where \(K_h(u-t)=K((u-t)/h)\). The bandwidth determines how local the event-time neighborhood is. An IPCW estimate of incident sensitivity is

The kernel weights observed events near \(t\); the event-time censoring weights correct their unequal observation probabilities. The incident specificity uses the same dynamic controls as the cumulative definition:

Thus the controls are event-free beyond \(t\), while incident cases are events locally around \(t\). Estimates can be sensitive to bandwidth: a narrow window may contain too few events and be unstable; a wide window can average over genuine changes in discrimination over time.

Diagnostics, Alternatives, and Limitations

Before interpreting an estimated curve, inspect the effective information behind it. Few events near a time point make incident estimates unstable, while few controls remaining beyond a late horizon make either type of specificity imprecise. Very small estimated censoring probabilities produce large weights, allowing a small number of observations to dominate. Report the horizon, threshold convention, censoring model, and—for incident curves—the kernel and bandwidth. Uncertainty should reflect estimation of the ROC quantities and censoring distribution, commonly through a bootstrap that repeats the relevant fitting steps.

Check whether the censoring model includes the score, or variables that jointly predict censoring and the score, as required by the conditional independence assumption. Compare predicted censoring probabilities with observed censoring patterns across relevant predictor groups, and examine the largest weights and their influence on the curve. Such diagnostics cannot prove conditional independence, but they can reveal poor support or a visibly inadequate model. If a meaningful group has near-zero probability of remaining observed, positivity fails; reweighting cannot recover its missing outcome distribution reliably.

Cumulative ROC curves provide a direct answer about classifying event occurrence by a specified horizon. Incident curves focus on discrimination among events occurring around a particular time, which can be useful when performance changes over follow-up. Neither replaces a single summary of pairwise ranking such as the concordance index: the concordance index aggregates comparable event-time pairs and does not directly state sensitivity and specificity at a chosen horizon and threshold. ROC analysis also does not assess whether predicted absolute risks agree with observed risks; that is a distinct calibration question.

These definitions assume a single event endpoint with right censoring. If a competing event changes the outcome definition, cases and controls must be defined for the intended competing-risk estimand rather than automatically treating the competing event as ordinary censoring. Finally, discrimination measured on the data used to fit a high-dimensional multi-omics model can be optimistic. Use independent validation data or appropriately constructed out-of-fold predictions, and keep model development separate from evaluation.