Predictive Checks for Time-to-Event Outcomes

Why Time-to-Event Predictive Checks Need Special Care

A time-to-event record does not always reveal when the event occurred. A participant may leave follow-up or reach the study’s administrative end before the event; the record then gives a censoring time, not an observed event time. A predictive check that treats every recorded time as an event can therefore make a model appear to predict events too early.

The central task is to compare observed and simulated summaries that represent the same time-dependent quantity. For survival outcomes, a useful starting point is the survival curve: the probability of remaining event-free beyond time \(t\). The comparison must also respect how censoring and follow-up were generated. If replicated data are fully observed while the real data are heavily censored, the two summaries do not describe comparable data.

Definition: For subject \(i\), let \(T_i\) be the time to the event of interest and \(C_i\) the censoring time. The observed time is \(Y_i=\min(T_i,C_i)\), and the event indicator is \(\delta_i=I(T_i\le C_i)\), where \(\delta_i=1\) means the event was observed and \(\delta_i=0\) means the time is censored. The survival function is \(S(t)=P(T_i>t)\); the event probability by time \(t\) is \(F(t)=P(T_i\le t)=1-S(t)\).

A basic Kaplan–Meier summary estimates survival while accounting for the changing number of subjects still at risk. At each distinct observed event time \(t_j\), let \(n_j\) be the number at risk just before that time and \(d_j\) the number of events at that time. Then the estimated survival through time \(t\) is the product of the estimated probabilities of surviving each event time up to \(t\).

\widehat S_{\mathrm{KM}}(t)=\prod_{t_j\le t}\left(1-\frac{d_j}{n_j}\right).

A censored subject contributes to the risk set until their censoring time, then leaves it. Their censoring time is not counted as an event. With ties between events and censoring, the usual product-limit convention counts subjects censored at a time as at risk immediately before that time, while only observed events contribute to \(d_j\).

Assumptions for the basic comparison: The event and censoring times are conditionally independent given the variables used to define the prediction, or the censoring mechanism is otherwise appropriately represented. The event of interest and time origin are defined consistently in observed and simulated data. The model and simulation procedure generate event times for the intended population or conditional prediction target. The Kaplan–Meier curve is a suitable summary under the censoring assumptions; it does not, by itself, correct for informative censoring.

From a Fitted Model to a Predictive Check

For each simulated replicate, generate event outcomes using the fitted model and the prediction target established for the check. Then represent censoring in a way that matches the observed study design and censoring process. For administrative censoring, this may mean applying the same planned follow-up limit. For variable follow-up or dropout, the censoring times or mechanism need to be simulated or otherwise handled in a way that makes the comparison appropriate.

Calculate the same summary from the observed data and every replicate. For example, at selected times \(t\), calculate the Kaplan–Meier estimate \(\widehat S_{\mathrm{KM}}(t)\), or its complement \(1-\widehat S_{\mathrm{KM}}(t)\). Across replicates, plot the median and selected quantiles of that summary. These form a predictive envelope: they describe variation in the summary across model-generated replicates under the specified design and censoring. They are not automatically a confidence interval for a model parameter or a formal global test.

Define the outcome and time scale.
State the event, time origin, time units, and whether the summary is survival \(S(t)\) or event probability \(F(t)\).

Specify censoring for the prediction.
Decide how administrative end-of-study censoring, dropout, or other follow-up limits are represented in each replicate.

Calculate matched summaries.
Estimate the observed curve and the corresponding curve for each replicate using the same definitions and time points.

Compare patterns and supporting information.
Look for sustained departures, but also inspect risk-set sizes and censoring patterns before interpreting a tail discrepancy.

Why the Kaplan–Meier Product Is Appropriate

At an event time \(t_j\), the fraction \(d_j/n_j\) estimates the conditional probability of an event at that time among those who remained event-free and under observation just beforehand. The corresponding conditional probability of surviving past that time is \(1-d_j/n_j\). Survival through several event times requires surviving each one, so multiplying these conditional probabilities gives the product-limit estimate.

\widehat P(T>t) \approx \prod_{t_j\le t} \widehat P(T>t_j\mid T\ge t_j,\ \text{event-free before }t_j) = \prod_{t_j\le t}\left(1-\frac{d_j}{n_j}\right).

Censoring does not appear as an event factor in this product. Instead, a censored subject reduces later risk sets. This construction relies on censoring being non-informative for the event time, at least conditional on the variables used: among subjects still at risk at a given time, those censored should not have systematically different unobserved event prospects from those who remain under observation.

Worked Example: Calculating an Observed Survival Summary

Consider eight hypothetical subjects followed for time in months. Their recorded times and statuses are: event at 2, censoring at 3, event at 4, event at 5, censoring at 6, event at 7, event at 8, and censoring at 9. There are no tied times.

At month 2, all eight are at risk and one event occurs, so the survival factor is \(1-1/8=7/8=0.875\). At month 3, one subject is censored and leaves the risk set. Thus six are at risk immediately before the event at month 4; the factor there is \(1-1/6=5/6\). At month 5, five are at risk and one event occurs, giving a factor of \(4/5\).

Therefore, through month 5,

\widehat S_{\mathrm{KM}}(5) = \frac{7}{8}\times\frac{5}{6}\times\frac{4}{5} = \frac{7}{12} \approx 0.5833.

The estimated event probability by month 5 is \(1-0.5833=0.4167\), or about 41.7%. The censoring at months 3 and 6 was not counted as an event. At month 7, three subjects remain at risk, so the factor is \(2/3\); hence \(\widehat S_{\mathrm{KM}}(7)=(7/12)(2/3)=7/18\approx0.3889\). The estimated median event time is 7 months, the first observed time at which estimated survival is at or below 0.5.

Reading Predictive Envelopes in Context

At a fixed time \(t\), the observed Kaplan–Meier estimate can be compared with the distribution of estimates at \(t\) from replicated data. A pointwise envelope might be the 5th and 95th percentiles across replicates. An observed value outside that envelope signals a local mismatch worth investigating; it does not alone establish a global model failure. Examining a curve at many time points also creates many opportunities for an observed estimate to fall outside a pointwise envelope by chance.

Always identify which quantity is plotted. A survival curve decreases as events accumulate, whereas an event-probability curve increases. A plotted event count by time is a different quantity again, and depends on the number of subjects and their follow-up. Risk-set size, number censored, and number of events by time help explain why a tail estimate may be unstable or unavailable.

Worked Example: Comparing Observed and Simulated Survival at Fixed Times

Suppose a hypothetical study follows subjects for time in days. The observed Kaplan–Meier survival estimates at days 30, 60, and 90 are 0.84, 0.55, and 0.31. From 500 simulated replicates, the 5th–95th percentile ranges for the corresponding estimates are 0.65–0.80, 0.42–0.68, and 0.35–0.62.

At day 30, the observed estimate, 0.84, is above the simulated range’s upper end, 0.80. The observed data therefore show higher estimated survival by day 30 than is typical in these replicates: events appear less frequent or later than predicted in this early period. At day 60, 0.55 lies within the range 0.42–0.68, so it is not a pointwise departure from this envelope. At day 90, 0.31 is below 0.35, suggesting lower observed survival than predicted by the replicate range at that time.

These comparisons do not prove a particular model mechanism is wrong. First check risk-set sizes and censoring near days 30 and 90. If few subjects remain under observation by day 90, the tail estimate may be imprecise. Also note that the early and late departures point in different directions; they should not be compressed into the statement that the model uniformly overpredicts or underpredicts survival.

Censoring, Diagnostics, and Alternatives

The censoring process is part of the predictive-check design, not a detail to ignore. If follow-up ends at different times across subjects, applying a common censoring time to every replicate can create a different risk-set pattern from the observed one. If dropout depends on evolving health or latent event risk, simple independent censoring may fail. In that setting, a standard Kaplan–Meier comparison can be biased, and the censoring or dropout process may need to be modeled jointly or the check framed conditionally on relevant observed information.

Inspect the number at risk and the number of events and censorings over time alongside the survival summaries. Warning signs include a long tail based on very few subjects, a marked difference between observed and simulated censoring, or an apparent curve departure that occurs only after most subjects have left follow-up. With rounded event times, ties are common; use the same tie-handling convention for observed and simulated data. A comparison of raw observed times that labels every \(Y_i\) an event is not a substitute for a censoring-aware summary.

The Kaplan–Meier curve is often more informative than a single median because it shows when the observed and simulated event patterns diverge. But a median is undefined if estimated survival never falls to 0.5 during follow-up, and medians alone can conceal differences in the early or late parts of the curve. A restricted mean survival time up to a prespecified horizon can summarize area under the survival curve over that finite interval, but it answers a different summary question and still requires appropriate censoring assumptions.

For competing events, the event of interest must be defined carefully. Treating a competing event as ordinary censoring in a Kaplan–Meier calculation does not estimate the real-world cumulative probability of the event of interest in the presence of competing events. A predictive check of that probability requires a competing-risk summary, such as a cumulative incidence function, calculated consistently for observed and simulated data. Likewise, a conditional check within covariate groups answers a different question from an overall check.

Worked Example: Detecting a Censoring Mismatch

In a hypothetical cohort of 20 subjects, two events occur at months 2 and 4, and two subjects are censored at months 3 and 5. Assume the remaining subjects are still followed at month 6. The Kaplan–Meier estimate at month 4 is calculated from the event risk sets: at month 2, \(n_1=20\) and \(d_1=1\); after that event and the censoring at month 3, 18 subjects are at risk just before month 4. Thus

\widehat S_{\mathrm{KM}}(4) = \left(1-\frac{1}{20}\right) \left(1-\frac{1}{18}\right) = \frac{19}{20}\times\frac{17}{18} \approx 0.8972.

Suppose replicated event times are generated from the fitted model, but the replicates are left uncensored even though the observed cohort has early loss to follow-up. The replicate curves then use more complete event information and may have substantially different risk sets. A discrepancy between their curves and 0.8972 could reflect the unmatched follow-up rather than event-time misfit.

If the observed censoring pattern is appropriate to condition on, apply corresponding censoring times in each replicate; if censoring is generated by a stochastic mechanism, simulate that mechanism consistently. Then compare the matched summaries. Also compare censoring counts over time. If observed censoring is informative, matching only its marginal frequency is not enough: a model or analysis that addresses the dependence between censoring and event risk is needed.

Common Mistakes

Wrong: “A subject censored at month 5 had an event at month 5.” Correction: The event is only known to occur after the observed follow-up, if it occurs at all; month 5 is a censoring time.
Wrong: “The observed curve is outside the 90% pointwise envelope, so the model fails globally.” Correction: It is a local signal for investigation. Consider multiple time points, risk-set sizes, censoring, and the overall pattern before drawing a broader conclusion.
Wrong: “Simulate events but ignore censoring because the model predicts event times.” Correction: The check compares summaries of observed data with summaries of replicated data under a comparable observation and censoring process.
Wrong: “A Kaplan–Meier curve estimates the event probability under any dropout mechanism.” Correction: Its usual interpretation depends on suitable non-informative censoring assumptions; informative dropout can invalidate that interpretation.

Key takeaway: Define the time-dependent quantity, preserve the distinction between events and censorings, and compare observed and simulated summaries under a compatible follow-up process. Interpret departures alongside risk sets and censoring diagnostics.

Check Your Understanding

Answer each question, then compare with the solutions below.

A subject is observed event-free until day 12, when follow-up ends. What are \(Y_i\) and \(\delta_i\), and how does this record enter a Kaplan–Meier estimate?
At a particular event time, 25 subjects are at risk and 3 events occur. What is the Kaplan–Meier survival factor at that time?
At day 60, observed survival is 0.40 and the simulated 5th–95th percentile range is 0.45–0.70. State the pointwise comparison correctly.
Why is it problematic to compare observed Kaplan–Meier curves with fully observed simulated event-time curves when the real data have substantial early dropout?
When would a Kaplan–Meier curve that treats other event types as censoring fail to answer the desired question?

Solutions: (1) \(Y_i=12\) days and \(\delta_i=0\). The subject contributes to the risk set through day 12 and is then removed; day 12 is not counted as an event. (2) The factor is \(1-3/25=22/25=0.88\). (3) The observed estimate is below the pointwise simulated range at day 60, indicating lower observed survival than the central simulated range there; this is a local discrepancy, not by itself a global test. (4) The observed and simulated summaries reflect different follow-up and risk-set patterns, so a difference may be due to censoring rather than event-time prediction. (5) If the target is the real-world probability of a particular event in the presence of competing events, censoring competing events in a Kaplan–Meier calculation does not estimate that cumulative incidence; a competing-risk summary is needed.