Horizon-Specific Calibration Plots

Why Plot Calibration at a Chosen Horizon?

A joint model can produce an individual’s predicted probability of experiencing the event within a specified period, conditional on being event-free at a prediction time and on the longitudinal history observed up to that time. A horizon-specific calibration plot asks whether those predictions agree with event probabilities observed in a validation cohort. The plot is a practical summary: it places groups of similar predicted risks on one axis and censoring-adjusted estimates of their event probabilities on the other.

The horizon matters. A prediction for the next six months is not the same target as a prediction for the next two years, even for the same patient at the same prediction time. Choose the landmark time and horizon before constructing the plot, following the earlier material on Choosing Prediction Times and Horizons. The earlier material on Calibration of Conditional Survival Predictions establishes the conditional prediction target; here the focus is the group-based plot and its censoring adjustment.

Constructing the Plot

Begin with a validation cohort whose members are eligible at the landmark: their event-free status through \(s\) is known, and their longitudinal history through \(s\) is available. Compute each patient’s predicted risk \(p_i(s,h)\) from the fitted joint model. The plot evaluates those predictions in the validation cohort; fitting the joint model and evaluating it on the same individuals can make apparent calibration look better than it is.

Divide the predicted risks into a small number of groups, often using quantiles so that group sizes are roughly balanced. For group \(g\), calculate the average predicted risk, \(\bar p_g\). The horizontal coordinate is \(\bar p_g\), not the group’s midpoint or an arbitrary label. The vertical coordinate estimates the event probability by \(s+h\) in that group, accounting for patients whose event status at the horizon is unknown because they were censored earlier.

Let \(W_i\) collect variables used to model censoring and the prediction or grouping, including relevant baseline covariates and the observed history \(\mathcal{Y}_i(s)\). If prediction-group membership is not already determined by these variables, include the group or predicted risk as well. Among landmark-eligible patients, define the conditional censoring survival \(G_s(t\mid W_i)=P(C_i\ge t\mid C_i>s,T_i>s,W_i)\), for \(t\ge s\). A patient who has an event by the horizon contributes an event outcome if that event is observed; a patient known to remain event-free through the horizon contributes a non-event outcome. Patients censored before the horizon without an event do not have a known binary outcome for this plot.

For a group \(g\), a useful inverse-probability-of-censoring-weighted (IPCW) estimate is the weighted proportion among members whose binary outcome is observed by the horizon. Let \(R_i=1\) if patient \(i\) has an observed event by \(s+h\) or is known to be event-free through \(s+h\), and let \(D_i=1\) if an event was observed by \(s+h\). Then

For an observed event, the weight uses the probability of remaining uncensored through that event time. For a patient observed event-free through the horizon, it uses the probability of remaining uncensored through \(s+h\). The estimate \(\widehat q_g\) is the vertical coordinate; \(\bar p_g\) is the horizontal coordinate. This ratio form is a Hájek-type weighted estimate. Under the assumptions above, it is consistent for the group’s event probability; in a finite sample it is not generally exactly unbiased.

Why the Weights Recover Group Risk

The justification is that inverse censoring weights correct the unequal chance that a patient’s binary outcome is observed. Conditional on \(W_i\), an event at time \(t\) is observable through \(t\) with probability \(G_s(t\mid W_i)\). Thus weighting an observed event by the reciprocal of that probability recovers its expected event contribution. The same reasoning applies to a patient event-free through the horizon.

These equalities use \(T_i\perp C_i\mid W_i\) in the landmark-eligible cohort and the definition of \(G_s\). Summing the first contributions recovers the group’s event count in expectation; summing both contributions recovers its size. Their ratio therefore estimates the event proportion. If independence fails after conditioning on \(W_i\), or if censoring survival is essentially zero for some relevant patients, the argument does not justify the plot’s vertical coordinates.

Reading the Plot and Checking Its Construction

Draw the 45-degree line as a reference for agreement. A point above the line has a larger estimated observed event probability than mean predicted risk in that group, indicating underprediction there; a point below the line indicates overprediction. The reference is not a fitted model, and proximity to it does not prove that predictions are accurate for every individual. Grouping compresses information: different prediction values and histories are represented by one point.

Check the plot’s construction before interpreting its shape. Verify that the prediction time and horizon are identical for every patient, that risks are calculated from information available by \(s\), and that the endpoint definition is applied consistently. Inspect group sizes and the spread of predicted risks within each group. Also inspect censoring survival estimates and the distribution of IPCW weights: very small \(G_s\) values generate large weights, so a few patients can dominate a point. The censoring model should reflect the information needed for conditional independent censoring, not merely whichever variables are easiest to include.

Alternatives, Limitations, and Common Pitfalls

A raw event fraction among patients with known horizon status is simple, but it can be biased when outcome observability is related to event risk. A Kaplan–Meier estimate within each prediction group is another option under appropriate censoring assumptions; it gives a group-level survival estimate that can be converted to event risk. IPCW makes the role of measured predictors of censoring explicit and can accommodate conditional censoring models, but it depends on those models and on adequate positivity. Neither approach removes bias from unmeasured dependence between event and censoring times.

A grouped plot is also sensitive to choices about the number and boundaries of groups. Too few groups can hide meaningful differences; too many produce noisy points, especially when censoring is heavy. The x-coordinate is a group mean, so a wide range of risks within a group may conceal individual-level miscalibration. A plot is a descriptive diagnostic, not a single definitive test or a substitute for checking the prediction target and validation design.

• Wrong: “Censoring is independent of event time given baseline covariates, so IPCW is automatically valid within each predicted-risk group.”
• Correction: Independence must hold conditional on variables sufficient for the censoring adjustment in the landmark cohort, including relevant longitudinal history and prediction or grouping information. Conditioning on history or group membership can alter the relationship between event and censoring times.
• Wrong: Treating a patient censored before the horizon without an event as event-free at the horizon.
• Correction: That patient’s binary status is unknown. Exclude that outcome from the complete-endpoint sums and use valid censoring weights for observed endpoints.
• Wrong: Reading points above the diagonal as evidence that the joint model causes more events.
• Correction: Above-diagonal points indicate that estimated observed risk exceeds predicted risk in those groups; they describe calibration, not causation.