Calibration Slopes for Survival Risk Predictions

Why Estimate a Calibration Slope?

A survival risk model can rank people reasonably well and still produce predictions that are too spread out or not spread out enough. At a fixed horizon, a calibration slope summarizes this aspect of agreement: it compares how strongly predicted risk varies with how strongly observed risk varies, on the log-odds scale. It is a compact summary, not a replacement for examining calibration across the prediction range.

Let \(\tau\) be the prediction horizon, \(T_i\) the event time for person \(i\), and \(p_i\) the risk of the event by \(\tau\) predicted before examining that person’s validation outcome. Define \(Y_i(\tau)=I(T_i\leq\tau)\). If everyone’s event status at \(\tau\) were known, a logistic recalibration model would relate the outcome to the logit of predicted risk. With censoring, the endpoint is not known for everyone, so its estimation requires additional assumptions and censoring adjustment.

The slope is estimated in a validation sample, using risks produced without fitting the prediction model to those same validation outcomes. Risks must be strictly between zero and one for the logit to be finite. In practice, predictions at or extremely near the boundaries require attention: arbitrary clipping changes the predictor values and can affect the slope.

Estimating the Slope With Censored Outcomes

Write \(X_i=\min(T_i,C_i)\) for the observed follow-up time, where \(C_i\) is the censoring time, and \(\Delta_i=I(T_i\leq C_i)\) for the event indicator. An event by \(\tau\) is observed when \(\Delta_i=1\) and \(X_i\leq\tau\). An event-free status at \(\tau\) is observed when \(X_i>\tau\). People censored before or at \(\tau\) without an observed event have unknown status at the horizon; treating them as event-free would generally bias calibration.

For censoring adjustment, let \(Z_i\) denote variables used in the censoring model, and define \(G(t\mid Z_i)=P(C_i\geq t\mid Z_i)\). The inverse-probability-of-censoring weights (IPCW) below use the following convention: censoring times are continuous around the horizon and event-censoring ties have probability zero. Thus the probability of remaining uncensored strictly beyond \(\tau\) agrees with \(G(\tau\mid Z_i)\). If censoring can have a point mass at the horizon, the event-status rule and denominator must instead use a consistent tie convention.

For an observed case, use \(w_i=1/G(X_i\mid Z_i)\); for an observed control, use \(w_i=1/G(\tau\mid Z_i)\). Let \(R_i\) indicate that the person’s status at \(\tau\) is observed, and let \(q_i=\operatorname{expit}\{a+b\operatorname{logit}(p_i)\}\), where \(\operatorname{expit}(v)=1/(1+\exp(-v))\). A censoring-weighted logistic log-likelihood is

Maximizing this weighted objective estimates \(a\) and \(b\); the estimate of \(b\) is the calibration slope. Equivalently, the fitted coefficients solve weighted logistic score equations. The censoring survival probabilities are usually estimated from a censoring model, so their estimation is itself a potential source of error.

Why the Weighting Works

The justification is that an observed outcome represents both itself and people who would have had the same status but were censored. Conditional on \(Z_i\), independent censoring makes the probability of observing a case with event time \(t\) equal to \(G(t\mid Z_i)\); the probability of observing an event-free control through \(\tau\) is \(G(\tau\mid Z_i)\) under the stated continuous-time convention. In either group, multiplying by the inverse of that probability removes the observation probability in expectation.

For example, if \(T_i=t\leq\tau\), the first indicator is one exactly when censoring has not occurred by \(t\), an event with conditional probability \(G(t\mid Z_i)\). Dividing by that probability gives conditional expectation one; if \(T_i>\tau\), the first indicator is zero. For a control with \(T_i>\tau\), the probability of \(X_i>\tau\) is \(G(\tau\mid Z_i)\), so the second weighted indicator has conditional expectation one. If \(T_i\leq\tau\), it is zero. Thus the weighted contributions recover the complete-data case and control contributions in expectation. Applying this argument to the logistic likelihood and its score explains why the weighted fit targets the complete-outcome recalibration relationship, provided the assumptions hold.

Calculating and Interpreting the Slope

The slope is a coefficient for the logit of predicted risk, not a ratio of predicted and observed probabilities. In a full data analysis, it is obtained by fitting the weighted logistic model. A small two-risk illustration shows the calculation and interpretation directly. With only two distinct prediction values and a free intercept and slope, the logistic line can pass through both corresponding observed event proportions.

To see why the intercept matters, consider recalibrating a prediction of \(p=0.25\) using the estimates above. The predicted logit is \(\operatorname{logit}(0.25)=-1.098612\), so the recalibrated logit is approximately \(0.3137+0.7737(-1.098612)=-0.5362\). Its corresponding risk is \(\operatorname{expit}(-0.5362)\approx0.3691\). Thus a slope below one does not imply that every recalibrated risk is lower than its original prediction; both slope and intercept determine the risk at a given value.

Diagnostics, Alternatives, and Limitations

A calibration slope is useful because it compresses a potentially complex relationship into one number, but that compression can hide important departures. Inspect the horizon-specific calibration plot or smoothed calibration curve alongside the slope. Look for curvature, a pattern that changes across the risk range, sparse observations at high predicted risks, extreme weights, and sensitivity to the censoring model. A single straight-line slope can look acceptable even when the calibration curve bends substantially.

The slope is distinct from calibration-in-the-large, which focuses on an overall level discrepancy while holding the predictions’ relative spread fixed. The slope instead measures spread on the log-odds scale while allowing an intercept to shift. Neither coefficient by itself describes all calibration features. A calibration curve is more flexible and can reveal nonlinearity, but is less compact and can be unstable where data are sparse. The weighted logistic slope is a useful summary when a linear recalibration relationship is a reasonable approximation.

Weight instability is a practical warning sign: if estimated \(G(t\mid Z_i)\) is very small, the corresponding inverse weight is large and a few observations can dominate the fit. This may reflect inadequate follow-up, a censoring model that needs review, or a target population with poor support in the validation data. Restricting the horizon or changing the target population may be more defensible than simply truncating weights without explanation. Uncertainty in the slope should also account for the validation sample and, where relevant, estimation of the censoring model.

A calibration slope is an association-based validation summary, not evidence that changing a model coefficient will causally change outcomes. It can also be a poor summary if the recalibration relationship is nonlinear, if the validation population differs materially from the intended target, or if the censoring assumptions fail. Report the horizon, the validation setting, the censoring adjustment, and the uncertainty estimate with the slope.

Common Mistake

• Wrong reasoning: “The slope is 0.77, so every predicted risk should be multiplied by 0.77.” The slope is not a probability-scale multiplier; it multiplies the logit of predicted risk, and the intercept also affects the recalibrated probability.
• Correction: Interpret the slope as the change in observed event log-odds associated with a one-unit increase in predicted-risk log-odds under the recalibration model. A value below one indicates overly extreme predictions on that scale, not a uniform reduction in risk.
• Another failure: Labeling everyone censored before \(\tau\) as event-free. Their status at \(\tau\) is unknown; excluding them without adjustment or using IPCW under its assumptions is necessary to avoid treating incomplete follow-up as evidence of no event.