Recalibrating Prediction Slopes

Why Recalibrate Prediction Slopes?

A joint model can produce dynamic predictions of a future event using a subject’s longitudinal history up to a landmark time. Even when the predictions rank subjects usefully, their spread may not match the spread of observed risks in a new population. Predictions may be too extreme, separating subjects more than the data support, or too compressed, understating real differences. A calibration slope measures this mismatch on a specified prediction scale.

The slope is not a measure of discrimination, and it does not establish that the prediction model’s associations are causally correct. It is a validation-based summary of how strongly predicted risk relates to observed risk. The slope correction described here is horizon-specific; it does not replace a full evaluation of calibration or overall predictive performance.

Definition: At landmark time $s$ and prediction horizon $u$, let $p_i$ be subject $i$’s predicted probability of the event between $s$ and $s+u$, conditional on being event-free at $s$ and on the longitudinal history available by $s$. Define $\eta_i=\log\{p_i/(1-p_i)\}$. A logistic calibration model is $$ \operatorname{logit}\{P(Y_i=1\mid \eta_i)\}=a+b\eta_i, $$ where $Y_i=1$ indicates an event by $s+u$. The calibration slope is $b$; $a$ is an intercept nuisance parameter in this slope assessment.

The ideal values are $a=0$ and $b=1$. When $b<1$, predicted risks tend to be too spread out on the log-odds scale: high risks are often too high and low risks too low. When $b>1$, predicted risks tend to be too compressed. These are population-level summaries, not claims that every individual prediction errs in the same direction.

Estimating a Slope With Complete Horizon Outcomes

If every validation subject’s event status by the horizon is known, fit the logistic calibration model using the held-out predictions as the sole predictor. Estimate $a$ and $b$ together by maximum likelihood. The intercept must generally be included even when the main interest is the slope: forcing $a=0$ can make an overall risk-level error distort the slope estimate. The earlier material on recalibrating prediction intercepts treats an intercept-only correction, which fixes the slope at one; here the slope is estimated instead.

A slope estimate below one can be used to shrink the spread of predictions. With the fitted intercept and slope, the recalibrated probability is

p_i^{\mathrm{recal}}= \operatorname{expit}(\hat a+\hat b\operatorname{logit}(p_i)), \qquad \operatorname{expit}(x)=\frac{1}{1+\exp(-x)}.

This transformation acts on log odds, not directly on probabilities. It is not generally equivalent to multiplying each risk by $\hat b$. A slope correction also does not ensure perfect calibration: the logistic form may be inadequate, and the estimate is subject to sampling uncertainty.

Worked Example: Estimating a Slope From Two Risk Groups

Suppose a hypothetical validation set has no censoring before the horizon. In a group of 100 subjects predicted to have risk 0.10, 8 experience the event by the horizon. In a separate group of 100 predicted risk 0.40, 25 experience the event. For illustration, fit the calibration model using these two predictor values.

The observed proportions are 0.08 and 0.25. Their logits are $\operatorname{logit}(0.08)=\log(0.08/0.92)=-2.442$ and $\operatorname{logit}(0.25)=\log(0.25/0.75)=-1.099$. The prediction logits are $\operatorname{logit}(0.10)=-2.197$ and $\operatorname{logit}(0.40)=-0.4055$.

With only two distinct predictor values, the fitted logistic model matches the two group proportions. Thus,

\hat b= \frac{-1.099-(-2.442)}{-0.4055-(-2.197)} =\frac{1.343}{1.792} =0.750.

The intercept is $\hat a=-2.442-(0.750)(-2.197)=-0.795$. The slope below one indicates that the predicted risk contrast is too large on the log-odds scale in this example. This two-point calculation is illustrative, not a substitute for fitting individual-level data with appropriate uncertainty estimates.

Why the Slope Has This Interpretation

The logistic calibration model provides a direct justification. A change of one unit in $\eta_i$ corresponds to multiplying the predicted odds by $\exp(1)$. Under the calibration model, the corresponding odds ratio for the observed horizon event is $\exp(b)$, holding the model’s linear form fixed. If $b=1$, the observed log-odds change at the same rate as the predicted log odds. If $b=0.75$, a one-unit increase in predicted log odds corresponds to an observed log-odds increase of only 0.75.

The recalibration transformation follows by applying the fitted model to each prediction. If $b<1$, deviations of $\eta_i$ from zero are reduced in magnitude, while $a$ shifts the overall level. This explains why the intercept is retained as a nuisance term: changing the level and changing the spread are distinct operations.

\operatorname{logit}(p_i^{\mathrm{recal}})=\hat a+\hat b\eta_i.

For example, if $\hat a=0.10$ and $\hat b=0.70$, a prediction of 0.10 has $\eta=\log(0.10/0.90)=-2.197$, so its recalibrated logit is $0.10+0.70(-2.197)=-1.438$, giving $p^{\mathrm{recal}}=1/\{1+\exp(1.438)\}=0.192$. A prediction of 0.40 has $\eta=-0.4055$, giving logit $-0.184$ and recalibrated risk $0.454$. The low prediction moves upward and the higher one slightly upward in this case because the positive intercept shifts the level; the reduced slope still narrows their logit separation.

Censoring Before the Prediction Horizon

In survival validation, an event-free subject censored before $s+u$ has unknown event status at the horizon. Treating such a subject as event-free biases the calibration analysis. One standard approach is inverse-probability-of-censoring weighting (IPCW). Among subjects at risk at the landmark, let $C_i$ be their censoring time measured from $s$, $T_i$ their event time from $s$, and $G_i(t)=P(C_i\ge t)$ the censoring survival probability. Under conditional independent censoring and positivity, subjects with observed outcomes receive weights inverse to their probability of remaining uncensored long enough to establish that outcome.

For a fixed horizon $u$, an event by $u$ observed at time $T_i\le u$ receives weight $1/\hat G_i(T_i-)$; a subject known to be event-free through $u$ receives weight $1/\hat G_i(u)$. A subject censored before $u$ without an earlier event contributes no known binary outcome. The weighted logistic calibration model can then be fitted to the observed outcomes. The censoring model should account for predictors of censoring when needed; assuming a single common censoring distribution is not justified if censoring depends on measured covariates.

Worked Example: Applying an IPCW Contribution

Consider a hypothetical landmark validation set with horizon $u=12$ months. One subject experiences the event at month 6, and the estimated probability of remaining uncensored to month 6 is $\hat G(6-)=0.80$. A second subject is event-free and observed through month 12, with $\hat G(12)=0.75$. A third is censored event-free at month 8.

The first subject’s event indicator is known to be one, so the weight is $1/0.80=1.25$. The second subject’s event indicator is known to be zero through the horizon, so the weight is $1/0.75=1.333$. The third subject’s status at month 12 is unknown and is censored before the horizon, so that subject contributes neither a zero outcome nor a standard horizon-status logistic contribution.

These weights do not make censoring irrelevant by definition. Their justification requires that, conditional on variables used in the censoring model, censoring be independent of the event process, and that censoring probabilities remain positive over the relevant follow-up. Inference should account for estimation of the censoring distribution.

Validation Design and Diagnostics

Estimate the recalibration parameters in data separate from the data used to develop or tune the prediction model. Evaluate the transformed predictions in further data, or use an appropriate resampling or cross-validation design when data are limited. If the same validation outcomes are used both to estimate $\hat a,\hat b$ and to report the corrected model’s performance, the reported performance is apparent and can be optimistic. For dynamic predictions with multiple landmarks per person, those records are not independent; uncertainty procedures should preserve subject-level clustering.

A slope is an estimated quantity, not a guaranteed correction. Examine its uncertainty interval and compare the fitted calibration relationship with a flexible plot of observed versus predicted risk where the outcome and censoring permit. Check whether the relationship is approximately linear on the logit scale and whether a few extreme predictions dominate the fit. A slope near one with a wide interval is inconclusive, not proof of perfect calibration.

Very few events or a narrow prediction range: the slope can be unstable or poorly identified. Report uncertainty and avoid a strong correction based on a weakly supported estimate.
Predictions at zero or one: their logits are infinite. Numerical clipping may be needed for computation, but should be prespecified and should not hide a prediction-model failure.
Misspecified censoring model: IPCW may not remove bias if censoring depends on factors omitted from the censoring model.
Nonlinear calibration: one slope may summarize a curved relationship poorly. A single linear slope cannot describe every local error.
Changing prediction setting: a slope estimated at one landmark, horizon, or population need not apply at another.

What Slope Recalibration Does—and Does Not Do

Approach	What is changed	Main distinction
Intercept-only recalibration	Overall level; slope fixed at one	Does not correct excessive or insufficient spread
Slope-based recalibration	Log-odds spread, with an intercept nuisance term	Targets the strength of the prediction-risk relationship at the chosen horizon
Full model redevelopment	Predictor effects and potentially the model structure	More extensive; requires appropriate development data and validation

A slope-based correction is economical when the existing ranking contains useful information but its risk contrasts are too strong or too weak. It is limited to the chosen scale and validation setting, and it does not repair omitted predictors, changing associations, or time-varying calibration patterns. In particular, this horizon-specific logistic formulation should not be interpreted as updating the survival model’s baseline hazard. The later treatment of baseline-hazard updating addresses that distinct survival-scale operation.

Common pitfall: “The estimated slope is 0.7, so multiply every predicted probability by 0.7.” This is wrong because the slope acts on log odds, not on probability. Use $p_i^{\mathrm{recal}}=\operatorname{expit}(\hat a+\hat b\operatorname{logit}(p_i))$, and state the landmark and horizon to which the estimate applies.

Worked Example: Choosing and Interpreting a Correction

Suppose a hypothetical independent validation analysis at a 6-month horizon gives $\hat a=-0.20$ and $\hat b=0.60$. For a subject with predicted risk 0.20, $\eta=\log(0.20/0.80)=-1.386$. The corrected logit is $-0.20+0.60(-1.386)=-1.032$, so the corrected risk is $1/\{1+\exp(1.032)\}=0.263$. For a subject with predicted risk 0.80, $\eta=1.386$, giving corrected logit $-0.20+0.60(1.386)=0.632$, or corrected risk $1/\{1+\exp(-0.632)\}=0.653$.

The two original risks span 0.60, whereas the corrected risks span about 0.390. The correction compresses the contrast and shifts the level downward through the negative intercept. The estimated slope suggests over-dispersed predictions at this horizon; it does not show that either individual’s corrected risk is the true risk, nor does it establish that the recalibration will work in another cohort.

Check Your Understanding

Answer each question, then compare with the solutions below.

What does a calibration slope below one indicate on the logit scale?
Why is an intercept usually included when estimating the slope?
A subject is censored event-free at month 5, before a 12-month horizon. Is that subject’s 12-month event indicator known?
If $\hat a=0$, $\hat b=0.5$, and the original risk is 0.50, what is the recalibrated risk?
Name one condition required for IPCW to address censoring in the horizon-specific analysis.

Solutions. (1) Predicted log-odds contrasts are generally too large relative to the observed event-risk relationship. (2) The intercept accounts for an overall level shift so that it is not incorrectly forced into the slope estimate. (3) No; the subject’s status at 12 months is unknown, so the record should not be treated as event-free at that horizon. (4) The original logit is zero, so the corrected logit is $0+0.5(0)=0$, and the recalibrated risk is 0.50. (5) For example, censoring must be conditionally independent of the event process given the variables in the censoring model, with positive probability of remaining uncensored through the required time.