Recalibrating Prediction Intercepts

Why Recalibrate the Prediction Intercept?

A joint model can rank people well by predicted event risk yet systematically predict risks that are too high or too low in a new validation population. If the main problem is an overall shift in risk, rather than a change in how predictors relate to risk, an intercept adjustment can correct that level without changing the ordering of predictions. It is a targeted recalibration, not a refit of the joint model.

Consider predictions for a common, prespecified horizon \(t\), such as the probability of an event within two years after a defined prediction time. Write \(p_i\) for person \(i\)'s predicted event probability at that horizon. The validation data provide outcomes against which those predictions can be compared. The target population and prediction horizon must be the ones for which the adjusted predictions are intended.

This is equivalent to fitting a logistic calibration model with the original prediction log-odds as an offset and its coefficient fixed at one. In the calibration relation \(\operatorname{logit}\{P(Y_i=1)\}=\delta+\operatorname{logit}(p_i)\), the ideal intercept is zero. Fixing the coefficient at one is essential: estimating a different coefficient changes the slope as well, which is a separate recalibration task.

Estimating the Shift With Complete Outcomes

First suppose every validation participant's event status by horizon \(t\) is known. Let \(Y_i=1\) if the event occurred by \(t\), and \(Y_i=0\) otherwise. Assume the participants are independent, the predictions were formed without using their validation outcomes, and each \(p_i\) is a valid probability strictly between zero and one. Treating the predictions as fixed, the Bernoulli log-likelihood for the intercept shift is

Differentiating gives the score equation. Because the second derivative is nonpositive, any finite solution to the score equation maximizes this log-likelihood.

Thus the fitted recalibrated risks sum to the number of observed events: \(\sum_i p_i^{*}=\sum_iY_i\). Equivalently, their mean equals the observed event proportion in the validation sample. This is calibration in the large for these data; it does not establish that predicted and observed risks agree at every risk level or in every subgroup. In general, the equation must be solved numerically. If all outcomes are zero or all are one, there is no finite maximum-likelihood shift: the fitted shift tends toward negative or positive infinity, respectively.

What the Adjustment Changes—and What It Does Not

For finite \(\delta\), the derivative of the recalibrated probability with respect to the original probability is

Therefore, a common intercept shift is a strictly increasing transformation: if one person's original risk is higher than another's, their adjusted risk remains higher. Ties also remain ties. Consequently, rank-based discrimination measures, including the area under the receiver operating characteristic curve when defined on the same outcomes, do not change. The numerical probabilities do change, so decisions based on a fixed risk threshold can change.

This adjustment has a deliberately limited objective. It aligns the average adjusted risk with the observed event proportion in the estimation data, but it does not fix an incorrect relationship between prediction level and outcome. For example, if predictions are too low for low-risk people and too high for high-risk people, one common shift may make the overall average agree while leaving substantial miscalibration across the range.

Fixed-Horizon Outcomes With Right Censoring

At a fixed horizon, some participants may be censored before their event status at that horizon is known. A participant who is censored event-free before \(t\) must not simply be coded as a non-event: the event could have occurred later, before \(t\). One option is to use inverse-probability-of-censoring weights in the intercept score equation, under assumptions that make the observed outcomes representative after weighting.

Let \(R_i=1\) when a participant's status by \(t\) is observed: either an event occurs by \(t\), or the participant is known to be event-free through \(t\). Let \(G(u\mid Z_i)\) denote the probability of remaining uncensored through time \(u\), conditional on the variables \(Z_i\) used to model censoring. For an observed event at time \(T_i\leq t\), use \(w_i=1/G(T_i-\mid Z_i)\); for an observed event-free participant through \(t\), use \(w_i=1/G(t\mid Z_i)\). The weighted intercept equation is

This weighted equation requires a correctly specified or otherwise adequately estimated censoring distribution, conditional independent censoring given \(Z_i\), and positivity: the relevant probabilities of remaining uncensored must be greater than zero. The event-risk predictions and censoring weights must refer to the same target population and horizon. Large weights can make the estimate unstable. The weighted solution matches the weighted event total, not necessarily the unweighted event proportion.

Diagnostics, Limitations, and Common Pitfalls

A fitted shift should be accompanied by checks on the data and the predictions. Confirm the horizon, prediction time, event definition, and target population match. Check that probabilities are finite and strictly between zero and one before taking logits; predictions exactly zero or one require a justified handling strategy, such as applying a prespecified small bound, because their logits are infinite. Inspect whether the estimated shift is driven by a few observations or large censoring weights, and quantify uncertainty using an approach appropriate to the validation design.

A common mistake is to see agreement between the adjusted mean risk and the observed event proportion and conclude that the predictions are now calibrated. The score equation guarantees only this overall agreement in the estimation data. It does not guarantee calibration within risk ranges, centers, or clinically relevant subgroups. A further diagnostic is to compare observed and predicted risks across groups or along the prediction range, with uncertainty accounted for; systematic patterns remaining after the shift indicate that an intercept-only correction is inadequate.

Another pitfall is estimating the shift and reporting performance on the same validation outcomes as if the adjustment had been fixed in advance. Because those outcomes were used to choose \(\widehat{\delta}\), performance assessed on them can be optimistic. Use an independent validation sample, or use a resampling or cross-validation design that repeats the recalibration within each training split and evaluates it on held-out data. Also remember that a common shift cannot correct changes in risk ordering, and an intercept estimated in one population may not transport to another.

Intercept recalibration is most useful when evidence suggests a relatively uniform difference in baseline risk while the original relative ordering remains appropriate. It is not a substitute for diagnosing why predictions fail. If it does not adequately describe the calibration problem, a more flexible update may be considered, with its additional estimation and validation requirements made explicit.