Recalibrating Competing Destinations Without Breaking Coherence

Why destination probabilities must be recalibrated together

At a fixed prediction time, competing destinations are not separate events that can each happen independently. For a person starting in a specified state, exactly one outcome category applies at the prediction horizon: for example, a particular destination is reached first, or no transition has occurred. A recalibration that improves one destination's predictions but makes the categories add to more than one is not a valid update to the predicted outcome distribution.

The target categories must also match the prediction being validated. A model might predict the first transition destination by time \(\tau\), or the state occupied at \(\tau\) after allowing intermediate transitions. These are different outcomes. Recalibrate predictions against the same mutually exclusive outcome definition, starting state or landmark, and horizon that the model was designed to predict.

This tutorial concerns updating the complete vector \(p_i=(p_{i0},p_{i1},\ldots,p_{iK})\), not recalibrating each destination as if it were an unrelated binary outcome. The earlier material on calibration slopes concerns calibration for transition predictions; here the central requirement is that any adjustments across destinations remain jointly coherent.

A multinomial recalibration model

A standard approach models the observed category using a multinomial logistic recalibration model. It uses the original prediction vector as its input and returns a new vector that sums to one by construction. Choose category \(0\) as the reference. For positive predicted probabilities, define the log-ratios \(z_{il}=\log(p_{il}/p_{i0})\), for \(l=1,\ldots,K\). Then specify

Here \(\alpha_j\) is the recalibration intercept for destination \(j\), and \(B_{jl}\) describes how the log-ratio for predicted category \(l\) contributes to the recalibrated log-odds of category \(j\) versus reference category \(0\). The model is fitted using outcomes and their corresponding original prediction vectors. In the fully specified multinomial model, \(\alpha_j=0\) for every \(j\), and \(B\) is the identity matrix. Departures from those values indicate that the original vector does not follow this particular calibration relationship exactly; they do not, by themselves, demonstrate why it is miscalibrated.

The model requires a clearly defined categorical outcome and comparable predictions for the population and horizon of interest. Its parameter estimates also require adequate data for the number of coefficients being fitted. If an outcome is censored before its category can be determined, it must not simply be treated as category \(0\). The outcome needs to be ascertained or handled with a valid censoring-adjusted method. For example, an inverse-probability-of-censoring weighted analysis requires censoring to be independent of the outcome conditional on the variables used for weighting, and requires adequate estimation of the relevant censoring probabilities.

That guarantee is mathematical, not empirical: a coherent vector can still be poorly calibrated. The model form must be suitable, and its performance should be assessed on data not used to estimate its recalibration parameters when an independent assessment is available.

Worked recalibrations

Derivation and practical checks

The normalization follows directly from the multinomial-logit definition. Write \(\eta_{ij}=\alpha_j+\sum_{l=1}^{K}B_{jl}z_{il}\) and \(w_{ij}=\exp(\eta_{ij})\), so every \(w_{ij}>0\). The reference category has weight \(1\). The probability for each destination is its weight divided by the common total \(1+\sum_j w_{ij}\); the reference probability is \(1\) divided by the same total. Adding the probabilities gives

This also shows why the shared denominator matters. A change that increases one destination's relative weight changes the denominator and therefore affects the probabilities assigned to the other categories.

Before interpreting a fitted update, check the target definition, the data used to fit it, and the resulting vectors. Useful checks include:

• Outcome definition: confirm that the categories are mutually exclusive and exhaustive for the same horizon, starting condition, and prediction target as the original model.
• Numerical coherence: verify that probabilities are finite, nonnegative, and sum to one before rounding. Unexpected totals often signal a coding or category-mapping error.
• Fit and support: examine whether some destinations are rare, whether coefficients are unstable, and whether recalibrated predictions rely on log-ratios far outside those seen in the fitting data.
• Censoring: check whether endpoint categories are observed for everyone included. A subject censored before the horizon does not automatically belong to the no-transition category.
• Performance after updating: compare category-specific observed outcomes with recalibrated predictions using a suitable method for the data structure. A valid sum-to-one constraint alone does not establish good calibration.

Predicted probabilities equal to zero cause a special problem: \(\log(p_{il}/p_{i0})\) may be undefined. A zero may be a structural impossibility specified by the model, or it may arise from rounding or numerical underflow; these cases should not be treated as interchangeable. If a small positive floor is used for numerical reasons, renormalize the prediction vector afterward and report the choice. If a category is structurally impossible, the model and target categories need to be handled so that no fitted recalibration assigns it positive probability without justification.

Alternatives, limitations, and a common mistake

Independent binary recalibrations can be useful when the scientific targets are genuinely separate binary outcomes. But if destinations are mutually exclusive categories in a single prediction, those fits do not automatically produce a coherent joint distribution. A post hoc normalization of separately fitted probabilities can enforce a sum of one when the values are nonnegative, but it does not generally reproduce the estimates from a jointly fitted multinomial model or ensure that the normalized predictions are well calibrated.

A simpler alternative is an intercept-only multinomial adjustment, which changes category proportions without changing the way predictions vary across subjects. It can be more stable when validation data are limited, but it cannot correct an incorrect relationship between the original predictions and outcomes. Adding many destination-specific slopes or interactions increases flexibility but can overfit, especially when some destinations are rare. Recalibration also cannot repair a mismatch in outcome definition, eliminate bias from an invalid censoring adjustment, or guarantee transportability to a population with different case mix.

• Common mistake: “Each destination's binary calibration curve looks acceptable, so the updated competing-risk probabilities must form a valid prediction.”
• Correction: Binary calibration assessed separately does not impose a shared total. Fit a joint categorical recalibration model, or otherwise use a justified method that explicitly preserves the sum-to-one constraint, and then assess calibration of the complete outcome distribution.

The practical aim is not merely to obtain numbers between zero and one. It is to update the probabilities for a well-defined set of competing outcomes in a way that remains mathematically coherent, is supported by the validation data, and is checked against observed outcomes using methods appropriate to censoring and the prediction target.