Harrell's Concordance for Survival Predictions

Why Pairwise Concordance Matters

A survival prediction can rank people by how soon they are expected to experience an event. Harrell’s concordance index, often written as Harrell’s C, evaluates that ranking by asking whether the person who experiences an event earlier was assigned the higher predicted risk. It is a measure of discrimination: how well predictions order individuals, rather than how close predicted probabilities are to observed probabilities.

For joint-model predictions, the risk score may summarize information from a participant’s longitudinal history and survival model. To evaluate rankings meaningfully, compare predictions made for the same prediction setting—for example, at the same landmark time and over the same prediction horizon. A score for a different horizon may encode a different prediction target.

Observed Follow-Up and Comparable Pairs

For individual \(i\), let \(T_i\) be the event time and \(C_i\) the censoring time, measured from the prediction origin. The observed follow-up time is \(Y_i=\min(T_i,C_i)\), and \(\delta_i=I(T_i\leq C_i)\) indicates whether the event was observed: \(\delta_i=1\) for an event and \(\delta_i=0\) for censoring. Let \(r_i\) be the predicted risk score, oriented so that larger values indicate greater risk and, in general, an earlier event.

A censored individual does not automatically make every pair involving them unusable. If an event is observed before that individual’s censoring time, the ordering is known: that event occurred first. But if censoring happens before the other individual’s observed event, their eventual event ordering is unknown, so that pair does not contribute. This rule makes the calculation possible without pretending that a censored observation is an event.

If \(\mathcal{P}\) denotes the set of comparable pairs, Harrell’s estimate is the concordant-pair proportion, with ties in risk score receiving half-credit:

In each pair \((i,j)\), the indexing is chosen so that \(i\) is the person with the earlier observed event and \(j\) is the person whose follow-up extends beyond that event. The index ranges from 0 to 1 when at least one comparable pair exists. A value near 1 indicates mostly correctly ordered pairs, a value near 0 indicates mostly reversed pairs, and a value near 0.5 indicates little pairwise ordering ability in that evaluation set. A value of 0.5 is not proof that the prediction contains no useful information.

Ties, Score Direction, and a Short Justification

Risk scores do not need to be probabilities. They can be linear predictors or another continuous model output, provided their direction is clear. For example, a model may output predicted survival rather than predicted event risk. At a fixed horizon, larger survival probabilities imply lower risk, so those values must be reversed in direction before applying the higher-risk-is-earlier-event rule. One option is to use \(r_i=1-\widehat S_i\), where \(\widehat S_i\) is predicted survival at that horizon.

Half-credit for a tied score is a convention with a useful symmetry: if the tied pair is ordered arbitrarily, reversing that arbitrary ordering would turn a concordant pair into a discordant one. Giving half-credit avoids favoring either direction. It does not make tied predictions informative; it records that the model did not distinguish the pair.

What Censoring Changes

Harrell’s C uses only pairs whose ordering is known from the observed follow-up. As censoring increases, fewer pairs may be comparable. More importantly, the set of retained pairs can become selective: it depends on when follow-up ends, and therefore on the censoring process as well as the event times and scores. The resulting estimate is directly a summary of the observed comparable pairs. It need not equal the concordance that would be calculated if every participant’s event time were known.

Independent censoring is a useful condition in survival analysis, but it does not make the set of comparable pairs identical to all pairs with known latent event ordering. Even when censoring is independent of event time under an appropriate conditioning set, the probability that a particular pair becomes comparable can vary with its event times. Heavy censoring can therefore make Harrell’s C unstable or make it reflect a different mix of pairs than a less-censored evaluation would. If censoring depends on participant characteristics or predicted risk, the observed comparisons can be especially selective.

Always report the number of comparable pairs alongside the estimate when possible. A high C based on very few pairs is not as informative as the same value supported by many pairs. Check follow-up and censoring across ranges of predicted risk, and investigate whether early censoring is concentrated in particular groups. Such checks do not repair selection, but they help reveal when the estimate may be difficult to interpret.

Using Concordance for Joint-Model Predictions

For a joint model, a prediction may be conditional on surviving to a landmark and on the longitudinal history observed up to that point. Harrell’s C should be computed using a clearly defined cohort and a common prediction target. For instance, if the target is event risk over a particular horizon after a landmark, generate the risk predictions at that landmark using information available by that time, and define follow-up from that landmark. Do not use later biomarker measurements to construct predictions that are meant to represent what could have been predicted at the landmark.

For a fixed horizon, higher predicted event probabilities should generally correspond to higher risk scores. Harrell’s C then evaluates how well those scores order observed event times among comparable pairs. It does not check whether a predicted risk of 0.30 corresponds to an observed event frequency of 30%; that is a calibration question. Nor is it the same as the time-dependent AUC, which evaluates discrimination using a specified time and case/control definitions. The measures can answer related but distinct questions.

Compared with a horizon-specific discrimination measure, Harrell’s C summarizes ordering across the comparable event-time pairs available in the data. That can be useful as a broad summary, but it can also obscure changes in prediction performance over time. Under heavy censoring, a censoring-adjusted concordance approach may be preferred; the choice of method should match the evaluation target and censoring conditions.

Common Mistakes and Limitations

• Counting every pair. A pair where the earlier observed follow-up is censored does not reveal the event ordering. Include it only if the other person’s observed event occurred before that censoring time.
• Reversing the score direction. If larger values mean better survival rather than greater event risk, the resulting C will be interpreted backwards unless the score is reoriented.
• Calling C an accuracy percentage. A value of 0.75 means 75% pairwise credit among comparable pairs, not that 75% of individual predictions are correct or calibrated.
• Ignoring censoring and sample size. The estimate depends on the observed comparable-pair set. Report its size and assess whether censoring patterns could make that set unrepresentative.
• Comparing unlike targets. Predictions from different landmarks or horizons need not be comparable as if they were one score. Define the target before ranking.

Harrell’s C is most useful when its pairwise meaning is kept explicit. It reduces a complex prediction output to an interpretable ranking summary, but it does not by itself establish calibration, clinical usefulness, or causal effects. Its value should be considered alongside the prediction target, censoring pattern, and other performance summaries.