Why Competing-Risks AUCs Need an Explicit Control Definition
A time-dependent area under the receiver operating characteristic curve (AUC) measures how well a prediction ranks people who experience an event by a chosen horizon above people who do not, under a specified definition of “do not.” With competing risks, that definition is consequential: a person who experiences a competing event before the horizon cannot subsequently experience the event of interest as their first event. Counting that person as a control answers a different question from restricting controls to people who remain event-free through the horizon.
For a joint-model prediction made at landmark time \(s\), consider people who are event-free at \(s\) and have observed longitudinal history \(H_i(s)\). Let \(T_i\) be time to the first event and \(J_i\) its cause. For cause \(k\) and horizon \(\tau>0\), the predicted cumulative incidence is the probability of experiencing cause \(k\) by \(s+\tau\), conditional on being event-free at \(s\) and on the prediction information available then. The introductory material on evaluating cumulative incidence predictions covers that prediction target; here the focus is how to evaluate its ranking ability.
Write \(\widehat R_{ik}(s,\tau)\) for person \(i\)'s predicted cumulative incidence of cause \(k\) over the horizon. Higher values indicate greater predicted risk. For any case \(i\) and control \(j\), the score comparison is 1 if \(\widehat R_{ik}>\widehat R_{jk}\), 0 if it is lower, and one-half for a tie. Giving ties half credit is the standard convention: a tied pair is neither correctly nor incorrectly ordered.
Two Useful Cause-Specific AUC Estimands
The event-free-control AUC compares cause-\(k\) cases with people who remain free of every event through the horizon. It asks whether the prediction separates people who experience cause \(k\) from those who experience no event by that time. The non-case-control AUC compares the same cases with everyone who does not experience cause \(k\) by the horizon, including people with competing events. It asks whether the prediction distinguishes cause-\(k\) cases from all people who avoid that particular cause during the horizon.
Both are valid discrimination summaries, but they concern different case-control populations. Neither definition should be left implicit. In particular, a competing event is a non-case for the second estimand, but not an event-free control for the first. Terminology for these competing-risk AUC variants is not completely uniform across applications, so a report should state the case and control definitions directly.
The estimands can be written using a comparison function \(\phi(a,b)\), equal to 1 when \(a>b\), one-half when \(a=b\), and 0 when \(a<b\). Let \(K\) denote the event-free controls and \(N\) the non-case controls. Then:
These are ranking measures, not calibration measures. An AUC does not tell us whether predicted risks are numerically close to observed risks, and it does not establish that using a prediction improves decisions or outcomes. Calibration and discrimination answer different validation questions.
Worked Example: Changing the Control Group Changes the AUC
Suppose six hypothetical people are event-free at a landmark. A model predicts each person's cause-1 risk over the next year. At one year, two people experience cause 1, two experience a competing cause, and two remain event-free. The predicted risks are:
| Outcome by one year | Predicted cause-1 risks |
|---|---|
| Cause-1 cases | 0.45, 0.30 |
| Competing events | 0.40, 0.10 |
| Event-free controls | 0.20, 0.15 |
Event-free-control AUC. Compare each of the two cases with each of the two event-free controls, giving four pairs. The case with risk 0.45 outranks both controls, and the case with risk 0.30 also outranks both. Thus all four pairs are concordant:
Non-case-control AUC. There are now four controls: both competing-event subjects and both event-free subjects, giving \(2\times4=8\) case-control pairs. The case with risk 0.45 outranks all four controls, contributing 4 concordant pairs. The case with risk 0.30 outranks the controls with risks 0.20, 0.15, and 0.10, but not the competing-event control with risk 0.40, contributing 3. Therefore:
The values differ because the second definition includes a competing-event subject whose predicted cause-1 risk, 0.40, exceeds one case's score. The event-free-control AUC of 1.000 does not mean perfect separation from all people who avoid cause 1; it means perfect separation from this example's event-free controls.
Estimation When Follow-Up Is Censored
In validation data, a person's status at \(s+\tau\) may be unknown because follow-up ends first. Simply dropping everyone censored before the horizon can bias the AUC: those retained may not represent all cases and controls. One standard approach is inverse-probability-of-censoring weighting (IPCW). Let \(D_i\) be the censoring time and let \(G_i(v)\) be the conditional probability of remaining uncensored through time \(v\), given the covariates or history used in the censoring model. A case whose cause and event time are observed at \(T_i\) receives a status weight \(1/G_i(T_i)\). A control whose status requires confirming event-free follow-up through \(s+\tau\) receives weight \(1/G_i(s+\tau)\). For a non-case whose competing event is observed before the horizon, the relevant status is observed at that event time, so its censoring weight is evaluated there.
For observed case-control pairs, the pair weight is the product of the case and control status weights. The weighted AUC is the weighted sum of comparison credits, divided by the total pair weight. The following arithmetic illustrates the ratio; its pair-weight totals are hypothetical.
Worked Example: A Pair-Weighted AUC With Censoring
Suppose the observed case-control pairs have total pair weight 5.904. Among the comparisons, two concordant pairs each have weight \((1/0.90)(1/0.80)\), and one other concordant pair has weight \((1/0.80)(1/0.80)\). All remaining pairs together contribute weight 1.5637 to the denominator but zero concordance credit. There are no ties in this example.
The weighted concordance numerator is calculated without rounding its components:
Dividing by the total pair weight gives:
In this hypothetical validation sample, the IPCW estimate is about 0.735: a randomly selected case-control pair, weighted to account for censoring, receives concordance credit about 73.51% of the time, where a tie would count as one-half. This is a sample estimate of the stated population AUC, not a probability that the model is correct.
The simple pair-product calculation depends on appropriate censoring weights. A key assumption is independent censoring conditional on the variables included in the censoring model: among people with the same modeled information, censoring must not carry further information about their event status. The estimated probability \(G_i(v)\) must also be positive over follow-up times needed for the analysis. If it is near zero, weights become very large and a few observations can dominate the estimate. In practice, uncertainty may be assessed with a suitable bootstrap that repeats the estimation procedure, including any censoring-model fitting.
Why the Risk Score Must Match the Target Event
For cause-specific discrimination, the natural score is predicted cumulative incidence for cause \(k\) over the stated horizon. A cause-specific hazard is an instantaneous event rate among people still event-free; it is not the same quantity as the probability of experiencing cause \(k\) by a future time. Competing causes affect that probability by removing people from the event-free risk set.
Worked Example: A Higher Cause-Specific Hazard Can Mean Lower Risk
Consider two hypothetical profiles with constant cause-specific hazards over one time unit. For profile A, the cause-1 hazard is 0.80 per time unit and the competing-event hazard is 2.00. For profile B, the corresponding hazards are 0.50 and 0.05. With constant hazards, the cause-1 cumulative incidence by time 1 is:
For A, substitution gives \(0.80/2.80\{1-\exp(-2.80)\}\approx0.2683\). For B, it gives \(0.50/0.55\{1-\exp(-0.55)\}\approx0.3846\). Although A has the higher cause-1 hazard, B has the higher probability of experiencing cause 1 by time 1 because A also has a much higher competing-event hazard. Ranking these profiles by cause-1 hazard would therefore reverse their ranking by predicted cause-1 risk. An AUC intended to evaluate cumulative-incidence predictions should use those predictions as its scores.
Assumptions, Diagnostics, and Limitations
An interpretable AUC requires a clearly defined target population, cause, landmark, horizon, case definition, control definition, and prediction score. The predictions must use information available at the landmark, rather than future measurements or event information. Event causes must be classified adequately, and censoring must be addressed rather than silently treated as event-free follow-up. If predictions are estimated on the same data used to evaluate them, apparent discrimination can be optimistic; validation should respect the prediction setting and the data structure.
Useful checks include tabulating the numbers of cases, controls, competing events, and censored observations at each horizon; examining the distribution of censoring weights; and checking whether a few extreme weights dominate the estimate. Report the control definition and horizon alongside the AUC. A time-dependent AUC can change over time, and a single value does not summarize every horizon. AUCs also do not convey clinical consequences at a particular risk threshold, nor do they provide the calibration information needed to judge absolute risk predictions.
The main alternative within competing-risks discrimination is not a universally superior AUC, but a different case-control estimand. Event-free controls isolate separation from those with no event by the horizon. Non-case controls incorporate competing events and may better match a question about distinguishing the target event from its absence. A cause-specific hazard AUC instead targets ranking of a hazard-related quantity and should not be substituted when the scientific target is cumulative incidence. The estimand should follow the intended prediction question.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- At horizon \(\tau\), a person experiences cause 2 but not cause 1. Are they a control for the cause-1 event-free-control AUC, the cause-1 non-case-control AUC, both, or neither?
- There are two cause-\(k\) cases with scores 0.60 and 0.40, and one event-free control with score 0.40. What is the event-free-control AUC using half credit for ties?
- Why can censoring people before the horizon and calculating an unweighted AUC from those remaining produce bias?
- Two profiles have cause-\(k\) hazards of 0.7 and 0.5, but the first has a much larger competing-event hazard. Which score should be used to evaluate prediction of cause-\(k\) cumulative incidence?
Solutions. (1) The person is a non-case control but not an event-free control, because cause 2 occurred by the horizon. (2) There are two pairs: the score 0.60 case wins, while the 0.40 case ties, so the AUC is \((1+0.5)/2=0.75\). (3) Remaining uncensored people may differ systematically from those censored, so the observed cases and controls need not represent the target populations; IPCW can correct this under its censoring and positivity assumptions. (4) Use predicted cause-\(k\) cumulative incidence over the target horizon, because it incorporates the competing event's effect on the probability of cause \(k\) occurring first.