Why Control Definition Matters
A transition-specific prediction assigns each person a predicted probability of a particular transition by a chosen horizon. An ROC analysis asks how well those predictions rank people who experience that transition against people who do not. The phrase “do not experience the transition” is not enough to define the comparison when another destination can occur first: a person may leave the origin state by a competing transition, or may still be in the origin state at the horizon.
These two groups answer different validation questions. Counting competing destinations as controls evaluates ranking of the target transition against all outcomes that are not that target transition by the horizon. Excluding those destinations evaluates ranking among people who either make the target transition or remain in the origin state. Neither definition is universally preferable. The intended use of the prediction determines which comparison is relevant.
This definition treats the target as the first outgoing transition during the prediction window. A subject who first enters a competing destination belongs to \(C\), even if a later path could eventually return to \(r\). If the scientific target instead counts a later \(r\to s\) transition after returning to \(r\), the outcome groups must be redefined to match that target. The horizon convention here includes a transition occurring exactly at \(h\). Simultaneous outgoing transitions require a prespecified adjudication rule; the simple three-group classification assumes such ties do not occur or have been resolved.
Two Control Definitions and Their AUCs
Write \(R_i(h)\) for subject \(i\)'s predicted probability of the target transition \(r\to s\) by horizon \(h\). Larger values are interpreted as stronger predictions of the target transition. The same case group \(D\) is used for both definitions; the difference is which subjects serve as controls.
For a case and a control selected from the specified groups, the AUC is the probability that the case has the higher score, with a tie counted as one-half. The empirical estimate is a pairwise ranking average. In the formula, \(K\) is whichever control group the analysis has chosen, and \(I(\cdot)\) is 1 when its condition is true and 0 otherwise.
The all-noncase version uses \(K=C\cup N\); the event-free version uses \(K=N\). An AUC of 0.5 indicates no better pairwise ranking than chance under the specified case–control comparison, while 1 indicates that every case scores higher than every control (apart from the tie convention). These interpretations are conditional on the chosen groups; an AUC is not a measure of calibration or of the absolute accuracy of predicted probabilities.
Worked Example: Competing Destinations Count as Controls
Suppose six hypothetical validation subjects begin in state \(r\). At \(h=12\) months, two have made the target transition, two have made a competing transition, and two remain in \(r\). Their predicted probabilities are as follows.
| Subject | Outcome by 12 months | Predicted target probability |
|---|---|---|
| A | Target transition (case) | 0.80 |
| B | Target transition (case) | 0.60 |
| C | Competing transition | 0.90 |
| D | Competing transition | 0.50 |
| E | Remains in origin state | 0.40 |
| F | Remains in origin state | 0.30 |
For the all-noncase definition, the four controls have scores 0.90, 0.50, 0.40, and 0.30. Case A outranks three of four controls, and case B also outranks three of four. There are \(2\times4=8\) case-control pairs and six wins, with no ties:
This estimates ranking of target-transition cases against all subjects who do not make the target transition by 12 months, including those who go to a competing destination. The high score for subject C counts against the model under this definition.
For event-free controls, only E and F are controls. Both target cases score higher than both controls, so all four pairs are wins:
This perfect value describes only the case-versus-remains-in-\(r\) comparison. It does not show that the model separates target cases from competing transitions.
How the Two AUCs Are Related
The difference between these summaries can be expressed as a mixture. Suppose both \(C\) and \(N\) contain at least one subject, and consider the empirical case-control comparisons. Let \(m_C=|C|\), \(m_N=|N|\), and define \(\widehat{\mathrm{AUC}}(C)\) as the pairwise AUC comparing cases with competing-transition subjects. Each all-noncase pair belongs either to the case-versus-\(C\) comparisons or to the case-versus-\(N\) comparisons. Counting those pairs gives:
The factors are the proportions of all-noncase controls in each group. Thus, an all-noncase AUC is a weighted average of ranking against competing-transition subjects and ranking against event-free subjects. Its value depends not only on the two types of ranking, but also on how common each control type is in the validation sample. If one group is empty, the all-noncase AUC reduces to the AUC for the other group, provided that group and the case group are nonempty.
Worked Example: A Tie Changes the Comparison
In a second hypothetical sample, two cases have scores 0.70 and 0.50. Competing-transition subjects have scores 0.70 and 0.90, while event-free subjects have scores 0.40 and 0.50. The target outcome and horizon are fixed, and all six subjects have known outcomes through that horizon.
With all noncases as controls, compare each case with the four control scores. The case scoring 0.70 ties one control, loses to 0.90, and beats 0.40 and 0.50, for \(0.5+0+1+1=2.5\) pairwise points. The case scoring 0.50 loses to 0.70 and 0.90, beats 0.40, and ties 0.50, for \(0+0+1+0.5=1.5\) points. Therefore:
With event-free controls, the two control scores are 0.40 and 0.50. The 0.70 case contributes \(1+1=2\) points; the 0.50 case contributes \(1+0.5=1.5\). Hence:
The tie contributes one-half, not zero or one. The large difference between the two AUCs reflects the high-scoring competing-transition subjects, not a change in the target cases or their predictions.
Choosing the Definition for a Validation Question
Use all noncases as controls when the practical question is whether predicted target-transition risk ranks people who will make that transition above everyone who will not make it by the horizon. This can be useful when the prediction is meant to distinguish the target outcome from all alternative observed outcomes. A competing destination is then a genuine noncase for the stated target, even though it is not an event-free outcome.
Use event-free controls when the intended comparison is target transition versus remaining in the origin state through the horizon. This can be useful when the clinical or operational question concerns which subjects leave the origin state by the target route rather than who experiences any non-target outcome. The interpretation is narrower: it says nothing directly about how target cases rank against people who make competing transitions.
There is no general rule that the larger AUC is the better summary. In the first worked example, excluding competing events raises the AUC from 0.750 to 1.000. That increase does not demonstrate improved prediction under a common estimand; the control population changed. Report the case definition, control definition, prediction origin, target transition, and horizon alongside the AUC. When both questions matter, report both AUCs and the numbers of subjects in \(D\), \(C\), and \(N\).
Worked Example: A High AUC Can Hide Poor Ranking of Competing Events
Suppose two hypothetical cases have predicted probabilities 0.80 and 0.70. Two competing-transition subjects have scores 0.99 and 0.95, while two event-free subjects have scores 0.30 and 0.20. The prediction probabilities are for the target transition, and all outcomes are observed through the same horizon.
Against event-free controls, each case beats both subjects, giving four wins among four pairs:
Against all noncases, each case beats the two event-free subjects but loses to both competing-transition subjects. There are eight pairs and four wins:
The event-free AUC alone would suggest perfect separation for its restricted comparison, while the all-noncase AUC shows no net pairwise ranking advantage against all noncases. This pattern is plausible if the score is high for subjects predicted to leave \(r\) by either route, or if predictions rank competing outcomes highly. It is not evidence that either AUC is calculated incorrectly: they answer different questions.
Assumptions, Diagnostics, and Limitations
The classification and interpretation above require a clearly specified prediction origin, target transition, and horizon. Subjects must be eligible for the prediction at that origin, and the prediction score must refer to the same target and horizon for every subject. The observed outcome groups must represent the stated first-transition definitions. These requirements are substantive: an ambiguous outcome label cannot be repaired by the AUC calculation.
The simple empirical formulas also presume outcomes are known through \(h\). If follow-up ends before \(h\), a subject with no recorded transition cannot automatically be assigned to \(N\): that subject may experience either the target or a competing transition later. Restricting analysis to subjects with complete follow-up can produce selection bias if completion is related to scores or outcomes. Use an appropriate censoring-adjusted approach when censoring prevents outcome classification; the earlier material on inverse-probability-of-censoring-weighted validation addresses that problem. Such adjustment does not remove the need to define which outcomes count as controls.
A useful audit is to tabulate the number of cases, competing transitions, event-free subjects, and censored subjects. Then inspect the score distributions separately for \(D\), \(C\), and \(N\), rather than only reporting a combined AUC. Check for an empty group, very few case-control pairs, score ties, unexpected score directions, and implausibly high or low predictions. AUCs can be unstable when a group is small, and a high AUC does not establish calibration, clinical utility, or accurate probabilities.
A common mistake is to call every subject without the target transition an “event-free control.” A subject who experiences a competing transition is not event-free; treating that person as a control is valid only for the all-noncase definition, whose target is explicitly “target transition versus any other outcome by \(h\).” Conversely, excluding competing events is not neutral missing-data handling: it changes the population being compared and the meaning of the result.
Check Your Understanding
Use the definitions in this tutorial. For each question, a solution follows so you can check the reasoning.
- At a 6-month horizon, a subject leaves origin state \(r\) by a non-target transition. Is this subject a control under the all-noncase definition, the event-free definition, both, or neither?
- There is one case with score 0.8 and two event-free controls with scores 0.8 and 0.4. Calculate the event-free AUC.
- In a validation sample, the case-versus-competing-transition AUC is 0.40 and the case-versus-event-free AUC is 0.80. There are 30 competing-transition controls and 70 event-free controls. Calculate the all-noncase AUC.
- Why is it incorrect to interpret an event-free-control AUC of 0.95 as evidence that cases are well ranked above people who experience competing transitions?
Solutions. (1) The subject is a control under the all-noncase definition because the target transition did not occur by 6 months; the subject is not an event-free control because a transition occurred. (2) The case ties the 0.8 control for \(0.5\) points and beats the 0.4 control for 1 point, so the AUC is \((0.5+1)/2=0.750\). (3) The competing-transition proportion among all controls is \(30/100=0.30\), and the event-free proportion is \(70/100=0.70\). Thus the all-noncase AUC is \(0.30(0.40)+0.70(0.80)=0.12+0.56=0.680\). (4) Competing-transition subjects were excluded from that AUC's control group, so the value describes ranking only against event-free subjects and provides no direct comparison with competing-transition subjects.