Why censoring matters for a horizon-specific AUC
A prediction model may assign each patient a risk score for an event by a chosen horizon \(\tau\). To assess its ranking accuracy, we would like to compare scores for patients who experience the event by \(\tau\) with scores for patients who remain event-free beyond \(\tau\). In a validation cohort, however, some patients are censored before their status at \(\tau\) can be determined. Excluding them as if they were a random subset, or treating their last observed status as their status at \(\tau\), can distort the comparison.
This tutorial focuses on the cumulative/dynamic AUC at a fixed horizon. It is a probability of correctly ranking a randomly selected case above a randomly selected control, with half-credit for tied scores. It measures discrimination, not whether the predicted risks are numerically well calibrated. The incident-dynamic AUC uses a different case definition; the horizon-specific target here is not that measure.
The target concerns the population distribution of scores among all cases and controls at \(\tau\), including people whose outcomes may be censored in the validation data. The notation \(R_1\) and \(R_0\) describes the two population groups; it does not mean that the two scores come from the same patient. AUC values closer to 1 indicate better ranking in the specified direction. An AUC of 0.5 indicates chance-level pairwise ranking in the absence of ties or with ties counted by the stated convention.
Inverse-probability weighting for cases and controls
For an observed event at time \(t\), the patient is eligible to be counted as a case only if censoring has not occurred by \(t\). For an observed control, follow-up must extend strictly beyond \(\tau\). These observation probabilities are not necessarily the same at the boundary. Define
where \(W\) contains the information used to model censoring and all variables determining the score. Under a continuous censoring-time distribution, the relevant left- and right-limit probabilities agree at a fixed time. If censoring can have an atom at \(\tau\), they need not agree, so the case and control probabilities must be kept distinct.
The condition involving the score is important. If censoring depends on a predictor of the score that is omitted from \(W\), weighting only for variables in \(W\) may not restore the ranking distribution. A model’s prediction can be treated as fixed when it is computed from information included in \(W\), for example by applying a previously fitted model to validation patients. When a prediction procedure is itself estimated using the validation data, its construction must also be accounted for in the analysis.
For subject \(i\), define the case and control weights
An observed event by \(\tau\) contributes as a case; a patient observed beyond \(\tau\) contributes as a control. A patient censored by \(\tau\) without an observed event contributes to neither group because their case/control status is unknown. The weights give greater influence to observations that had a lower chance of being observed in their relevant group.
Why the weights recover the pairwise target
The key justification is an inverse-probability identity. Conditional on \(T=t\) and \(W\), the event indicator \(\Delta I(Y\le\tau)\) equals \(I(T\le\tau)I(C\ge T)\). By conditional independence, the probability of \(C\ge t\) is \(G_{\mathrm{case}}(t\mid W)\). Therefore,
For a patient with \(T>\tau\), the observed-control indicator is \(I(Y>\tau)=I(C>\tau)\). Thus, conditioning on \(T,W\),
Because the score is determined by \(W\), it is fixed in these conditional expectations. For two independent patients, conditional independence of their censoring processes and independence across individuals let the product of their case and control weights recover the complete-data pair contribution. This yields a weighted estimate of both the number of correctly ranked case-control pairs and the total number of case-control pairs.
The sum runs over ordered pairs with \(i\) as the potential case and \(j\) as the potential control. The denominator normalizes the weighted comparisons, so the estimate lies between 0 and 1 when the weights are nonnegative and the denominator is positive. In practice, \(G\) may be estimated with a censoring survival model, possibly conditional on \(W\). The estimator’s validity depends on that model representing the relevant censoring probabilities and on adequate positivity; choosing a more elaborate model does not by itself guarantee either condition.
Worked Example: Calculating a Weighted AUC
Consider a hypothetical validation sample at \(\tau=12\) months. The fitted risk scores are higher for patients predicted to be at greater risk. For simplicity, suppose the censoring probabilities below are known and correctly specified for the patients’ \(W\) values.
| Patient | Observed time \(Y\) (months) | \(\Delta\) | Score \(R(\tau)\) | Relevant censoring probability |
|---|---|---|---|---|
| A | 4 | 1 | 0.80 | \(G_{\mathrm{case}}(4)=0.80\) |
| B | 9 | 1 | 0.60 | \(G_{\mathrm{case}}(9)=0.50\) |
| C | 7 | 0 | 0.90 | Censored before \(\tau\) |
| D | 13 | 0 | 0.30 | \(G_{\mathrm{control}}(12)=0.40\) |
| E | 16 | 1 | 0.70 | \(G_{\mathrm{control}}(12)=0.40\) |
Patients A and B are observed cases, with weights \(1/0.80=1.25\) and \(1/0.50=2.00\). Patients D and E are observed controls, each with weight \(1/0.40=2.50\). Patient C is censored before 12 months, so the data do not establish whether C is a case or a control; C contributes to neither group.
There are four case-control pairs. A’s score exceeds both control scores, so A contributes \(1.25(2.50+2.50)=6.25\) weighted pair units to the numerator. B’s score exceeds D’s but not E’s, so B contributes \(2.00(2.50)=5.00\). The denominator is the product of the total case and control weights:
The estimated AUC is about 0.692, indicating that, under the stated censoring assumptions, a randomly selected patient who experiences the event by 12 months has a higher score than a randomly selected patient event-free beyond 12 months in about 69% of comparisons when ties receive half-credit. It is not a statement about the probability that any particular patient will experience the event.
Worked Example: Tied Scores and Unequal Case Weights
In another hypothetical validation sample, let \(\tau=10\) months. There are two observed cases: one has score 0.70 and event time 8 months, with \(G_{\mathrm{case}}(8)=0.50\); the other has score 0.50 and event time 3 months, with \(G_{\mathrm{case}}(3)=1.00\). There are two observed controls, with scores 0.70 and 0.40, each observed beyond 10 months and each having \(G_{\mathrm{control}}(10)=0.40\). The case weights are 2 and 1; both control weights are 2.50.
The case with score 0.70 ties one control and beats the other, contributing \(2(2.50)(0.5+1)=7.50\) to the numerator. The case with score 0.50 loses to the 0.70 control and beats the 0.40 control, contributing \(1(2.50)(0+1)=2.50\). The total denominator is \((2+1)(2.50+2.50)=15\). Therefore,
The estimated AUC is 0.667. The tied pair received half-credit, and the lower censoring observation probability for the first case gave that case twice the weight of the second. This example also checks the censoring probabilities’ direction: the probability at 8 months (0.50) is no greater than the probability at 3 months (1.00), as a censoring survival probability should be.
Boundary handling, diagnostics, and alternatives
Worked Example: Censoring Exactly at the Horizon
Suppose a patient is known to have \(T>\tau\), and the conditional censoring distribution has \(P(C=\tau\mid W)=0.20\) and \(P(C>\tau\mid W)=0.80\). With the control indicator \(I(Y>\tau)\), this patient is observed as a control only when \(C>\tau\), which occurs with probability 0.80. The correct weight is therefore \(1/0.80=1.25\), and the expected weighted indicator is \(0.80(1.25)=1\).
If one instead divided \(I(Y>\tau)\) by \(P(C\ge\tau\mid W)=1\), its expectation would be only \(0.80\), not 1. The strict control indicator and its observation probability must match. An alternative consistent rule counts a patient censored exactly at \(\tau\) as observed event-free through \(\tau\): use \(I(Y>\tau)+I(Y=\tau,\Delta=0)\) and divide by \(P(C\ge\tau\mid W)=1\). Either convention can be used when defined consistently; mixing the indicator from one convention with the probability from the other causes bias.
Before interpreting an estimate, check that censoring probabilities are compatible with the observation rules, conditional on the variables in \(W\). In particular, a case observed at time \(t\) uses the probability of remaining uncensored through \(t\), while a control under the strict \(Y>\tau\) rule uses the probability of remaining uncensored beyond \(\tau\). If there is an atom at the horizon, use the appropriate left- or right-limit probability rather than assuming they coincide.
Very small estimated censoring probabilities create very large weights. Inspect their distribution, the largest weights, and how much of the weighted total is supplied by a few observations. For weights \(w_k\) within a group, an effective sample size summary is \((\sum_k w_k)^2/\sum_k w_k^2\); a low value relative to the number observed signals that the estimate is relying heavily on a small number of records. Also check whether the censoring model is plausible for the predictors of censoring and whether there are regions of \(W\) with almost no chance of observing a case or control. If positivity fails, the target is not reliably identified from those data. Truncating extreme weights may stabilize a calculation, but changes the estimator and can introduce bias; it should not be presented as a cost-free repair.
The main simple alternative is to calculate an AUC using only observed cases and controls without weighting. That is appropriate only when this restriction does not distort the score distributions for the target groups—for example, under suitable noninformative observation conditions. If censoring is related to score or outcome after accounting for the variables used, the unweighted observed pairs can be systematically unrepresentative. A model-based approach that predicts or imputes missing case/control status is another possibility, but it introduces assumptions about the outcome model; it does not eliminate the need to justify how censoring is handled. The censoring-adjusted AUC is also distinct from summaries that compare risk scores across event times rather than at one specified horizon.
Finally, a high AUC does not establish that predicted risks are accurate in absolute terms: a model can rank patients well while systematically overpredicting or underpredicting their event probabilities. Nor does an AUC identify a causal effect. It summarizes ranking performance for the selected horizon, population, score, and case/control definition. It can be unstable when either cases or controls are scarce, when censoring is heavy, or when many scores are tied. Report the horizon, score direction, censoring model, and censoring assumptions alongside the estimate. Sampling uncertainty is a separate issue from the point estimate and should be assessed when drawing conclusions from validation data.
Check Your Understanding
Answer each question, then compare your reasoning with the brief solutions.
- At horizon \(\tau\), what defines a case and a control for this AUC?
- An observed case has \(G_{\mathrm{case}}(Y\mid W)=0.60\). What is its case weight, to three significant figures?
- A case score is 0.80, while a control score is also 0.80. How much does this pair contribute to the weighted numerator relative to the product of its case and control weights?
- Why must the censoring information \(W\) account for variables determining the risk score, not merely variables associated with event time?
- For the strict control rule \(I(Y>\tau)\), which censoring probability belongs in the denominator when censoring may occur exactly at \(\tau\)?
Solutions. (1) Cases have \(T\le\tau\), and controls have \(T>\tau\). (2) The weight is \(1/0.60=1.67\). (3) The tied pair contributes one-half of the product \(w^{\mathrm{case}}w^{\mathrm{control}}\). (4) Otherwise, censoring can remain associated with the score after conditioning, so weighted score comparisons need not recover the complete-data ranking target. (5) Use \(P(C>\tau\mid W)\), because \(Y>\tau\) for a control requires censoring strictly beyond \(\tau\); alternatively, change both the observation indicator and its probability consistently to include eligible censoring at \(\tau\).