ROC Dominance and Model Comparison

Why Compare ROC Curves?

A classifier can be used at multiple score thresholds, and each threshold gives an operating point: a false positive rate paired with a true positive rate. Comparing classifiers at just one operating point may not describe how they perform over the full range of thresholds. ROC dominance addresses that broader question: at a given false positive rate, does one classifier achieve at least as high a true positive rate as another?

For a ROC curve, the horizontal coordinate is the false positive rate and the vertical coordinate is the true positive rate. In the standard binary-classification setting, these are \(1-\text{specificity}\) and sensitivity, respectively. The curve runs from \((0,0)\) to \((1,1)\) when the endpoint operating points are included. For comparisons and trapezoidal AUC calculations below, treat the plotted curve as linearly interpolated between consecutive listed points.

Definition: Let \(T_A(r)\) be classifier A's true positive rate at false positive rate \(r\), for \(0\le r\le 1\). Classifier A weakly dominates classifier B if \(T_A(r)\ge T_B(r)\) at every false positive rate \(r\). It strictly dominates B if it weakly dominates B and is strictly higher at at least one false positive rate. If each curve is higher over some part of the range, the curves cross and neither dominates the other.

The comparison is made at the same false positive rate, not necessarily at the same numerical score threshold. Two classifiers can have different score scales, so matching thresholds would not generally match operating points. ROC dominance asks whether one curve is at least as high as the other across the horizontal axis.

Comparing Curves With Different Operating Points

ROC curves need not list points at identical false positive rates. When the curve is treated as linear between listed points, find its height at a requested rate by linear interpolation. If a segment joins \((r_1,t_1)\) and \((r_2,t_2)\), then for \(r_1\le r\le r_2\):

T(r)=t_1+\frac{r-r_1}{r_2-r_1}(t_2-t_1).

For example, moving a fraction \((r-r_1)/(r_2-r_1)\) of the way across the segment moves the same fraction of the vertical distance from \(t_1\) to \(t_2\). To establish dominance for piecewise-linear curves, compare their heights over the full range—not merely at one convenient threshold. It is sufficient to compare at the union of both curves' breakpoints: between consecutive breakpoints, their height difference is linear, so if it is nonnegative at both ends, it remains nonnegative throughout that interval.

Worked Example: One Curve Dominates at Every False Positive Rate

A hypothetical screening team compares two classifiers, E and F. Their ROC coordinates, listed in increasing false positive rate, are E: \((0,0)\), \((0.50,0.60)\), \((1,1)\); and F: \((0,0)\), \((0.50,0.70)\), \((1,1)\). Determine whether either curve dominates and calculate both AUCs by the trapezoidal rule.

Method. Compare the curves at their common breakpoints, then sum the area of every trapezoid. On the first segment, both curves are linear from the origin. At any \(r\) between 0 and 0.50, F's height is \(0.70/0.60\) times E's height, so F is higher. On the second segment, both end at \((1,1)\), but F starts at height 0.70 rather than 0.60. The linear difference between their heights is positive at \(r=0.50\) and zero at \(r=1\), so it is nonnegative throughout. Therefore F weakly dominates E and is strictly higher for interior false positive rates.

For E, the first trapezoid has width \(0.50\) and endpoint heights 0 and 0.60:

A_{E,1}=0.50\frac{0+0.60}{2}=0.150.

The second trapezoid has width \(1-0.50=0.50\) and endpoint heights 0.60 and 1:

A_{E,2}=0.50\frac{0.60+1}{2}=0.400, \qquad \operatorname{AUC}_E=0.150+0.400=0.550.

For F, the two trapezoid areas are:

A_{F,1}=0.50\frac{0+0.70}{2}=0.175, \qquad A_{F,2}=0.50\frac{0.70+1}{2}=0.425,

so \(\operatorname{AUC}_F=0.175+0.425=0.600\). Check: both areas are between 0 and 1, and F's AUC exceeds E's, consistent with F being higher throughout. The common mistake is to report only the second trapezoid as the AUC; the AUC is the sum of the areas of all segments.

Why Dominance Implies an AUC Ordering

AUC is the area under the ROC curve, so it summarizes vertical height across the full range of false positive rates. If \(T_A(r)\ge T_B(r)\) for every \(r\), then the area under A cannot be less than the area under B. This follows directly from comparing the curves' heights at each horizontal position.

\operatorname{AUC}_A-\operatorname{AUC}_B =\int_0^1 \bigl(T_A(r)-T_B(r)\bigr)\,dr \ge 0.

For piecewise-linear curves, the same justification can be written as a finite sum. Split the range at every breakpoint of either curve. On each resulting interval, the difference \(T_A(r)-T_B(r)\) is linear. If A is never below B, each interval contributes a nonnegative area difference, so their sum is nonnegative. If A is strictly higher over an interval of positive width, the area difference is strictly positive.

The converse does not hold: a larger AUC does not imply dominance. A classifier can be higher over much of the range yet lower over another part. AUC totals those gains and losses, while dominance requires an advantage at every false positive rate.

Worked Example: Crossing Curves With Equal AUC

Two hypothetical claim-screening classifiers have these ROC coordinates: G: \((0,0)\), \((0.25,0.50)\), \((0.75,0.80)\), \((1,1)\); H: \((0,0)\), \((0.25,0.60)\), \((0.75,0.70)\), \((1,1)\). Determine whether either dominates, calculate each AUC, and explain why one AUC does not resolve the pointwise comparison.

Method. Compare heights at the shared breakpoints. At \(r=0.25\), H is higher by \(0.60-0.50=0.10\). At \(r=0.75\), G is higher by \(0.80-0.70=0.10\). Thus the curves cross, and neither dominates. In fact, between 0.25 and 0.75 their height difference changes linearly from \(-0.10\) (G minus H) to \(+0.10\), so it equals zero halfway across, at \(r=0.50\).

For G, compute each trapezoid:

\begin{aligned} A_{G,1}&=0.25\frac{0+0.50}{2}=0.0625,\\ A_{G,2}&=(0.75-0.25)\frac{0.50+0.80}{2} =0.50(0.65)=0.3250,\\ A_{G,3}&=(1-0.75)\frac{0.80+1}{2} =0.25(0.90)=0.2250,\\ \operatorname{AUC}_G&=0.0625+0.3250+0.2250=0.6125. \end{aligned}

For H:

\begin{aligned} A_{H,1}&=0.25\frac{0+0.60}{2}=0.0750,\\ A_{H,2}&=0.50\frac{0.60+0.70}{2} =0.50(0.65)=0.3250,\\ A_{H,3}&=0.25\frac{0.70+1}{2} =0.25(0.85)=0.2125,\\ \operatorname{AUC}_H&=0.0750+0.3250+0.2125=0.6125. \end{aligned}

The AUCs are equal even though H is higher near \(r=0.25\) and G is higher near \(r=0.75\). Check: G's first and last areas sum to \(0.2875\), and H's sum to \(0.2875\); their middle areas are both \(0.3250\). Equal AUC therefore does not mean identical curves or equivalent performance at every operating point. The common mistake is to infer dominance from an AUC comparison alone.

Dominance When Breakpoints Differ

When two curves have different listed operating points, compare both at a common set of false positive rates. Include every breakpoint from either curve, interpolate the other curve where needed, and check the height difference at each such location. As noted above, for piecewise-linear curves this checks the entire intervals between them as well.

Worked Example: Comparing Curves With Unequal Breakpoints

A hypothetical fraud-review team considers classifier A with coordinates \((0,0)\), \((0.40,0.60)\), \((1,1)\) and classifier B with coordinates \((0,0)\), \((0.20,0.20)\), \((0.70,0.65)\), \((1,1)\). Determine whether A dominates B, and calculate both AUCs.

Method. Use the combined breakpoint set \(0, 0.20, 0.40, 0.70, 1\). At \(r=0.20\), A is on its first segment, so \(T_A(0.20)=(0.20/0.40)(0.60)=0.30\); B is at 0.20. At \(r=0.40\), A is at 0.60. B is on its segment from \((0.20,0.20)\) to \((0.70,0.65)\), giving:

T_B(0.40)=0.20+\frac{0.40-0.20}{0.70-0.20}(0.65-0.20) =0.20+\frac{0.20}{0.50}(0.45)=0.38.

At \(r=0.70\), A is on its second segment: \(T_A(0.70)=0.60+(0.30/0.60)(0.40)=0.80\), while B is at 0.65. At \(r=0\) and \(r=1\), the curves coincide. A is at least as high at every combined breakpoint; since the difference is linear between consecutive breakpoints, A is at least as high throughout. Thus A dominates B.

A's two trapezoids have areas:

A_{A,1}=0.40\frac{0+0.60}{2}=0.120, \qquad A_{A,2}=0.60\frac{0.60+1}{2}=0.480, \qquad \operatorname{AUC}_A=0.600.

For B:

\begin{aligned} A_{B,1}&=0.20\frac{0+0.20}{2}=0.0200,\\ A_{B,2}&=0.50\frac{0.20+0.65}{2}=0.2125,\\ A_{B,3}&=0.30\frac{0.65+1}{2}=0.2475,\\ \operatorname{AUC}_B&=0.0200+0.2125+0.2475=0.4800. \end{aligned}

Check: A's total exceeds B's, as dominance requires. The common mistake is to compare only the classifiers' originally listed coordinates; A's height at 0.20, for instance, must be interpolated before comparing it with B at that false positive rate.

Interpreting a Crossing

A crossing curve means the preferred classifier depends on the false positive rate being considered. Before the operating point is chosen, the ROC comparison alone does not establish a universally superior classifier. The curves still provide useful information: they identify the ranges where one classifier has the higher true positive rate at equal false positive rate. If one AUC is larger, that gives an aggregate area comparison, but it does not remove the local trade-off indicated by a crossing.

The comparison is also limited to the ROC information shown. Dominance does not by itself select a threshold or account for consequences not represented by the two rates. Those questions require an explicit decision objective. For an exam response, state the pointwise relationship first, then report AUC separately if it is requested.

Key takeaway: ROC dominance is a point-by-point claim: one curve must be at least as high at every false positive rate. A larger AUC is an overall area result, not proof of dominance. When curves cross, neither has a universal ROC advantage, even if their AUCs differ.

Check Your Understanding

For each problem, use linear interpolation between listed ROC coordinates and include every trapezoid when calculating AUC.

Classifier M has coordinates \((0,0)\), \((0.40,0.50)\), \((1,1)\); classifier N has \((0,0)\), \((0.40,0.40)\), \((1,1)\). Which dominates, and what are the AUCs? (Answer: M dominates N; \(\operatorname{AUC}_M=0.550\) and \(\operatorname{AUC}_N=0.500\).)
Classifier P has coordinates \((0,0)\), \((0.30,0.40)\), \((0.80,0.75)\), \((1,1)\); Q has \((0,0)\), \((0.30,0.50)\), \((0.80,0.70)\), \((1,1)\). Do the curves cross, and which AUC is larger? (Answer: They cross, so neither dominates; \(\operatorname{AUC}_P=0.5225\) and \(\operatorname{AUC}_Q=0.5450\), so Q has the larger AUC.)
Classifier C has coordinates \((0,0)\), \((0.60,0.72)\), \((1,1)\); D has \((0,0)\), \((0.30,0.30)\), \((0.80,0.65)\), \((1,1)\). At false positive rate 0.45, calculate each true positive rate and state which is higher there. (Answer: \(T_C(0.45)=0.540\), \(T_D(0.45)=0.405\); C is higher at that rate.)
Classifier R has coordinates \((0,0)\), \((0.25,0.55)\), \((0.75,0.85)\), \((1,1)\); S has \((0,0)\), \((0.50,0.60)\), \((1,1)\). Does R dominate S, and what are their AUCs? (Answer: R dominates S; \(\operatorname{AUC}_R=0.650\) and \(\operatorname{AUC}_S=0.550\).)
Classifier K has coordinates \((0,0)\), \((0.40,0.70)\), \((1,1)\); L has \((0,0)\), \((0.40,0.50)\), \((0.70,0.90)\), \((1,1)\). Find their AUCs, determine whether they cross, and locate the crossing between 0.40 and 0.70. (Answer: They cross at false positive rate \(0.64\), so neither dominates; \(\operatorname{AUC}_K=0.650\) and \(\operatorname{AUC}_L=0.595\), so K has the larger AUC.)

What You'll Learn