Two Views of Classification Performance
A classifier that assigns scores to cases can be evaluated at many thresholds. The receiver operating characteristic (ROC) curve and the precision-recall (PR) curve summarize those threshold choices from different perspectives. This distinction matters when the positive class is rare: a false-positive rate can be numerically small while false positives still greatly outnumber true positives.
Both curves use the same underlying predictions. They do not change the classifier or the cases; they change which performance rates are placed on the axes. The ROC view compares detection of positives with the rate of false alarms among negatives. The PR view compares the fraction of positives retrieved with the reliability of positive predictions.
Assume throughout that a larger score indicates stronger evidence for the positive class and that a case is predicted positive when its score is at least the threshold. Moving the threshold downward can only add predicted-positive cases. As a result, both the number of true positives and the number of false positives can increase. Earlier material on confusion matrices and thresholds establishes how these counts produce the rates used here.
How Prevalence Connects the Two Curves
ROC coordinates are calculated separately within the actual positive and actual negative groups. The PR curve uses precision, whose denominator combines true and false positives. Consequently, the same TPR and FPR can produce different precision values when the positive prevalence differs.
This expression follows by considering a population of size \(N\). There are \(\pi N\) actual positives and \((1-\pi)N\) actual negatives. At a fixed threshold, the expected true-positive count is \(\pi N\,\mathrm{TPR}\), and the expected false-positive count is \((1-\pi)N\,\mathrm{FPR}\). Precision is the first count divided by their sum; cancelling \(N\) gives the formula. It applies when the denominator is positive, so there are predicted-positive cases.
For an uninformative ranking, positive and negative cases are intermixed rather than systematically ranked above one another. Its expected ROC curve is the diagonal \(\mathrm{TPR}=\mathrm{FPR}\), and its expected precision at a selection level is the positive prevalence \(\pi\). Thus, the ROC reference line does not depend on prevalence, while the PR reference level does. The PR reference is an expectation: a particular finite random ranking can fluctuate above or below it.
Worked Example: Constructing Both Curves From the Same Imbalanced Scores
Worked Example: Constructing Both Curves From the Same Imbalanced Scores
A hypothetical screening model scores 1,000 cases: 20 are positive and 980 are negative. The score bands contain the following counts. Calculate ROC and PR coordinates after including each band in turn, and identify the respective baseline levels.
| Score band added | Positive cases | Negative cases |
|---|---|---|
| At least 0.90 | 8 | 10 |
| 0.70 to less than 0.90 | 6 | 40 |
| 0.40 to less than 0.70 | 4 | 130 |
| 0.20 to less than 0.40 | 2 | 310 |
| Less than 0.20 | 0 | 490 |
Method. For each threshold, accumulate the positive and negative counts from the highest score band downward. Divide cumulative positives by 20 for TPR, cumulative negatives by 980 for FPR, and cumulative positives by the total selected cases for precision. The prevalence is \(20/1{,}000=0.0200\).
| Threshold rule | TP | FP | TPR / recall | FPR | Precision |
|---|---|---|---|---|---|
| At least 0.90 | 8 | 10 | 8/20 = 0.4000 | 10/980 = 0.0102 | 8/18 = 0.4444 |
| At least 0.70 | 14 | 50 | 14/20 = 0.7000 | 50/980 = 0.0510 | 14/64 = 0.2188 |
| At least 0.40 | 18 | 180 | 18/20 = 0.9000 | 180/980 = 0.1837 | 18/198 = 0.0909 |
| At least 0.20 | 20 | 490 | 20/20 = 1.0000 | 490/980 = 0.5000 | 20/510 = 0.0392 |
| All cases selected | 20 | 980 | 20/20 = 1.0000 | 980/980 = 1.0000 | 20/1,000 = 0.0200 |
The ROC coordinates are \((0.0102,0.4000)\), \((0.0510,0.7000)\), \((0.1837,0.9000)\), \((0.5000,1.0000)\), and \((1.0000,1.0000)\). The corresponding PR points are \((0.4000,0.4444)\), \((0.7000,0.2188)\), \((0.9000,0.0909)\), \((1.0000,0.0392)\), and \((1.0000,0.0200)\), where each pair is (recall, precision). The ROC random-ranking reference is the diagonal; the PR random-ranking reference is precision \(0.0200\).
The high-threshold point retrieves 40% of positives, and only about 1.02% of negatives are flagged. Yet fewer than half of the flagged cases are positive. This is not a contradiction: the negative group is 49 times larger than the positive group. The common mistake is to read a small FPR as proof that nearly all flagged cases are true positives. The correction is to calculate precision using both group sizes, as the table does.
Why the PR Baseline Moves With Prevalence
The ROC curve’s rates condition on the actual class: its denominators are the number of positives and the number of negatives. If the same conditional rates are evaluated in populations with different prevalences, the ROC coordinate remains the same. Precision instead conditions on a positive prediction. It depends on how many actual positives are available to be found relative to the negatives that can be falsely flagged.
Worked Example: Same ROC Point, Different Precision
At one threshold, a classifier has TPR \(0.8000\) and FPR \(0.0400\). Compare its PR coordinate when prevalence is \(0.0200\) with its coordinate when prevalence is \(0.2000\). The conditional rates are held fixed for this comparison.
Method. The ROC coordinate is \((0.0400,0.8000)\) in either population. Substitute each prevalence into the precision formula.
At prevalence \(0.0200\), the positive contribution is \(0.0200(0.8000)=0.0160\), and the false-positive contribution per case in the population is \(0.9800(0.0400)=0.0392\). Therefore,
At prevalence \(0.2000\), the corresponding contributions are \(0.2000(0.8000)=0.1600\) and \(0.8000(0.0400)=0.0320\). Thus,
The ROC point is unchanged, but precision rises from \(0.2899\) to \(0.8333\) because the positive class is more prevalent. The PR random-ranking baselines also differ: \(0.0200\) and \(0.2000\), respectively. The trap is to treat ROC and PR coordinates as interchangeable measures; recall is the same as TPR, but precision is not determined by TPR and FPR alone without prevalence.
What Each View Emphasizes
The ROC curve is useful for seeing how sensitivity changes as the false-positive rate among negatives changes. It separates the classifier’s ranking behavior across the two actual classes from their relative frequencies. This makes the ROC view informative when the central question concerns the trade-off between finding positives and falsely flagging negatives.
The PR curve focuses directly on the positive class. Recall shows how many positives are found; precision shows how trustworthy the predicted-positive group is. When positives are rare, this can make the practical burden of false positives more apparent than a low-looking FPR on its own. A PR curve should be interpreted relative to its prevalence baseline, not against a universal horizontal level.
Neither curve answers every question. A curve summarizes performance over thresholds, but selecting a particular threshold requires context about the intended use and the consequences of errors. Those considerations are distinct from simply reading the curve coordinates. For a fair visual comparison, make sure the positive class is defined consistently and report the prevalence relevant to the data being evaluated.
Worked Example: Interpreting a Small False-Positive Rate
A hypothetical model is evaluated on 20,000 cases, of which 100 are positive and 19,900 are negative. At a specified threshold it has TPR \(0.9500\) and FPR \(0.0500\). Calculate the confusion-matrix counts, ROC point, PR point, and PR baseline. State what a reader should notice.
Method. Apply each rate to the size of its actual class. There are 100 actual positives, so \(TP=0.9500(100)=95\) and \(FN=100-95=5\). There are 19,900 actual negatives, so \(FP=0.0500(19{,}900)=995\) and \(TN=19{,}900-995=18{,}905\).
The ROC coordinate is \((0.0500,0.9500)\). Recall is \(95/(95+5)=95/100=0.9500\). Precision is
The prevalence and thus the PR random-ranking baseline are \(100/20{,}000=0.0050\). The PR point is \((0.9500,0.0872)\). Although the ROC point has high TPR and a 5% FPR, there are 995 false positives for 95 true positives, so fewer than one in ten positive predictions is correct. The common mistake is to multiply the FPR by the positive count; its denominator is the actual negative count. Using 19,900 negatives gives the correct false-positive count and the appropriate precision.
Practical Reading Checks
When moving between ROC and PR displays, keep the denominators visible. ROC’s horizontal axis is \(FP/(FP+TN)\), not the fraction of all cases that are false positives. PR’s vertical axis is \(TP/(TP+FP)\), not sensitivity. The shared recall coordinate means that a given threshold has the same TPR and recall, while the other coordinate answers a different question.
- Confirm which class is designated positive and state its prevalence.
- Read an ROC point as FPR first and TPR second; read a PR point as recall first and precision second.
- Use the prevalence as the PR random-ranking reference, rather than assuming a fixed baseline such as 0.5.
- Translate rates back to counts when a small FPR may conceal many false positives.
- Do not infer precision from an ROC point without also knowing prevalence.
The key comparison is not that one curve is universally better. ROC highlights class-conditional detection and false-alarm rates; PR highlights the quality and coverage of positive predictions. For imbalanced classification, reading both views alongside prevalence helps explain how a model can look strong in ROC coordinates while still producing a positive-prediction group with modest precision. This interpretation is aligned with the SOA Exam PA (Predictive Analytics) syllabus.
Check Your Understanding
For each question, calculate the requested values and check the stated final answer.
- A classifier is evaluated where prevalence is \(0.0400\). At one threshold, TPR is \(0.7500\) and FPR is \(0.1000\). Give its ROC point and precision. (Answer: ROC \((0.1000,0.7500)\); precision \(0.2381\))
- A data set has 12 positives and 588 negatives. A threshold produces 9 true positives and 30 false positives. Calculate recall, precision, and FPR. (Answer: recall \(0.7500\); precision \(0.2308\); FPR \(0.0510\))
- Prevalence is \(0.0100\), TPR is \(0.9000\), and FPR is \(0.0200\). Calculate precision and state the PR random-ranking baseline. (Answer: precision \(0.3125\); baseline \(0.0100\))
- A random-ranking reference is shown for a population with 3 positives among 250 cases. State the expected PR baseline and the ROC diagonal coordinate at FPR \(0.6000\). (Answer: PR baseline \(0.0120\); ROC point \((0.6000,0.6000)\))
- At prevalence \(0.0800\), a threshold has TPR \(0.5000\) and FPR \(0.0400\). Find precision and the ROC point. (Answer: precision \(0.5208\); ROC point \((0.0400,0.5000)\))