Why Sensitivity and Specificity Matter
A binary classifier assigns each case to one of two predicted classes, such as positive or negative. To evaluate it, compare its predictions with the cases’ actual classes. Two useful questions are: among the actually positive cases, what fraction did the model identify? And among the actually negative cases, what fraction did it correctly leave negative? Sensitivity answers the first question; specificity answers the second.
These measures describe different aspects of classification performance. Sensitivity reflects how often an actual positive is detected, while specificity reflects how often an actual negative is correctly rejected. Neither rate is meaningful until the positive class and the denominator are clear. This tutorial is aligned with the SOA Exam PA (Predictive Analytics) syllabus.
Read the Confusion Matrix by Actual Class
For this tutorial, rows indicate the actual class and columns indicate the predicted class. The positive class is the outcome designated as positive for the analysis; it must be stated in context. For example, if positive means “a claim requires further investigation,” then a positive prediction means the classifier sends a claim for investigation.
| Predicted positive | Predicted negative | |
|---|---|---|
| Actual positive | True positive (TP) | False negative (FN) |
| Actual negative | False positive (FP) | True negative (TN) |
A true positive (TP) is an actual positive case predicted positive. A false negative (FN) is an actual positive case predicted negative. A false positive (FP) is an actual negative case predicted positive. A true negative (TN) is an actual negative case predicted negative. “False” means the predicted class disagrees with the actual class.
The denominator for sensitivity is all actual positives, whether they were predicted positive or negative. The denominator for specificity is all actual negatives, whether they were predicted negative or positive. Equivalently, sensitivity is the observed conditional proportion predicted positive given that the actual class is positive; specificity is the observed conditional proportion predicted negative given that the actual class is negative.
The hats emphasize that a rate calculated from a validation sample is an empirical estimate of performance on the relevant actual class. It is not automatically the exact rate for every future population. Also, if a denominator is zero—for example, a validation sample contains no actual negatives—specificity cannot be calculated from that sample.
Why These Denominators Are the Right Ones
The conditional interpretation gives a short justification. Restrict attention to the actual positive cases. Every one of these cases must have been assigned to exactly one of two predicted classes: positive or negative. The resulting groups are TP and FN, so the total number of actual positives is \(\mathrm{TP}+\mathrm{FN}\). The fraction correctly identified is therefore \(\mathrm{TP}/(\mathrm{TP}+\mathrm{FN})\).
For the actual negative cases, the two possible predictions similarly divide the cases into FP and TN. There are \(\mathrm{FP}+\mathrm{TN}\) actual negatives, and the correctly classified share is the number predicted negative divided by that total:
This reasoning explains why the denominators follow the actual class. The measures ask how predictions behave within each actual class; they do not ask what fraction of predicted-positive or predicted-negative cases are correct.
Worked Calculations
Worked Example: Detecting Claims Requiring Review
A hypothetical insurer validates a classifier that flags claims for additional review. In a set of 300 claims, 90 actually require review and 210 do not. The classifier flags 72 of the actual review cases and also flags 30 claims that do not require review. Calculate sensitivity and specificity, and state what each rate means.
Method. Define “requires review” as positive. The 72 flagged actual positives are TP, so \(\mathrm{TP}=72\). The other actual positives were missed: \(\mathrm{FN}=90-72=18\). Of the 210 actual negatives, 30 were incorrectly flagged, so \(\mathrm{FP}=30\), and the remaining \(\mathrm{TN}=210-30=180\).
The actual-positive total checks as \(\mathrm{TP}+\mathrm{FN}=72+18=90\). The actual-negative total checks as \(\mathrm{TN}+\mathrm{FP}=180+30=210\). Thus all 300 claims are accounted for.
Sensitivity is \(72/90=0.8000\), or 80.00%. In this validation set, the classifier flagged 80.00% of claims that actually required review. Specificity is \(180/210=0.8571\), or 85.71% when rounded to two decimal places. It correctly left 85.71% of claims that did not require review unflagged.
Check and trap avoided. The missed actual positives are the 18 false negatives; the wrongly flagged actual negatives are the 30 false positives. Using 72 divided by the total number flagged would not calculate sensitivity, because that denominator includes predicted positives rather than all actual positives.
Worked Example: Classifying Late Policy Payments
A hypothetical insurer defines a positive case as a policy payment that will be more than 30 days late. In a validation sample, 40 payments are actually more than 30 days late and 160 are not. The model predicts positive for 28 late payments and predicts negative for 128 payments that are not late. Find both rates and identify the error counts.
Method. The 28 correctly flagged late payments give \(\mathrm{TP}=28\). The remaining actual late payments are false negatives: \(\mathrm{FN}=40-28=12\). Among the 160 payments that are not late, 128 were correctly predicted negative, so \(\mathrm{TN}=128\); the other \(160-128=32\) were false positives.
Check the row totals: \(28+12=40\) actual late payments and \(32+128=160\) actual non-late payments. The full sample total is \(40+160=200\), which also equals \(28+12+32+128=200\).
Sensitivity is \(28/(28+12)=28/40=0.7000\), or 70.00%. Specificity is \(128/(128+32)=128/160=0.8000\), or 80.00%. The classifier misses 12 actual late payments, while 32 payments that are not late are incorrectly flagged.
Check and trap avoided. The 128 true negatives alone are not the denominator for specificity. The denominator must also include the 32 false positives, because both groups are actual non-late payments.
Worked Example: Comparing Two Review Rules
A hypothetical claims team tests two classification rules on the same 100 actual positive cases and 200 actual negative cases. Rule A produces 82 true positives and 30 false positives. Rule B produces 90 true positives and 50 false positives. For each rule, calculate sensitivity and specificity, then describe the difference in the errors measured by these rates.
Rule A. There are 100 actual positives, so \(\mathrm{FN}=100-82=18\). There are 200 actual negatives, so \(\mathrm{TN}=200-30=170\). The row totals check: \(82+18=100\) and \(30+170=200\). Sensitivity is \(82/100=0.8200\), or 82.00%. Specificity is \(170/200=0.8500\), or 85.00%.
Rule B. The false-negative count is \(100-90=10\), and the true-negative count is \(200-50=150\). The totals check: \(90+10=100\) and \(50+150=200\). Sensitivity is \(90/100=0.9000\), or 90.00%. Specificity is \(150/200=0.7500\), or 75.00%.
Rule B has higher sensitivity: it identifies a larger share of the actual positives, with 10 false negatives rather than 18. It has lower specificity: it correctly rejects a smaller share of the actual negatives, with 50 false positives rather than 30. These rates make the two types of classification error visible. They do not, on their own, decide which rule is preferable; that depends on the decision context and the consequences of each type of error.
Check and trap avoided. Compare each rate using its own actual-class denominator: 100 for sensitivity and 200 for specificity. Comparing the raw numbers 90 and 50, without their denominators and roles, would not give the two rates.
Common Denominator Errors
The most frequent error is to divide by a total from the wrong direction of the matrix. Sensitivity is not the share of predicted positives that are correct; it is the share of actual positives that were predicted positive. For example, in the first worked example, a tempting but incorrect calculation is:
That calculation uses the predicted-positive total. It therefore does not answer the sensitivity question. The correct calculation is:
Specificity has the parallel denominator trap. It is not the share of predicted negatives that are correct. Its denominator is all actual negatives, so it includes both TN and FP. A reliable check is to label the denominator in words before substituting counts: “all actual positives” for sensitivity, “all actual negatives” for specificity.
- False negatives lower sensitivity. They are actual positives the classifier failed to identify.
- False positives lower specificity. They are actual negatives the classifier incorrectly predicted positive.
- Switching the positive class changes the interpretation. Always state which outcome counts as positive before interpreting either rate.
- Do not infer a population rate from a zero denominator. If no actual positives appear in the evaluation sample, sample sensitivity is undefined; the same applies to specificity if no actual negatives appear.
Check Your Understanding
For each problem, define the positive class from the context, use actual-class totals as denominators, and check that the counts reconcile.
- A hypothetical screening model is evaluated on 60 actual positive cases and 140 actual negative cases. It identifies 45 actual positives and correctly predicts 130 actual negatives as negative. Calculate sensitivity and specificity. (Answer: Sensitivity \(=45/60=0.7500\); specificity \(=130/140=0.9286\).)
- A hypothetical payment-monitoring model has 28 true positives, 12 false negatives, 24 false positives, and 96 true negatives. Calculate both rates and name the error count that reduces each one. (Answer: Sensitivity \(=28/40=0.7000\), reduced by 12 false negatives; specificity \(=96/120=0.8000\), reduced by 24 false positives.)
- A hypothetical classifier is tested on 100 actual positives and 300 actual negatives. Its sensitivity is 0.8200 and its specificity is 0.9000. Find TP, FN, TN, and FP. (Answer: TP \(=82\), FN \(=18\), TN \(=270\), FP \(=30\).)
- On the same validation set, Rule A has 72 true positives out of 90 actual positives and 170 true negatives out of 200 actual negatives. Rule B has 81 true positives and 160 true negatives, with the same actual-class totals. Calculate both rates for each rule and identify which rule has greater sensitivity and which has greater specificity. (Answer: Rule A sensitivity \(=0.8000\), specificity \(=0.8500\); Rule B sensitivity \(=0.9000\), specificity \(=0.8000\). Rule B has greater sensitivity; Rule A has greater specificity.)
- A hypothetical classifier is evaluated on 240 actual positive and 760 actual negative cases. Its sensitivity is 0.8750 and its specificity is 0.9500. Reconstruct all four confusion-matrix counts and verify the row totals. (Answer: TP \(=210\), FN \(=30\), TN \(=722\), FP \(=38\); actual-positive total \(=210+30=240\), and actual-negative total \(=722+38=760\).)