Why Predictive Values Need Their Own Denominators
A binary classifier assigns each case to one of two predicted classes. For example, a hypothetical claims model might flag a claim for review or leave it unflagged. Once the true class is known for an evaluation set, a confusion matrix records how the predictions compare with the observed outcomes.
Precision and negative predictive value answer questions about the cases that received a particular prediction. Precision asks, among cases predicted positive, what fraction were actually positive? Negative predictive value (NPV) asks, among cases predicted negative, what fraction were actually negative? Their denominators therefore come from predicted-positive and predicted-negative groups, respectively.
Use a consistent confusion-matrix orientation: rows show the actual class and columns show the predicted class. The positive class must be identified in the problem; switching which outcome is called positive changes the interpretation of the metrics.
| Predicted positive | Predicted negative | |
|---|---|---|
| Actual positive | TP | FN |
| Actual negative | FP | TN |
The predicted-positive total is \(TP+FP\), while the predicted-negative total is \(TN+FN\). Thus the formulas are
These metrics differ from sensitivity and specificity, which condition on the actual class. Sensitivity asks what fraction of actual positives were predicted positive; specificity asks what fraction of actual negatives were predicted negative.
A reliable way to select a denominator is to read each question literally. “Among predicted positives” selects the predicted-positive column, \(TP+FP\); “among actual positives” selects the actual-positive row, \(TP+FN\). Precision and sensitivity share a numerator but answer different conditional questions.
Conditional Interpretation and a Short Derivation
Suppose each evaluated case has a known actual class and a model prediction. Regard a case as selected uniformly at random from the evaluation set. The empirical probability of an event is then its count divided by the total number of cases. Precision is the empirical conditional probability that a case is actually positive given that it was predicted positive. NPV is the empirical conditional probability that it is actually negative given that it was predicted negative.
The numerator counts cases satisfying both conditions, which are the true positives. The conditioning event, predicted positive, includes both true positives and false positives, so its count is \(TP+FP\). Applying the same reasoning to a predicted-negative case gives
Here, the cases satisfying both conditions are true negatives, and all predicted negatives consist of true negatives plus false negatives. This count-based derivation also shows when a metric is undefined: precision cannot be calculated if \(TP+FP=0\), and NPV cannot be calculated if \(TN+FN=0\). A zero denominator means there are no cases in the group about which the metric asks; it does not mean the metric equals zero.
Worked Example: Predictive Values for a Claim-Review Flag
A hypothetical insurer evaluates a flag that predicts whether a claim will require a particular type of review. Among 500 claims, 80 actually require review. The model flags 60 of those 80 claims and also flags 42 claims that do not require review. Calculate precision and NPV, then distinguish them from sensitivity and specificity.
Method. First complete the confusion matrix by finding each actual-class total and then the remaining cell count. This avoids treating an actual-class total as a predicted-class denominator.
There are 80 actual positives. Of these, 60 are predicted positive, so \(TP=60\), and the remainder are false negatives:
The number of actual negatives is \(500-80=420\). The problem states that 42 of these are flagged, so \(FP=42\). The remaining actual negatives are true negatives:
The predicted-positive total is \(TP+FP=60+42=102\). Therefore,
To four decimal places, precision is 0.5882, or 58.82%. Among claims flagged by this model, about 58.82% actually require review.
The predicted-negative total is \(TN+FN=378+20=398\). Therefore,
To four decimal places, NPV is 0.9497, or 94.97%. Among claims not flagged, about 94.97% actually do not require review.
For comparison, sensitivity is \(60/(60+20)=60/80=0.7500\), and specificity is \(378/(378+42)=378/420=0.9000\). These answer different questions: sensitivity is conditional on actually requiring review, while specificity is conditional on actually not requiring review. The precision and NPV calculations are checked against the column totals: \(102+398=500\).
Worked Comparisons
Worked Example: Correcting a Denominator Mix-Up
In a hypothetical validation set of 300 applications, 60 are actually positive for a specified outcome. A model correctly predicts 45 of those positives and incorrectly predicts positive for 30 actual negatives. Calculate precision and NPV. Also identify the error in using the actual-positive count as the precision denominator.
Method. Build all four cells from the stated actual totals. The false-negative count is \(60-45=15\). There are \(300-60=240\) actual negatives. Since 30 actual negatives were predicted positive, \(FP=30\), leaving
The predicted-positive total is \(TP+FP=45+30=75\), so
The predicted-negative total is \(TN+FN=210+15=225\), so
To four decimal places, precision is 0.6000 and NPV is 0.9333. The cell counts check: \(45+15=60\) actual positives, \(30+210=240\) actual negatives, and \(60+240=300\) total applications.
Common mistake and correction. The incorrect calculation \(45/60=0.7500\) uses all actual positives as the denominator. It is sensitivity, not precision: it asks what fraction of actual positives the model found. Precision instead conditions on predicted positive, so its denominator is \(45+30=75\). Likewise, NPV uses \(TN+FN=225\), not just the actual-negative total of 240, because it conditions on predicted negative.
Worked Example: Comparing Predictive Values Across Two Models
Two hypothetical models are evaluated on the same 300 cases, of which 100 are actually positive and 200 are actually negative. Model A has \(TP=72\) and \(FP=18\). Model B has \(TP=64\) and \(FP=8\). Calculate each model’s precision and NPV, and explain which model has the higher value for each metric.
Method. Since both models use the same actual cases, calculate each model’s false negatives from the 100 actual positives and true negatives from the 200 actual negatives.
For Model A,
Its predicted-positive total is \(72+18=90\), and its predicted-negative total is \(182+28=210\). Thus,
For Model B,
Its predicted-positive total is \(64+8=72\), and its predicted-negative total is \(192+36=228\). Therefore,
Model B has higher precision, because a larger fraction of its predicted positives are actually positive. Model A has higher NPV, because a larger fraction of its predicted negatives are actually negative. Neither comparison can be replaced by looking at the number of true positives alone: the relevant predicted-group denominators differ. As a check, each model’s four cells sum to 300.
Reading and Reporting the Metrics Carefully
Precision and NPV are useful when the practical question concerns what a model’s predictions mean for the cases receiving them. In a hypothetical screening or claim-triage setting, precision describes the reliability of a positive flag, while NPV describes the reliability of a negative result. These interpretations should be reported in context, rather than as bare percentages.
A value of 0.80 for precision does not mean the classifier correctly identifies 80% of all actual positives. It means 80% of the cases predicted positive are actually positive in the evaluated set. The first statement describes sensitivity; the second describes precision. Similarly, an NPV of 0.90 does not mean the classifier correctly identifies 90% of actual negatives; it means 90% of predicted negatives are actually negative.
Always state which class is positive before interpreting any of these quantities. For example, if “requires review” is positive, precision refers to the share of flagged claims that require review. If the positive label were reversed, the terms and the relevant counts would change.
Translate positive and negative into the actual outcome named in the problem.
For precision, select predicted positives; for NPV, select predicted negatives.
Precision uses TP, and NPV uses TN. Divide by the total count of the corresponding predicted group.
Describe the fraction in words, check that the denominator is nonzero, and verify the confusion-matrix totals.
A common exam trap is to select a denominator by matching the word “positive” or “negative” in the metric name, rather than by identifying the conditioning event. Precision does not condition on actual positives, and NPV does not condition on actual negatives. Sensitivity and specificity do condition on the actual class.
Check Your Understanding
For each original scenario, identify the requested predicted-group denominator before calculating. Report probabilities to four decimal places.
- A hypothetical inspection model is tested on 200 devices. Of 50 devices that actually need repair, it flags 40. Of the 150 that do not need repair, it incorrectly flags 30. Calculate precision and NPV. (Answer: \(TP=40\), \(FN=10\), \(FP=30\), \(TN=120\); precision \(=40/70=0.5714\); NPV \(=120/130=0.9231\).)
- In a hypothetical set of 200 accounts, there are 72 actual positives and 128 actual negatives. The model predicts positive for 60 accounts, of which 48 are actual positives. Calculate precision and NPV. (Answer: \(TP=48\), \(FP=12\), \(FN=24\), \(TN=116\); precision \(=48/60=0.8000\); NPV \(=116/140=0.8286\).)
- A model makes 100 positive predictions in a validation set. Of those, 70 are actual positives. The model predicts negative for the other 400 cases, among which 16 are actually positive. Calculate precision and NPV, and state which predicted group each denominator represents. (Answer: \(TP=70\), \(FP=30\), \(FN=16\), \(TN=384\); precision \(=70/100=0.7000\), using predicted positives; NPV \(=384/400=0.9600\), using predicted negatives.)
- Two models are evaluated on the same 400 cases, with 100 actual positives and 300 actual negatives. Model X has \(TP=90\) and \(FP=30\); Model Y has \(TP=80\) and \(FP=10\). Calculate precision and NPV for each, then identify which model has the higher value for each metric. (Answer: X has \(FN=10\), \(TN=270\), precision \(=90/120=0.7500\), and NPV \(=270/280=0.9643\); Y has \(FN=20\), \(TN=290\), precision \(=80/90=0.8889\), and NPV \(=290/310=0.9355\). Y has higher precision; X has higher NPV.)
- A hypothetical model is configured to predict every case as negative. In its evaluation set, 20 cases are actually positive and 80 are actually negative. Find NPV and determine whether precision is defined. (Answer: \(TP=0\), \(FN=20\), \(TN=80\), \(FP=0\); NPV \(=80/(80+20)=0.8000\); precision is undefined because its denominator \(TP+FP=0\).)