Why Prevalence Changes What a Test Result Means
A positive classification does not have the same meaning in every population. If an outcome is common, a positive result is more likely to come from an individual who truly has the outcome. If the outcome is rare, even a modest false-positive rate can produce many false positives relative to the number of true positives. This matters whenever a model or screening rule is applied to a population whose outcome prevalence differs from the one used to assess it.
The key distinction is between a test’s performance conditional on the actual outcome and the probability of the actual outcome conditional on the test result. Sensitivity and specificity are of the first kind; precision and negative predictive value are of the second. Bayes’ rule connects them.
These definitions use a fixed convention: \(D\) is the positive outcome, and \(T+\) is the positive classification. A confusion matrix makes the conditional directions visible.
| Actual positive \(D\) | Actual negative \(D^c\) | |
|---|---|---|
| Classified positive \(T+\) | True positives (TP) | False positives (FP) |
| Classified negative \(T-\) | False negatives (FN) | True negatives (TN) |
Sensitivity is \(\mathrm{TP}/(\mathrm{TP}+\mathrm{FN})\), while precision is \(\mathrm{TP}/(\mathrm{TP}+\mathrm{FP})\). Specificity is \(\mathrm{TN}/(\mathrm{TN}+\mathrm{FP})\), while negative predictive value is \(\mathrm{TN}/(\mathrm{TN}+\mathrm{FN})\). The denominators differ because the first pair conditions on the actual outcome and the second pair conditions on the classification.
Bayes’ Rule for Predictive Values
Write sensitivity as \(Se=P(T+\mid D)\) and specificity as \(Sp=P(T-\mid D^c)\). The false-positive rate is \(1-Sp=P(T+\mid D^c)\), and the false-negative rate is \(1-Se=P(T-\mid D)\). The law of total probability gives the overall chance of a positive classification as \(Se\pi+(1-Sp)(1-\pi)\). Applying Bayes’ rule gives precision:
For a negative classification, the two ways to obtain \(T-\) are a false negative among actual positives and a true negative among actual negatives. Thus \(P(T-)=(1-Se)\pi+Sp(1-\pi)\), which gives negative predictive value:
The formulas show the prevalence effect directly. If \(Se>0\) and \(Sp<1\), increasing \(\pi\) increases precision: the numerator’s true-positive contribution grows relative to the false-positive contribution. If \(Sp>0\) and \(Se<1\), increasing \(\pi\) decreases negative predictive value: a negative classification is more likely to be a false negative when actual positives are more common. These directions concern fixed sensitivity and specificity; they do not say that either predictive value must have the same numerical value across populations.
Worked Examples Using Cohort Counts
A useful check on Bayes’ rule is to imagine a cohort of a convenient size, divide it into actual positive and negative outcomes using prevalence, and then apply sensitivity and specificity within those groups. The resulting counts should produce the same predictive values as the probability formulas.
Worked Example: A Rare Outcome in a Screening Cohort
A hypothetical screening rule is assessed in a population of 10,000 people. The outcome prevalence is 1%, sensitivity is 90%, and specificity is 95%. Find precision and negative predictive value. State why precision is not 90%.
Method. Use prevalence to form the actual-outcome groups, then apply sensitivity to actual positives and specificity to actual negatives. This avoids treating sensitivity as if it were the probability of the outcome after a positive classification.
There are \(10{,}000(0.01)=100\) actual positives and \(10{,}000-100=9{,}900\) actual negatives. Among the 100 actual positives, sensitivity of 90% gives \(100(0.90)=90\) true positives and \(100-90=10\) false negatives.
Among the 9,900 actual negatives, specificity of 95% gives \(9{,}900(0.95)=9{,}405\) true negatives. The false-positive rate is \(1-0.95=0.05\), so there are \(9{,}900(0.05)=495\) false positives.
There are \(90+495=585\) positive classifications. Therefore precision is \(90/585=0.153846\ldots\). There are \(9{,}405+10=9{,}415\) negative classifications, so negative predictive value is \(9{,}405/9{,}415=0.998938\ldots\). Rounded to four decimal places, precision is 0.1538 and negative predictive value is 0.9989.
The result is checked by Bayes’ rule: precision is \(0.90(0.01)/[0.90(0.01)+0.05(0.99)]=0.009/0.0585=0.153846\ldots\). The high specificity still leaves 495 false positives because it is applied to 9,900 people without the outcome. The common mistake is to report sensitivity, 0.90, as precision; sensitivity answers \(P(T+\mid D)\), not \(P(D\mid T+)\).
Worked Example: Comparing Two Prevalence Levels
A hypothetical model has sensitivity 90% and specificity 95% at a specified classification rule. Assume both rates remain applicable in two populations. Population A has prevalence 1%; Population B has prevalence 20%. Calculate precision and negative predictive value in each population and compare them.
Method. Apply the same conditional rates separately to each population’s actual positive and actual negative groups. The populations need not have the same total size, so use proportions rather than raw counts.
For Population A, the precision calculation is \(0.90(0.01)/[0.90(0.01)+0.05(0.99)]=0.009/0.0585=0.153846\ldots\). Its negative predictive value is \(0.95(0.99)/[0.10(0.01)+0.95(0.99)]=0.9405/0.9415=0.998938\ldots\). Rounded to four decimals, these are 0.1538 and 0.9989.
For Population B, prevalence is 0.20 and the proportion without the outcome is 0.80. Precision is \(0.90(0.20)/[0.90(0.20)+0.05(0.80)]=0.18/0.22=0.818181\ldots\). Negative predictive value is \(0.95(0.80)/[0.10(0.20)+0.95(0.80)]=0.76/0.78=0.974358\ldots\). Rounded to four decimals, these are 0.8182 and 0.9744.
From A to B, precision rises from 0.1538 to 0.8182, while negative predictive value falls from 0.9989 to 0.9744. The sensitivity and specificity used in the calculations did not change; the composition of each population did. This comparison is checked against the Bayes formulas and illustrates why predictive values should be interpreted with the relevant population prevalence. The common mistake is to assume a fixed sensitivity and specificity imply fixed precision and negative predictive value.
Why the Prevalence Effect Holds
The core justification is that the number of true positives and false positives respond differently to the population composition. In a population of size \(N\), the expected proportions in the four confusion-matrix cells are:
Precision is true positives divided by all positive classifications, so substituting the first and fourth proportions yields its Bayes formula. Negative predictive value is true negatives divided by all negative classifications, so substituting the third and second proportions yields its formula. This derivation also explains why prevalence matters: changing \(\pi\) changes how much weight the actual-positive and actual-negative groups contribute to each classification group.
Worked Example: Prevalence Needed for a Target Precision
A hypothetical classification rule has sensitivity 90% and specificity 98%. What prevalence is required for its precision to be 75%, assuming the sensitivity and specificity apply at that prevalence? Then calculate precision and negative predictive value if prevalence is instead 5%.
Method. First solve the precision formula algebraically for prevalence. Then check the result by substituting it back into the formula. This avoids guessing a prevalence from the target predictive value.
Let target precision be \(q=0.75\), with \(Se=0.90\) and \(1-Sp=0.02\). Starting from \(q=Se\pi/[Se\pi+(1-Sp)(1-\pi)]\), multiply both sides by the denominator: \(qSe\pi+q(1-Sp)(1-\pi)=Se\pi\). Rearranging gives \(q(1-Sp)=\pi[Se(1-q)+q(1-Sp)]\). Therefore:
The required prevalence is 0.0625, or 6.25%. Checking: the true-positive contribution is \(0.90(0.0625)=0.05625\), and the false-positive contribution is \(0.02(0.9375)=0.01875\). Thus precision is \(0.05625/(0.05625+0.01875)=0.75\), as required.
At prevalence 5%, precision is \(0.90(0.05)/[0.90(0.05)+0.02(0.95)]=0.045/0.064=0.703125\), or 0.7031 to four decimals. Negative predictive value is \(0.98(0.95)/[0.10(0.05)+0.98(0.95)]=0.931/0.936=0.994658\ldots\), or 0.9947 to four decimals. The calculation is checked by the same Bayes formulas. The common mistake is to use specificity, 0.98, as the false-positive rate; the correct false-positive rate is \(1-0.98=0.02\).
Interpretation and Common Traps
Predictive values answer practical questions about people who receive a particular classification: among those classified positive, what fraction truly have the outcome? Among those classified negative, what fraction truly do not? Their dependence on prevalence means a value reported for one population should not automatically be carried to another. When the prevalence changes, recompute predictive values using the new prevalence and the applicable sensitivity and specificity.
The formulas assume that the population prevalence and the conditional rates refer to the same target population and classification rule. If a model is moved to a new setting, a changed prevalence may be only one difference. For example, the mix of outcome severity or the way data are collected may also differ. In that case, sensitivity and specificity should not be treated as fixed without justification. This is a transportability issue, not a correction that Bayes’ rule can solve on its own.
- Reversing the conditioning. Sensitivity is \(P(T+\mid D)\); precision is \(P(D\mid T+)\). They are not interchangeable.
- Forgetting the negative class size. False positives equal \((1-Sp)\) times the number without the outcome, not \((1-Sp)\) times the number with it.
- Using prevalence as a count. A prevalence of 2% means \(0.02N\) actual positives in a cohort of size \(N\); it is not itself the number of positives.
- Assuming predictive values are model constants. They depend on prevalence as well as sensitivity and specificity.
When a problem supplies counts, calculate the confusion-matrix cells and form the appropriate ratios. When it supplies probabilities, use Bayes’ rule. Both methods should agree, and checking the denominator against the total number of positive or negative classifications is an effective way to catch errors. This emphasis on conditional interpretation is aligned with the SOA Exam PA (Predictive Analytics) syllabus.
Check Your Understanding
For each problem, use the stated population prevalence and assume the sensitivity and specificity given apply to that population. Round probabilities to four decimal places.
- A hypothetical rule has sensitivity 95%, specificity 90%, and prevalence 2%. Find precision and negative predictive value. (Answer: precision \(=0.1624\); negative predictive value \(=0.9989\).)
- In a hypothetical cohort of 10,000, prevalence is 8%, sensitivity is 85%, and specificity is 97%. Find the true-positive, false-negative, true-negative, and false-positive counts, then find precision and negative predictive value. (Answer: TP \(=680\), FN \(=120\), TN \(=8{,}924\), FP \(=276\); precision \(=0.7113\); negative predictive value \(=0.9867\).)
- A rule has sensitivity 80% and specificity 96%. What prevalence gives precision of 50%? (Answer: prevalence \(=0.0476\), or about \(4.7619\%\).)
- A rule has sensitivity 90% and specificity 99%. Compare precision at prevalences of 20% and 5%. (Answer: precision is \(0.9574\) at 20% prevalence and \(0.8257\) at 5% prevalence.)