Predictive Values Under Changing Event Prevalence

Why Predictive Values Depend on the Population

A safety classifier can have the same sensitivity and specificity in two populations but produce different proportions of correct positive and negative classifications. The reason is that a positive result can be a true positive or a false positive, and the balance between those possibilities depends on how common toxicity is in the population being evaluated. Predictive values answer a practical question: given a classifier result, how likely is the corresponding toxicity status in this population?

This dependence matters when applying a classifier in a population whose toxicity prevalence differs from the population used to evaluate it. A positive result that is usually correct in a high-prevalence group may be less convincing in a low-prevalence group, even if the classifier’s ability to distinguish event from non-event patients has not changed.

Definition: Let \(E\) denote the event of toxicity, \(+\) a positive classifier result, and \(-\) a negative result. The event prevalence is \(\pi=P(E)\). Sensitivity is \(Se=P(+\mid E)\), and specificity is \(Sp=P(-\mid E^c)\), where \(E^c\) means no toxicity. The positive predictive value is \(PPV=P(E\mid +)\); the negative predictive value is \(NPV=P(E^c\mid -)\).

The formulas below concern a defined target population and a fixed toxicity outcome over a specified assessment period. They assume that \(\pi\), \(Se\), and \(Sp\) refer to that same population, outcome definition, and classifier threshold. To compare predictive values across populations while holding \(Se\) and \(Sp\) fixed, we assume that sensitivity and specificity remain applicable in each population; only prevalence changes. This is a useful calculation, not a guarantee: changes in patient characteristics, measurement, or implementation can also alter sensitivity or specificity.

Bayes’ Rule and the Predictive-Value Formulas

Partition the population into four groups: toxicity with a positive result, toxicity with a negative result, no toxicity with a positive result, and no toxicity with a negative result. Their proportions follow from the definitions of prevalence, sensitivity, and specificity.

\begin{aligned} P(E\cap +)&=\pi Se, & P(E\cap -)&=\pi(1-Se),\\ P(E^c\cap +)&=(1-\pi)(1-Sp), & P(E^c\cap -)&=(1-\pi)Sp. \end{aligned}

For example, the proportion with a positive result is the sum of the true-positive and false-positive proportions: \(P(+)=\pi Se+(1-\pi)(1-Sp)\). Applying conditional probability gives the predictive values.

PPV=\frac{\pi Se}{\pi Se+(1-\pi)(1-Sp)}

NPV=\frac{(1-\pi)Sp}{(1-\pi)Sp+\pi(1-Se)}

These expressions are defined when their conditioning events have positive probability: \(PPV\) requires \(P(+)>0\), and \(NPV\) requires \(P(-)>0\). At ordinary interior prevalence values, \(0<\pi<1\), both denominators are positive unless the classifier is completely degenerate. At a boundary, a predictive value can be undefined if no one can have the corresponding classifier result.

Worked Examples Across Prevalence Values

Worked Example: Comparing Two Prevalences by Counts

Hypothetical data. Suppose a safety classifier has \(Se=0.90\) and \(Sp=0.95\). Compare its predictive values in populations of 1,000 patients with toxicity prevalences of 10% and 30%. Assume the same sensitivity and specificity apply in both populations.

At 10% prevalence, 100 of the 1,000 patients have toxicity and 900 do not. The expected counts are \(TP=100(0.90)=90\), \(FN=100(0.10)=10\), \(TN=900(0.95)=855\), and \(FP=900(0.05)=45\). Therefore,

PPV=\frac{90}{90+45}=0.6667,\qquad NPV=\frac{855}{855+10}=0.9884.

Among patients classified positive, about 66.7% have toxicity in this population; among those classified negative, about 98.8% do not have toxicity.

At 30% prevalence, 300 patients have toxicity and 700 do not. The expected counts are \(TP=270\), \(FN=30\), \(TN=665\), and \(FP=35\). Thus,

PPV=\frac{270}{270+35}=0.8852,\qquad NPV=\frac{665}{665+30}=0.9568.

With sensitivity and specificity held fixed, the PPV rises from 66.7% to 88.5%, while the NPV falls from 98.8% to 95.7%. The higher prevalence means that a positive result is more likely to come from a patient with toxicity; it also means that a negative result is less likely to come from a patient without toxicity.

Worked Example: A Low-Prevalence Safety Outcome

Hypothetical data. A classifier has \(Se=0.85\) and \(Sp=0.90\). Calculate predictive values at toxicity prevalences of 2% and 20%, using proportions of patients rather than rounding to whole people.

At \(\pi=0.02\), the true-positive proportion is \(0.02(0.85)=0.017\), and the false-positive proportion is \(0.98(0.10)=0.098\). The true-negative proportion is \(0.98(0.90)=0.882\), and the false-negative proportion is \(0.02(0.15)=0.003\). Hence,

PPV=\frac{0.017}{0.017+0.098}=0.1478,\qquad NPV=\frac{0.882}{0.882+0.003}=0.9966.

At 2% prevalence, about 14.8% of positive classifications correspond to toxicity, whereas about 99.7% of negative classifications correspond to no toxicity. The high NPV does not mean the classifier has perfect sensitivity; it reflects, in part, that most patients in this population do not have the event.

At \(\pi=0.20\), the corresponding proportions are \(TP=0.20(0.85)=0.17\), \(FP=0.80(0.10)=0.08\), \(TN=0.80(0.90)=0.72\), and \(FN=0.20(0.15)=0.03\). Therefore,

PPV=\frac{0.17}{0.17+0.08}=0.6800,\qquad NPV=\frac{0.72}{0.72+0.03}=0.9600.

The PPV is now 68.0%, while the NPV is 96.0%. The calculations describe different populations; neither value is an intrinsic, prevalence-free property of the classifier.

Odds Form and a Short Derivation

The odds form makes the role of prevalence especially clear. For a positive result, the odds of toxicity among positive patients are the true-positive proportion divided by the false-positive proportion. Dividing both by \((1-\pi)(1-Sp)\) gives the prior odds of toxicity multiplied by the positive likelihood ratio.

\frac{PPV}{1-PPV} =\frac{\pi Se}{(1-\pi)(1-Sp)} =\frac{\pi}{1-\pi}\times\frac{Se}{1-Sp}

The ratio \(Se/(1-Sp)\) is the positive likelihood ratio. Thus, when sensitivity and specificity are fixed, a positive result multiplies the prior odds by a fixed factor, but the prior odds themselves change with prevalence. In the same way, the odds of no toxicity among negative patients are

\frac{NPV}{1-NPV} =\frac{(1-\pi)Sp}{\pi(1-Se)} =\frac{1-\pi}{\pi}\times\frac{Sp}{1-Se}.

This also shows why predictive values should not be transported between populations simply because the classifier has the same sensitivity and specificity. The prior odds differ, so the resulting predictive-value odds differ.

Worked Example: Finding the Prevalence Needed for a PPV of 80%

Hypothetical calculation. Suppose \(Se=0.90\) and \(Sp=0.95\), and ask what prevalence would yield \(PPV\geq 0.80\), if these performance values remain fixed. This is a mathematical threshold, not a clinical action recommendation.

Starting from the PPV formula and writing \(q=0.80\), solve the inequality:

\begin{aligned} \frac{\pi Se}{\pi Se+(1-\pi)(1-Sp)}&\geq q\\ \pi Se(1-q)&\geq q(1-\pi)(1-Sp)\\ \pi\{Se(1-q)+q(1-Sp)\}&\geq q(1-Sp)\\ \pi&\geq\frac{q(1-Sp)}{Se(1-q)+q(1-Sp)}. \end{aligned}

Substituting \(q=0.80\), \(Se=0.90\), and \(1-Sp=0.05\),

\pi\geq\frac{0.80(0.05)}{0.90(0.20)+0.80(0.05)} =\frac{0.040}{0.220}=0.1818.

Under the stated fixed-performance assumption, prevalence must be at least 18.2% for the PPV to reach 80%. For a check, at \(\pi=0.20\), \(PPV=0.18/(0.18+0.04)=0.8182\), which exceeds 80%; at \(\pi=0.15\), \(PPV=0.135/(0.135+0.0425)=0.7606\), which does not.

Boundary Behavior, Diagnostics, and Limitations

The formulas have informative boundary behavior, but the conditions matter. As \(\pi\to 0\), \(PPV\to 0\) if \(Sp<1\), because the false-positive proportion approaches the positive-result denominator’s nonzero limit \(1-Sp\). In that case, \(NPV\to 1\) if \(Sp>0\), since the true-negative proportion approaches \(Sp\) while the false-negative proportion approaches zero. As \(\pi\to 1\), \(PPV\to 1\) if \(Se>0\), because the true-positive proportion approaches \(Se\) while the false-positive proportion approaches zero. Under that limit, \(NPV\to 0\) if \(Se<1\), because the false-negative proportion approaches \(1-Se\) while the true-negative proportion approaches zero. At a boundary where the corresponding classifier result has probability zero, its predictive value is undefined there.

For instance, if \(Sp=1\) and \(Se>0\), there are no false positives, so \(PPV=1\) for every interior prevalence; it does not approach zero as prevalence approaches zero. If \(Se=1\) and \(Sp>0\), there are no false negatives, so \(NPV=1\) for every interior prevalence; it does not approach zero as prevalence approaches one.

A practical diagnostic is to reconstruct the four cell proportions and check that they are nonnegative and sum to one. Then verify that the positive-result denominator equals \(P(+)\), and the negative-result denominator equals \(P(-)\). If a computed PPV or NPV is outside \([0,1]\), or if the denominator does not match the relevant marginal proportion, an arithmetic or formula error has occurred. When applying published sensitivity and specificity, also examine whether the target population and classifier implementation plausibly support the fixed-performance assumption.

Predictive values complement, rather than replace, sensitivity and specificity. Sensitivity and specificity condition on the true event status; predictive values condition on the classifier result. Accuracy, the overall proportion correctly classified, is another quantity and can be dominated by the more common status when prevalence is highly imbalanced. None of these summaries alone establishes whether using a classifier improves patient outcomes or is appropriate for a particular dosing decision.

Common mistake: “Sensitivity and specificity are unchanged, so PPV and NPV must also be unchanged.” This reverses the conditioning. Sensitivity and specificity describe result rates within event-status groups; PPV and NPV describe event-status rates within result groups. The latter depend on the proportions of event and non-event patients, so changing \(\pi\) generally changes them.

A second limitation is that prevalence is not always known precisely, especially for uncommon toxicities or outcomes defined over a particular time window. Predictive values calculated from an estimated prevalence inherit its uncertainty. Moreover, if the target population differs in ways that alter classifier performance, recalculating predictive values with the old \(Se\) and \(Sp\) may be misleading. The formulas isolate the effect of prevalence; they do not establish that performance is transportable.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

A classifier has \(Se=0.80\), \(Sp=0.90\), and event prevalence \(\pi=0.10\). Calculate PPV and NPV.
With sensitivity and specificity fixed, what usually happens to PPV and NPV as prevalence increases? Explain using the formulas or the four cell proportions.
At prevalence 5%, a classifier has \(Se=0.90\) and \(Sp=0.95\). Calculate its PPV. Why can it be well below 90%?
State the conditions for the limits \(PPV\to0\) as \(\pi\to0\) and \(NPV\to0\) as \(\pi\to1\). What must be said if the corresponding result has probability zero at the boundary?

Solutions. (1) The true-positive proportion is \(0.10(0.80)=0.08\), the false-positive proportion is \(0.90(0.10)=0.09\), the true-negative proportion is \(0.90(0.90)=0.81\), and the false-negative proportion is \(0.10(0.20)=0.02\). Thus \(PPV=0.08/(0.08+0.09)=0.4706\) and \(NPV=0.81/(0.81+0.02)=0.9759\). (2) PPV generally increases and NPV generally decreases: a higher \(\pi\) increases the event share among positive results and increases the event share among negative results. Degenerate performance values can produce equality or boundary exceptions. (3) \(PPV=[0.05(0.90)]/[0.05(0.90)+0.95(0.05)]=0.045/0.0925=0.4865\). The false-positive group can be large relative to the true-positive group when most patients do not have the event. (4) \(PPV\to0\) as \(\pi\to0\) if \(Sp<1\); \(NPV\to0\) as \(\pi\to1\) if \(Se<1\). If the positive-result or negative-result event has probability zero at the boundary, the corresponding conditional predictive value is undefined there, even though a one-sided limit from interior prevalence values may exist.