Why imperfect classification changes clinical utility
A biomarker-guided policy does not act on a patient’s true biological status directly. It acts on a test result. When the test misclassifies patients, some people who might benefit from treatment are missed, while some who might be harmed or receive little benefit are treated. The clinical value of the policy therefore depends not only on treatment effects in the true biomarker groups, but also on prevalence, sensitivity, specificity, and the consequences of treatment in each group.
The calculations below express treatment effects as changes in a common utility scale, such as quality-adjusted life-years (QALYs). A positive value favors treatment over control; a negative value means treatment produces net harm on that scale. These are expected values for a specified target population, not guarantees for individual patients.
Set up the classification and utility model
For instance, \(\Delta_1=0.12\) QALY means that, on average, treatment rather than control adds 0.12 QALY per true-positive patient over the specified follow-up and utility horizon. A negative \(\Delta_0\) represents net harm in the true-negative group. Benefits and harms can be reported separately, but the net utility contrast must use a consistent scale and include relevant treatment burden. If a test or treatment has a cost not already included in the utility contrast, subtract it explicitly.
Sensitivity and specificity determine the joint proportions in the target population:
A policy that treats only patients with a positive test therefore treats both true positives and false positives. If it does not treat test-negative patients, its expected incremental utility relative to treating nobody is:
The first term is the expected treatment effect contributed by correctly identified true positives. The second is the effect among false positives, which is harmful when \(\Delta_0<0\). False negatives do not appear as treated patients in this expression, but they matter because their potential benefit is forgone. The utility of a perfect rule that treats true positives and not true negatives is \(\pi\Delta_1\). This is an oracle benchmark: it is useful for quantifying the cost of classification errors, but it is not an implementable policy when true status is unknown.
Deriving a treatment threshold for test-positive patients
The positive predictive value is the probability that a patient is truly biomarker-positive given a positive test. By the joint probabilities above,
Among test-positive patients, the expected incremental utility per treated patient is a weighted average of the two true-stratum treatment effects. If \(\Delta_1>0\) and \(\Delta_0<0\), treatment has positive expected utility in this group exactly when:
To obtain the threshold, move the negative term to the other side: \(PPV(\Delta_1-\Delta_0)>-\Delta_0\). Since \(\Delta_1-\Delta_0>0\), division preserves the inequality. If there is an additional treatment cost \(c\) per treated patient on the same utility scale, replace the left-hand side by \(PPV\Delta_1+(1-PPV)\Delta_0-c\); the threshold then becomes \((-\Delta_0+c)/(\Delta_1-\Delta_0)\), provided the resulting decision is feasible. Test costs incurred for everyone screened affect the overall policy comparison, even though they do not change this conditional treatment threshold.
Worked Example: Expected Utility and Harm From a Test-Positive Policy
Suppose a target population has biomarker prevalence \(\pi=0.30\). A test has sensitivity \(Se=0.80\) and specificity \(Sp=0.90\). Treatment adds an average of \(\Delta_1=0.20\) QALY for true positives and changes utility by \(\Delta_0=-0.05\) QALY for true negatives. Assume these effects apply within each true-status group regardless of the test result, and initially ignore test costs.
The proportion of the population that is a true positive and tests positive is \(0.30(0.80)=0.24\). The false-positive proportion is \((1-0.30)(1-0.90)=0.70(0.10)=0.07\). Thus 0.31 of the population tests positive. Under a policy treating only test-positive patients, the expected utility per person screened is:
The correctly identified patients contribute 0.048 QALY per person screened, while treatment of false positives contributes a loss of 0.0035 QALY. Among the 0.31 who test positive, \(PPV=0.24/0.31=0.7742\). Their mean treatment utility is \(0.7742(0.20)+0.2258(-0.05)=0.1435\) QALY per treated test-positive patient, consistent with \(0.0445/0.31\).
The perfect true-status rule would yield \(0.30(0.20)=0.0600\) QALY per person screened. The gap, \(0.0600-0.0445=0.0155\) QALY, is the net utility lost relative to that oracle rule. Treating everyone would yield \(0.30(0.20)+0.70(-0.05)=0.0250\) QALY, and treating nobody yields zero. Under these assumptions, the test-positive policy has greater expected utility than either of those implementable alternatives, though less than perfect classification.
Compare policies on the same population scale
A fair policy comparison uses the same target population, time horizon, utility scale, and treatment regime. The following expressions give expected incremental utility per person in that population relative to treating nobody, before any policy-specific test costs:
| Policy | Expected incremental utility | Interpretation |
|---|---|---|
| Treat nobody | \(0\) | Reference policy; no treatment benefit or treatment harm is incurred. |
| Treat only test-positive patients | \(\pi Se\Delta_1+(1-\pi)(1-Sp)\Delta_0\) | Treats true positives and false positives; misses true positives with negative tests. |
| Treat everyone | \(\pi\Delta_1+(1-\pi)\Delta_0\) | Applies treatment effects to both true-status groups. |
| Treat true positives only | \(\pi\Delta_1\) | Oracle benchmark requiring perfect knowledge of true status. |
For a more general rule, let \(d_+\) and \(d_-\) be the probabilities of treating patients with positive and negative test results. They can be 0 or 1 for deterministic policies, or intermediate values for randomized decisions. Expected incremental utility is:
This form makes clear that treating test-negative patients may recover benefit among false negatives, but may also expose true negatives to treatment harm. If testing costs \(c_{\text{test}}\) per person screened, subtract \(c_{\text{test}}\) from the utility of every policy that requires testing. A test-guided strategy should be selected only if its expected utility, including such costs, exceeds the relevant alternatives.
Worked Example: Applying the Positive Predictive Value Threshold
Suppose \(\pi=0.20\), \(Se=0.60\), and \(Sp=0.85\). Treatment changes utility by \(\Delta_1=0.12\) QALY in true positives and \(\Delta_0=-0.04\) QALY in true negatives. A test costs 0.001 QALY per screened patient, expressed here as a common utility-equivalent burden. Assume the stratum-specific effects do not vary by test result.
The true-positive, test-positive proportion is \(0.20(0.60)=0.12\), and the false-positive proportion is \(0.80(0.15)=0.12\). Thus \(PPV=0.12/(0.12+0.12)=0.5000\). The treatment threshold is:
Because \(0.5000>0.2500\), treating test-positive patients has positive expected treatment utility. Per test-positive patient, it is \(0.50(0.12)+0.50(-0.04)=0.0400\) QALY. Per person screened, the treatment contribution is \(0.12(0.12)+0.12(-0.04)=0.0144-0.0048=0.0096\) QALY. After the test cost, policy utility is \(0.0096-0.0010=0.0086\) QALY per person screened.
For comparison, treating everyone yields \(0.20(0.12)+0.80(-0.04)=0.0240-0.0320=-0.0080\) QALY before testing costs, while treating nobody yields zero. The test-guided rule is better than these alternatives under the stated inputs. The positive threshold establishes that treating test-positive patients is worthwhile as a treatment decision; the additional population-level comparison establishes whether the full testing policy is worthwhile after its cost.
Worked Example: When a Test’s Treatment Gain Does Not Cover Its Cost
Consider a population with \(\pi=0.10\), \(Se=0.70\), and \(Sp=0.80\). Let \(\Delta_1=0.10\) QALY and \(\Delta_0=-0.03\) QALY. Suppose testing costs 0.002 QALY per person screened. As before, assume treatment effects are constant within true-status groups across test classifications.
The true-positive, test-positive proportion is \(0.10(0.70)=0.07\), and the false-positive proportion is \(0.90(0.20)=0.18\). Hence \(PPV=0.07/(0.07+0.18)=0.2800\). The treatment threshold is \(0.03/(0.10+0.03)=0.2308\), so treatment has positive expected utility among test-positive patients. The treatment contribution per person screened is:
Subtracting the test cost gives \(0.0016-0.0020=-0.0004\) QALY per person screened. Thus the test-positive treatment decision is favorable conditional on a positive result, but the complete test-and-treat policy has lower expected utility than treating nobody after the assay burden is counted. Treating everyone yields \(0.10(0.10)+0.90(-0.03)=0.0100-0.0270=-0.0170\) QALY before testing costs. The oracle rule would yield \(0.10(0.10)=0.0100\) QALY, illustrating how much could be gained under perfect classification, not what this imperfect test can deliver.
Assumptions, diagnostics, and limitations
The calculation is only as credible as its inputs and assumptions. It requires a defined target population and time horizon; prevalence and test performance representative of that population; and treatment effects expressed on a common utility scale. The \(\Delta_t\) values should represent appropriate treatment-versus-control contrasts for the intended clinical regime. In a randomized trial, the subgroup contrasts should be estimated with their uncertainty rather than treated as known constants.
A key assumption is that, within each true biomarker stratum, the average treatment effect is the same for patients with positive and negative test results. If the test result carries prognostic or treatment-effect information beyond true status, a single \(\Delta_t\) is insufficient. The calculation must then use treatment effects for the joint groups \((T,Z)\), or a model that captures the additional heterogeneity. Likewise, if the test result affects clinical care other than through treatment eligibility, those consequences must be included in the policy utilities.
Before using a policy estimate, check the four joint proportions: they must be nonnegative and sum to one. Check that sensitivity and specificity are conditional on true status, not reversed. Confirm that a negative treatment effect is counted as a loss, that costs use compatible units, and that all policies use the same population denominator. Sensitivity analyses over plausible prevalence, test performance, and treatment-effect values can reveal when a decision changes; a recommendation that reverses under small input changes is fragile.
A common mistake is to treat the positive predictive value as the sensitivity, or to claim that a test with high sensitivity identifies most patients who test positive as true positives. Sensitivity is \(P(Z=1\mid T=1)\); positive predictive value is \(P(T=1\mid Z=1)\). The latter depends on prevalence as well as sensitivity and specificity. Another error is to compare utility per treated test-positive patient with utility per person screened. Those quantities have different denominators; policy decisions should generally be compared per person in the same target population, with testing costs included where relevant.
The oracle rule is also easy to misuse. Its utility is a useful upper benchmark for a rule that treats true positives only, but it is not an achievable test-guided result. Nor does this framework by itself account for uncertainty in estimated effects, long-term consequences beyond the chosen horizon, changing test performance, or constraints on testing and treatment capacity. Those features require additional modeling before a policy is implemented.
Check Your Understanding
Answer each question, then compare with the worked solutions below.
- A biomarker prevalence is 0.25, sensitivity is 0.80, and specificity is 0.90. What proportion of the population is test-positive, and what is the positive predictive value?
- If \(\Delta_1=0.15\) QALY and \(\Delta_0=-0.05\) QALY, what positive predictive value is required for treatment to have positive expected utility among test-positive patients?
- Using the inputs in question 1, suppose \(\Delta_1=0.10\) QALY and \(\Delta_0=-0.02\) QALY. Calculate the test-positive policy’s treatment contribution per person screened, before test costs.
- Explain why a test-positive policy with positive expected treatment utility can still have lower overall utility than treating nobody.
Solutions: (1) The true-positive test-positive proportion is \(0.25(0.80)=0.20\); the false-positive proportion is \(0.75(0.10)=0.075\). Therefore the test-positive proportion is \(0.20+0.075=0.275\), and \(PPV=0.20/0.275=0.7273\). (2) The threshold is \(-\Delta_0/(\Delta_1-\Delta_0)=0.05/(0.15+0.05)=0.2500\); treatment has positive expected utility when \(PPV>0.25\). (3) The contribution is \(0.20(0.10)+0.075(-0.02)=0.0200-0.0015=0.0185\) QALY per person screened. (4) A positive treatment contribution can be smaller than the assay cost, or other policy-specific costs. After subtracting those burdens, total expected utility may be negative even though treatment itself is favorable among test-positive patients.