Statistical Evidence for Clinical Use of Polygenic Risk

From Prediction Performance to Clinical Evidence

A polygenic risk score may separate people who will and will not develop an outcome, yet still be unsuitable for guiding care. Its absolute-risk estimates may be inaccurate for the patients who would receive them; the decisions it prompts may add little beyond current clinical predictors; or using it may cause burdens or harms that outweigh any benefit. Clinical use therefore requires evidence about a chain of questions: are the risks useful and calibrated in the intended population, do they change decisions in a beneficial way, how uncertain are those findings, and what happens when the score is actually used in care?

These questions concern a defined prediction target: the outcome, time horizon, and patient population for which a risk is intended. Evidence for one target does not automatically establish usefulness for another. Calibration and transport are considered here as requirements to assess, not as substitutes for evaluating what happens after a clinical decision changes.

A sound assessment distinguishes three claims. First, a score can predict outcomes in a specified population. Second, adding it to existing information can improve a decision analysis at a chosen decision threshold. Third, implementing score-guided care can improve patient outcomes or other prespecified clinical consequences. Each claim needs evidence suited to it; success at one stage does not prove the next.

Evidence About Risks in the Intended Population

Absolute-risk estimates should be evaluated in a population resembling the patients for whom they are proposed, using the intended outcome definition and time horizon. A basic calibration question is whether predicted risks agree with observed frequencies. Calibration-in-the-large compares the average predicted risk with the overall observed event frequency. It is a useful summary, but agreement on average does not ensure that risks agree across clinically relevant risk ranges.

Evidence should also be reported with uncertainty. An observed proportion is subject to sampling variation, and uncertainty in estimated score effects or other model components can add further uncertainty. If a confidence interval for a calibration discrepancy is wide, apparent agreement may simply reflect limited information. If the interval excludes discrepancies that would be clinically acceptable, that is more informative than a point estimate alone. The acceptable range should be justified for the intended decision, not chosen after seeing the results.

Calibration should be assessed for the stated target population and time horizon. A result from another setting may not apply when baseline incidence, clinical practice, or the distribution of relevant characteristics differs. Earlier material on calibration and transporting absolute risk gives tools for those evaluations. The central evidential question here is whether the resulting risk estimates are sufficiently reliable for the proposed use, not merely whether a model can be recalibrated.

Incremental Decision Benefit and Its Uncertainty

A score should be compared with the best relevant alternative, often current clinical predictors or current practice. Incremental benefit asks whether using the score improves decisions at a threshold that reflects the consequences of acting and not acting. A higher area under a discrimination curve, improved accuracy, or changed risk classification does not by itself show that a decision is better.

Decision-curve analysis expresses the trade-off between true-positive and false-positive classifications through net benefit. For a defined threshold probability \(p_t\), a common form is \(NB=TP/n-(FP/n)\{p_t/(1-p_t)\}\), where \(TP\) and \(FP\) are the numbers of true-positive and false-positive decisions among \(n\) people. The threshold weight reflects the relative consequences encoded by the threshold; net benefit should be interpreted at that threshold and for the stated outcome and action.

The score's relevant comparison is the difference in net benefit from the comparator, not its net benefit in isolation. A positive difference may be too small to matter clinically, and an estimated difference can be imprecise. Because both strategies are often evaluated on the same people, uncertainty for the difference should preserve that pairing. A patient-level bootstrap can resample individuals and recompute both strategies and their difference in each replicate.

The threshold weight has a decision interpretation. Let \(B\) be the benefit of correctly acting for someone who would experience the outcome, and \(H\) the harm of acting for someone who would not. At the decision boundary, expected benefit and harm are equal: \(p_tB=(1-p_t)H\). Dividing by \(B(1-p_t)\) gives \(H/B=p_t/(1-p_t)\). Thus the false-positive weight in net benefit corresponds to the threshold's implied relative harm. This also explains why decision benefit must be reported at a clinically justified threshold rather than as a universal property of the score.

Prospective Evidence About Patient-Care Consequences

Even a calibrated model with favorable decision-curve results may not improve care when implemented. Clinicians may not use the result as intended; patients may decline recommended actions; follow-up and treatment may differ from assumptions in the analysis; and testing can introduce costs, anxiety, or other burdens. A prospective evaluation assesses the consequences of a defined care strategy rather than only the behavior of a prediction model in an existing data set.

A strong evaluation specifies in advance the intended patients, score, risk threshold, clinical action, comparator, follow-up period, primary outcome, and important harms. A randomized comparison of score-guided care with usual care can estimate the effect of implementing the strategy when randomization is properly conducted and outcomes are measured comparably. If randomization is not feasible, the design must address how differences between people receiving each strategy could distort the comparison. In either case, the analysis should retain all eligible patients in the assigned strategy where appropriate, report missing outcomes and adherence, and present uncertainty for the effect estimate.

A prospective study should distinguish an impact question from a prediction question. A prediction evaluation asks how well risks and classifications match outcomes. An impact evaluation asks whether offering and using the score changes health outcomes, decisions, or harms compared with the alternative. The latter can be influenced by the whole implementation pathway, not only by predictive performance.

A Practical Evidence Standard

Clinical readiness is not established by one favorable metric or one small p-value. A defensible assessment aligns the target, comparator, decision, and outcome across the evidence. Before implementation, the evidence should make clear what the score is intended to change, what would count as a worthwhile improvement, and what uncertainty or harm would make the change unacceptable.

These steps also clarify what evidence is missing. A favorable retrospective decision analysis can justify a prospective evaluation, but it does not guarantee that patients benefit. Conversely, a trial of an implementation strategy estimates the consequences of that strategy in its setting; it does not make the underlying risks universally calibrated. Clinical use is supported when the total evidence is coherent, sufficiently precise, relevant to the intended patients, and consistent with a prespecified standard of worthwhile benefit.

Common Pitfall

Wrong reasoning: “The score has better discrimination and its decision curve is above the comparator, so it should be used clinically.” This skips whether absolute risks are reliable for the intended patients, whether the difference is precise and large enough to matter, and whether score-guided care actually improves outcomes without unacceptable harms.

Correction: Treat discrimination, calibration, decision analysis, and prospective impact as complementary evidence. State the target and comparator; quantify uncertainty; interpret benefit at the relevant threshold; and distinguish estimated decision benefit from demonstrated consequences of implementation. A result that is statistically distinguishable from zero may still be too small, uncertain in important subgroups, or outweighed by harms to support routine use.