Start With the Decision, Not the Model
A mechanistic model can connect drug action to biological changes and clinical outcomes, but that connection alone does not make the model valuable for development. The practical question is whether improved biological understanding could change a consequential choice: for example, which regimen to advance or which patient context to investigate. If all plausible biological explanations point to the same action, additional mechanistic insight may be scientifically interesting without changing the decision.
The earlier material on counterfactual questions established that mechanistic models can predict outcomes under interventions or conditions that have not yet been observed. Here, the focus is how to judge whether such predictions matter to a decision. The answer depends on three elements: the available actions, uncertainty about relevant biology, and the consequences of acting under each possible biological state. A model is one possible source of evidence about those elements; it does not define the decision or establish the consequences by itself.
A useful starting point is to write down the decision before specifying model outputs. Name the actions that are genuinely available, such as advancing one of two regimens, and the biological states or explanations that could affect their outcomes. Then state what counts as a desirable consequence. That may be clinical benefit, safety, development time, cost, or a combination. If consequences are combined into a numerical utility, the scale and priorities must be explicit; utility units need not be physical units.
Expected Consequences and Decision Thresholds
Let \(a\) denote an available action and \(\theta\) an uncertain biological state. Let \(U(a,\theta)\) be the consequence of taking action \(a\) if state \(\theta\) applies, expressed on a common utility scale. Given current evidence, let \(P(\theta)\) represent the probability assigned to each state. The expected utility of an action is its consequence averaged over those states, weighted by their probabilities. The action with the highest expected utility is preferred under this decision framework.
This calculation is only as meaningful as its inputs. The states should capture uncertainty relevant to the choice; probabilities should be supported by evidence rather than chosen to favor an action; and consequences should reflect the decision objective. In practice, a model may help explain how a state would affect outcomes or predict outcomes under a new regimen. But the probabilities and utilities usually require evidence and judgment beyond the model itself.
A decision threshold is a value of an uncertain quantity at which the preferred action changes. If plausible evidence could move that quantity across the threshold, further information may have decision value. If the threshold is far outside the plausible range, learning more about that quantity is less likely to change the action.
Worked Example: A Regimen Choice Near a Biological Threshold
Hypothetical example. A development team is choosing between regimens A and B. A biological state, “high pathway dependence,” is thought to make A particularly effective. The team assigns a probability \(p\) to that state. The following illustrative utilities are in decision points; higher values are preferred.
| Action | High pathway dependence | Low pathway dependence |
|---|---|---|
| Regimen A | 8 | -4 |
| Regimen B | 5 | 2 |
For A, the expected utility is \(8p+(-4)(1-p)\). For B, it is \(5p+2(1-p)\). Substituting and simplifying gives:
The actions have equal expected utility when \(12p-4=3p+2\), so \(9p=6\) and \(p=2/3\). Below that threshold, B has higher expected utility; above it, A does. Suppose current evidence supports \(p=0.40\). Then \(EU(A)=12(0.40)-4=0.80\) points, while \(EU(B)=3(0.40)+2=3.20\) points. The current preferred action is B.
A QSP analysis could be valuable if credible biological evidence could materially change the estimate of \(p\) and place it on the other side of \(2/3\). It would not be enough for the model to predict different outcomes in the two states: the uncertainty about which state applies must be relevant to the choice. Nor does this calculation establish that QSP can estimate \(p\) accurately. That depends on evidence supporting the model and its connection to the decision.
Why Information Has a Limit on Its Value
A short derivation makes clear why uncertainty matters most when it can change the preferred action. Imagine, as an ideal upper bound, that the biological state could be learned perfectly before acting. With current evidence, the best expected utility is \(\max_a \sum_{\theta}P(\theta)U(a,\theta)\). With perfect information, the action can be selected separately for each state, giving expected utility \(\sum_{\theta}P(\theta)\max_a U(a,\theta)\). The difference is the expected value of perfect information:
The inequality holds because, for every fixed action \(a\), the state-specific maximum satisfies \(\max_{a'}U(a',\theta)\geq U(a,\theta)\). Multiplying by each nonnegative \(P(\theta)\) and summing gives \(\sum_{\theta}P(\theta)\max_{a'}U(a',\theta)\geq\sum_{\theta}P(\theta)U(a,\theta)\). This is true for every \(a\), including the action with the largest current expected utility, so the difference cannot be negative.
For the regimen example at \(p=0.40\), perfect information would mean choosing A in the high-dependence state and B in the low-dependence state. Its expected utility would be \(0.40(8)+0.60(2)=4.40\) points. The best current expected utility is 3.20 points, so \(EVPI=4.40-3.20=1.20\) points. This is an upper bound on the value of any imperfect information, not an estimate of the value of a QSP model. Model predictions are not perfect information, and the bound does not account for the cost or time required to produce evidence.
A model can therefore have high scientific detail and little decision value if it cannot resolve uncertainty that affects the choice. Conversely, a focused model may be useful when it addresses a pivotal uncertainty, the decision is consequential, and the predictions can be checked against relevant evidence. The key is not the number of modeled mechanisms; it is whether the model informs a decision-relevant uncertainty credibly enough to improve the action.
Worked Examples Across Development Contexts
Worked Example: Deciding Whether to Treat a Patient Context
Hypothetical example. A team is considering whether a therapy should be evaluated in a particular patient context. In this illustration, treatment has a utility of \(+6\) points for a patient whose disease is responsive through the proposed mechanism, and \(-3\) points for a patient whose disease is not responsive. No treatment has utility 0 in either state. Let \(p\) be the probability that a patient in the context is responsive.
The expected utility of treatment is \(6p-3(1-p)=9p-3\) points. The expected utility of no treatment is 0. Treatment is preferred when \(9p-3>0\), or \(p>1/3\). At \(p=0.25\), treatment has expected utility \(9(0.25)-3=-0.75\) points, so no treatment is preferred under these assumptions. At \(p=0.50\), treatment has expected utility \(9(0.50)-3=1.50\) points, so treatment is preferred.
Mechanistic evidence may add value if it helps estimate whether the responsive state is sufficiently common in this context to cross the threshold. The calculation assumes that the two utility values apply to the population and that the probability is relevant to the decision. It omits such factors as treatment burden, uncertainty in the utilities, and the cost of identifying responsive patients. Those factors could change the threshold, so the calculation is a structured illustration rather than a clinical recommendation.
Worked Example: Recognizing a Decision That Is Robust
Hypothetical example. A development team must choose between actions C and D. It considers two plausible biological states, with probabilities \(0.30\) and \(0.70\). The utility of C is 6 points in either state. The utility of D is 2 points in the first state and 4 points in the second.
The expected utilities are \(EU(C)=0.30(6)+0.70(6)=6.00\) points and \(EU(D)=0.30(2)+0.70(4)=3.40\) points. C is preferred. More importantly, C has higher utility in each state: \(6>2\) and \(6>4\). Thus, changing the probability assigned to either state cannot make D preferable, as long as the stated utilities and available actions remain unchanged. Even perfect information would not change the action, and its decision value is 0 points in this simplified comparison.
A model might still improve scientific understanding, but this particular uncertainty does not appear to affect this choice under the assumptions shown. The team should check whether relevant states, actions, or consequences have been left out before concluding that the decision is robust. Robustness is always relative to the options and uncertainties considered.
Assessing Whether the Case for QSP Is Convincing
Decision value should be argued from a traceable chain: the model represents a biological process; available evidence supports the relevant links; the model predicts an outcome relevant to the options; and that outcome could alter a consequential choice. Earlier material on evidence chains and integrating evidence across assays and biological scales helps frame this reasoning. If a link is weak, the decision claim should be correspondingly cautious.
A useful comparison is with decisions based on direct clinical observations, empirical exposure–response relationships, or expert judgment. Each can support a decision when it addresses the relevant question. Mechanistic QSP may add a structured way to reason about biological pathways and unobserved conditions, while existing clinical data may provide more direct evidence about outcomes in observed settings. These are not automatically competing sources: they can inform different parts of the same decision. The important comparison is whether the model contributes credible, decision-relevant information beyond what the team already knows.
In practice, check whether the proposed model output connects to an action and whether plausible changes in its inputs or assumptions could alter the choice. Be explicit about the development consequence of being wrong. A small change in an outcome prediction may not matter if the action remains the same; a change that reverses a high-stakes decision deserves closer scrutiny. Also distinguish the model’s conditional prediction—what follows if its structure and inputs are accepted—from evidence that those assumptions are appropriate for the intended decision.
A disciplined case for QSP states the decision, the relevant uncertainty, the action threshold or other decision rule, the evidence supporting the prediction, and the consequence of changing the action. It also states what the analysis cannot resolve. If the decision is unchanged across plausible states, or if the model cannot credibly address the uncertainty near a threshold, its decision value may be limited even when the biology is complex.
Check Your Understanding
Use the decision-value framework to answer each question. Solutions follow the questions.
- In the regimen example, which probability threshold separates the preferred actions?
- In the patient-context example, calculate treatment’s expected utility when \(p=0.40\). Which action is preferred?
- Why is the expected value of perfect information an upper bound rather than the expected value of a QSP model?
- In the robust-decision example, what feature shows that the action does not depend on the probabilities assigned to the two states?
Solutions. (1) Set \(12p-4=3p+2\), giving \(p=2/3\); B is preferred below the threshold and A above it. (2) \(EU(\text{treatment})=9(0.40)-3=0.60\) points, which exceeds the 0 points for no treatment, so treatment is preferred under the stated assumptions. (3) Perfect information reveals the relevant state without error before the action is chosen; a QSP model provides predictions with uncertainty and may not resolve the state. (4) C has greater utility than D in each state, so changing their probabilities cannot reverse the ranking while the stated utilities and options remain fixed.