From Model Output to a Decision
A QSP model can generate detailed trajectories for pathway activity, biomarkers, disease measures, and safety variables. A development team still needs to answer a practical question: what result would change the program’s next action? A plot or a large collection of simulated trajectories does not, by itself, specify whether to advance, select a dose, gather more information, or stop.
A decision rule makes that connection explicit. It names the model outputs to be evaluated, defines how each output is summarized, sets thresholds before interpreting results, and states what action follows from each possible combination of results. The rule should be specific enough that a second analyst using the same inputs and outputs would reach the same recommendation.
The examples use a hypothetical target program and a virtual population of 100 simulated patient profiles. The thresholds are illustrative program choices, not general standards. They are meaningful only for the endpoint definitions, population, dose regimens, and prediction horizon specified in the rule.
Specify the Metrics Before Applying Thresholds
A decision rule must separate three things that are easy to blur: what the model predicts, how that prediction is summarized, and what action the summary triggers. For this example, define the quantities as follows.
- Efficacy: percentage reduction from baseline in a disease-activity score at week 12. A larger reduction is favorable.
- Efficacy responder: a simulated profile with at least a 20% reduction in that score at week 12.
- Safety: whether a simulated profile experiences at least one specified serious toxicity by week 12. The safety summary is the proportion of profiles with that event.
- Biomarker: percentage suppression of a prespecified pathway biomarker at week 4 relative to that profile’s baseline. A larger suppression indicates greater predicted biomarker modulation; it is not, by itself, evidence of clinical benefit.
For equally weighted profiles, the model-based responder proportion is the number of profiles meeting the responder definition divided by the total number of profiles. The safety proportion is calculated in the same way, using the event definition. These are summaries of the simulated virtual population under the specified model and regimen. They are not automatically estimates of real-world clinical probabilities.
Before applying either proportion, confirm that profiles represent the intended decision population and that the simulation protocol, model version, regimen, and output definitions are the intended ones. The decision rule is conditional on those choices. Reproducible model release and traceable inputs make it possible to identify which choices produced the result; they do not make a prediction clinically certain.
Worked Example: Writing the Advancement Gates
Suppose a hypothetical team will consider advancing a target program only if all three criteria are met at the proposed dose: mean disease-activity reduction at week 12 is at least 25%; at least 60% of profiles are efficacy responders; and no more than 10% of profiles experience the specified serious toxicity by week 12. The team also requires median week-4 biomarker suppression of at least 70% as evidence that the regimen produces the intended modeled biomarker change.
The rule is conjunctive: every criterion must pass. In symbols, let \(E\) denote the mean efficacy criterion, \(R\) the responder criterion, \(S\) the safety criterion, and \(B\) the biomarker criterion. Advancement requires \(E\land R\land S\land B\). For 100 equally weighted profiles, the percentage thresholds translate into counts: at least 60 responders and no more than 10 profiles with toxicity. The mean, median, and proportions remain distinct summaries; passing one does not substitute for passing another.
This structure makes the intended trade-off explicit. Under a conjunctive rule, an unusually strong efficacy prediction cannot compensate for failing a prespecified safety ceiling. That is appropriate when each gate represents a required condition, rather than a preference that can be traded against another outcome.
Apply a Rule Across Candidate Doses
The table gives hypothetical model summaries at three evaluated doses. Each column refers to the same defined virtual population and prediction horizon. The responder and toxicity counts have 100 profiles as their denominator.
| Evaluated dose | Mean disease-score reduction at week 12 | Efficacy responders | Profiles with serious toxicity | Median biomarker suppression at week 4 |
|---|---|---|---|---|
| 10 mg | 18% | 42 of 100 (42%) | 2 of 100 (2%) | 48% |
| 30 mg | 29% | 68 of 100 (68%) | 6 of 100 (6%) | 74% |
| 60 mg | 34% | 76 of 100 (76%) | 18 of 100 (18%) | 88% |
Worked Example: Selecting the Lowest Dose That Passes
Apply the advancement gates from the previous example, then use the prespecified selection rule: among evaluated doses that pass every gate, choose the lowest dose. This rule is only being applied to the doses and regimen already evaluated in this hypothetical model analysis.
At 10 mg: the mean reduction is 18%, below 25%; 42% of profiles respond, below 60%; and median biomarker suppression is 48%, below 70%. The safety criterion passes because \(2/100=0.020=2\%\), which is no more than 10%. The dose fails the full advancement rule.
At 30 mg: the mean reduction is 29%, which is at least 25%. The responder proportion is \(68/100=0.68=68\%\), which is at least 60%. The toxicity proportion is \(6/100=0.06=6\%\), which is no more than 10%. Median biomarker suppression is 74%, which is at least 70%. All four gates pass.
At 60 mg: the mean reduction is 34%, the responder proportion is \(76/100=76\%\), and median biomarker suppression is 88%; these three criteria pass. But the toxicity proportion is \(18/100=0.18=18\%\), exceeding the 10% ceiling. The dose fails the full rule.
Therefore, 30 mg is the lowest evaluated dose that passes all four criteria. This is a model-based recommendation under the specified rule, not a claim that 30 mg is clinically optimal or that its real-world toxicity rate will be 6%.
Make the Action Depend on the Pattern of Results
A useful rule describes what happens when a gate fails, not just what happens when every gate passes. In particular, failure to reach the biomarker criterion and failure to achieve predicted efficacy are not the same result. The former says the simulated regimen did not meet a chosen biomarker threshold. It does not, on its own, establish why the prediction fell short.
The following example defines a small set of possible actions for a hypothetical program. “Hold for review” means the rule does not recommend advancement or termination from this result alone; the team must first check whether the modeled inputs, scenario, and outputs are suitable for the intended decision. It is not an automatic instruction to modify the dose or to run a particular experiment.
Worked Example: Distinguishing a Failed Gate from a Termination Rule
Suppose the program rule is: advance if all four gates pass; recommend termination of this modeled program if the biomarker gate passes but the mean efficacy reduction is below 25%; otherwise, hold for review when a required gate fails in a pattern not covered by those rules. Consider two hypothetical result sets.
Result set A: mean disease-score reduction is 19%, median biomarker suppression is 55%, responder proportion is 45%, and toxicity proportion is 4%. The mean efficacy and biomarker gates fail, while the toxicity gate passes. Because the biomarker gate does not pass, the stated termination condition is not met. The action is hold for review, not terminate: the outputs do not meet the stated rule for interpreting a biomarker-engaged efficacy failure.
Result set B: mean reduction is 19%, median biomarker suppression is 76%, responder proportion is 48%, and toxicity proportion is 4%. The biomarker gate passes, the mean efficacy gate fails, and safety passes. The defined termination condition is met, so the rule recommends termination of this modeled program under the specified assumptions. This recommendation does not prove that the biological target cannot affect disease; it is the action specified for these model predictions and criteria.
The distinction follows from the rule’s logic: termination requires both biomarker suppression of at least 70% and mean efficacy reduction below 25%. In result set A, \(55\%<70\%\), so the conjunction is false. In result set B, \(76\%\geq70\%\) and \(19\%<25\%\), so both conditions hold. A rule should not be broadened after seeing outputs merely to make a preferred action seem justified.
Justify the Logic and Check the Analysis
Decision rules commonly use either a conjunctive gate or a scoring approach. With conjunctive gates, advancement requires every required criterion to pass. With a scoring approach, metrics contribute to a combined score, so one favorable result can offset another unfavorable result unless additional constraints are imposed. Neither form is universally correct; the choice depends on whether the criteria are all mandatory or can reasonably be traded against one another.
For mandatory criteria, the conjunctive form follows directly from their meaning. If advancement requires efficacy, safety, and biomarker conditions simultaneously, the logical event for advancement is their intersection:
If any one condition is false, the intersection is false. This is why a weighted average can misrepresent a hard safety limit: a high efficacy score might raise the average even when the safety condition is violated. A composite score can still be useful for ranking options after mandatory gates have been applied, but its weights and interpretation need their own justification.
Before communicating a recommendation, check that every summarized output matches the rule’s definition: same time point, endpoint direction, population, denominator, and regimen. Inspect the underlying simulations for missing or non-finite outputs and confirm that each profile was classified consistently against the responder and toxicity definitions. A proportion calculated from the wrong denominator, or a score whose direction was reversed, can change the action even when the arithmetic is correct.
- Use fixed thresholds: record criteria before evaluating candidate results. Changing a threshold after seeing the result makes the rule difficult to interpret as a decision standard.
- Keep units and time points visible: “29% reduction at week 12” is interpretable; “29%” without the endpoint and time point is not.
- Separate model failure from criterion failure: invalid inputs, incomplete outputs, or a mismatched simulation protocol call the analysis into question. They should not be silently counted as a program’s biological failure.
- Report close calls plainly: a result just above a chosen threshold passes the numerical rule, but the threshold does not make the prediction precise or certain.
Common Mistake and Limitations
A frequent mistake is to write, “The model predicts a 6% chance of serious toxicity,” solely because 6 of 100 simulated profiles had toxicity. The arithmetic is \(6/100=6\%\), but the interpretation is too strong unless the model, virtual population, and weighting support that clinical probability interpretation. A safer statement is: “Six of the 100 equally weighted simulated profiles met the specified toxicity definition under this model and regimen.” This precisely describes the output without implying a guarantee about patients.
Likewise, a model-based rule is not a substitute for judgment about the decision context. Its thresholds may be sensitive to endpoint definitions, the simulated population, model structure, and the range of regimens considered. The rule cannot establish that an unmodeled outcome is acceptable, and passing every gate does not guarantee clinical success. Document the assumptions and the action mapping alongside the outputs so that the recommendation remains interpretable when the model or decision context changes.
Check Your Understanding
Answer each question using the definitions and hypothetical rule in this tutorial. Brief solutions follow the questions.
- For 100 equally weighted profiles, 63 meet the efficacy-responder definition. What is the model-based responder proportion?
- A dose meets efficacy and biomarker thresholds but has a 12% toxicity proportion when the safety ceiling is 10%. Does it pass a conjunctive rule requiring all three criteria?
- Why should the rule specify the endpoint time point as well as the threshold?
- A result has 17% mean efficacy reduction and 74% median biomarker suppression. Under the example termination condition, which requires biomarker suppression of at least 70% and efficacy reduction below 25%, what action is indicated?
- What is the careful interpretation of 4 toxicity-positive profiles among 100 equally weighted simulated profiles?
Solutions. (1) \(63/100=0.63=63\%\). (2) No. The safety criterion fails because \(12\%>10\%\), so the conjunction is false even though the other two criteria pass. (3) The prediction and decision threshold refer to a specific point in the disease course; changing the time point can change the value and therefore the decision. (4) The stated termination condition is met: \(74\%\geq70\%\) and \(17\%<25\%\), so the rule recommends termination of the modeled program under its specified assumptions. (5) State that 4 of the 100 simulated profiles met the defined toxicity criterion under the specified model and regimen; do not automatically call this a guaranteed 4% clinical risk.