Why Credibility Conclusions Need More Than “Pass” or “Fail”
A PBPK model can be informative for one decision and inadequate for another. A model that predicts a moderate interaction adequately in adults with normal organ function may not support a high-consequence decision in a population or dosing situation with little relevant evidence. Credibility is therefore a judgment about a model for a defined use, not a permanent label attached to the model.
A useful conclusion does more than summarize favorable evidence. It states which prediction and decision are supported, which claims are restricted, what important evidence is missing, and what risk remains if the prediction is used. The earlier material on context-of-use statements and decision robustness provides the framing: here the task is to turn that framing into a graded, decision-facing conclusion.
The conclusion should be proportionate to the decision. A prediction used to prioritize further study can tolerate more unresolved uncertainty than one used as the sole basis for a consequential dose or safety decision. That does not mean that low-consequence use requires no evidence; it means that the consequences of an error help determine what evidence and controls are adequate.
A Practical Grading Structure
There is no single mandatory set of credibility grades for every PBPK application. The following three-level structure is a practical way to make the extent of support explicit. The grade is always tied to the stated context of use and its decision criteria.
| Conclusion level | What it means | What the conclusion must still state |
|---|---|---|
| Supported for the defined use | Relevant evidence and mechanistic support are adequate for the particular prediction and decision, and important gaps are not expected to change the decision materially. | The exact use, population, conditions, and decision covered; remaining limitations and residual risk. |
| Supported with restrictions | The evidence supports a narrower use, or reliance is reasonable only with stated controls, additional evidence, or decision safeguards. | The boundary of the supported claim, conditions that must hold, and what the prediction cannot establish. |
| Not currently supported for the proposed use | Important evidence is missing, discordant, or insufficiently applicable, or the decision consequence is too large for the unresolved uncertainty. | Why the proposed use is not supported, what evidence or model changes could address the gap, and whether a lower-stakes use remains reasonable. |
These levels are not scores and should not be averaged across evidence categories. Strong clinical fit in one setting does not automatically compensate for a major mechanistic gap in the population of interest. Nor does an overall grade remove the need to identify the specific evidence that supports or weakens the claim.
A defensible conclusion links four elements: the prediction and decision; the evidence that bears on them; the limitations or gaps; and the consequences and controls associated with relying on the result. The reasoning is clearer when each element is stated separately before the final grade is chosen.
From Evidence to a Graded Conclusion
The conclusion should follow from the decision question rather than from a generic checklist. Begin with the intended use and its decision threshold. Then judge whether the evidence is relevant to that use, whether the remaining uncertainty could alter the decision, and whether restrictions or controls can reduce the risk to an acceptable level. A prediction interval can help describe uncertainty, but it does not by itself establish credibility: its relevance depends on how it was constructed and whether it covers the uncertainty important to the decision.
Name the prediction, population, conditions, and decision it is intended to inform. A conclusion cannot be interpreted independently of this scope.
State which clinical, mechanistic, and technical evidence is relevant to the prediction. Explain important evidence that is absent, discordant, or outside the intended domain.
Ask whether plausible alternative predictions or unresolved gaps could cross the decision threshold or change the action. If so, say what control or additional evidence is needed.
State whether the use is supported, supported with restrictions, or not currently supported. Make the boundary and residual risk explicit.
A simple threshold argument illustrates why the decision context matters. Suppose an action is triggered when a predicted exposure ratio \(R\) is at least \(T\). If a relevant predictive interval has upper endpoint \(U\), then \(U<T\) indicates that the interval does not reach the trigger under the stated uncertainty analysis. If \(U\geq T\), the analysis cannot rule out a trigger-level value. This comparison is meaningful only if the interval represents the uncertainties relevant to the use and the threshold is the applicable decision criterion; it does not prove that the model is credible or not credible by itself.
The logic is direct: if every value in the interval is below \(T\), none of those represented values reaches the action threshold. If its upper endpoint is equal to or above \(T\), at least part of the represented range reaches that threshold. Neither statement assigns a probability to the model being wrong, and neither addresses uncertainty omitted from the interval.
Worked Example: A Drug-Interaction Prediction Near an Action Threshold
Hypothetical scenario. A PBPK analysis is intended to determine whether a co-administered drug could raise exposure enough to trigger an additional clinical management step. The decision trigger is an exposure ratio of \(T=2.0\). The model predicts an AUC ratio of 1.46, with a 90% predictive interval from 0.98 to 2.05. Relevant interaction data support the main metabolic pathway in the studied adult population, but evidence is sparse for a subgroup with a potentially important difference in pathway activity.
Decision comparison. The interval’s upper endpoint is \(U=2.05\). Substituting into the threshold comparison gives \(2.05\geq2.0\). Thus, the interval reaches the action threshold; the analysis cannot rule out an exposure ratio at or above 2.0. The point prediction, 1.46, is below the threshold, but using that point estimate alone would conceal the decision-relevant uncertainty.
Graded conclusion. The model is supported with restrictions for informing the decision in the adult population represented by the relevant evidence, but it does not establish that the trigger can be excluded. Use in the subgroup with sparse pathway evidence is not supported by this analysis alone. A suitable restriction is to retain the clinical management step, or obtain additional evidence, rather than treating the point prediction as proof that no action is needed.
Residual risk. The remaining risk is that actual exposure in the intended use could reach the trigger, particularly if the subgroup’s pathway activity differs from the modeled assumptions. The 90% predictive interval describes a range under the analysis; it is not a 90% probability that the model is correct or that the true ratio lies in this interval.
Worked Examples of Different Grades
Worked Example: Support for a Narrowly Defined Prediction
Hypothetical scenario. A model is proposed to predict AUC for a specified oral drug in adults under a defined dosing condition. For three relevant clinical comparisons, the observed AUC values are 10, 25, and 40 arbitrary concentration-time units, while the model predictions are 9.2, 26.5, and 47.2 in the same units. The pre-specified decision criterion for this illustration is that each prediction be within 25% of its corresponding observed value. Assume the comparisons are relevant to the defined use and there is no known systematic failure in the model configuration.
Check the criterion. For each comparison, calculate relative prediction error as \((\text{predicted}-\text{observed})/\text{observed}\times100\%\). For the first comparison, \((9.2-10)/10\times100\%=-8.0\%\). For the second, \((26.5-25)/25\times100\%=6.0\%\). For the third, \((47.2-40)/40\times100\%=18.0\%\). Each absolute error is at most 25%: \(8.0\%\leq25\%\), \(6.0\%\leq25\%\), and \(18.0\%\leq25\%\).
Graded conclusion. These hypothetical results support the defined prediction use against the stated criterion. The conclusion should remain narrow: three comparisons do not establish support for different populations, doses, or conditions that were not represented. Residual risk includes untested conditions and possible limitations not exposed by these comparisons. The calculation shows consistency with this criterion; it does not prove that all future predictions will meet it.
Worked Example: A High-Consequence Use With a Major Evidence Gap
Hypothetical scenario. A PBPK model predicts an exposure ratio of 1.60 for a proposed high-consequence dosing decision. A predictive interval is 0.70 to 3.40, and the decision trigger is 2.00. The intended population has a physiological feature that could alter a clearance pathway, but that pathway has not been adequately characterized for the population. No independent clinical evidence in that population is available.
Decision comparison. Here, \(U=3.40\) and \(T=2.00\), so \(3.40\geq2.00\). The interval spans values on both sides of the trigger: \(0.70<2.00<3.40\). Therefore, the prediction does not establish which side of the trigger is relevant for the decision. The wide interval also cannot compensate for a missing mechanism if that mechanism is not represented in the uncertainty analysis.
Graded conclusion. The proposed use is not currently supported as the sole basis for this decision. The reasons are the interval’s overlap with the trigger, the consequential nature of the decision, and the unresolved clearance-pathway gap. The model may still be useful for exploratory scenario analysis or prioritizing data collection, provided those uses are clearly distinguished from decision support.
Residual risk. If the model were used as the sole basis for action, the prediction could lead to a decision on the wrong side of the trigger. The size of that risk cannot be quantified from the interval alone, particularly because the missing pathway evidence may represent uncertainty not included in the interval. Relevant clinical or mechanistic evidence, or a justified restriction to a lower-stakes use, would be needed before reconsidering the proposed use.
Diagnostics, Alternatives, and Common Pitfalls
A conclusion may be too strong or too vague even when its supporting analysis is sound. Useful warning signs include a grade that is not tied to a named context of use; a limitation listed without explaining how it could affect the decision; an interval compared with a threshold without checking that it covers relevant uncertainties; and a favorable average that obscures a decision-relevant subgroup or discordant evidence. If a gap could plausibly change the action, the conclusion should address that possibility directly.
A binary pass/fail statement is concise, but it can hide meaningful differences between uses and does not naturally express restrictions or residual risk. A numerical score can aid comparison when its components and decision rules are defined in advance, but a single score may conceal a critical weakness and should not substitute for a reasoned conclusion. A purely narrative judgment can capture context, but without explicit scope, criteria, and gaps it may be difficult to interpret consistently. A graded, evidence-linked conclusion combines the useful features of these approaches while avoiding the claim that one label applies to every use.
- Pitfall: “The point prediction is below the threshold, so the decision trigger is excluded.” This ignores the uncertainty range. Compare the relevant range with the threshold and state whether it reaches or crosses it.
- Pitfall: “The model is validated, so it can be used for this population.” Evidence for one context does not automatically support another. State the population and conditions the evidence covers.
- Pitfall: “A 90% predictive interval means there is a 90% chance the model is correct.” It does not. Describe what the interval represents under the specified analysis, and identify important uncertainties it may not cover.
- Pitfall: “A limitation exists, therefore the model is unusable.” A limitation may justify a narrower use or a control rather than rejecting every use. Explain its decision relevance before assigning the grade.
Check Your Understanding
Answer each question, then compare with the brief solutions below.
- A prediction interval for an exposure ratio is 0.85 to 1.70. The action trigger is 1.50. Can the analysis rule out reaching the trigger? What comparison supports your answer?
- Why is “the model is credible” less useful than a statement of credibility for a specified context of use?
- A model meets its stated prediction criterion in the studied population, but the intended decision concerns an unstudied population with a potentially relevant physiological difference. Which grade is most defensible without further evidence?
- Does a 90% predictive interval mean that the model has a 90% probability of being correct? Explain.
- Give two elements that should be included in a residual-risk statement.
Solutions. (1) No. The upper endpoint \(U=1.70\) is at least the trigger \(T=1.50\), since \(1.70\geq1.50\); the interval reaches values at or above the trigger. (2) Credibility depends on the intended prediction, population, conditions, and decision, so a conclusion for one use need not apply to another. (3) Support with restrictions, or not currently supported for the proposed use if the gap is consequential; the evidence does not directly cover the intended population. (4) No. The interval is an uncertainty summary under its specified analysis, not a probability that the model is correct. (5) State the decision-relevant uncertainty that remains and the consequence if the prediction is wrong; also identify any restriction or control that reduces reliance risk.