What Confidence in a Predicted Ratio Means
A PBPK model may predict a DDI exposure ratio that appears clearly above or below a decision threshold. That numerical separation does not, by itself, establish that the prediction is dependable. Confidence concerns the strength and relevance of the evidence supporting the ratio for a particular use: a model may be well supported for one drug, endpoint, dose, or population and weakly supported for another.
For a victim drug, a common endpoint is the ratio of exposure with the perpetrator present to exposure without it. For example, the AUC ratio is \(R_{\mathrm{AUC}}=\mathrm{AUC}_{\mathrm{with}}/\mathrm{AUC}_{\mathrm{alone}}\). A Cmax ratio is defined analogously. The ratio is dimensionless, but its interpretation depends on the endpoint, regimen, population, and comparison specified in the model. Confidence in an AUC ratio does not automatically establish confidence in a Cmax ratio.
This assessment is distinct from classifying the predicted DDI magnitude. The same ratio can merit different confidence depending on the evidence behind it. Likewise, a ratio with strong support can still imply a clinically important exposure change, while a weakly supported ratio near a decision boundary should not be presented as conclusive. The introductory material on interpreting victim and perpetrator label implications helps specify whose exposure is being predicted; here the focus is whether the evidence supports that prediction in its intended context.
Four Evidence Questions
A useful assessment begins with four linked questions. They are not a scoring system: a strength in one area does not necessarily compensate for a critical weakness in another. Explain the evidence and its limits rather than assigning an unsupported label such as “high confidence.”
| Evidence area | Question to ask | Examples of a limitation |
|---|---|---|
| Validation coverage | Has the model been tested against relevant observations across the mechanisms, drugs, endpoints, and conditions that support this prediction? | Validation covers a different pathway or dose range, or only parent-drug AUC when the claim concerns Cmax. |
| Input-data quality | Are the compound and clinical inputs suitable, traceable, and sufficiently measured for this application? | Uncertain fraction metabolized, missing concentration data, or reliance on an unverified input. |
| Key-parameter uncertainty | Which uncertain parameters materially change the ratio, and how well are they constrained? | A sensitive enzyme contribution is poorly estimated, or plausible parameter values move the ratio across a decision threshold. |
| Scenario match | Does the simulation represent the population, regimen, route, duration, and endpoint relevant to the label question? | The simulated perpetrator exposure or dosing schedule differs from the intended clinical use. |
Validation coverage is about relevance, not simply the number of model checks. Evidence from a different compound can be informative when it tests the same mechanism and model components, but it does not directly establish performance for every drug or context. Similarly, a good fit to one observed concentration-time profile is not, on its own, validation of the predicted DDI ratio. The comparison should address the quantities and mechanisms that drive the intended prediction.
Why Distance from a Decision Boundary Matters
A small prediction error matters most when it could change the interpretation. This can be made explicit on the log-ratio scale, where multiplicative differences become additive. Let \(R_{\mathrm{pred}}\) be the predicted ratio and \(T\) a relevant decision threshold. Their separation is \(d=|\log(R_{\mathrm{pred}})-\log(T)|\). If there is a defensible bound \(e\) on the absolute log-scale prediction error for the relevant use, then \(e<d\) implies that an error no larger than \(e\) cannot carry the ratio across \(T\).
The justification is the triangle inequality. If \(R_{\mathrm{pred}}>T\), then \(\log(R_{\mathrm{pred}})-\log(T)=d\). A prediction error of at most \(e<d\) leaves \(\log(R_{\mathrm{true}})-\log(T)\ge d-e>0\), so \(R_{\mathrm{true}}>T\). The argument is symmetric when \(R_{\mathrm{pred}}<T\). This is a sensitivity-based robustness check, not a confidence interval or a claim that a suitable error bound is known. Without a defensible bound, distance from the boundary alone cannot establish confidence.
Worked Example: Validation Coverage Is Relevant, Not Merely Numerous
Hypothetical case. A model predicts a victim-drug AUC ratio of 2.30 for a specified inhibitor regimen. The decision question uses a 2.00-fold threshold. The team reviews five validation domains: (1) victim-drug disposition, (2) inhibitor exposure, (3) the affected enzyme mechanism, (4) AUC ratios under comparable inhibition, and (5) the target population. Four domains have relevant validation evidence; the target population does not.
A simple coverage inventory gives \(4/5=0.800\), or 80% of the listed domains represented. This is a description of the review checklist, not an 80% probability that the prediction is correct. The missing population domain may matter substantially if the population changes clearance or perpetrator exposure; the four covered items do not erase that limitation.
The predicted ratio is \(2.30/2.00=1.15\) times the threshold. Its log-scale distance is \(\log(2.30)-\log(2.00)=\log(1.15)=0.1398\). If a relevant error bound of 0.10 log units had been justified, then \(0.10<0.1398\), so that bound would not permit threshold crossing. But the checklist alone does not establish such an error bound. The proper conclusion is that validation is substantial in several relevant domains but incomplete for the target population; the ratio’s classification is not automatically secure.
Trace Input Quality and Influential Parameters
Input quality refers to the evidence used to specify the model, not only whether a value has been entered. Record the source, measurement conditions, units, and uncertainty for inputs such as clearance pathways, enzyme or transporter contributions, binding, permeability, and perpetrator concentration profiles. A precise-looking parameter can remain weakly supported if it was inferred from indirect evidence or measured under conditions unlike the simulated clinical setting.
Parameter uncertainty matters most when it affects the endpoint materially. A parameter can be uncertain but practically unimportant for a particular ratio; another can be estimated fairly precisely yet dominate the prediction. Use sensitivity analyses or parameter variation to ask whether plausible alternatives change the ratio enough to alter its interpretation. This complements propagation of PBPK uncertainty: the present assessment asks whether that uncertainty is understood and relevant to the label use, rather than supplying a new interval-based decision procedure.
Worked Example: An Uncertain Enzyme Contribution
Hypothetical case. A victim drug is predicted to have an AUC ratio of 2.20 under a perpetrator regimen, with a 2.00-fold threshold used for this example. The fraction of baseline clearance attributed to the affected enzyme, \(f_m\), is uncertain. A sensitivity analysis gives ratios of 1.85, 2.20, and 2.55 at \(f_m=0.40\), 0.60, and 0.80, respectively. Assume all other inputs are held fixed in these runs.
Across the examined range, the ratio changes by \(2.55-1.85=0.70\), and the predictions range from below to above the threshold. The central prediction exceeds the threshold by \(2.20-2.00=0.20\), but the parameter sensitivity shows that plausible \(f_m\) values could change the threshold classification. Because the ratio increases across these runs as \(f_m\) increases, the affected pathway contribution is decision-relevant in this model.
The result does not show that the true ratio is between 1.85 and 2.55 with a stated probability. It is a sensitivity range for the specified input values, with other assumptions fixed. Confidence would be strengthened by better evidence for \(f_m\), or by evidence that the range considered is not plausible for the intended population. Until then, the threshold conclusion should be qualified.
Check Whether the Simulated Scenario Matches the Label Question
A model can be internally consistent and still answer the wrong clinical question. Compare the simulated conditions with the intended labeling context: victim and perpetrator roles, dose and route, dosing interval, treatment duration, timing of the interaction, relevant patient population, and endpoint. For a label statement about a particular regimen, a simulation at a different perpetrator dose is not automatically transferable. For a claim about Cmax, a convincing AUC prediction alone is insufficient.
When conditions differ, describe the mismatch and why it could matter. For example, a shorter perpetrator treatment duration may not produce the same enzyme inhibition or induction as the intended regimen. A population with different organ function may have different baseline exposure or pathway contributions. A mismatch does not prove the prediction is wrong, but it weakens direct support unless the model or evidence justifies transfer to the target context.
Worked Example: Strong Model Evidence, Wrong Regimen for the Question
Hypothetical case. A PBPK model predicts a victim-drug AUC ratio of 1.70 when a perpetrator is given at 100 mg once daily for 7 days. The proposed label question concerns 200 mg once daily for 14 days. The model has relevant validation for the enzyme mechanism and the victim drug, but the available validation used the lower perpetrator dose and shorter duration.
The ratio of 1.70 describes the simulated 100-mg, 7-day regimen; it is not the prediction for the proposed 200-mg, 14-day regimen. Dose and duration could alter perpetrator concentrations or the degree of enzyme effect, so simple reuse of 1.70 is not justified. The favorable mechanistic validation supports the model’s general relevance, but the regimen mismatch limits confidence for the label question. A simulation of the target regimen, supported by appropriate input data, is needed before treating the ratio as directly applicable.
Integrated Assessment, Alternatives, and Failure Modes
A practical assessment can be written as a short evidence chain: state the ratio and endpoint; identify validation that directly supports its main mechanisms and conditions; identify influential uncertain inputs; compare the simulated scenario with the intended use; and state whether remaining uncertainty could change the label-relevant interpretation. This gives reviewers a reasoned account without turning unlike evidence dimensions into an arbitrary total score.
Name the victim, perpetrator, endpoint, ratio direction, regimen, and population; do not assess a context-free number.
Separate direct validation from mechanistic or indirect support, and identify missing domains relevant to this application.
Identify influential uncertain inputs and whether plausible alternatives could alter the threshold interpretation.
Explain what the evidence supports, what it does not establish, and which mismatch or uncertainty remains material.
This evidence-based assessment differs from three tempting alternatives. A threshold classification answers which category the point prediction falls into; it does not establish its reliability. A model-fit statistic or successful check may support one component but cannot substitute for relevant DDI validation. A single confidence score can conceal a critical gap, such as poor data for the pathway that drives the result. Predictive intervals provide a different way to represent uncertainty and are addressed separately; they do not remove the need to assess input quality, validation relevance, or scenario match.
Warning signs include a ratio reported without a precise regimen or endpoint, validation described only as “successful” without specifying what was tested, a sensitive parameter with no credible evidence source, and a simulation that differs from the intended label scenario. Another warning sign is a confident conclusion based only on the ratio being far from a threshold when model error has not been characterized. The remedy is not to invent a confidence percentage: state the gap, show its possible consequence, and narrow or qualify the claim.
Check Your Understanding
Answer each question, then compare with the brief solutions.
- A model is validated for AUC ratios under one inhibitor, but the intended conclusion concerns Cmax under a different inhibitor. What does the validation establish, and what remains uncertain?
- A predicted ratio is 1.80 and the relevant threshold is 2.00. Calculate the log-scale distance to the threshold.
- A sensitivity run changes a ratio from 1.60 to 1.75 when a parameter is varied over plausible values, and both remain below a threshold of 2.00. Does this prove the true ratio is below 2.00?
- Why should a ratio predicted for a 7-day perpetrator regimen not automatically be used for a 14-day label regimen?
Solutions. (1) It supports AUC prediction under the tested inhibitor conditions; it does not directly validate Cmax or transfer to the different inhibitor. (2) \( |\log(1.80)-\log(2.00)|=|\log(0.90)|=0.1054 \) log units. (3) No. It shows that this particular parameter variation, with other assumptions fixed, leaves the predictions below the threshold; other uncertainty or model error could still change the result. (4) Duration may change perpetrator exposure or the extent of the interaction, so the two regimens are not the same simulated scenario; target-regimen support is needed.