Why Discordant Evidence Needs Diagnosis
A PBPK prediction that disagrees with an observation is a signal to investigate, not a diagnosis by itself. A discrepancy may arise because the observed quantity was measured or processed incorrectly, because an input parameter does not represent the population or conditions of interest, because the model structure cannot represent an important process, or because evidence challenges the proposed mechanism. These explanations have different implications and call for different checks.
The purpose of diagnosis is to determine which explanations are consistent with the evidence and what additional evidence would distinguish them. A single mismatch rarely identifies its cause. For example, changing a clearance parameter may improve an AUC prediction, but that improvement does not establish that clearance was the source of the original mismatch. Independent information about clearance and the shape of the concentration-time profile matter too.
Start by confirming that the prediction and observation refer to the same quantity, units, population, dose, route, sampling interval, and conditions. This is particularly important when applying prediction evidence to its intended use: evidence can appear inconsistent when the comparison actually crosses populations or contexts. The earlier material on prediction evidence and context of use provides the basis for that comparison.
A Common Scale for Describing Disagreement
For positive quantities such as AUC, concentration, or clearance, a ratio is often easier to interpret than a raw difference. Let \(O\) be an observed value and \(P\) the corresponding model prediction. The ratio \(O/P\) is 1 when they agree; values above 1 mean the observation is larger, and values below 1 mean it is smaller. Its logarithm, \(\log(O/P)\), measures discrepancy symmetrically: a twofold excess gives \(\log(2)\), while a twofold deficit gives \(\log(1/2)=-\log(2)\).
When uncertainty is quantified on the log scale, a standardized discrepancy can help describe how large the mismatch is relative to that uncertainty. If the log observation and log prediction have standard errors \(SE_O\) and \(SE_P\), and their errors are independent, then the standard error of their difference is the square root of the sum of their variances:
This follows because the variance of the difference between independent quantities is the sum of their variances. If the errors are correlated, the correct variance includes their covariance: \(\operatorname{Var}(\log O-\log P)=SE_O^2+SE_P^2-2\operatorname{Cov}(\log O,\log P)\). The standardized discrepancy is a descriptive diagnostic, not a universal pass/fail rule or a substitute for pre-specified acceptance criteria. It also depends on the uncertainty estimates being credible.
Worked Example: Quantifying an AUC Discrepancy
Suppose a hypothetical PBPK simulation predicts an AUC of \(100\ \text{mg}\cdot\text{h}/\text{L}\), while a comparable clinical observation is \(125\ \text{mg}\cdot\text{h}/\text{L}\). Assume, for this diagnostic calculation, independent log-scale standard errors of 0.10 for both the observation and prediction.
The observation-to-prediction ratio is \(125/100=1.25\). Thus the observation is 25% higher than the prediction. On the log scale,
The standard error of the log discrepancy is
Therefore \(z_d=0.2231/0.1414=1.578\). This summarizes the discrepancy relative to the assumed uncertainty; it does not identify its cause. In particular, it does not show whether the observation, a parameter value, or the model structure is responsible.
Four Explanations to Investigate
The categories below are practical hypotheses, not mutually exclusive labels. A biased assay and a mismatched parameter could both contribute to one prediction error. Work from checks that are inexpensive and directly testable toward more consequential explanations, while keeping multiple explanations open when the evidence does not distinguish them.
| Candidate explanation | Evidence pattern to investigate | Useful diagnostic checks |
|---|---|---|
| Measurement or data problem | Discrepancy tracks an assay, batch, unit conversion, sampling, or data-processing feature | Review calibration, quality controls, raw data, units, sample handling, and protocol |
| Parameter mismatch | A relevant parameter differs in the target conditions, and a plausible change predicts the direction and size of the discrepancy | Check provenance, population relevance, independent parameter evidence, and sensitivity |
| Structural limitation | Systematic residual patterns remain despite defensible parameters, especially across time, dose, or conditions | Compare profiles and conditions; test whether an omitted or simplified process could explain the pattern |
| Mechanism challenged | High-quality evidence contradicts a necessary prediction of the proposed mechanism under conditions where it should operate | Check whether the mechanism is present, engaged, and measurable; seek independent or perturbation evidence |
Worked Diagnostics
Worked Example: Checking an Apparent Assay-Related Discrepancy
In a hypothetical study, the model predicts a concentration of \(10.0\ \text{mg}/\text{L}\) at a specified time, while the reported measurement is \(12.0\ \text{mg}/\text{L}\). A blinded analytical quality check indicates that the assay reads 1.20 times the true concentration in this range. Assume this bias factor applies to the reported sample and that the check is reliable.
If \(M\) is the measured value and \(b\) is the multiplicative assay bias, the bias-adjusted estimate is \(M/b\). Substitution gives
The adjusted value matches the prediction. This supports an assay-related explanation for this discrepancy, conditional on the bias check applying to the sample. It does not prove that every prediction from the model is accurate: the correction should be checked against calibration data and, where feasible, an independent method. If the assay bias were uncertain, that uncertainty should also be carried into the comparison.
Worked Example: Testing a Parameter-Mismatch Explanation
Consider a hypothetical, approximately linear dose range in which \(AUC \approx \text{Dose}/CL\). At a dose of \(480\ \text{mg}\), a model using clearance \(CL=6.00\ \text{L}/\text{h}\) predicts an AUC of
A comparable hypothetical observation is \(100\ \text{mg}\cdot\text{h}/\text{L}\). Under the same approximation, the apparent clearance corresponding to that AUC is
A lower clearance would increase AUC, so the direction is consistent with the observed excess. Suppose an independent, population-relevant clearance estimate is \(4.90\ \text{L}/\text{h}\). The corresponding prediction is \(480/4.90=97.96\ \text{mg}\cdot\text{h}/\text{L}\), close to the observation. This makes parameter mismatch plausible, but does not establish it: the approximation may not hold outside the stated dose range, and other parameters can affect AUC. Check the clearance estimate's provenance and uncertainty, and assess whether the revised value also improves other relevant predictions without contradicting independent evidence.
Worked Example: Recognizing a Profile-Level Structural Signal
In a hypothetical concentration-time comparison, predictions and observations at 0, 2, 4, 6, and 8 hours are, respectively, predicted: \(0, 8, 6, 4, 2\ \text{mg}/\text{L}\); observed: \(0, 4, 8, 6, 2\ \text{mg}/\text{L}\). Using the trapezoidal rule with 2-hour intervals, the predicted AUC from 0 to 8 hours is
The observed AUC over the same interval is
The totals agree, but the profiles do not: at 2 hours the observation is lower, and at 4 and 6 hours it is higher. AUC agreement therefore does not establish that the model reproduces the time course. The pattern motivates checking sampling times, absorption-rate information, and other processes that could shift concentration over time. If defensible parameter changes cannot reproduce the pattern across relevant conditions, structural limitation becomes a stronger candidate. The profile alone does not identify which process is missing.
Worked Example: Challenging a Proposed Mechanism
Suppose a hypothetical model predicts that a co-administered inhibitor will double the substrate drug's AUC, giving an AUC ratio of 2.00 relative to the control condition. A well-controlled, comparable experiment instead estimates a ratio of 1.05. Assume the inhibitor exposure is confirmed, the assay is qualified, and the experiment has adequate sampling to estimate AUC.
The log discrepancy between observed and predicted ratios is
This is a substantial disagreement in direction and magnitude relative to the model's predicted interaction. It warrants checking whether the inhibited pathway materially contributes to clearance in the tested conditions, whether inhibition was sufficient at the relevant site, and whether compensating pathways or other processes are represented. If independent evidence confirms pathway engagement and rules out measurement or parameter explanations, the result challenges the proposed mechanism or its implementation. It does not, by itself, show that the mechanism is absent in every population or context.
A Disciplined Diagnostic Sequence
The order below helps prevent an adjustable parameter from being used to conceal a data problem or structural mismatch. It is a reasoning sequence, not a requirement to stop at the first plausible explanation.
Check the quantity, units, population, dose, route, conditions, and time window. Confirm that the evidence is relevant to the prediction's intended use.
Review assay performance, data handling, protocol deviations, input data, and numerical implementation. A measurement or processing issue should be supported by a specific check, not inferred merely because it would explain the mismatch.
Identify parameters that could influence the discrepant output. Check whether their evidence is applicable and whether plausible changes move the prediction in the required direction and by a credible amount.
Look beyond a single summary statistic. Residuals that vary systematically with time, dose, or population may be more informative than an overall mean or AUC alone.
Ask whether the model can represent the process needed to explain the pattern, and whether the evidence directly challenges a necessary mechanistic prediction. State what remains uncertain.
A useful diagnostic record links each explanation to evidence that could support or weaken it. For example: “A clearance mismatch is plausible because an independent estimate in the target population is lower, and this change predicts higher AUC; it is not yet established because the time-course residuals remain.” This is more informative than either declaring the model invalid or quietly tuning a parameter until one output matches.
Failure Modes and Limits of Diagnosis
- Explaining the mismatch by assumption. Saying “the assay must be wrong” is not a diagnosis. Identify a calibration, handling, or processing check that can test the claim.
- Tuning a parameter to fit one endpoint. A revised value may improve AUC while worsening peak concentration or another population prediction. Evaluate the consequences across relevant evidence.
- Treating uncertainty as a cause. Wide uncertainty can make a discrepancy less informative, but it does not explain why the central prediction differs from the observation.
- Calling any persistent mismatch a failed mechanism. First check whether the comparison is valid, the observation reliable, and relevant parameters adequately supported. A mismatch may instead expose a structural limitation or unrepresented condition.
- Relying on one summary measure. Similar AUCs can conceal different concentration-time profiles, as the worked example shows. Use evidence at a resolution suited to the model claim.
These categories organize investigation; they do not guarantee that a unique explanation can be identified. Several explanations may remain consistent with the available evidence, or the evidence may be insufficient to distinguish them. The important result of diagnosis is a transparent account of what the evidence supports, what it rules out, and what targeted check would be most informative—not an unsupported single-cause verdict.
Check Your Understanding
Answer each question, then compare with the solutions below.
- A predicted AUC is \(50\ \text{mg}\cdot\text{h}/\text{L}\), and a comparable observation is \(40\ \text{mg}\cdot\text{h}/\text{L}\). What is the observation-to-prediction ratio, and what does it mean?
- Why does improving an AUC prediction by changing clearance not, by itself, establish a clearance mismatch?
- A model and observations have similar AUCs but show consistently early predicted peaks and later observed peaks. Name two diagnostic directions to investigate.
- What additional evidence would make a discrepancy more persuasive as a challenge to a proposed mechanism rather than a measurement problem?
Solutions. 1. The ratio is \(40/50=0.800\). The observed AUC is 20% lower than the prediction; the ratio alone does not identify the cause. 2. Multiple parameters or structural features can influence AUC, and adjusting a parameter to fit an endpoint is not independent evidence that its original value was wrong. Check parameter provenance and other predictions. 3. Check sampling and time alignment, and investigate absorption or distribution parameters and whether the model structure can represent the observed timing. A systematic profile pattern is more informative than AUC alone but does not uniquely identify a cause. 4. Confirm comparable conditions, reliable measurements, and sufficient engagement of the mechanism; then seek independent evidence that the mechanism should operate and that plausible parameter or structural explanations do not account for the result.