Decision-Relevant Acceptance Criteria for PBPK

Why acceptance criteria must be decision-relevant

A PBPK prediction can be numerically close to an observed exposure and still be unsuitable for a particular decision. The relevant question is not simply whether the model is “accurate,” but whether its predictions are sufficiently reliable for the decision they are intended to inform. A tolerable error for screening a low-consequence scenario may be unacceptable when a small exposure difference could change a dose or safety decision.

Acceptance criteria make that judgment explicit before the prediction evidence is reviewed. They should identify the exposure endpoint, the clinical decision threshold, the exposure difference considered meaningful, and the largest prediction error that would still be acceptable for the intended use. The values are context-specific; there is no universal PBPK error percentage that establishes acceptability for every endpoint and decision.

These criteria are not conclusions about whether a model has passed. They are rules for evaluation, set before looking at the results being evaluated. They also do not replace mechanistic support, verification, or evidence that the model applies to the population and scenario of interest. Instead, they clarify what predictive performance would be adequate for a specified decision.

Specify the decision, endpoint, and error scale

Start by stating the decision in clinical terms. For example, a decision might depend on whether a maximum concentration exceeds a specified threshold, or whether exposure changes enough to warrant a dose adjustment. Then define the predicted quantity: an AUC, a maximum concentration, a trough concentration, or a ratio to a reference scenario. State units, time interval, population, regimen, and relevant assumptions so that the criterion has an unambiguous target.

For a positive exposure measure, prediction error is often most interpretable as a ratio. Let \(E_{\mathrm{pred}}\) denote predicted exposure and \(E_{\mathrm{obs}}\) the corresponding observed or reference exposure used for evaluation. Define the log prediction error as:

An error of zero means exact agreement. A positive error means the prediction is higher than the observed value; a negative error means it is lower. A prespecified symmetric tolerance on the log scale, \(|e|\leq \tau\), corresponds to a multiplicative range:

This symmetry is multiplicative, not additive: a prediction 20% higher and one 20% lower do not have exactly equal-magnitude log errors. If the intended tolerance is instead defined as a conventional percentage difference, specify the denominator and the asymmetric bounds explicitly. Do not use “within 20%” without saying how that percentage is calculated.

Connect error tolerance to a clinical threshold

Suppose a clinical decision changes at exposure \(T\), and the error tolerance is \(\tau\) on the log scale. If the prediction is \(E_{\mathrm{pred}}\) and the log error is no greater than \(\tau\) in magnitude, then the exposure compatible with that tolerance lies between \(E_{\mathrm{pred}}\exp(-\tau)\) and \(E_{\mathrm{pred}}\exp(\tau)\). A prediction alone is robustly below the decision threshold only if the upper end of this error range is below \(T\). It is robustly above the threshold only if the lower end is above \(T\).

If neither inequality holds, the tolerated error range overlaps the threshold. That does not prove the prediction is wrong. It means the prediction, at the specified tolerance, cannot reliably distinguish which side of the threshold the exposure lies on. This gives a practical way to set a threshold guard band: predictions close enough to \(T\) that their permitted error could cross \(T\) require a different decision rule, a tighter error tolerance, or additional evidence.

The justification follows directly from the error definition. If \(|e|\leq\tau\), then \(-\tau\leq\log(E_{\mathrm{pred}}/E_{\mathrm{obs}})\leq\tau\). Exponentiating gives \(\exp(-\tau)\leq E_{\mathrm{pred}}/E_{\mathrm{obs}}\leq\exp(\tau)\). Rearranging yields:

Thus, requiring the entire tolerated range to lie on one side of \(T\) is a sufficient condition for the decision classification to be unchanged by any error within the prespecified bound. It is a design criterion, not a claim that all actual errors will fall within that bound.

Make the rule operational before evaluation

A usable criterion should say what will be calculated and what constitutes passing. For example, a team might prespecify that the absolute log error for a particular endpoint must not exceed \(\log(1.25)\), and that the tolerated range around a prediction must not cross a specified decision threshold for scenarios in which the model is being used to classify that threshold. If several endpoints or subgroups matter, each should have an explicit rule rather than being hidden in an overall average.

When criteria apply to multiple predictions, state the aggregation rule as well. A limit on the mean error alone can conceal large errors in opposite directions that cancel. A rule may therefore include a bias limit and a separate bound on individual or subgroup errors. If a proportion of predictions must fall within a tolerance, specify the denominator, the required proportion, and the treatment of missing or unevaluable cases. These details are part of the acceptance criterion, not administrative afterthoughts.

Diagnostics, alternatives, and limitations

Several checks help reveal whether an acceptance rule is poorly specified or poorly matched to its purpose:

• Check the endpoint and units. A threshold defined for a maximum concentration cannot be applied directly to an AUC prediction, and a ratio is uninterpretable unless its numerator and denominator are clear.
• Check the decision margin. If the tolerated error is as large as, or larger than, the smallest clinically meaningful change, the model may be unable to distinguish a decision-relevant change from prediction error.
• Check threshold proximity. If many predictions fall in the guard band, the criterion may be too permissive for threshold classification, or the decision may require additional information.
• Check for concealed subgroup failures. An acceptable pooled average can coexist with unacceptable errors in a relevant population or scenario. Define subgroup rules when the decision depends on them.
• Check whether the error measure fits the endpoint. Relative or log-scale errors are often natural for positive exposures; an absolute difference may be more interpretable when the clinical threshold is expressed in fixed units.

A global fold-error rule is easy to communicate and compare across exposure magnitudes, but it may ignore whether a prediction lies close to a decision threshold. A threshold-focused rule directly addresses the decision, but it applies only where that threshold and decision are meaningful. A fixed absolute error can be clinically intuitive in units such as \(\mathrm{mg/L}\), yet the same absolute error may represent very different relative errors at low and high exposures. These approaches are alternatives to select deliberately, not interchangeable descriptions of one criterion.

A further limitation is that a tolerated error bound is a decision choice, not an estimated guarantee. It does not by itself quantify the probability that a prediction will meet the bound. Nor does a prediction interval automatically become an acceptance limit: the interval describes uncertainty in the prediction under its assumptions, whereas the tolerance defines what error the decision can accept. When an uncertainty interval crosses a threshold, that is a reason to acknowledge limited decision resolution, not to alter the threshold after seeing the result.

Common mistake