Setting Credibility Goals Before Reviewing Evidence

Why Set Credibility Goals in Advance?

A PBPK model is not credible in the abstract. Its credibility is judged in relation to a particular use: a prediction, in a specified population and set of conditions, intended to inform a decision. Before examining the available evidence, the modeler and decision makers therefore need to agree on what predictions require support and what evidence would be persuasive for those predictions.

This is the purpose of prospective credibility goals. The context-of-use statement identifies the decision claim; the goals make the credibility question specific enough to guide evidence review. Without them, teams may focus on familiar outputs rather than decision-relevant ones, overlook important scenarios, or unconsciously change expectations after seeing favorable or unfavorable results. The earlier material on context-of-use statements and the risk-informed credibility matrix provides the basis for this step: goals should reflect both the intended use and the consequences of being wrong.

Definition: A prospective credibility goal is a statement, made before reviewing the evidence, of (1) the model predictions and scenarios that must be supported for the intended use and (2) the qualities of evidence that would be persuasive in judging that support. A goal describes what the review needs to establish; it is not itself evidence, a credibility conclusion, or necessarily a numerical pass-or-fail rule.

From Decision Claim to Specific Goals

A useful goal begins with the decision, not with the model’s available outputs. Work backward: identify which decision could change, determine what prediction informs it, specify the conditions under which that prediction is needed, and then consider what evidence could meaningfully test the prediction. This sequence prevents a common mismatch: collecting evidence about a convenient endpoint while the decision depends on another.

\text{decision claim} \longrightarrow \text{decision-relevant prediction} \longrightarrow \text{population and scenario set} \longrightarrow \text{prospective evidence goals}

For example, if a model is intended to predict whether a concomitant drug changes exposure enough to affect dosing, “predicts pharmacokinetics” is too broad. The goal should identify the exposure quantity or quantities that inform dosing, the relevant victim-drug and perpetrator conditions, and the range of scenarios in which the prediction is meant to apply. The goal should also say what would make evidence informative: for instance, whether it directly tests the predicted interaction, covers the conditions of use, and is sufficiently reliable and interpretable.

A compact way to organize the planning is to record four elements:

Use: the decision and the claim the model is meant to inform.
Predictions: the outputs needed for that decision, including the population, time period, and units where relevant.
Scenarios: the conditions or subgroups within the intended use for which support is needed.
Evidence qualities: the attributes that would make evidence probative, such as relevance to the target prediction, suitable coverage, adequate measurement, and an interpretable comparison.

These elements are connected. If a dosing decision depends on a lower exposure bound in a subgroup, a goal focused only on the typical exposure for the overall population is incomplete. Likewise, a goal that names a prediction but leaves the intended scenarios unspecified cannot tell reviewers whether the eventual evidence speaks to the intended use.

Worked Examples: Making Goals Concrete

Worked Example: Pediatric Exposure Extrapolation

Hypothetical use. A team plans to use a PBPK model to support a proposed dose in children aged 2 to 6 years when direct pharmacokinetic data are limited. The decision concerns whether the proposed dose is expected to produce exposure comparable to that associated with an effective and tolerated adult regimen.

Step 1: Identify the prediction. The decision is about exposure, not simply whether the model can reproduce adult concentration-time profiles. The team therefore names the relevant exposure measures, such as AUC over the dosing interval and peak concentration, with units and the dosing interval specified. If a trough concentration informs the decision, it belongs in the goal too; an output should not be omitted merely because it is less convenient to evaluate.

Step 2: Identify the scenarios. The population is children aged 2 to 6 years, not “pediatrics” without qualification. The proposed dosing regimen and the intended range of body sizes within that age group are included. If the decision will be applied to children receiving a common concomitant medication, that condition is included as a scenario rather than silently assumed to be covered by the base case.

Step 3: State persuasive evidence qualities. Evidence would be more persuasive if it bears on the exposure quantities and age range in the goal, allows a meaningful comparison with observed or otherwise justified reference exposures, and represents more than one convenient point in the intended range. Adult agreement alone may support part of the model’s use but does not directly establish predictive performance for 2- to 6-year-olds. The prospective goal is therefore explicit about what the review must address without claiming that any particular evidence has already met it.

Worked Example: Predicting a Drug Interaction

Hypothetical use. A model will inform whether a dose adjustment is warranted when a substrate drug is administered with a strong inhibitor. For the intended decision, a hypothetical model predicts that the AUC ratio, inhibitor condition divided by control condition, is 2.4. This number is a planned prediction in the example, not an observed result.

Step 1: Make the prediction interpretable. The goal identifies the AUC ratio and specifies the inhibitor, substrate, dosing conditions, and population relevant to the decision. “Predicts the interaction” alone would not say whether the target is the AUC ratio, peak concentration, or some other quantity.

Step 2: Bound the scenarios. The goal states whether the claim concerns the named strong inhibitor only or is meant to cover a broader class of inhibitors. It also states whether the model is intended for a particular formulation, dosing schedule, or patient group. A result for one inhibitor should not automatically be described as support for all inhibitors.

Step 3: Specify persuasive evidence. Before reviewing results, the team can state that evidence most directly relevant to the goal would test the same interaction and exposure measure under conditions sufficiently close to the intended use. Evidence should be interpretable as a comparison between inhibitor and control conditions; a single concentration measurement at an unrelated time point would not, by itself, establish support for the AUC-ratio prediction. The goal does not decide whether 2.4 is acceptable for dosing—that is a separate decision question.

Worked Example: Food-Effect Prediction Across Formulations

Hypothetical use. A PBPK analysis is intended to predict whether taking a proposed oral formulation with food materially changes systemic exposure. In a hypothetical planning exercise, the team expects to examine AUC and peak concentration ratios for fed versus fasted administration.

Step 1: Name the outputs. The prospective goal includes both the AUC ratio and the peak-concentration ratio because the planned decision considers overall exposure as well as peak-related concerns. It defines the comparison as fed administration divided by fasted administration, so the direction of each ratio is unambiguous.

Step 2: Specify what is covered. The goal identifies the proposed formulation and the food conditions relevant to its use. If the claim is intended to cover only a particular meal condition, it should not be broadened later to all meals. If different formulations are in scope, each must be named rather than treated as interchangeable.

Step 3: Define persuasive evidence qualities. Evidence would be persuasive to the stated goal when it can inform both named ratios and the stated comparison, and when its conditions allow the fed–fasted contrast to be interpreted. Evidence that addresses AUC but not peak concentration might be informative but would leave part of the goal unresolved. This illustrates why a goal should specify all decision-relevant outputs before the evidence review begins.

What Makes Evidence Persuasive?

“Persuasive” does not mean “favorable.” It means that the evidence is capable of changing a careful reviewer’s view about the particular prediction and scenarios in the goal. Several qualities are useful to specify in advance, with their importance depending on the context of use:

Relevance: Does the evidence address the prediction, population, and conditions named in the goal?
Coverage: Does it include the scenarios or range for which support is required, rather than only a narrow, convenient case?
Interpretability: Can the comparison be understood in relation to the prediction, including the outcome measured and the conditions under which it was measured?
Reliability: Are the observations and analyses sufficiently dependable for the question being asked?
Independence or diversity: Does the evidence add information beyond what was used to develop the model, or does it provide a meaningfully different test? Repeated use of the same information is not automatically multiple independent confirmations.

These are prompts for setting goals, not a universal checklist with equal weight in every application. A high-consequence use may justify goals that call for broader coverage or stronger independent testing than a lower-consequence use. Conversely, a large evidence set can still be weak for a specific goal if it does not address the prediction that matters.

Justification: Why Goals Must Precede Evidence Review

The logic can be stated directly. Let the intended decision depend on a prediction \(Y\) under scenarios \(S\). The evidence review is meant to inform whether the model can support that prediction for those scenarios. If \(Y\) or \(S\) is chosen only after seeing the evidence, the review can inadvertently select whichever output or scenario looks most favorable. Fixing the goal first makes the question stable:

\text{goal fixed before review} \ \Longrightarrow\ \text{evidence judged against a specified prediction and scenario set}

This does not eliminate scientific judgment, and it does not guarantee that the evidence will be sufficient. It makes the judgment more transparent by separating two questions: “What did we need evidence to address?” and “What did the evidence actually show?” Any later change to the goal should be documented and justified, rather than quietly substituted for the original target.

Diagnostics, Alternatives, and Limitations

Warning signs of poorly set goals include broad phrases such as “validate the model,” predictions without a defined population or conditions, and evidence requirements phrased as “use whatever data are available.” Another warning sign is a goal that addresses only the model’s easiest-to-predict output while omitting an output that could change the decision. A practical diagnostic is to ask whether a reviewer who has not seen the results could identify, from the goal alone, what prediction is being assessed, for whom, under what conditions, and what kinds of evidence would be informative.

The main alternative is to begin by inventorying available data and then formulate the credibility question around those data. That approach can efficiently describe what is known, but it risks making evidence availability determine the scope of the claim. Prospective goals reverse the order: first define what the intended use requires, then judge how well available evidence addresses it. Evidence inventories remain useful, but they answer a different question.

Goals also have limits. They cannot make weak evidence strong, remove uncertainty from a prediction, or guarantee that a model is suitable for a decision. They do not replace the later task of specifying decision-relevant acceptance criteria or judging the resulting evidence. Nor do they require that every imaginable scenario be covered: the goal should bound the intended use in a way consistent with its consequences and practical constraints. When the model’s intended use changes, its goals may need to change too, but the change should be explicit.

Common Mistakes

Writing “validate the model” as the goal. This does not name the prediction, population, or intended scenarios. Replace it with specific prediction and use statements.
Setting goals after seeing results. Choosing only outputs with good agreement makes the review vulnerable to selective emphasis. Record the goals before examining the evidence and explain any later revision.
Treating all evidence as equally relevant. Evidence can be reliable but still address a different population, condition, or outcome from the target prediction.
Confusing a credibility goal with a pass-or-fail threshold. A goal describes what needs support and what evidence could be informative; a quantitative decision rule is a distinct element of a qualification plan.
Overstating what one scenario supports. Support in one condition does not automatically establish support across a broader class of conditions.

Key takeaway: Before reviewing evidence, state the decision-relevant PBPK predictions, the population and scenarios for which support is needed, and the evidence qualities that would make a review informative. Keep those goals distinct from both the evidence itself and any later pass-or-fail criteria.

Check Your Understanding

Answer each question, then compare with the solutions below.

A model will inform a dose decision in adults with moderate renal impairment. What elements should a prospective credibility goal specify?
Why is “show that the model is accurate” an insufficient credibility goal?
A model predicts both AUC and peak concentration, but the intended decision depends only on peak concentration. What should guide which output is named in the goal?
A team sees favorable results for one inhibitor and then narrows its original goal from several inhibitors to that one. What is the concern, and how should the change be handled?

Solutions. (1) Specify the decision, the exposure prediction or predictions that inform it, the adult population and moderate-impairment conditions, and the evidence qualities needed to assess those predictions. (2) It does not identify the outcome, population, or conditions, so a reviewer cannot tell what evidence would address it. (3) Decision relevance should guide the goal; peak concentration should be specified if that is the output that informs the decision, while other outputs need not be made central without a reason. (4) Narrowing the goal after seeing favorable evidence risks selective reporting. The team should retain the original goal as the reference and document and justify any revised scope explicitly.