From Conflicting Evidence to a Defensible Input
Even after measurements have been checked for comparability and their disagreements investigated, more than one plausible value may remain. The modeler must then decide whether the evidence supports a single input or whether the conflict should remain visible in the model analysis. This is a decision about fitness for the intended use, not simply about which paper is newest or which estimate is numerically central.
Begin by defining the target: the parameter, the population or system to which it applies, and the conditions under which it is needed. For example, “fraction unbound in plasma” is not yet a complete target if the model represents a particular species, age group, disease state, or assay condition. A value can be measured carefully and still be a poor choice if it does not describe the modeled target.
Three questions organize the decision. Is the evidence relevant to the modeled population and conditions? Is the measurement method valid for the quantity and range at issue? Is the evidence sufficiently well supported and reported to justify using it? These questions overlap, but they should not be collapsed into a vague judgment that one source “looks better.”
Separate Eligibility From Ranking
A useful rule has two stages. First, apply eligibility gates: minimum requirements that a candidate must meet before it can be selected. Examples include a measurement of the intended quantity, a sufficiently documented method, and assay performance that meets criteria set in advance. A candidate that fails a critical validity requirement should not recover eligibility merely because it is highly relevant or widely cited.
Second, rank the eligible candidates using explicit preferences. Relevance may include species, population, biological matrix, and experimental conditions. Assay validity may include whether the method measures the intended property and whether controls support its performance. Evidence quality may include direct access to the original measurement, adequate replication, reported variability, and enough methodological detail to assess limitations. The relative priority of these preferences depends on the parameter and model purpose, so it should be stated rather than assumed.
This ordered approach prevents an important logical error: allowing a strength in one dimension to compensate for a fatal weakness in another. For example, a highly relevant measurement from an assay with demonstrated failure may not be rescued by a large sample size. Conversely, a technically excellent measurement in a different species may be less relevant than a sound, direct measurement in the modeled species.
A Short Justification for the Two-Stage Rule
The logic can be stated compactly. Suppose candidate \(i\) has an eligibility indicator \(e_i\), equal to 1 if it passes every minimum requirement and 0 otherwise. Suppose eligible candidates are then compared by an ordered set of criteria \(r_i\), with the first criterion given priority over the second, and so on. Selection is made only from candidates with \(e_i=1\); therefore a failure of a minimum condition cannot be offset by a favorable rank on another criterion. If the highest-ranked candidate is unique, the rule selects it. If the top rank is shared, the available evidence does not distinguish those candidates under the stated rule, so selection of just one would require an additional justified criterion.
This is not a statistical theorem about which measurement is closest to truth. It is a justification for a transparent decision process: minimum validity is protected, priorities are visible, and unresolved ties are not disguised as certainty. If no candidates pass the gates, the correct result is that the evidence is inadequate for the intended input—not that the least-bad value automatically becomes reliable.
Worked Example: Selecting a Human Plasma Binding Estimate
A hypothetical PBPK model is intended to represent adults, and the target input is the fraction unbound in human plasma under the model’s specified conditions. Three sources report candidate values. Before examining the values, the team sets a minimum assay recovery requirement of 80% to 120%, requires adequate documentation of the measurement, and gives direct adult human evidence priority over evidence from another species.
| Candidate | Reported value | Evidence details | Decision |
|---|---|---|---|
| A | \(f_u=0.018\) | Adult human plasma; equilibrium dialysis; recovery 96%; method and replicate results reported | Eligible |
| B | \(f_u=0.031\) | Adult human plasma; ultrafiltration; recovery 62%; filter adsorption noted | Fails assay gate |
| C | \(f_u=0.026\) | Mouse plasma; method documented and recovery 94% | Not preferred for adult human target |
Candidate B fails the prespecified recovery requirement because \(62\%<80\%\). Its numerically higher estimate does not override that failure; the noted adsorption also raises a method-specific concern. Candidate C passes the stated recovery criterion, but it does not directly represent the human target. Candidate A is eligible, relevant to the modeled population, and adequately documented. Under the stated rule, select \(f_u=0.018\) as the input, while recording that the choice is supported by the evidence available rather than guaranteed to be the true value.
The decision does not require averaging the three reported values. In particular, including a value that fails an eligibility gate would make the resulting average depend on evidence judged unsuitable for the target measurement.
When Assay Validity Changes the Decision
Assay validity concerns whether the method can support the reported estimate for the intended measurement. Check whether the method is appropriate for the property, whether its operating conditions match the target, and whether reported controls and performance measures support interpretation. Relevant issues can include recovery, nonspecific binding, sample stability, equilibrium, concentration dependence, and the assay’s measurement range. The applicable checks depend on the assay; no single performance threshold is universal.
Set quantitative acceptance limits before selecting among conflicting results whenever a defensible standard or study protocol supplies them. If no universal threshold exists, state the criterion used and why it is appropriate. Do not treat an arbitrary cutoff as a law of measurement. A result just outside a chosen boundary may warrant review, but changing the boundary only after seeing which candidate it favors undermines the decision rule.
Worked Example: Choosing Between Two Assay Methods
In a hypothetical comparison for an adult human plasma input, equilibrium dialysis reports \(f_u=0.022\), with recovery of 93% and evidence that the measurement reached equilibrium. Ultrafiltration reports \(f_u=0.041\), with recovery of 65% and evidence of appreciable binding to the filter. The analysis plan requires recovery between 80% and 120% and a method without an identified bias that invalidates the estimate.
The equilibrium-dialysis result passes the recovery gate because \(80\%\leq93\%\leq120\%\), and the reported equilibrium check supports interpreting it as an estimate of the target quantity. The ultrafiltration result fails because \(65\%<80\%\); filter binding also threatens the measurement’s validity. Thus, \(0.022\) is selected under the stated rule. The comparison is not decided by taking the arithmetic midpoint:
The value \(0.0315\) is merely the midpoint of two reported numbers. It is not a supported estimate from either assay, and including the result that failed the assay gate would contradict the stated decision rule.
When the Evidence Does Not Support a Unique Choice
Sometimes multiple candidates pass the eligibility gates and are similarly relevant, valid, and well documented. In that situation, a single “best” value may not be identifiable from the evidence. Do not use the number of decimal places, publication date, or convenience for model implementation as an unspoken tie-breaker. Retaining alternatives is a substantive conclusion: the evidence review has not justified choosing one candidate over another.
The alternatives can be carried forward as distinct, documented input scenarios. This makes the consequence of the unresolved choice visible in subsequent PBPK analyses. It also avoids implying that a value formed by averaging is itself measured evidence. An average may be reasonable in some settings if a meaningful estimand and a justified pooling method support it, but it does not follow automatically from disagreement.
Worked Example: Retaining Two Equally Defensible Values
A hypothetical model requires adult human plasma \(f_u\). Two independent sources report \(0.016\) and \(0.024\). Both use documented methods, meet the prespecified assay criteria, represent the target population, and provide comparable evidence quality. The decision rule has no additional criterion that distinguishes them.
Both candidates pass eligibility, and neither ranks above the other on the declared priorities. Therefore, the rule does not identify a unique input. Retain two scenarios, \(f_u=0.016\) and \(f_u=0.024\), and report the resulting PBPK predictions separately. The midpoint is
But \(0.020\) is not selected merely because it lies between the estimates: no source reported it, and the stated evidence does not establish it as a better estimate. If the model’s predictions differ materially between the two scenarios, that difference is decision-relevant and should be communicated. If they are nearly unchanged, the input conflict may have little consequence for the model outputs considered, but the evidence disagreement still exists.
Compare the Main Decision Strategies
| Strategy | When it is defensible | Main risk |
|---|---|---|
| Select the highest-ranked eligible source | A prespecified rule identifies one candidate as uniquely better suited to the target | Unstated priorities can make the selection appear objective when it is not |
| Average the reported values | A defined estimand and justified method support combining the measurements | Can hide differences in relevance or validity and create a value unsupported by any source |
| Retain discrete alternatives | Multiple candidates remain credible and the rule does not distinguish them | Requires transparent scenario reporting rather than a falsely precise single input |
| Choose the most conservative value | A defined decision objective makes a particular direction conservative | “Conservative” is context-dependent and may not mean conservative for every PBPK prediction |
A strategy based on fitting model predictions also needs care. Agreement with an observed clinical profile can help evaluate a model, but choosing an input solely because it improves fit may confound input selection with other model assumptions. A better fit does not, by itself, prove that the chosen input is the correct measured value. The intended use of the model and the independent evidence for the parameter still matter.
Diagnostics, Limitations, and Common Mistakes
Before finalizing a choice, check whether the decision remains stable under reasonable changes to the ranking criteria, whether any exclusion depends on a poorly justified cutoff, and whether a candidate is being favored for a reason unrelated to its relevance or measurement quality. Confirm that the evidence actually measures the target parameter, not a related quantity that requires additional interpretation. If the result depends entirely on one subjective criterion, say so.
The rule cannot eliminate limitations in the evidence. A relevant assay can still have unrecognized bias; a well-documented source can still have a narrow population; and evidence of similar quality can leave genuine disagreement unresolved. A decision rule makes those judgments inspectable, not infallible. The introductory material on ranking evidence sources and the earlier work on diagnosing measurement disagreement provide the context for evaluating candidates; the present decision step asks what those evaluations imply for the model input.
- Mistake: “The newest estimate must be best.” Newness alone does not establish relevance or validity. Check what was measured, in whom, and by what method.
- Mistake: “Use the mean of all reported values to be neutral.” An average can combine unsuitable evidence and suggest a precision the sources do not support. Select it only when combining the measurements is justified.
- Mistake: “A high-quality study can compensate for a mismatch to the target.” Study quality and target relevance are distinct. A strong measurement in the wrong population may not be the best input for the modeled population.
- Mistake: “If the model runs, the input choice is resolved.” Numerical feasibility is not evidence that a candidate is well supported. Separate implementation convenience from evidential justification.
Check Your Understanding
For each question, decide what the stated rule implies. Solutions follow the questions.
- A candidate is from the correct species but fails a prespecified assay-validity gate. Can it be selected because it is the most relevant?
- Two candidates pass all gates and tie on every declared ranking criterion. What should the decision rule return?
- Two eligible estimates are \(0.010\) and \(0.030\). Is \(0.020\) automatically the correct input?
- Why should a recovery threshold ordinarily be specified before comparing candidate values?
Solutions. (1) No. The candidate is excluded by the minimum gate; relevance does not compensate for failure of a required validity condition. (2) Retain both as alternatives unless a further defensible criterion distinguishes them. (3) No. The midpoint is \( (0.010+0.030)/2=0.020\), but that arithmetic alone does not justify it as a measurement or as the model input. (4) Prespecification reduces the risk of choosing a threshold after seeing which candidate it favors and makes the decision reproducible. The threshold still needs a rationale appropriate to the assay.