Why Rank Evidence Sources for a Specific Input?
A PBPK input is not supported merely because a number appears in a paper, database, review, or software interface. The source needs to support the particular parameter being entered, for the intended model context. A carefully measured value for the wrong species or parameter definition may be less useful than a well-documented curated value that matches the intended use. Conversely, a database label such as “validated” does not by itself show that a value applies to the model in front of you.
The task is therefore not to declare one source type universally superior. It is to identify which available source best supports a specified input, make the reasons visible, and state what remains uncertain. The anatomy of a PBPK parameter record provides a place to capture that choice and its provenance; source ranking supplies a disciplined way to make it.
Set the Target Before Comparing Sources
First specify what the model needs: the parameter name and definition, the units or scale, and the intended species, population, route, or other relevant context. Then identify the candidate sources and ask whether each can support that target. This is an eligibility screen, not a score. For example, an untraceable default should not become eligible just because it is the only candidate or has a convenient value.
Eligibility criteria should be stated before ranking. A practical screen asks whether the source identifies the input well enough to interpret it, has a traceable basis, and is sufficiently relevant to the intended context. The exact minimum depends on the use. A preliminary sensitivity analysis may tolerate a provisional source that would not be adequate for a decision-critical prediction. State the minimum rather than silently changing it to admit a preferred candidate.
Among eligible candidates, compare fit-for-purpose support. Relevant considerations often include directness of evidence for the input, match to the intended context, traceability to supporting evidence, and quality control or validation. These are distinct: direct evidence may be context-mismatched, while a carefully curated database may be traceable and validated but less direct than a primary measurement.
A Transparent Ranking Aid
One way to make the comparison explicit is to rate each eligible source against criteria chosen for the input. The following illustrative scheme uses ratings from 0 to 4, with higher ratings indicating stronger support, and weights that sum to 1. The weights express the priorities for this particular decision; they are not universal PBPK constants. A weighted fit score can be written as follows.
Here \(r_j(s)\) is the rating for source \(s\) on criterion \(j\), and \(w_j\) is that criterion’s predeclared weight. The score is a structured summary of stated judgments, not a probability that a value is correct and not a statistical estimate of measurement error. Eligibility remains a separate gate: a high score cannot rescue a source that fails an essential criterion.
The calculation is straightforward. If one source has ratings \(r_j\), its score is the sum of each rating multiplied by its weight. Comparing two sources gives \(F(s_1)-F(s_2)=\sum_j w_j[r_j(s_1)-r_j(s_2)]\). Thus the ranking depends on both the ratings and the chosen weights. That is useful for transparency, but it also means a small score difference should not be presented as an objective or precise measure of source superiority.
State the parameter definition and intended context before opening the candidate-source comparison.
Apply predeclared minimum requirements. Set aside sources that fail, recording why.
Use explicit criteria and, if useful, a disclosed scoring scheme. Explain close rankings and ties.
State the selected source if one is selected, alternatives considered, and any remaining uncertainty or lack of suitable evidence.
Worked Source Comparisons
Worked Example: Primary Measurement Versus Validated Database
Hypothetical case. A model requires the fraction unbound in plasma, \(f_{u,p}\), for an adult human context. Two eligible candidates report values of 0.076 and 0.081. Candidate A is a primary study with direct measurements in the intended context. Candidate B is an entry in a validated database; its record identifies supporting evidence and describes a quality-control process. For this decision, use the criteria directness (\(D\)), context match (\(C\)), traceability (\(T\)), and quality control (\(Q\)), with weights 0.35, 0.25, 0.25, and 0.15, respectively. Ratings are hypothetical and illustrative.
Candidate A receives ratings \(D=4,\ C=4,\ T=3,\ Q=3\). Its score is \(F(A)=0.35(4)+0.25(4)+0.25(3)+0.15(3)=1.40+1.00+0.75+0.45=3.60\).
Candidate B receives \(D=3,\ C=4,\ T=4,\ Q=4\). Its score is \(F(B)=0.35(3)+0.25(4)+0.25(4)+0.15(4)=1.05+1.00+1.00+0.60=3.65\).
Under this declared scheme, B ranks slightly higher, so it is the selected source for this example, with \(f_{u,p}=0.081\) as the candidate input. The 0.05-point difference is small, and the result reflects the stated weights and ratings; it does not establish that the database value is biologically closer to the truth. Record A as a plausible alternative and explain why B was preferred. If directness had been weighted more heavily, the ranking might change.
Worked Example: Primary Study Versus Review
Hypothetical case. A PBPK model requires a human blood-to-plasma concentration ratio, \(R_b\). An eligible primary study reports \(R_b=0.92\) in a context judged relevant to the intended model. An eligible review reports a summary value of 1.05 and clearly identifies the evidence it summarizes. Use the same four criteria and weights as in the previous example. For illustration, rate the primary study \(D=4,\ C=4,\ T=3,\ Q=3\), and the review \(D=2,\ C=3,\ T=4,\ Q=3\).
The primary-study score is \(F(\text{primary})=0.35(4)+0.25(4)+0.25(3)+0.15(3)=1.40+1.00+0.75+0.45=3.60\). The review score is \(F(\text{review})=0.35(2)+0.25(3)+0.25(4)+0.15(3)=0.70+0.75+1.00+0.45=2.90\).
The primary study ranks higher under these assumptions and is selected for the input in this example. The review is still useful as a secondary source: it indicates a reported synthesis and may help identify other evidence. The difference between 0.92 and 1.05 does not, by itself, establish why the values differ or whether they are directly comparable. The ranking answers which source better supports this specified input under the stated criteria; it does not settle the scientific interpretation of the discrepancy.
Worked Example: A Model Default That Fails Eligibility
Hypothetical case. A model requires a tissue-to-plasma partition coefficient for a specified human population. The only candidate is a software default of 1.0. The interface labels it a default but provides no supporting basis, intended population, or indication that it applies to this input. Suppose the predeclared eligibility requirements include traceable support and adequate context for the intended use.
The default fails both requirements, so \(E(\text{default})=0\). With \(S=\{\text{default}\}\), the eligible set is \(S_E=\varnothing\). There is no eligible source to score and no selected source \(s^*\). The correct report is that no suitable source is available under the stated requirements. The modeler may separately document that the software offers 1.0, but must not present that number as a supported input merely because it is available.
If a different default were documented, traceable, and applicable to the target context, it could be assessed as a candidate and might be eligible for a limited or provisional use. “Default” is a source description, not an automatic rejection or endorsement.
How Source Types Differ
A primary study can provide direct evidence for a measured quantity, but the measurement may not match the model’s intended species, population, or parameter definition. A validated database can organize evidence, apply quality controls, and make source records easier to find; its value still depends on what the entry represents and whether its basis is traceable. A review can synthesize a field or identify evidence, but a summarized value is not automatically a direct measurement. A model default may be useful when its basis and applicability are documented, but an unexplained software value is not evidence of fit.
This is a comparison of roles, not a fixed ladder. A relevant and well-supported review may be more useful than a weak or context-mismatched primary study. A database entry may be preferable when its curation is transparent and the underlying record fits the target. A default may be defensible for a clearly limited purpose if its basis is known. In every case, justify the choice for the input rather than relying on the source label.
Diagnostics, Pitfalls, and Limits
A ranking is becoming unreliable when its eligibility rules shift after candidates are seen, ratings cannot be explained, weights are chosen to force a preferred answer, or tiny score differences are described as decisive. Check the table of criteria and ratings against the original rationale: can another modeler see why a source earned each rating? Would a modest, defensible change to a weight reverse the ranking? If so, report that sensitivity and avoid claiming a uniquely superior source.
- Wrong reasoning: “The primary study must be best because it is primary.” Correction: Assess whether its evidence is relevant, traceable, and adequate for the particular input.
- Wrong reasoning: “The database is validated, so its value is automatically suitable.” Correction: Check the entry’s meaning, context, and supporting basis.
- Wrong reasoning: “The default is better than having no value.” Correction: An unsupported default does not become eligible through convenience. Report the gap if no source meets the minimum requirements.
- Wrong reasoning: “A higher weighted score proves the value is more accurate.” Correction: The score summarizes disclosed judgments under chosen criteria and weights; it is not a probability or validation statistic.
Ranking also has limits. The criteria and ratings involve judgment, and a single score can conceal important differences between sources. Eligibility thresholds depend on the intended use, and a decision that is adequate for exploratory modeling may be inadequate for a consequential application. Ranking is not a substitute for evaluating the underlying evidence or documenting the eventual input and its limitations. Where candidates are nearly tied, retain the alternatives and describe the basis of the choice rather than implying a sharp distinction.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- A direct primary measurement has a weaker context match than a curated database entry. Does “primary” automatically determine the selection?
- Using weights 0.50 and 0.50, a candidate has ratings 4 and 2. Calculate its weighted score.
- Two eligible sources have the same highest score. What should the modeler do?
- A software default is the only candidate, but it fails a predeclared traceability requirement. What is the eligible set and what should be reported?
Solutions. 1. No. Compare their fit to the specified input using the predeclared criteria; source type alone does not decide. 2. The score is \(0.50(4)+0.50(2)=2+1=3.00\). It is a decision aid, not a probability. 3. Report the tie, or apply a tie-break rule declared in advance and explain it. Do not imply that the score distinguishes the sources. 4. The eligible set is empty because the sole candidate fails an eligibility requirement. Report that no suitable source is available under the stated criteria; do not select the default as supported evidence.