Constructing an Evidence Map for QSP Calibration

Why Build an Evidence Map?

A QSP calibration may draw on experiments conducted in cell systems, animal models, and clinical studies. These sources can differ in biological context, measured outcomes, design, and reporting. If their results are combined before those differences are documented, it becomes difficult to see which evidence supports which part of the calibration, what assumptions are being made, or where the evidence is incomplete.

An evidence map is a structured inventory of the measurements available for a model, organized so that each source and its conditions remain identifiable. Its purpose is not to fit the model, resolve differences among studies, or decide final source weights. It makes those later decisions traceable by recording what was observed, how it was obtained, what uncertainty was reported, and how the source is currently intended to be used.

The map is an inventory, not a claim that all listed measurements are comparable or mutually consistent. It should preserve differences rather than hide them. For example, a response measured in cultured cells and a biomarker measured in patients may both be relevant to a pathway, but listing them together does not establish that they measure the same quantity or should enter calibration in the same way. The later work of formally linking measurements to model quantities, harmonizing study definitions, and specifying a joint calibration objective depends on having this inventory first.

Define the Unit of an Evidence-Map Entry

A practical default is one entry per source, measured outcome, and materially distinct study condition. A source might be a paper, a study report, a laboratory dataset, or a clearly identified analysis within a report. Split entries when the same source reports distinct outcomes or conditions that matter for interpretation. Keep them together when a report presents the same outcome under a common design and clearly describes the summary as a group.

Do not use one row for an entire publication if it contains several distinct assays, arms, populations, or time points. Conversely, avoid making separate rows for every individual measurement when the purpose is to inventory a reported group summary. Record the level of detail actually available, and make the grouping rule explicit. If the source provides individual-level data, note that availability; the map itself need not reproduce every observation.

Useful fields commonly include:

• Source identifier and provenance: citation or internal identifier, report version, data location, and the specific table, figure, supplement, or page supporting the entry.
• System and population: for example, cell line and passage range, animal species and disease model, or clinical population and relevant inclusion criteria.
• Measured outcome: the reported endpoint and its recorded unit or scale, using the source's terminology. Include the assay or measurement method when it affects how the result should be understood.
• Study conditions: treatment or perturbation, comparator, dose as reported, sampling schedule, duration, and other conditions needed to distinguish this entry from another. Record unavailable details as unavailable, rather than guessing.
• Result and uncertainty: the reported summary, sample size, and the type of uncertainty provided, such as standard deviation, standard error, confidence interval, or a qualitative statement. Keep distinct quantities distinct.
• Data access and quality notes: whether individual observations or only a published summary are available; missing reporting; exclusions; and any limitations stated by the source.
• Proposed calibration use: a provisional role such as calibration candidate, contextual constraint, held-out evaluation source, or not currently usable, with a brief rationale.

A field marked “not reported” is more informative than an empty field: it distinguishes missing documentation from an oversight in constructing the map. Likewise, record study conditions in the source's own terms at this stage. Converting units, aligning time origins, or redefining dose is separate work and should not be concealed inside an inventory entry.

Represent Uncertainty Without Over-Interpreting It

An evidence map should record what a source reports, not replace it with a generic “error” value. A standard deviation describes variation among observations in the reported group; a standard error describes uncertainty in an estimated mean under the stated sampling framework; and a confidence interval is an interval produced by a particular procedure. These quantities are not interchangeable. If the source does not state which one is reported, note the ambiguity rather than assigning an interpretation.

When a reported standard deviation \(s\) and sample size \(n\) are available for a mean, the standard error can be calculated as \(SE=s/\sqrt{n}\) under the usual independent-sampling interpretation. Record this as a derived value, label it as derived, and retain the source's original summary and uncertainty. The calculation does not account for every uncertainty relevant to calibration: it does not, for example, resolve possible assay bias, study-design limitations, or differences between study systems.

A useful map also distinguishes known uncertainty from unknown or unreported uncertainty. An assay's replicate variation, variability among animals, and variability among patients arise at different levels. The map records which level the source describes; it does not assume that one reported dispersion measure captures all of them. This care is essential before decisions about assay-specific observation error, replicate correlation, or source weighting are made.

Assign a Provisional Use, Not a Final Weight

The intended-use field answers a practical question: what role might this source play, given the calibration objective and what is currently known? It is a documented proposal, not a statistical decision. A source may be relevant without being ready for direct quantitative use. For example, a study can provide useful context about a response direction while lacking the numerical detail required for a calibration target.

Roles can change as the model purpose becomes clearer and missing documentation is resolved. Record the rationale and, when useful, the person or date associated with the decision. Do not assign a numerical weight merely because a source has a large sample size or appears precise. Whether sources can be combined and how their contributions should be balanced are separate questions.

Compare Sources While Keeping Their Context

An evidence map is most useful when it allows a reviewer to compare entries without implying that unlike measurements are equivalent. A compact overview can show which source types and outcomes are represented, while detailed entries preserve the design information behind each result. A gap in the overview is a prompt for investigation, not evidence that the corresponding biological process is absent.

For example, an in vitro assay may report response at several concentrations, a preclinical study may report a time course, and a clinical trial may report a change from baseline at a small number of visits. The map can list all three, but should not convert them into a common response scale or claim a direct match just because their labels sound similar. At this stage, record the measured endpoint, its study context, and the proposed relevance. Formal measurement-to-model mapping is a distinct task.

A Short Justification for Source-Level Organization

Suppose two reports give the same numerical mean but use different assays or populations. If the map stores only the mean, the source distinction is lost. That makes it impossible to trace a discrepancy or decide whether an entry is relevant to a particular calibration use. Keeping source identity and conditions attached to each measurement preserves that information.

More formally, let an entry be identified by the tuple \(r=(s,y,c)\), where \(s\) is a source, \(y\) is a measured outcome, and \(c\) is a defined study condition. If two entries differ in any component, then \(r_1\ne r_2\), so they remain separately traceable even when their reported numerical summaries coincide. This simple indexing rule does not prove the measurements are biologically different; it prevents the inventory from treating them as identical without justification.

The result is a reviewable chain from a calibration decision back to the reported evidence. It also exposes information gaps early: a missing sample size, ambiguous endpoint definition, or unreported uncertainty can be addressed before the source is used as if fully characterized.

Diagnostics, Pitfalls, and Limits

• Blank fields. A blank does not reveal whether information was absent from the source or simply not extracted. Use explicit labels such as “not reported,” “not applicable,” or “not yet checked.”
• One row per paper. This can conceal different outcomes, study arms, or sampling times. Split entries when those distinctions matter, and state the grouping rule.
• Unlabelled uncertainty. A number beside a mean is not self-explanatory. Record whether it is an SD, SE, interval, range, or another statistic, along with its source and level of variation when known.
• Premature equivalence. Similar endpoint names do not establish that different assays or populations measure interchangeable quantities. Preserve the source terms and conditions; do not infer comparability from labels alone.
• Premature weighting. A larger \(n\) does not automatically justify a larger calibration contribution. The map supports later decisions but is not itself a weighting rule.
• Confusing relevance with usability. A biologically relevant source may lack enough information for quantitative use. Record both its relevance and the specific limitation.

The evidence map cannot eliminate study bias, recover unreported design details, or establish that a model is identifiable. It also cannot decide how to represent observation error, account for correlation, or combine sources in a calibration objective. Its value is narrower and foundational: it makes the evidence and its limitations visible, so that later modeling choices can be explicit rather than implicit.