Why Reporting Deficiencies Matter
A PBPK submission can contain a plausible model and a large volume of output yet still be difficult to review. The central problem is often not that a particular result is wrong, but that the report does not let a reviewer determine which question was answered, how the result was generated, and whether the conclusion follows from the evidence. A reporting deficiency therefore creates uncertainty about interpretation, reproducibility, or relevance to the stated context of use.
This tutorial focuses on five recurring review problems: unclear objectives, untraceable inputs, incomplete simulation settings, selective plots, and unsupported conclusions. The remedy is not simply to add pages. It is to connect each claim to the objective, scenario, inputs, execution record, and results that support it. Reporting needs vary with the analysis and applicable regulatory expectations; the checks below are a practical diagnostic framework, not a universal checklist of formal requirements.
A Claim Must Have a Traceable Evidence Path
A useful way to diagnose a report is to follow a claim backwards. A claim such as “the predicted exposure change is not clinically meaningful” should connect to a specified decision question, the simulated population and scenario, the model and parameter set, the simulation settings, the displayed or tabulated result, and the criterion used to interpret that result. If one link is absent, the reviewer may be unable to tell whether the claim answers the stated question.
The logic can be summarized as a dependency, rather than as a numerical score:
The order matters. A result can be reproduced but irrelevant to the decision if the objective was not defined. Conversely, a clearly stated objective does not make a result interpretable if the settings or input values cannot be identified. The earlier material on report organization, parameter provenance, simulation records, verification evidence, uncertainty, and clinical evaluation addresses these records in depth. Here the emphasis is on recognizing when their absence or disconnection becomes a review problem.
Worked Example: Repairing an Unclear Objective
Hypothetical report statement: “The model was used to evaluate the new oral product.” This does not state which decision the analysis is intended to inform, what comparison is being made, or which outcome matters. A reviewer cannot tell whether the analysis concerns exposure, a dosing recommendation, or another question.
Repair the objective. Suppose the intended question is whether the proposed product changes systemic exposure relative to the reference product in adults under fed conditions. State the population, products, condition, exposure measure, and decision use explicitly. For example: “This analysis estimates the effect of the proposed product, compared with the reference product, on steady-state AUC in adults under fed conditions to inform whether the products can be bridged for the stated use.”
Specify what must be reported with it. The report should identify the administered regimens, the model outputs used to represent AUC, the comparison or ratio being interpreted, and the prespecified basis for deciding whether the difference matters. It should also clarify whether this is a prediction, an analysis of observed clinical data, or a comparison of both.
Review diagnosis. The repaired wording makes the intended question testable, but it does not establish that the prediction is credible or that the products can be bridged. Those conclusions still require the relevant model, input, simulation, and evaluation evidence. A precise objective prevents an analysis from appearing to answer a broader question than it actually addresses.
Inputs and Simulation Conditions Must Be Traceable
An input is not fully traceable merely because a report gives a parameter name and a numerical value. A reviewer may need to know the value’s source, units, population or experimental context, any conversion or adjustment, the rationale for selecting it, and how uncertainty or alternative evidence was handled. The report should make clear which values are measured, taken from literature or product information, estimated, or assumed. When evidence is conflicting, the choice and its rationale should be visible rather than hidden in an unexplained final value.
Simulation settings have a parallel role. They define what calculation was performed and under what conditions. Depending on the analysis, relevant records may include the simulated population and demographic characteristics, dose and schedule, route, formulation, co-administered substances, sampling or summary times, number of virtual individuals, variability assumptions, software and model version, and execution settings such as a random seed when stochastic sampling is used. Not every item is relevant to every analysis; the report should identify the settings needed to interpret and reproduce its particular result.
Worked Example: Making a Parameter Value Traceable
A hypothetical report gives an unbound fraction of 0.12 for a compound but does not state its source or units. A reviewer cannot determine whether 0.12 is a measured value, a selected literature value, or an assumed value, nor whether it applies to the population and conditions in the analysis.
Document the record. A usable parameter entry might identify the parameter as fraction unbound in plasma, give the selected value as 0.12 (dimensionless), name the source and the experimental conditions, and record any transformation used to obtain the value entered in the model. If the selected estimate was taken from a hypothetical source reporting 12%, the conversion is \(12\% = 12/100 = 0.12\), a dimensionless fraction.
Explain the choice. If two candidate sources reported 0.10 and 0.12, the report should say why 0.12 was selected—for example, because the stated assay conditions were judged more applicable to the modeled population. It should identify whether the alternative value was examined and, if it could materially affect the conclusion, report the corresponding sensitivity analysis or explain why one was not performed.
Review diagnosis. The entry now lets a reviewer trace the numerical input and assess its relevance. It does not prove that the source is appropriate; that judgement depends on the reported evidence and rationale. A table of values without provenance remains incomplete, while a source citation without the actual selected value and transformation may also be insufficient.
Simulation Settings and Displays Must Be Complete Enough to Review
A result table or plot is not self-explanatory. It should be possible to identify the population, treatment or exposure scenario, endpoint, summary statistic, units, and relevant comparator without guessing. If only a mean is shown, for example, the reader may not learn whether the mean represents individuals, virtual trials, or another level of aggregation. If a plot shows a simulated profile, its axes, time units, dose conditions, and description of what the displayed bands or symbols represent must be stated.
A selective plot is a display that presents only a subset of results in a way that could hide important information—for instance, showing one representative profile while omitting the range across virtual individuals, or showing a favorable scenario but not the comparator needed to interpret it. Concision is appropriate; selective omission is a problem when it prevents evaluation of the stated claim. A complete plot inventory or a clearly described selection rule can help a reviewer understand what is shown and what is not.
When a result cannot be reproduced or interpreted, first ask whether the problem is missing documentation or a deeper analytical concern. These are different findings. A missing random seed may matter for reproducing a stochastic run; it does not automatically mean that the predicted quantity is wrong. A mismatch between reported settings and the execution record, on the other hand, raises a consistency concern that should be resolved before relying on the result.
Worked Example: Checking an Incomplete Simulation Record
A hypothetical report states that a virtual population produced a predicted exposure ratio of 1.42, but gives no population size, dosing schedule, summary definition, or software run identifier. A bar chart shows the ratio but not the underlying distribution or its comparison conditions.
Check numerical consistency. Suppose the report’s supporting table gives a simulated mean AUC of \(1{,}420\ \text{ng·h/mL}\) for the test condition and \(1{,}000\ \text{ng·h/mL}\) for the reference condition. The ratio is:
The units cancel, so the ratio is dimensionless. The calculation agrees with the stated value, but agreement of the arithmetic does not resolve missing conditions. The report should identify which AUC summary is being divided, the regimen and conditions in both arms, the population and simulation settings, and the run or configuration record linked to the output.
Repair the display and interpretation. Add a clearly labeled comparison of the relevant results and a display that makes the variability or distribution relevant to the claim inspectable. State how the displayed summaries were calculated and whether the plotted data represent individuals or aggregate values. Do not call a ratio of 1.42 “clinically unimportant” unless a suitable decision criterion has been stated and the evidence supports that interpretation.
Conclusions Must Not Exceed the Evidence
A conclusion is unsupported when it is broader, more certain, or more causal than the results and their limitations allow. Examples include claiming that a model “proves safety” from an exposure prediction alone, describing a prediction as observed clinical evidence, or generalizing from one population and scenario to others without supporting analyses. The report should distinguish what was directly calculated from what is inferred for a decision, state the comparison and decision criterion, and describe uncertainty or unresolved limitations that could change the interpretation.
Reviewers can test a conclusion by asking three questions: Does the result address the stated objective? Is the displayed evidence sufficient to check the result and its variability? Does the wording remain within the population, regimen, formulation, and use actually evaluated? If the answer to any question is no, the appropriate response is a focused request for clarification, an additional record or analysis, or a narrower conclusion—not an automatic assumption that the analysis is invalid.
Record the exact conclusion and the decision it is intended to support.
Identify the objective, scenario, model and inputs, simulation settings, and output linked to the claim.
Confirm that the comparison, units, summary, and stated decision criterion match the presented result.
Specify the missing record, display, analysis, or narrower wording needed to resolve the particular gap.
Comparing Remedies and Recognizing Failure Modes
There is no single best remedy for every deficiency. A concise main report supported by organized appendices can work well if cross-references make the evidence easy to locate. A machine-readable archive can aid reproducibility, but it does not replace a readable account of the objective, assumptions, and interpretation. Adding more plots may increase volume without fixing an unclear objective; adding more citations may not explain why a particular input was selected.
| Deficiency | What the reviewer cannot judge | Useful remedy |
|---|---|---|
| Objective is vague | Whether the analysis answers the decision question | State population, scenario, comparator, endpoint, and intended use |
| Input lacks provenance | Where the value came from and whether it applies | Report source, units, selected value, transformation, and rationale |
| Settings are incomplete | What conditions generated the result | Link the output to a complete, analysis-relevant execution record |
| Plots are selective or unlabeled | Whether the display represents the evidence relevant to the claim | Show necessary comparisons and variability; state units and selection rules |
| Conclusion overreaches | Whether the claim follows from the result and its scope | Provide supporting analysis or narrow the wording to the evidence |
A common mistake is to treat every reporting deficiency as proof that the model failed, or, in the opposite direction, to dismiss a gap as “only documentation.” The correction is to identify what the gap prevents the reviewer from evaluating. Missing provenance may block assessment of input suitability; missing settings may block reproduction; a selective plot may block assessment of variability; and an overbroad conclusion may require narrower language even if the underlying calculation is correct. Each problem calls for a remedy matched to its consequence.
Check Your Understanding
For each case, identify the reporting deficiency and the most direct remedy.
- A report says a PBPK analysis “supports the proposed dose” but does not identify the population or outcome used to reach that conclusion.
- A parameter table lists a numerical value and units but gives no source or selection rationale. What information should be added?
- A stochastic simulation is described, but the report omits the population size and the run record. Why can this matter?
- A plot presents only the mean predicted profile, while the conclusion refers to performance across individuals. What is missing?
- A report calculates a test-to-reference AUC ratio of 1.42 and concludes that the products are interchangeable, without stating a decision criterion. What is the problem?
Solutions. (1) The objective and context are underspecified; state the population, scenario, endpoint, comparator, and intended decision use. (2) Give the source and conditions, selected value, units, transformations, and rationale, including how relevant alternatives were handled. (3) The population size and run record help establish what was simulated and connect the reported output to its execution; provide analysis-relevant settings and a traceable record. (4) The display does not show the individual-level variability needed for the claim; add an appropriate distribution or variability display and describe its summaries. (5) The conclusion exceeds the evidence because the ratio alone does not establish interchangeability; provide a relevant prespecified criterion and supporting evidence, or narrow the conclusion.