Why Review in Stages?
A model-informed dossier may contain extensive technical detail, but a reviewer’s central task is not to count analyses or confirm that every expected section is present. It is to determine whether the evidence, taken together, supports the specific claim being made for a decision. A staged review makes that task more consistent: it starts with the claim and works through the evidence needed to evaluate it, rather than letting a prominent result dictate what the reviewer notices next.
The stages below follow a dependency order: define what the dossier is meant to establish; establish which data support the analysis; inspect the model and its numerical and empirical evidence; evaluate the results; then assess how well those results support the decision and what limitations remain. A concern discovered late may require returning to an earlier stage. The order is a way to organize scrutiny, not a rule that evidence can only be considered once.
The review assumes that the dossier states its context of use and intended decision, identifies the analyses and data being relied on, and provides enough information to assess them. If a required item is unavailable, record that as an evidence gap; do not infer that the analysis passed simply because no contrary evidence is visible. The reviewer may also need to examine referenced protocols, analysis outputs, or computational artifacts to resolve questions that cannot be answered from the main report alone.
A Staged Checklist
At each stage, record the question, the evidence examined, the reviewer’s assessment, and any unresolved uncertainty. The checklist is most useful when it preserves the connection between an assertion and its support.
Identify the context of use, the decision the analysis is intended to inform, the population and setting, and the exact claim. Check that the dossier does not imply a broader claim than its evidence can address.
Trace the source data and exclusions into each analysis set. Confirm that counts, inclusion rules, and relevant data transformations are sufficiently clear, and that differences between analysis sets are explained.
Check whether the stated model, assumptions, and estimation approach are appropriate to the question and data. Review convergence and numerical stability before interpreting parameter estimates or derived predictions.
Consider whether parameters are identifiable and estimated with useful precision. Inspect goodness-of-fit, predictive checks, and validation evidence for agreement with the intended uses of the model.
Trace reported results to the relevant model outputs. Verify estimates, units, uncertainty summaries, and consistency among text, tables, and figures. Distinguish a point estimate from the uncertainty around it.
Compare the result and uncertainty with the decision rule or threshold. Assess whether sensitivity analyses change the decision, and whether limitations restrict the population, conditions, or strength of the conclusion.
This sequence has a simple justification. A decision claim is supported only if each essential link in its evidence chain is adequate. If the data do not represent the stated analysis population, a technically good fit does not repair that mismatch. If estimation is unstable, a precise-looking downstream simulation may not be trustworthy. Therefore, assessing later evidence without first checking its prerequisites risks interpreting results whose inputs or meaning are not established.
The arrows indicate dependence, not a numerical formula. When a link is unclear, the reviewer should trace it back to the earliest stage that could explain the problem. The introductory material on context of use and claim scope provides the foundation for the first stage; the review here applies that framing consistently across the dossier.
Worked Applications
Worked Example: Checking the Claim Before the Results
Hypothetical dossier. A sponsor uses a population pharmacokinetic model and exposure-response simulations to support a 40 mg dose for adults with a specified disease. The stated decision criterion is that at least 70% of the target population should attain a prespecified exposure target. The report’s executive summary says, “The model demonstrates that 40 mg is appropriate for all adults.”
Stage 1: define the claim. The quantitative criterion concerns target attainment in the specified population. “Appropriate for all adults” is broader: it may imply other populations, clinical outcomes, or safety considerations that the stated target-attainment criterion does not establish. The first review finding is therefore a scope mismatch. This observation does not by itself show that the 40 mg dose fails the criterion; it shows that the broad wording is not supported by the evidence described.
Stage 2: identify what must be checked next. The reviewer should verify that the simulated population corresponds to the specified adults, that the exposure target is defined and justified, and that the model evidence supports predictions for the intended use. A result about target attainment cannot be interpreted until these links are clear.
Conclusion. The dossier’s stated decision question is narrower than its executive-summary claim. The review should assess the 70% target-attainment claim in its specified population and should not treat that result as evidence for a universal dose recommendation.
Worked Example: Reconciling Analysis Populations
Hypothetical dossier. A trial enrolled 240 participants. The report states that 18 participants had no postbaseline measurements and were excluded from the efficacy analysis, leaving 222. A pharmacokinetic analysis uses 210 participants with at least one concentration measurement. The dossier gives no explanation for the difference between these two analysis sets.
Check the arithmetic. For the efficacy analysis, \(240-18=222\), consistent with the stated count. For the pharmacokinetic analysis, the additional difference is \(222-210=12\). Thus, 12 participants included in the efficacy set are not represented in the reported pharmacokinetic set.
Assess what is established. The counts are arithmetically consistent, but the reason for the additional 12 exclusions is not established by the information given. The reviewer cannot assume that these participants lacked samples, had unusable records, or were excluded under a prespecified rule. Those are possible explanations, not evidence.
Review implication. The data-provenance stage is incomplete until the dossier explains the 12-person difference and its handling. If the exposure-response analysis relies on the 210-person set, the reviewer should also consider whether the analysis population remains suitable for the stated claim. No conclusion about bias follows from the count difference alone, but the unexplained difference limits assessment of representativeness and analysis reproducibility.
Worked Example: Reading Results Against a Decision Threshold
Hypothetical dossier. For the specified adult population, a simulation estimates target attainment of 0.72 at 40 mg, with a 90% model-based uncertainty interval from 0.61 to 0.81. The prespecified decision criterion is target attainment of at least 0.70. The dossier describes the result as “meeting the criterion.”
Compare the point estimate with the criterion. The estimate is \(0.72\), or 72%, which is above the 70% threshold by \(0.72-0.70=0.02\), equivalent to 2 percentage points. On the point estimate alone, the criterion is met.
Consider uncertainty. The interval extends from 61% to 81%, so it includes values below as well as above 70%. This interval is an uncertainty summary for the model-based estimate under the stated analysis; it is not a 90% probability that the fixed true target-attainment value lies in this particular interval. The interval makes clear that the point estimate alone does not describe the uncertainty relevant to the decision.
Review conclusion. The reviewer should distinguish “the point estimate exceeds 70%” from “the evidence establishes target attainment above 70%.” Whether the dossier’s decision rule requires only a point estimate or requires a stronger degree of assurance must be checked in the stated decision framework. Without that rule, the reviewer cannot decide that the criterion is robustly met merely because 0.72 exceeds 0.70.
Interpreting Evidence Across the Dossier
The stages are related, but they address different questions. A successful estimation run does not demonstrate that the model is empirically adequate. Goodness-of-fit plots do not by themselves validate predictions for a new population or decision. A predictive check can provide evidence about specified predictive features, but it does not establish every possible use of the model. Similarly, sensitivity analyses help show how conclusions change under selected alternatives; they cannot guarantee robustness to assumptions that were not examined.
A reviewer should distinguish an absent analysis from a negative result. If no predictive check is reported, the concern is that the relevant evidence is not available in the dossier; it is not correct to report that the model failed the check. If a check is reported and shows systematic discrepancy for a feature central to the claim, that is different evidence and should be assessed in context. The same distinction applies to convergence, parameter precision, and data reconciliation.
A practical record for each stage can use three plain-language statuses: adequate for the stated review question, concern identified, or not assessable from the available evidence. These are organizational labels, not universal regulatory categories. They help prevent “not reported” from being mistaken for “passed,” while leaving room to describe the reason and significance of a concern. A concern’s importance depends on its connection to the claim: a discrepancy in a peripheral analysis may matter less than one affecting a decision-critical population or endpoint.
Review findings should also be checked for internal consistency. For example, a reviewer should not describe parameter precision as adequate and then treat large parameter uncertainty as immaterial without explaining why. Nor should the conclusion state that a decision is robust if a sensitivity analysis changes the decision under a plausible alternative. The earlier material on consistency checks across tables, figures, and text is useful here: the same principle applies to the review’s own reasoning.
Diagnostics, Alternatives, and Limitations
A structured checklist is especially helpful when a dossier is long, when several model components feed a common decision, or when different reviewers focus on different sections. It reduces the chance that a polished summary or a favorable headline result will displace basic checks of scope, data, or estimation. It also creates a traceable route from a decision claim to the evidence examined.
The checklist can fail if it becomes a box-ticking exercise. Warning signs include marking a stage complete because a heading exists, accepting a diagnostic without considering the claim it is meant to support, and recording conclusions without noting which evidence was examined. Another warning sign is a mismatch between stages: for example, a conclusion about a broad population despite data or validation evidence that covers only a narrower one. When this occurs, return to the claim and trace the dependency chain rather than adding more checklist marks.
A free-form narrative review may be more flexible and can be appropriate for a focused question or a short dossier. However, it can make omissions and differences in reviewer emphasis harder to detect. A checklist improves coverage and comparability, but may oversimplify complex evidence or give equal visual weight to issues with very different decision relevance. A strong review uses the staged structure to ensure coverage and a narrative explanation to preserve context and judgment.
The process has important limits. It does not replace independent scientific judgment, reproduce the analysis, or establish that every unexamined assumption is acceptable. The reviewer may lack access to data, code, or sufficient detail to resolve a concern. Also, no single ordering eliminates all feedback: a result may reveal a claim ambiguity, and a sensitivity analysis may prompt renewed scrutiny of the model specification. Record such returns explicitly, so the final assessment reflects the evidence chain rather than a mechanical pass through stages.
Common Pitfall
Correction: Presence is not adequacy. For each item, ask whether the reported evidence answers the relevant question, whether it applies to the stated context of use, and whether remaining uncertainty changes the decision interpretation. A checklist documents coverage; the reviewer must still evaluate the substance and connection of the evidence.
Check Your Understanding
Answer each question, then compare with the solutions below.
- A dossier reports an exposure-response result before explaining who was included in the analysis. Which stage should the reviewer check first, and why?
- A report gives a target-attainment estimate of 0.68 against a decision threshold of 0.65, with a 90% uncertainty interval of 0.54 to 0.77. What can be concluded from the point estimate, and what does the interval add?
- A predictive check is not included in the dossier. Should the reviewer record that the model failed the check? Explain.
- Why might a reviewer return to the claim-scope stage after completing the model-diagnostics stage?
Solutions. (1) Check the analysis population and data reconciliation first, because the applicability of a result depends on which data and participants generated it. (2) The point estimate exceeds the threshold by 0.03, or 3 percentage points. The interval includes values below and above the threshold, so the point estimate alone does not characterize decision-relevant uncertainty; interpretation also depends on the stated decision rule. (3) No. The check is absent, so the reviewer should record that predictive evidence is not assessable from the dossier, not claim that a check was performed and failed. (4) Diagnostics may show that the evidence applies only to a narrower population or use than the dossier’s claim. The reviewer should then reassess whether the claim is appropriately scoped to the evidence.