Why Cross-Document Consistency Matters
A population pharmacokinetic submission is assembled from materials with different purposes: an analysis plan describes intended methods, datasets and programs support the work, and reports and summaries communicate results. Those materials may use different levels of detail, but they should describe a traceable analysis rather than several subtly different ones. If a summary reports a parameter from a different model version than the report, or a report describes a population that does not match the analysis dataset, a reviewer cannot confidently connect the conclusion to its evidence.
Cross-checking is therefore not a search for identical wording or numbers in every file. It is a structured audit of whether each statement refers to the same analysis, whether any difference is explained, and whether the result and conclusion remain accurate in every place they appear. The analysis plan, its amendments and deviations, data-lineage records, model-development records, and reporting materials provide the context for that audit.
Build the Audit Around a Traceable Analysis
Start by identifying the version that supports the reported primary results. Do not assume that the newest file, a similarly named dataset, or a final-looking summary is authoritative. Use the documented lineage and decision records to establish which analysis was performed and which outputs were used. Then compare statements across the documents against that reference.
An audit matrix makes this practical. For each important item, record the statement in each relevant document, its source, and the result of reconciliation. Include at least:
- Population and data: analysis-set definition, data cutoff, dataset version, participant counts, and usable concentration records.
- Analysis implementation: model identifier or version, program and computational settings where relevant, and any documented amendments or deviations.
- Results: parameter estimates, units, uncertainty summaries, and any transformations or rounding used in summaries.
- Interpretation: the conclusion drawn, the result it refers to, and whether the wording preserves the result's scope and uncertainty.
Counts deserve particular care. A number of participants, a number of concentration records, and a number of records used by a particular model run are different quantities. A count difference is not automatically an error, but it should be definable and traceable. Similarly, a model run can legitimately differ from the primary analysis if it is identified as a sensitivity analysis; presenting its results as if they came from the primary analysis is not a harmless formatting difference.
Use the analysis plan and documented changes, data-lineage information, and model records to identify the population, cutoff, dataset, and model behind the result.
For counts, estimates, intervals, and conclusions, identify the dataset, output, table, or documented decision that supports the statement.
Decide whether the difference is explained by a definition, unit conversion, rounding, or distinct analysis, or whether it is an unresolved contradiction.
Correct an inaccurate statement, explain a justified difference where it appears, and retain a record of what was checked and changed.
A useful audit trail records the item checked, the documents or outputs compared, the discrepancy, its disposition, and the responsible correction or explanation. This makes the check reproducible: another reviewer can follow the same links and reach the same reconciliation. The audit is complete only when a material mismatch has been resolved or clearly bounded as an intentional difference.
Worked Example: Reconciling Population and Record Counts
Suppose a hypothetical analysis plan defines the primary analysis population as participants with at least one usable concentration observation by a stated data cutoff. A lineage summary lists 312 enrolled participants. It records 14 exclusions before the analysis-set definition is applied, and the model input contains 1,842 concentration records from 298 participants. The report says 298 participants, while a submission summary says 296.
Check the population count. The plan's definition and lineage summary imply:
The 298 participants in the model input therefore agree with the documented analysis-set count. The 1,842 figure is a count of records, not participants, and should not be compared as though it were a participant count.
Resolve the mismatch. The report agrees with the lineage summary, but the submission summary does not. The difference is \(298 - 296 = 2\) participants. The auditor should check the submission summary's source and confirm whether it used a different population or an outdated count. If the source documents confirm 298 for the primary analysis, correct the summary to 298 and verify the corrected summary against the same source. If the 296 count belongs to a separately defined analysis, label that population and its rationale rather than silently replacing one count with the other.
Conclusion. On the stated hypothetical evidence, the primary-analysis participant count is 298 and the record count is 1,842. The summary's 296 is unresolved until its source is checked; it cannot be justified simply by noting that counts sometimes vary across documents.
Check Model Versions, Values, and Units
A numerical result is meaningful only when it can be linked to the analysis that produced it. Compare model identifiers in the report, tables, and supporting records; then verify that each quoted value came from the identified run. Where the model changed during development, distinguish the primary model from alternatives and sensitivity analyses. Do not infer that two outputs are interchangeable merely because their labels are similar.
Next, check whether values agree after accounting for units, rounding, and transformations. Unit conversions should be explicit and arithmetically verifiable. A rounded value may differ slightly from an unrounded value without being a contradiction, but the reported precision should not imply more agreement than the source supports. When uncertainty is presented, verify that the estimate, interval, and units belong to the same result and analysis.
Worked Example: Checking a Model Version and Unit Conversion
In a hypothetical submission, the report identifies model PK-04 as the primary analysis and gives clearance as 12.4 L/h. A one-page summary also identifies PK-04 but reports 0.207 L/min. The output record for PK-04 gives an unrounded estimate of 12.40 L/h.
To check the units, use \(1\) hour \(= 60\) minutes:
The summary's value is consistent with the report after conversion and rounding. The output record also supports the report's rounded 12.4 L/h value. Record the conversion and rounding as the explanation; do not treat the different numeric formats as distinct estimates.
Now suppose a table elsewhere reports 12.1 L/h but is labelled PK-03. That value should not be averaged with 12.4 L/h or substituted into the PK-04 summary. The auditor should check whether PK-03 is an alternative analysis and, if so, ensure that its label and role are clear. If the table claims to report PK-04, the discrepancy is unresolved and requires checking against the underlying output before correction.
Check Whether Conclusions Match Results
Consistency is not only numerical. A conclusion can be inconsistent with its supporting estimate even when every table is copied correctly. Compare the population, analysis, result, uncertainty, and scope in the conclusion with those in the underlying report. Watch for summaries that turn a qualified finding into an unqualified one, omit a relevant limitation, or state a stronger interpretation than the cited result supports.
Also check whether a statement is about the primary analysis, an alternative analysis, or a general claim. If a conclusion depends on a prespecified threshold or decision rule, confirm that the rule is identified and that the reported result is interpreted against it. If the threshold or analysis context is absent, the auditor may be able to identify the mismatch but should not invent a rule to resolve it.
Worked Example: Aligning a Summary Conclusion With Its Result
Consider a hypothetical report that gives an exposure ratio of 1.18 for a specified comparison, with a 90% confidence interval from 1.05 to 1.32. A short submission summary calls this “no difference in exposure.” Assume the report and summary refer to the same population and model.
First, express the point estimate as a relative percentage difference from 1:
The point estimate corresponds to 18% higher exposure for the comparison as defined. The interval corresponds to ratios from 1.05 to 1.32, or relative differences from:
Thus, the reported estimate and interval do not support the summary's unqualified phrase “no difference.” They describe an estimated increase, with uncertainty spanning relative increases of 5% to 32%. The auditor should correct the summary to state the result with appropriate uncertainty, or identify a different documented analysis if the summary was intended to describe one. Without a prespecified interpretation rule and its context, the auditor should not make an additional claim about clinical importance or equivalence.
Distinguishing Differences From Errors
Not every difference needs correction. The key question is whether a reader can understand why the materials differ and which analysis a statement describes. Common legitimate differences include:
| Observed difference | Possible explanation | Audit check |
|---|---|---|
| Participant counts differ | Distinct, defined analysis populations | Verify the population definitions and label each count. |
| Values use different units | Explicit conversion | Recalculate the conversion and check the units. |
| Values differ slightly | Rounding or display precision | Compare the unrounded source and the rounding rule. |
| Estimates differ | Different documented model versions or analyses | Check run identifiers and ensure each result is labelled correctly. |
| Conclusions differ | Different evidence or interpretation | Trace each conclusion to its analysis and stated decision context. |
An unexplained mismatch is not made acceptable by being small, and an explained difference should not be concealed by forcing identical numbers into every document. The audit should preserve distinctions that matter while removing accidental contradictions. If source records cannot establish which statement is correct, mark the issue unresolved and seek the needed confirmation rather than selecting the most convenient value.
Common Pitfall and Limitations
Common mistake: “The report and summary give different numbers, so one must be wrong; I will replace the summary number with the report number.” This skips the possibility of a legitimate unit conversion, rounding, or distinct analysis—and it does not verify that the report itself is correct. Correction: trace both numbers to their sources, compare analysis identities, and calculate any stated conversion. Correct only after establishing which value belongs to the analysis the document claims to summarize.
A consistency audit checks traceability and agreement; it does not, by itself, establish that the analysis methods are scientifically adequate or that the model is fit for its intended purpose. It also cannot resolve undocumented decisions or missing source outputs. Those require further review or clarification. Treat the audit as complementary to technical review, not a substitute for it.
The main alternative to a structured cross-document audit is an informal read-through. That can catch obvious wording problems, but it is less reliable for tracking counts, versions, conversions, and repeated conclusions across many files. A source-to-statement matrix is more systematic and easier to reproduce. Conversely, a matrix cannot judge an interpretation mechanically: reviewers still need to assess whether the wording accurately reflects the evidence and its scope.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- A model input contains 1,200 concentration records from 240 participants. A report says “1,200 participants.” What should the audit check?
- A report gives a parameter as 18.0 L/h and a summary gives 0.300 L/min. Are these values consistent? Show the conversion.
- A primary analysis uses model PK-02, while a table reports results from PK-01 without labelling it as an alternative. What is the issue?
- A result is described as “no difference” in one summary, while its estimate is 1.10 and its interval is 1.02 to 1.20. What should be checked before deciding how to revise the wording?
- Why is a consistency audit not a substitute for technical review of the model?
Solutions. (1) The statement likely confuses records with participants. Check the source and report 240 participants and 1,200 records as distinct quantities, if those counts are verified. (2) Yes: \(18.0 \div 60 = 0.300\) L/min. The values agree after conversion. (3) Trace the table to its model output and determine whether PK-01 is an alternative analysis; label it accordingly or resolve the mismatch if the table claims to summarize PK-02. (4) Verify the analysis identity and any prespecified interpretation rule. The estimate is 10% above 1, and the interval ranges from 2% to 20% above 1, so an unqualified “no difference” needs review; do not invent a clinical threshold. (5) Consistency checks whether statements and sources agree. It does not establish that the model assumptions, methods, or scientific interpretation are adequate.