Why an Audit Trail Matters
A population PK report may state that a model estimated a typical clearance of 8.40 L/h, describe a covariate relationship, and summarize predictive checks. A reviewer should be able to determine which analysis dataset and model run support each statement, which program created each output, and how the data were prepared. If a number in a report cannot be linked to a specific result artifact and run, checking it may require guesswork or an informal reconstruction.
An audit trail is the documented chain of relationships among analysis inputs, transformations, software and configuration, model runs, output artifacts, quality-control checks, and report statements. It is not simply a folder of files, nor does it mean that every intermediate file belongs in the report. Its purpose is to make the path from evidence to conclusion identifiable and checkable.
The chain is best treated as a set of linked records rather than as a single filename. Each record should have a stable identifier, a meaningful description, and links to its immediate inputs and outputs. For example, a model run should identify the exact analysis dataset, model-control program, software version, execution settings, and output files used. A report statement should identify the result table, figure, or other artifact on which it relies.
Build the Chain From Data to Claim
A practical audit trail follows the direction of analysis while preserving enough information to follow it in reverse. The usual links are:
Record the source extract or transfer, its receipt date, version or snapshot identifier, and relevant file fingerprints. Keep the source artifact unchanged.
Identify the programs and rules that transform source records into the analysis dataset, including exclusions, derived variables, and handling decisions that affect observations.
Record the analysis dataset identifier, model program version, software and environment, run settings, and any random-number seed needed for stochastic procedures.
Link the run to its parameter estimates, diagnostics, logs, and other outputs. Record which quality-control checks were performed and their outcomes.
Give each important claim a location in the report and a reference to the exact output artifact and run that support it.
Use identifiers that remain unique when filenames, locations, or labels change. A human-readable identifier such as RUN-042 is useful, but it should be paired with enough recorded context to distinguish its inputs and configuration. A cryptographic hash can help detect whether a file's contents changed: identical bytes produce the same hash, while a changed file will ordinarily produce a different one. A hash alone does not explain what a file means, who created it, or whether it is the correct input. It complements, rather than replaces, descriptive metadata and documented links.
The audit trail also depends on explicit assumptions about record keeping. Identifiers must be unique within the project; the referenced files and logs must be retained and accessible; and the recorded versions must describe the files actually used. If a model program changes after a run, for example, the updated program must receive a new version identifier rather than silently overwriting the old one.
Worked Examples of Traceability
Worked Example: Linking Source Records to an Analysis Dataset
Suppose a hypothetical population PK analysis begins with source extract SRC-017, containing 128 subjects and 1,024 concentration records. The preparation program is identified as PREP-03. Its documented rules exclude five subjects: three withdrew before any usable post-dose observation, and two have no post-dose records. The program also identifies 16 duplicate concentration records and removes one copy of each. Below-quantification records are retained and flagged under the analysis plan rather than removed.
The subject count reconciliation is \(128 - 5 = 123\) subjects in analysis dataset ADS-021. The concentration-record reconciliation is \(1{,}024 - 16 = 1{,}008\) records after duplicate removal. The 24 records flagged below quantification remain among those 1,008 records. That count should be confirmed from the dataset, not inferred from the fact that the records were flagged.
A useful lineage record for ADS-021 would name SRC-017 and PREP-03 as inputs, identify the preparation-program version and its configuration, and record the output dataset fingerprint. A reviewer can then check that the subject and record counts match the documented transformations. If the dataset has 1,008 records but the run manifest says it used a different dataset identifier, the count agreement does not repair the broken link: the run's actual input must be resolved.
This example illustrates a key distinction. A reconciliation checks whether expected transformations are reflected in counts; it does not prove that every record was transformed correctly. Record-level checks, such as inspecting duplicate keys and verifying the below-quantification flag against the source status, provide additional evidence.
Worked Example: Comparing Two Model Runs
A hypothetical model run, RUN-042, uses ADS-021, model program MODEL-08, software version 7.4, and the recorded estimation settings. It produces output set OUT-042. The run manifest records the input and program fingerprints, run date, software version, settings, log location, and output fingerprints. The report gives a typical clearance estimate of 8.40 L/h from OUT-042.
A second run, RUN-043, is created as a reproducibility check. Before comparing estimates, the analyst checks that both manifests identify the same dataset, model program, software version, estimation settings, and relevant environment. If any of those differ, the runs are not a like-for-like rerun; the difference must be documented and its effect evaluated rather than treated as unexplained numerical variation.
For a deterministic estimation run with the same inputs and settings, compare the key output fingerprints and numerical results. Suppose RUN-043 also reports clearance of 8.40 L/h and its other designated parameter estimates match to the recorded output precision. This supports consistency of those results, while matching output fingerprints would provide stronger evidence that the output files themselves are identical. If the estimation procedure is stochastic, record the random-number seed and compare results using a prespecified tolerance appropriate to the procedure; do not require byte-for-byte identity when stochastic variation is expected.
A successful rerun does not establish that the model is scientifically adequate or that the inputs are correct. It establishes only a limited reproducibility claim: under the recorded conditions, the specified process generated results that meet the stated comparison criteria.
Worked Example: Tracing a Reported Clearance Estimate
Suppose a report sentence states, “The typical clearance estimate was 8.40 L/h.” Its traceability record should point to the specific table and row containing that estimate, such as TABLE-12, row CL, and identify OUT-042 and RUN-042 as the source. The run record in turn points to MODEL-08 and ADS-021. The table's units and the report sentence must agree.
For a simple consistency check, suppose the stored table value is 8.397 L/h and the reporting rule is to round to three significant figures. The digits are 8, 3, and 9; the next digit is 7, so rounding gives \(8.40\) L/h. The report value is therefore consistent with the table under the stated rule. The trailing zero communicates the reported precision; it does not imply that the estimate is known without uncertainty.
If the table instead contains 8.34 L/h, the sentence cannot be justified by the stated rounding rule. The discrepancy should be resolved at the source: confirm the table version and run identifier, then correct either the report or the linked output reference. Manually editing the sentence to appear consistent without recording which value is authoritative would weaken the audit trail.
Quality-Control Checks and Failure Modes
A useful audit trail includes checks designed to catch breaks in the chain, not just a list of identifiers. Checks should be defined before they are needed, recorded with their results, and linked to the artifact or run they assess. At minimum, consider the following:
- Input identity: verify that the run manifest names the intended dataset and model-program versions, and that fingerprints match the retained files.
- Data reconciliation: compare subject counts, record counts, and key status counts before and after documented transformations. Investigate unexplained differences rather than forcing counts to agree.
- Run completeness: confirm that the run has a usable log, the expected outputs, and no unaddressed termination or execution errors.
- Output integrity: check that output files are linked to the correct run and have not been replaced after review. Record any later changes as new versions.
- Report consistency: verify that values, units, labels, and referenced figures or tables in the report agree with the identified output artifacts.
- Independent review: have a second person or a separate QC program check selected links and report values, where the project process permits.
Common warning signs include a report citing an output file with no run identifier, a run manifest pointing to a mutable filename without a version or fingerprint, unexplained differences between reruns, and a figure whose displayed dataset cannot be established. Missing links should be treated as uncertainty about provenance, not repaired by assuming that the newest file is the right one.
Another mistake is to treat a matching hash as proof of analytical correctness. A hash can help establish that two retained files have the same contents; it cannot show that the data are appropriate, that the model was specified correctly, or that a report interpretation follows from the output. Those questions require substantive review of the analysis and its evidence.
Choosing the Right Level of Documentation
An audit trail complements technical appendices and the main report, but it serves a different purpose. The report presents findings and interpretation; an appendix can give detailed model specifications and supporting material; the audit trail establishes which versions and artifacts connect those materials. Putting every file and log in the main report would obscure the scientific argument. Providing only a polished report without traceable sources, however, makes its numerical claims difficult to verify.
A compact run manifest and an output inventory are often more useful than a long narrative. A manifest might include a run identifier, input dataset identifier, program version, software and environment details, settings, seed when relevant, timestamps, and links to logs and outputs. An output inventory can identify which tables and figures support which claims. For high-impact results, record the QC checks and their outcomes alongside these links.
Traceability has practical limits. Systems may not retain all execution details, legacy files may lack fingerprints, and proprietary or changing software environments may be difficult to reconstruct exactly. State these limitations plainly. A documented limitation is more useful than implying that a run is exactly reproducible when its dependencies were not preserved. Where exact rerunning is not feasible, preserve the available inputs, outputs, logs, and version information and explain what could and could not be independently checked.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- A run manifest identifies the dataset only as “final.csv,” and that filename has been overwritten twice. What information is missing, and what should be recorded?
- A source extract has 200 records. A documented step removes 12 duplicate records and retains 8 below-quantification records. How many records should remain after this step, assuming no other changes?
- Two model runs produce identical output fingerprints, but use different model-program versions. What can you conclude, and what can you not conclude?
- A report gives a value of 5.67 mg/L, while its linked output table reports 5.67 mg/h. What should be checked before the report is finalized?
Solutions. (1) The identifier is not sufficient because the filename was reused. Record a stable dataset version or identifier, its fingerprint, and the run's link to that exact version; also retain the relevant file. (2) \(200 - 12 = 188\) records remain. The eight below-quantification records are retained, so they are included in 188. (3) Identical output fingerprints show that the retained output files have identical contents, but the runs are not a controlled comparison of the same program version. This does not establish that the different program versions are equivalent or that either analysis is scientifically correct. (4) Check the source output, the quantity represented, and the units in both the table and report. Resolve the mismatch and correct the report or its evidence link; do not assume that the numerical agreement makes the units interchangeable.