Why Record Run Provenance?
A model result is useful only if its origin can be identified. A parameter estimate or diagnostic plot, considered by itself, does not say which source code, settings, software environment, or execution produced it. A run manifest is a compact record that links a specific execution to those details. It complements the project organization and pinned environment practices established in the earlier material on those subjects.
A manifest is not a model validation report, and it does not establish that an analysis is scientifically appropriate. Its purpose is narrower: to make a run identifiable and to preserve enough context for someone to understand what was executed and where its outputs were placed. A manifest is most useful when it is created as part of the run, rather than reconstructed from memory afterward.
What a Useful Manifest Records
The manifest should answer a practical question: which exact computational conditions correspond to this run? Each field should have a clear meaning, a stable format, and a value that can be checked. A field that is present but ambiguous—for example, “latest model” or “current settings”—does not provide reliable provenance.
| Information | What to record | Why it matters |
|---|---|---|
| Run identity | A unique run identifier and, where useful, the run directory | Distinguishes this execution from other runs, including repeated runs with the same settings |
| Source revision | The repository commit identifier and whether the working tree had uncommitted changes | Identifies the committed code and warns when the executed files may differ from that revision |
| Configuration | The configuration file or identifier, its checksum if used, and the resolved values supplied to the analysis | Records both the source of settings and the effective settings after defaults or overrides |
| Execution | Start and end timestamps, time zone, command or entry point, and completion status | Identifies when and how the run was launched and whether it completed |
| Environment | Relevant software names and versions, operating system, and the lock-file or environment specification identifier | Connects the run to the software context in which it executed |
| Inputs and outputs | Input identifiers and output paths; checksums may be recorded for files | Links the execution to the data and artifacts it consumed and created |
The source revision is not enough if code was modified but not committed. Record the revision and a clean-or-dirty status; if the working tree was dirty, preserve or identify the exact changes used. Similarly, a configuration file alone may not capture settings supplied through command-line arguments, environment variables, or defaults. Record the resolved configuration: the values the program actually used.
Execution timestamps should include a time-zone indicator, preferably a consistent standard such as UTC, and should use an unambiguous format. Do not infer elapsed time from timestamps that omit their time zone or refer to different clocks without qualification. Environment details should be sufficient for the analysis at hand: in a pharmacometric workflow, this could include the modeling software and version, the language runtime, relevant package versions, and the environment specification used to install them.
Worked Example: Assembling a Manifest for a Model Run
Suppose a hypothetical population pharmacokinetic analysis is run from a project repository. The run uses a committed source revision, a YAML configuration file, a pinned software environment, and writes a parameter-estimate table and diagnostic plots to a run-specific directory. A compact manifest might record the following.
| Field | Hypothetical value |
|---|---|
| Run ID | PK-2026-10-28-001 |
| Source revision | Commit 8f31c2a; working tree clean |
| Configuration | configs/base.yaml; resolved dose group = 100 mg; seed = 417; configuration checksum recorded |
| Execution | Entry point: scripts/run_model; started 2026-10-28T09:10:00Z; ended 2026-10-28T09:12:30Z; status: completed |
| Environment | Modeling software version 7.4.1; R version 4.4.1; environment lock-file checksum recorded |
| Outputs | runs/PK-2026-10-28-001/estimates.csv; runs/PK-2026-10-28-001/diagnostics/; output checksums recorded for files |
This record links the result to a specific commit and reports that the working tree was clean, so there is no known uncommitted source change to account for. It names the configuration file and includes example resolved settings; in an actual run, the resolved values should cover all analysis-relevant settings, not just the two shown here. The environment fields identify software versions, while the lock-file checksum helps distinguish environment specifications that might otherwise have the same filename.
The output entries are locations, not guarantees that the files remain available. Recording checksums for output files can help determine later whether a file at a recorded location has changed. The manifest should itself be stored with the run artifacts in a predictable location, for example as a manifest file inside the run directory.
Recording and Checking Execution Time
The elapsed duration of a run is the end time minus the start time, expressed in a consistent unit. The calculation is straightforward only when both timestamps refer to the same time standard and are parsed as dates and times rather than treated as unrelated text.
Worked Example: Calculating Elapsed Time From UTC Timestamps
For the hypothetical run above, the start time is 2026-10-28T09:10:00Z and the end time is 2026-10-28T09:12:30Z. Both timestamps are on the same day and use UTC.
Convert the difference to seconds. The hour fields match, and the minute and second fields give:
The elapsed time is therefore 150 seconds, or 2 minutes 30 seconds. This is a duration between the recorded timestamps; it is not evidence that the software used 150 seconds of processor time. If a run crosses midnight, the date must be included in the calculation. If the timestamps are from different time zones, convert them to a common time standard before subtracting.
Why These Fields Work Together
A result can be associated with a run only if the run is identified across several dimensions. The source revision identifies code; the resolved configuration identifies settings; the environment fields identify relevant software context; the timestamps and status describe the execution; and output locations identify the resulting artifacts. In combination, these fields provide a practical account of what produced the result.
This logic can be expressed as a record of the relationships that provenance must preserve:
Each component answers a different question. If two runs share a commit but use different resolved configurations, they are not the same computational setup. If source and configuration match but the software environment differs, the runs may still behave differently. If the first four components match but output paths are missing, a reviewer may not be able to identify which files correspond to the run.
A manifest therefore works as a set of linked identifiers rather than as a narrative description. To make the links dependable, use exact values where possible: a commit identifier rather than a branch name, a configuration checksum rather than “the current config,” and an explicit output path rather than “results folder.” Checksums can detect changes to files, but they do not explain what a file means or establish that its contents are correct.
Worked Example: Diagnosing a Configuration Mismatch
Two hypothetical runs report the same source commit and environment. Run A records a resolved dose group of 100 mg, while Run B records 200 mg. Both manifests point to a file named configs/base.yaml. The result tables differ.
The shared filename does not establish that the configurations were equivalent. The file may have changed between runs, or a command-line override may have supplied a different dose group. Because the resolved settings are recorded, the manifests show that these executions used different dose groups. Their differing results cannot be attributed to a software change on the basis of this comparison; the configuration also differs.
A stronger record would include the configuration checksum for each run and identify any overrides. If the checksums differ, the underlying configuration files differ. If checksums match but resolved settings differ, an override or another configuration mechanism must be investigated. The manifest does not by itself identify the cause, but it makes the discrepancy visible and narrows the investigation.
Diagnostics, Failure Modes, and Alternatives
A provenance record is suspect when important values are absent, mutable, or difficult to interpret. Check for the following failure modes when creating or reviewing one:
- Branch name recorded instead of a revision identifier. A branch can point to different commits over time. Record the commit identifier and the working-tree status.
- Dirty source state ignored. A commit identifies committed content, not uncommitted edits. If the working tree was dirty, the manifest must flag that fact and identify or preserve the executed changes.
- Only the configuration filename recorded. A filename can refer to different content at different times, and it may not reveal defaults or overrides. Record resolved settings and, where useful, a checksum of the configuration file.
- Environment described as “latest” or “current.” Such labels can change. Record software versions and the relevant pinned environment specification.
- Timestamps lack a zone or use inconsistent clocks. The start and end may appear out of order or yield an incorrect duration. Use explicit time zones and a consistent clock source.
- Output directory recorded without specific artifacts. A directory may contain files from multiple runs or be moved. Use run-specific paths and, when appropriate, output checksums.
- Sensitive information copied into the manifest. Do not place credentials, direct identifiers, or protected data in a provenance record. Use approved, access-controlled identifiers for inputs.
A short free-text log is an alternative for informal exploration, and a workflow system may capture many run details automatically. A free-text note is flexible but can omit fields or use ambiguous labels. Automated capture is more consistent, but it can faithfully record the wrong configuration or fail to detect uncommitted source changes. In either case, the same core information should be checked. A manifest is an aid to traceability, not a substitute for code review, scientific judgment, or data governance.
There are limits to what a manifest can establish. A recorded commit, configuration, and environment do not guarantee that another person can reproduce a result: required inputs may be unavailable, hardware or external services may differ, or software may contain nondeterministic behavior. A checksum can show that two files differ or match, but not whether either file is scientifically valid. The manifest’s role is to make the recorded provenance explicit enough to support investigation and interpretation.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- A run manifest records a branch name but not a commit identifier. What important ambiguity remains?
- A run begins at 14:05:10 UTC and ends at 14:08:00 UTC on the same day. Calculate its elapsed time in seconds.
- Two runs use the same configuration filename but have different resolved values. Are they documented as having the same effective configuration? Explain.
- Why should a manifest record whether the source working tree was clean?
- Does recording an output checksum establish that the model result is scientifically valid? Explain.
Solutions. (1) The branch name can refer to different commits at different times, so it does not uniquely identify the source revision used; record the commit identifier. (2) The duration is \((8-5)\times60+(0-10)=180-10=170\) seconds, or 2 minutes 50 seconds. (3) No. The resolved values are the settings actually used, so different values mean the effective configurations differ, even if the filename is the same. (4) A commit identifies committed content only; uncommitted edits may have affected the run and need to be flagged or identified. (5) No. A checksum helps identify whether a file has changed, but it does not assess the model, data, or scientific validity of the result.