Why Simulation Provenance Matters
A simulation result is not defined by its output table alone. The same table may depend on a particular model implementation, input data, scenario configuration, sampling procedure, random-number setup, software environment, and execution plan. If any of these are missing or ambiguous, a later analyst may be unable to determine what was run, assess whether a rerun is comparable, or investigate a discrepancy.
Simulation provenance is the recorded evidence linking a result to the computational conditions that produced it. Its purpose is practical: to make the run identifiable, its inputs verifiable, and its reproduction attempt well specified. Recording provenance does not establish that a model is appropriate or that a simulation is free of errors. Those are separate scientific and validation questions.
The record should distinguish a declared setting from evidence that the setting was used. For example, a configuration file may specify 1,000 replicates, while the run may stop after 640 because of an error. Preserve both the intended configuration and the observed execution status. Do not infer successful completion from the presence of a partially written output file.
What a Reproducible Record Must Identify
A practical record has several linked parts. These can be stored in one manifest or in related files, provided each link is stable and unambiguous.
- Run identity: a unique run identifier, creation time with time zone, operator or automated job identity, and a status such as completed, failed, or stopped.
- Model identity: model source version or revision, the exact source artifact or its cryptographic digest, and, when relevant, the compiled model artifact and build settings.
- Input identity: the exact input files or immutable data release, their versions or digests, preprocessing code and settings, and units or transformations that affect interpretation.
- Scenario definition: the actual values and units of scenario inputs, including regimen, population, duration, and any scenario-specific assumptions needed to interpret the run.
- Sampling and random-number configuration: replicate counts, virtual-population definition, sampling distributions and parameterization, sampling method, seed or seed identifiers, random-number generator and version, and the mapping of streams to work.
- Execution environment: software and package versions, operating system, relevant numerical libraries, hardware or accelerator when material, parallel worker count, and command or workflow configuration.
- Execution evidence and outputs: start and end times, completed and failed replicate counts, warnings or error logs, and paths plus digests for the outputs retained from that run.
Record values in machine-readable form where possible, but preserve enough human-readable context to make the fields interpretable. A field such as seed = 731 does not say which generator used it, whether it initializes one stream or many, or how streams were assigned to workers. Likewise, a model label such as “final model” is not a stable identifier unless it resolves to an exact revision or artifact.
Seed Records Are Necessary but Not Sufficient
A random seed initializes a pseudo-random number generator; it does not, by itself, specify the entire sequence of random values used by a simulation. Reproducing a stochastic run may also depend on the generator algorithm and its implementation, the number and organization of streams, the order in which random values are requested, and the allocation of work to workers. The earlier material on reproducible random-number streams and reproducible parallel simulation establishes why these choices matter.
For a useful record, write down the seed together with the generator name and version, stream or substream identifiers, and the rule used to assign streams to replicates or workers. Also record whether the run was serial or parallel and the relevant worker configuration. A statement such as “seed 731 used” is incomplete if a future analyst cannot determine which generator or stream arrangement it describes.
Even complete metadata cannot guarantee identical results on every machine. Different compiler settings, numerical libraries, hardware, or parallel execution order can change floating-point calculations. State the intended reproducibility target: for example, identical output files, identical numerical results on a specified environment, or agreement within a defined numerical tolerance. Do not claim bit-for-bit reproducibility unless it has been tested under the relevant conditions.
Worked Example: Assembling a Run-Specific Manifest
Consider a hypothetical simulation of exposure under two dosing regimens. The team runs 500 replicates per regimen, for a total of \(2 \times 500 = 1{,}000\) planned replicate-regimen combinations. The model is stored at revision pk-model-v3.2; the population input is the immutable release pop-2026-04. A provenance record could contain the following values.
| Field | Recorded value |
|---|---|
| Run ID and status | sim-2026-04-18-017; completed |
| Model | pk-model-v3.2; source revision 8f21c7a; compiled artifact digest recorded separately |
| Population input | pop-2026-04; exact input-file digest recorded |
| Scenario definitions | Regimen A: 100 mg every 24 h; Regimen B: 200 mg every 48 h; both for 14 days |
| Sampling | 500 replicates per regimen; parameter distributions and covariate sampling settings stored in the configuration file |
| Random numbers | Generator and version recorded; root seed 731; stream assignment recorded in the run configuration |
| Execution | Software build and package versions, operating system, and four-worker parallel configuration recorded |
| Outputs | Output-table path and digest recorded; 1,000 planned combinations and 1,000 completed |
The count check is \(2\) regimens \(\times 500\) replicates per regimen \(=1{,}000\) planned combinations. Since the recorded completed count is also 1,000, the run's count is consistent with its plan. This check does not prove that all outputs are scientifically correct, but it flags a discrepancy if, for example, only 980 combinations appear in the retained output.
The record identifies both the regimen values and the files that encode the model and population. If the model label were the only model information, later edits could make that label point to different code. The source revision and artifact identity instead provide a way to check what was used. The stream assignment and worker configuration are included because recording the root seed alone would not fully describe the random-number setup.
Linking Configuration, Execution, and Output
A provenance record should preserve the distinction among planned, executed, and retained information. Planned information comes from the configuration: requested scenarios, replicate count, sampling rules, and software settings. Executed information comes from the run: the start and end times, actual worker configuration, warnings, and completion counts. Retained information identifies what was saved: output files, logs, and any intermediate artifacts needed to interpret the run.
This distinction matters when a run is interrupted or partially rerun. Suppose a configuration requests 600 replicates, but the execution log shows 572 completed and 28 failed. A record that says only “600 replicates” is misleading if it implies 600 results were produced. Record the request and the outcome separately, and identify whether the final output includes only completed replicates or also contains failure records. This makes the run's status auditable without changing the output's meaning.
Use stable references to connect the parts: a run ID in the manifest, a configuration-file digest, and explicit output paths or digests. A digest can detect whether a file differs from the one originally recorded; it does not explain what the file means or whether its contents are valid. Retain the source files or ensure they are available from an immutable repository. A digest pointing to an unavailable input verifies identity only if the original artifact can still be retrieved.
Worked Example: Documenting an Interrupted Simulation
A hypothetical run requests 300 replicates for each of three scenarios. The intended total is \(3 \times 300 = 900\) scenario-replicate combinations. The run stops after completing 270 replicates in Scenario 1, 300 in Scenario 2, and 245 in Scenario 3; five attempted replicates in Scenario 3 fail.
The number of completed replicates is \(270 + 300 + 245 = 815\). Including the five failed attempts, the recorded attempts total \(815 + 5 = 820\), which is short of the planned 900 by \(900 - 820 = 80\) attempts. A useful record therefore reports 900 planned, 820 attempted, 815 completed, and five failed, rather than labelling the run simply “900 replicates.”
The record should also preserve the stop reason, relevant log, and whether the output file contains failed attempts or only successful results. If a later rerun completes the remaining work, it must be identified as a new execution or as a documented continuation with explicit replicate identities and stream assignments. Otherwise, an analyst cannot tell whether the results represent a fresh run, an appended continuation, or a mixture.
Justification: Why the Record Must Follow the Computational Chain
A simulation result can be viewed as the output of a computational mapping:
Here, \(Y\) is the generated result; \(M\) is the model implementation; \(I\) is the set of inputs; \(S\) is the scenario configuration; \(Q\) is the sampling specification; \(R\) is the random-number configuration; and \(E\) is the execution environment. The notation is schematic: some components may be files, settings, or execution events rather than single numerical values.
If two runs differ in any component that affects \(f\), their outputs may differ. Thus, identifying \(Y\) alone does not identify the computation. Recording only the seed leaves \(M\), \(I\), \(S\), \(Q\), and \(E\) unspecified; recording only the model revision leaves the other components unspecified. A provenance record is adequate for a reproduction attempt only to the extent that it identifies the inputs to the computation and the execution conditions relevant to the stated reproducibility target.
This argument also explains an important limit: a complete record supports reproduction and investigation, but does not prove that every setting was appropriate or that the implementation is correct. Reproduction asks whether the computation can be repeated under specified conditions. Validation asks whether the computation behaves as intended. Provenance helps make both inquiries possible, but is not a substitute for either.
Worked Example: Comparing Two Reproduction Attempts
A hypothetical exposure summary is reported as 12.40 mg·h/L. An analyst repeats the simulation and obtains 12.40 mg·h/L to the displayed precision. The recorded configuration shows the same model revision, population input, scenario, sampling specification, random-number generator and stream assignment, and software environment. The output agrees to the reported precision, although the unrounded values should be compared before claiming exact numerical identity.
A second analyst also reports 12.40 mg·h/L but used a different population release and did not record the generator version. The matching rounded summary is not enough to establish that this analyst reproduced the original computation. Different underlying replicate results can yield the same rounded summary. The missing input and generator information also prevent a complete comparison of computational conditions.
The unit is part of the interpretation: 12.40 mg·h/L is an exposure summary, not a dimensionless value. The provenance record should preserve the output definition and units alongside its file identity, so that identical numbers are not mistaken for identical quantities.
Diagnostics, Failure Modes, and Practical Limits
Before archiving a run, check that the record is internally consistent and complete enough for its intended use. At minimum, compare planned and observed counts, confirm that each scenario has explicit values and units, verify that input and model references resolve to retained artifacts, and check that the stated random-number details match the executed configuration. Confirm that output files and logs are associated with the same run ID.
Common warning signs include a generic model label without a revision, a seed without a generator, software recorded only as “current,” missing units in scenario inputs, and output files with no link to the run configuration. A record can also fail through ambiguity: “four workers” may mean four processes or four threads; “500 replicates” may mean per scenario or across all scenarios. Define such terms in the record rather than expecting a future reader to infer them.
Keep sensitive data and credentials out of manifests and logs. Record stable identifiers and access-controlled locations where needed, not passwords, tokens, or unnecessary personal information. Finally, do not treat a file digest as a replacement for an archived artifact, or a successful rerun as evidence that the model is scientifically valid. Provenance has a bounded role: it documents the computational chain and makes discrepancies diagnosable.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- Why is recording “seed 731” alone insufficient to specify a stochastic simulation?
- A run plans four scenarios with 250 replicates per scenario. It completes 980 replicates and records 12 failed attempts. How many attempts are unaccounted for relative to the planned total?
- Name two reasons to record both the model source revision and the compiled model artifact identity.
- Does a matching rounded output summary prove that two simulation runs are identical? Explain.
- What is the difference between planned, executed, and retained information in a provenance record?
Solutions. (1) The seed does not identify the generator and version, stream arrangement, stream-to-work assignment, or execution settings that can affect the random sequence used. (2) The planned total is \(4 \times 250 = 1{,}000\). Recorded attempts are \(980 + 12 = 992\), so \(1{,}000 - 992 = 8\) attempts are unaccounted for. (3) The source revision identifies the model code, while the compiled artifact identifies the executable actually used; build settings can also affect the artifact. (4) No. Rounding can conceal differences in underlying results, and matching summaries do not establish matching inputs or execution conditions. (5) Planned information describes the requested configuration; executed information records what happened, including completion and failures; retained information identifies the saved outputs, logs, and artifacts.