Creating Run Manifests

Why a Run Needs a Manifest

A model run is more than an estimate or an output file. To interpret or reproduce it, a reviewer needs to know which model and inputs were used, which software ran them, who or what initiated the run, when it started, and where its outputs were written. If these details are scattered across filenames, shell history, and personal notes, it is easy to confuse one run with another.

A run manifest puts these details in one structured, machine-readable record. It is useful both to a person reviewing a result and to software that checks, indexes, or summarizes runs. A manifest does not itself prove that a run succeeded, preserve every artifact, or capture every detail of the execution environment. It records the specified facts and references in a consistent form; those other responsibilities require separate workflow steps.

Designing the Record

Start with the questions a reader must be able to answer. Which run is this? Which model, data, and configuration were used? Which software was invoked? When did execution start, and who or what initiated it? Where should its outputs be found? Each answer should have a designated field rather than being inferred from a filename or a directory name.

Use the stable run identifier established for the workflow as the manifest's run identifier. Do not make the identifier do the work of the manifest: a compact identifier may help locate a run, but it cannot reliably encode every input, software version, or output path. In particular, avoid interpreting an identifier by splitting it into tokens unless its format explicitly defines that behavior.

A path is a reference, not the content of a file. Relative paths are compact, but their meaning depends on a documented base directory; absolute paths avoid that ambiguity on one system but may not resolve on another. A URI can be appropriate for a shared storage location. Choose a convention and apply it consistently. If a digest is recorded, specify its algorithm and the exact file bytes to which it applies. A digest can help detect a mismatch; it does not make the referenced file available.

For timestamps, include a timezone. A value such as 2026-05-12 09:30 is ambiguous because it does not say whether the time is UTC or local time. A value such as 2026-05-12T09:30:00Z identifies a UTC instant. For output locations, named roles such as estimates, log, and diagnostics are more informative than a single field called output.

Why Explicit Fields Improve Traceability

The minimum field set follows from the questions needed to distinguish and review a run. If the model reference is absent, a reader cannot identify the model input. If the data or configuration reference is absent, the model input alone does not determine the analysis that was run. If the software, start time, or operator is absent, those execution facts cannot be recovered reliably from the model and input references. If outputs are not located explicitly, a reviewer must guess where to look. Thus each required field closes a different information gap; duplicating information in a filename does not close a gap unless the filename convention is itself defined, stable, and parsed consistently.

This reasoning also explains why references should be specific. Suppose two files are both called analysis.csv. The filename alone does not identify which data were used. A path narrows the reference; a content digest can further distinguish files that happen to share a name. The same principle applies to a model or configuration. The manifest becomes useful when its references resolve to the intended objects under the workflow's stated conventions.

Worked Manifest Examples

Validation and Failure Modes

A manifest can be syntactically valid yet operationally misleading. Validation should therefore check both its structure and its references. Structural checks include whether the record parses as the chosen format, required keys are present, values have the expected types, timestamps follow the agreed format, and output roles use the schema's defined names. Reference checks can confirm that input paths resolve and, when recorded, that calculated digests match the referenced files.

Validation should happen at a well-defined point: after run-specific values are assembled and before the manifest is treated as the authoritative description of that run. A missing configuration should not be silently replaced with an empty string. If a field is genuinely inapplicable, the schema should define how to represent that case, such as an explicit null for an optional field. Required fields should not be made optional merely to let incomplete records pass.

• Ambiguous time: A local timestamp without a timezone may refer to different instants. Record an explicit offset or UTC.
• Unpinned software label: “Latest” can change between runs. Record a specific version rather than a moving label.
• Generic paths: A filename such as data.csv may resolve to different files in different working directories. Define the path base or use a resolvable URI.
• Output overclaim: A planned output path does not show that a file was created. Keep the manifest's location record distinct from claims about run completion.
• Schema drift: A parser may misread renamed or retyped fields. Increment and validate the schema version when the record structure changes.
• Unescaped or malformed JSON: A hand-edited file may fail to parse. Use a JSON serializer and validate the result rather than relying only on visual inspection.

Choosing a Format and Recognizing Limits

JSON is a practical choice when manifests need to be parsed by common programming languages and tools. YAML can be easier for people to edit, but its parsing rules and implicit type conversions require care. Either format can work if the schema is explicit, the same field meanings are applied consistently, and the records are validated. A spreadsheet may be convenient for browsing multiple runs, but it is not a substitute for a defined per-run record when fields, types, and parsing behavior matter.

A manifest is not the model, the data, or the configuration; references can break if the referenced materials move or become unavailable. Nor does recording a software version capture all execution details. The next workflow concern is recording the broader execution environment; here, record the software identity and version without claiming that this alone fully describes the environment. Likewise, recording output locations is not the same as preserving output artifacts or documenting run status.

The manifest is valuable because it reduces avoidable ambiguity. Its quality depends on a clear schema, accurate values collected for the run, consistent path and time conventions, and validation. It should be reviewed when the workflow changes: a new required input or a changed field meaning may require a schema update rather than an informal addition.