Auditable Reports With Parameters and Provenance

From Results to an Auditable Run

A single-cell report may show a clustering, a marker-gene table, or a cell-type annotation, but those results are difficult to assess if the reader cannot tell which inputs and execution choices produced them. A figure without traceable context can be visually clear yet leave basic questions unanswered: Which count matrix was used? Which version of the analysis code ran? Were the displayed parameters the values actually used, or merely the defaults in a configuration file?

An auditable report links results to the specific run that produced them. It does not replace the analysis, validation checks, or a retained workflow record. Its purpose is to expose a compact, interpretable account of the run: enough information for a reviewer to identify its inputs and choices, understand its status, and locate relevant outputs. The earlier material on computational and statistical reproducibility distinguishes reproducing computation from reproducing scientific conclusions; a run summary supports scrutiny of both, but cannot guarantee either.

What the Report Should Expose

A useful summary is compact, but it should not achieve compactness by hiding distinctions that affect interpretation. At minimum, a reader should be able to identify the following:

• Run identity and time. Give a unique run identifier and a timestamp with a stated time zone, or an unambiguous standard such as UTC. A timestamp helps distinguish executions; it does not identify the inputs by itself.
• Inputs. Name the input artifacts and record stable identifiers such as file checksums, object identifiers, or immutable data-version identifiers. State relevant dimensions and identifier conventions where needed to interpret the data. A checksum supports comparison of file identity, not validation of its scientific contents.
• Code. Record the repository and commit identifier, and report whether the code tree had uncommitted changes. A commit alone does not identify uncommitted modifications that affected execution.
• Software environment. Record the language and relevant package versions, ideally alongside a reference to the locked environment. Listing a few package versions is not equivalent to recording every system dependency, but it gives a reader important context.
• Resolved configuration. Report the actual values used for consequential parameters, not only a path to a configuration file. Include units, scale, and interpretation when a value could otherwise be ambiguous.
• Randomness. Record the seed values and, where relevant, the random-number-stream or parallel execution strategy. A seed is useful context, but does not by itself promise identical results across different software, hardware, or execution order.
• Status and outputs. Separate workflow execution status from validation status. Name important outputs and their locations or identifiers. Indicate warnings, failures, and skipped steps rather than presenting a partial run as a complete success.

The report should be generated from the values and events recorded by the actual execution. If its author manually retypes the parameters, versions, or status after the fact, a discrepancy can arise between the report and the run. This is especially likely when defaults, environment variables, or command-line overrides change the resolved values. The configuration file can remain a useful input, but the report should expose the resolved settings that governed the analysis.

A Compact Summary as a Structured Record

A run summary can be presented as a table, a structured text block, or a generated section in a longer report. What matters is that fields have clear meanings and that separate concepts are not collapsed into one label. For instance, “complete” might mean that all workflow tasks finished, whereas “validation passed” means that the specified checks met their criteria. Neither statement alone says that the analysis is statistically appropriate for its scientific question.

The following compact record is for a hypothetical single-cell workflow. The numbers and identifiers are illustrative. The input matrix is described as genes by cells; an actual report should specify its own dimension convention and what those dimensions count.

Why Resolved Values and Status Matter

A report that says only “parameters: config.yaml” leaves a reviewer dependent on a separate file and does not reveal whether the run used that file unchanged. A configuration may include defaults, substitutions, or overrides. Reporting resolved values gives the reader a direct account of the settings applied in that execution. The same logic applies to software: a package name without a version is often insufficient to distinguish behavior across installations.

Status also needs more than one field. Consider a workflow in which preprocessing and clustering finish, but a required validation check fails. “Workflow completed” could correctly describe execution of all tasks, yet “validation failed” must remain visible. Conversely, a task can be deliberately skipped under the recorded configuration; reporting it as successful would be misleading. A clear report states which steps ran, which checks were performed, and the outcome of each relevant check.

A practical design treats the summary as a record generated from run metadata, rather than as free-form narrative alone. In a simplified representation, let \(R\) denote a run record:

where \(I\) identifies inputs, \(C\) identifies code, \(E\) describes the software environment, \(\theta\) contains resolved analysis settings, \(S\) records seed and randomness details, \(W\) records workflow and validation status, and \(O\) identifies outputs. This notation is a way to organize information, not a claim that one fixed schema is suitable for every workflow.

A Justification for Recording the Run Components

The run-record structure can be justified by asking what a reviewer needs to distinguish two executions that yield different outputs. If the input identity is unknown, a changed output might reflect changed data. If code or environment differs, it might reflect changed implementation or dependencies. If resolved parameters or random-number handling differ, those are additional possible sources of variation. If status and output identity are absent, the reviewer may not know whether the reported result came from a complete run or which artifact to inspect.

This is not a mathematical guarantee that a particular list is sufficient for every software stack. It is a traceability argument: omitting a category can leave a plausible explanation for differences unrecorded. Recording the categories makes specific comparisons possible. For example, if two runs have matching input checksums, code commits, software environments, resolved settings, and recorded randomness details, then the report supports a more informative comparison than a pair of unlabeled figures. It still does not prove that the executions are identical in every unrecorded detail.

Diagnostics, Alternatives, and Limits

Warning signs include a report that lists only a configuration filename, an unversioned software stack, a seed without any description of which stochastic steps used it, output names without stable identifiers, or a single “success” field that conceals failed checks. Other warning signs are internally inconsistent counts, a checksum that was computed for a different file than the one analyzed, and a report that was edited manually without being regenerated after the run. A report should also be checked for accidental disclosure of access tokens, private file paths, or sensitive sample identifiers; auditability does not justify publishing secrets.

A concise prose summary is easy to read but can omit fields or drift from the run. A structured manifest is easier to validate and compare programmatically, but can be hard to interpret without a human-readable explanation. A robust approach uses structured metadata as the source of truth and renders a compact human-readable summary from it. For smaller workflows, a carefully maintained table may be adequate, provided it is generated or checked against the execution record. None of these formats is a substitute for retaining the input artifacts or a secure, stable reference to them.

Checksums have a specific, limited role. If the same checksum algorithm is applied to two files and the resulting digests match, this is strong evidence that the files have the same byte representation, subject to the properties of the algorithm and the exact hashing procedure. It does not prove that the data were collected correctly, that cell identifiers are valid, or that a particular analysis is suitable. Similarly, a recorded seed supports interpretation of a stochastic run, but exact equality of results is not guaranteed across changed software versions, parallel execution arrangements, or hardware. These limitations complement the earlier material on seed management and reproducible randomness.

The report’s role is therefore evidential and operational: it helps readers find what was used, what was run, and what happened. It should not be presented as proof that the results are unbiased, that assumptions are satisfied, or that observed biological patterns are causal. Those conclusions require appropriate scientific and statistical reasoning beyond provenance records.

Common Mistakes

• Reporting only a configuration filename. A file reference does not reveal overrides or resolved defaults. Report the effective values used in the run.
• Treating a checksum as a quality certificate. A checksum helps identify a file representation; it does not establish that the input is valid or scientifically appropriate.
• Using one status label for everything. A workflow can finish while validation fails. Report execution and checks separately.
• Interpreting a fixed seed as a guarantee. A seed is one reproducibility detail, not a guarantee of bit-for-bit agreement across environments or parallel execution choices.
• Inferring one-sided missing counts from a difference in set sizes. If two identifier sets have different sizes, their size difference is only a net difference. The number of identifiers present in one set but absent from the other requires a direct comparison or additional assumptions.
• Confusing a traceable result with a scientifically validated result. Provenance makes a result easier to inspect; it does not by itself establish the validity of the analysis or its interpretation.