Why Analysis Provenance Matters
A reported value is not self-explanatory. A table may show an estimated clearance, a figure may show a diagnostic, and a narrative summary may quote an objective-function value. For independent review, the question is not only whether the value looks reasonable. It is also which project artifacts produced it: which report code selected the value, which output file supplied it, and which run, model, and input artifacts are associated with that output.
Reconstructing provenance means establishing and documenting those links from the project files that are available. It is a documentary trace, not a fresh run of the analysis and not a judgement that the model or data are scientifically correct. Those are separate review questions. Earlier material on packaging report artifacts and source references can help locate the evidence; the task here is to connect the located artifacts into a defensible chain.
What Counts as a Supported Chain?
A typical chain for a reported parameter estimate might run from a report table, back through the code that creates the table, to a particular run output, and from that output to the run record, model file, and input file. The exact artifacts vary by project. Some reports read a structured results file; others read a database or a summary generated by another script. Record the actual path rather than assuming one standard arrangement.
A link is stronger when the project contains an explicit reference that connects artifacts: for example, a report script names a specific output file, a run record names the model and input files, or a manifest associates the report with a run identifier. A filename that merely looks similar, a modification time, or a matching number can help generate a lead, but on its own it does not establish that link.
The reconstruction also rests on practical assumptions. The files and records being examined are the relevant project materials; identifiers and paths have not been altered in a way that conceals their relationship; and logs or manifests accurately describe the run they purport to document. If those assumptions are not supported, state the limitation. Provenance is only as strong as its evidence.
A useful way to justify this criterion is to treat the chain as a sequence of links. Suppose the report value is linked to an output table, the output table to a run record, and the run record to a model and input file. If each link is supported, the links compose into a trace from the reported value to those run artifacts. If the output table could instead belong to either of two runs, the path branches and the trace is not uniquely established. Thus, confidence in the whole chain depends on evidence for every link, not merely on the number of files collected.
A Practical Reconstruction Sequence
Start with the reported item and work backward. Record its location and label, then find the code or configuration that supplies it. Follow explicit file references to the output, then use run records or manifests to identify the model and input artifacts associated with that output. Preserve enough detail that another reviewer can follow the same path in the project package.
Record the report file, section or table, row or label, units if displayed, and the exact printed value. A vague target such as “the clearance result” is difficult to trace when several results are reported.
Inspect the relevant report code or configuration to determine which field, row, or file supplies the displayed item. Note transformations such as rounding or unit conversion when the code shows them.
Follow the code’s file reference and record the output path and any stable identifier available. If more than one candidate file exists, do not select one just because its value matches.
Use explicit run identifiers, manifests, or recorded command details where available. Record which model and input files the run record associates with that output.
For every link, note the evidence location and what it establishes. Mark missing or conflicting links plainly; do not infer a unique source where the artifacts do not support one.
Worked Traces
Worked Example: Tracing a Parameter in a Report
Hypothetical project. A diagnostic report displays “Typical clearance” as 1.42 L/h. The report-generation script contains an instruction to read the field named CL from the file results/run_07/estimates.csv and print it to two decimal places. That file contains CL = 1.4176 L/h. The run manifest for run_07 names model file pk_model.mod and input file analysis_input.csv.
Step 1: Identify the report item. The target is the “Typical clearance” entry in the report, with displayed value 1.42 L/h. Its label alone would not identify a source; the script’s extraction instruction provides the next link.
Step 2: Follow the report logic. The script selects the CL field from results/run_07/estimates.csv and formats the value to two decimal places. Applying that formatting to the file value gives 1.4176 L/h, which rounds to 1.42 L/h because the third decimal place is 7. Thus, the displayed number is consistent with the identified output field and stated formatting.
Step 3: Follow the run record. The manifest associates run_07 with pk_model.mod and analysis_input.csv. The resulting documented chain is: report entry → script’s CL extraction and rounding → run_07/estimates.csv, CL field → run_07 manifest → named model and input files.
This chain supports the claim that the report value came from that field in that identified output and that the manifest associates the run with those files. It does not establish that the model was correctly implemented, that the input data were appropriate, or that a new execution would produce the same output.
Worked Example: Handling Two Candidate Output Files
Hypothetical project. A report shows an objective-function value of 812.4. The project directory contains output files run_12/fit_summary.csv and run_13/fit_summary.csv, and both contain 812.4. Their folder names and file modification times appear consistent with two separate analysis attempts. The report script refers only to fit_summary.csv by a relative path, and the saved report package does not include the working-directory record or a run identifier.
The matching values do not distinguish the candidates: both files could supply the displayed result. The relative path is also insufficient because its meaning depends on the working directory at the time the report script ran. The folder names and timestamps provide clues, but they do not establish which path the script resolved.
The appropriate conclusion is that the report value is consistent with both candidate outputs, but the available artifacts do not identify a unique source. Record the two candidates and the missing link—the resolved path or equivalent run association. Do not choose run_13 merely because its timestamp is later; that would turn a guess into an unsupported provenance claim.
Worked Example: Separating a Report Derivative From Its Source
Hypothetical project. A report states that 18 records were excluded. The report script calculates this number as the difference between 240 records in a preprocessing summary and 222 records in a model-input summary. The script reads both summaries from a folder labeled run_21. A run manifest in that folder names prepare_input.R, a preprocessing configuration file, and a model input file.
The arithmetic shown by the report logic is \(240 - 222 = 18\). The report’s displayed count is therefore supported as a derivative of the two named summary counts, provided those are the files the script actually reads. The folder’s manifest then provides a documented association between run_21 and the listed preparation artifacts.
The trace should preserve both source counts and the calculation, rather than treating “18” as if it were copied directly from one output. It supports how the report calculated the number and which files the manifest associates with the run. It does not, by itself, verify the individual records counted or the validity of the exclusion rule; those require different review checks.
Evidence, Diagnostics, and Failure Modes
A provenance record should let a reviewer distinguish direct evidence from inference. For each link, state the artifact examined, the specific reference or instruction observed, and the conclusion that reference supports. A checksum can help distinguish two files with the same name when the project records it, but a checksum alone says nothing about which file a script used. Similarly, a log may identify a command or output path, but it is only useful for this purpose if it can be tied to the report and run under review.
Common warning signs include a report script with a generic relative path and no recorded working directory; multiple candidate outputs with identical labels; a run manifest that names a model but not the output being reviewed; and summaries that have been copied into a report folder without retaining their source references. Conflicting run identifiers are especially important: do not silently prefer the identifier that makes the chain look complete.
When a path is missing, look for corroborating project evidence such as an explicit manifest entry, a report-generation record, or a saved command that names the relevant files. Corroboration is not the same as independent proof if both records repeat the same unverified claim. If the available evidence still cannot distinguish candidates, report the ambiguity rather than extending the chain by assumption.
How This Review Differs From Related Checks
Provenance reconstruction answers “which artifacts are linked to this reported result?” It is narrower than a complete evidence-package review and different from other review activities. The comparison helps keep the conclusion within scope.
| Review activity | Question it addresses | What provenance reconstruction adds or does not establish |
|---|---|---|
| Artifact inventory | Which files are present? | Connects relevant files with evidence; a file list alone does not establish lineage. |
| Provenance reconstruction | Which files and transformations are linked to this result? | Establishes supported artifact relationships, subject to gaps and assumptions. |
| Reproducing a run | Can the submitted analysis be executed again from a clean state? | Not performed here; provenance documentation does not show that an execution can be reproduced. |
| Checking model implementation | Does the model code implement its intended specification? | Not decided by finding which model file is associated with an output. |
The method has limits. A complete-looking chain can still rely on incomplete or inaccurate records; file association is not proof of scientific validity. Conversely, a missing link in an archived project does not prove that the reported result was fabricated or wrong. It means the available artifacts are insufficient to support a specific lineage claim. State the boundary precisely: identify what is supported, what remains uncertain, and what additional record would resolve the uncertainty.
Common Mistake
Wrong reasoning: “The output file contains the same value as the report, and its timestamp is closest to the report date, so it must be the source.” A matching value and nearby timestamp are compatible with that claim, but they do not rule out another output or establish which file the report code read.
Correction: Follow an explicit reference from the report logic or a reliable run record to the output. If the path or run association cannot be established, list the candidate files and mark the source as unresolved. Do not turn consistency into proof of lineage.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- A report script explicitly reads CL from estimates.csv and rounds to two decimals. The file contains 3.246 L/h. What value should the report show, and what does this establish?
- Two candidate output files contain the same reported value. The report uses a relative path, but no working directory is recorded. Can the source be uniquely identified? What should be documented?
- A manifest associates a run with a model file and an input file. Does that establish that the model file correctly implements its written specification? Explain.
- A report calculates a count as 96 minus 89. State the reported count and identify what must be traced to support its provenance.
Solutions. (1) The value is 3.25 L/h because the third decimal place in 3.246 is 6, so rounding to two decimal places increases the hundredths digit. This supports the report’s calculation from that field if the script’s file reference resolves to the identified file; it does not independently establish the run’s correctness. (2) No. Both files remain candidates because the relative path cannot be resolved without the working directory or equivalent evidence. Document the candidates, the missing path or run association, and the unresolved conclusion. (3) No. The manifest supports an association between the run and the model and input files. Assessing whether the model implements a specification is a separate check. (4) The reported count is 7. Trace the report logic to both source counts and establish which summary files supplied them; record the subtraction that produced the displayed count.