Why Input Provenance Matters
A single-cell analysis can be statistically careful and still be difficult to reproduce if its inputs are not identifiable. A matrix may have been downloaded from a public repository, supplied by a collaborator, or regenerated from an upstream workflow. A filename alone rarely records which source, release, or exact file was used. Sample metadata can also be revised independently of the count matrix, so the two inputs need separate records.
Earlier material on computational reproducibility motivates recording the computational context of an analysis. Here the focus is narrower: documenting the origin and identity of the input count matrix and sample metadata, then using checksums to detect whether the files differ from a recorded version.
Provenance is not just a path such as data/counts.mtx.gz. That path identifies a location in one working environment, not the source or version of the contents. Record the source details and a checksum alongside the working path. Keep separate records for the matrix and its metadata, and document how the sample or cell identifiers link them.
What a Checksum Comparison Says
Let \(B\) denote the bytes of one file and let \(H\) denote a specified hash function, such as SHA-256. The checksum is \(H(B)\). To compare a current file with a trusted recorded version, compute the same hash function on both files and compare the complete digest strings.
A cryptographic hash is deterministic: the same input bytes and the same algorithm produce the same digest. Consequently, identical files must have identical digests. The converse is not logically guaranteed: distinct byte sequences could, in principle, produce the same digest. Such an event is a hash collision. For a collision-resistant hash such as SHA-256, matching full digests provide very strong evidence that the files have the same bytes, subject to the possibility of a collision; they do not prove byte equality as a mathematical certainty.
The comparison is meaningful only when the algorithm is the same, the entire digest is retained, and both hashes refer to the intended files. A digest copied from a trusted repository or verified release can be a useful reference. If a team hashes a downloaded file only after the download, that new digest records the bytes received; without an independently trusted reference, it cannot show that the download matches the source’s intended release.
Build a Useful Input Record
A compact manifest can record one row per input file. For single-cell work, make the matrix and sample metadata separate entries even when they arrive in the same archive. Record enough detail to distinguish releases and to interpret how the files were read.
- Origin: repository, accession, provider, or collaborator; include a stable source identifier when available.
- Release and retrieval: version or release label, retrieval date, and source location or transfer method.
- File identity: original filename, format, compression status, size in bytes, and SHA-256 digest.
- Data meaning and linkage: matrix orientation, feature and cell identifiers, metadata identifiers, and any documented changes made before analysis.
- Local handling: the working path and whether the recorded digest applies to the original downloaded file or to a later transformed file.
The last distinction matters. A checksum belongs to the exact bytes hashed. Decompression, recompression, changing line endings, reordering rows, or saving the same values in another format changes bytes and usually changes the digest. A manifest should therefore label which stage of a file’s history it identifies rather than silently treating all representations as one file.
Worked Example: Recording a Matrix and Its Metadata
Suppose an analyst receives a hypothetical release called SCX-204, release 2.1, containing a matrix file and a sample metadata file. The matrix has three feature rows and four cell columns. Its columns are cell IDs C01, C02, C03, and C04. The metadata contains exactly those four IDs, one per row, along with sample labels. A useful input record would distinguish both files and state their relationship.
| Input | Source and release | File details | Linkage or interpretation |
|---|---|---|---|
| Count matrix | SCX-204, release 2.1; retrieved 2026-10-12 | counts.mtx.gz; Matrix Market, gzip-compressed; 8,420 bytes; SHA-256 recorded in manifest | Three feature rows by four cell columns; column IDs are C01–C04 |
| Sample metadata | SCX-204, release 2.1; retrieved 2026-10-12 | samples.tsv; tab-separated text; 196 bytes; separate SHA-256 recorded in manifest | Four rows; cell_id values C01–C04 join to matrix columns |
The byte sizes and dates here are hypothetical record values. Before analysis, the analyst checks that the matrix has four cell columns and the metadata has four unique cell IDs, and confirms that the identifier sets agree. These checks address consistency and linkage; the checksums address the identity of each file’s bytes. Neither replaces the other.
The manifest should store the complete digest strings produced by the chosen tool, not the phrase “SHA-256 recorded.” The table abbreviates them to emphasize which information belongs in the record; the actual manifest would retain every hexadecimal character.
Comparing a Current File With a Recorded Digest
For a practical comparison, compute a digest for the current file and compare it character for character with the recorded digest for that same file and algorithm. Preserve the original file and manifest so a changed file can be investigated rather than overwritten. If the digests differ, check whether the file was intentionally replaced, transformed, or transferred incorrectly before deciding what to do next.
Worked Example: Detecting a Changed Transfer
Suppose a trusted manifest records the hypothetical SHA-256 digest 4b1f9a2c7d3e8f605a1c9b2d6e4f70813c5a9d2e6b4f8017a3c9d5e2f6b4081c for counts.mtx.gz. After copying the file to an analysis server, the analyst computes the full digest and obtains 4b1f9a2c7d3e8f605a1c9b2d6e4f70813c5a9d2e6b4f8017a3c9d5e2f6b4081c. These are illustrative digest strings, not claimed calculations for a displayed file.
The comparison is direct: all 64 hexadecimal characters agree, so the digests match. This provides very strong evidence that the copied file’s bytes match those represented by the recorded digest, subject to the possibility of a hash collision. The result does not independently authenticate the source, verify that the manifest itself is trustworthy, or establish that the counts are correct.
Now suppose a later check gives 4b1f9a2c7d3e8f605a1c9b2d6e4f70813c5a9d2e6b4f8017a3c9d5e2f6b4081d. The final character differs, so the digests do not match. Under the stated assumptions, the current and recorded files have different bytes. The analyst should preserve both versions and investigate whether the cause was a changed release, an accidental transfer or storage problem, or an intentional transformation. The mismatch alone does not identify the cause.
Why Matching Data Can Have Different Checksums
A checksum tests file bytes, not the abstract count matrix obtained after parsing. Two files can encode equivalent counts yet have different bytes because they use different formats, compression settings, line endings, or row and column order. Conversely, a file can have a matching digest to a recorded file while still containing an incorrect matrix if the recorded file was already wrong.
Worked Example: Equivalent Counts, Different File Bytes
Consider a hypothetical matrix with two genes and two cells:
Suppose one file stores these counts as a Matrix Market file, while another stores the same values in a tab-separated table. The parsed matrices can agree entry by entry, but the files have different bytes because their encodings and text representations differ. A byte-level checksum comparison is therefore not a test of equality between parsed matrices: it compares the serialized files.
If the scientific question is whether the parsed count data are equivalent, compare the parsed values and the relevant identifiers under a clearly stated rule—for example, after matching rows by feature ID and columns by cell ID. That is a different comparison from checking file identity. Record a new checksum for any transformed or reserialized file, and describe its relationship to the source file.
Justification and Practical Limits
The basic logic of checksum comparison follows from the properties of a deterministic hash. If \(B_1=B_2\), applying the same function to both gives the same output, so \(H(B_1)=H(B_2)\). If the digests differ, the inputs cannot have been identical under that same deterministic function. But when the digests agree, the output alone does not rule out a collision.
The reverse implication, \(H(B_1)=H(B_2)\implies B_1=B_2\), is not guaranteed. A hash maps a very large set of possible files to a fixed-length set of digests, so distinct files can share an output. Collision resistance makes such a match very strong evidence in ordinary file-integrity checks, but it does not turn the implication into a proof.
Checksums also have practical boundaries. They do not explain why bytes differ, verify biological plausibility, show that sample labels are correct, or establish that an input came from the stated repository. For source authentication, use a trusted release record or other independently verifiable source information. For data integrity and linkage, inspect the file structure, dimensions, identifiers, and values as appropriate.
Diagnostics and Common Mistakes
- Comparing unlike digests. A SHA-256 value cannot be compared with a digest from a different algorithm. Record the algorithm beside every digest.
- Keeping only part of a digest. A shortened string can conceal differences. Retain and compare the complete digest.
- Hashing the wrong object. A digest for an archive is not the digest for a file extracted from it. State whether the checksum covers the archive, the extracted file, or a transformed output.
- Assuming a match proves scientific correctness. A matching digest supports file identity relative to a recorded digest, not correctness or trustworthy provenance.
- Assuming a mismatch means corrupted data. Recompression, reserialization, or an authorized update also changes file bytes. Investigate the file history before treating the difference as an error.
- Recording only a local path. A path may move or refer to different contents in another environment. Pair it with source, release, file details, and a full checksum.
A common incorrect argument is, “The current and recorded checksums match, so the source is authentic and the matrix is correct.” The correction is narrower: matching full digests provide very strong evidence that the bytes match, subject to a possible collision, assuming the recorded digest is trustworthy and both files were hashed with the same algorithm. Source authenticity and scientific validity require separate evidence.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- A metadata file has the same name in two releases. Name two provenance details, besides the filename, that help distinguish the inputs.
- A current file and a trusted SHA-256 record have different full digests. What can be concluded, and what cannot be concluded from the mismatch alone?
- Two count files parse to the same values and cell IDs but have different SHA-256 digests. Is this necessarily a contradiction? Explain.
- A team computes a checksum after downloading a file but has no checksum or other reference from the provider. What does the new checksum establish?
Solutions. (1) Examples include the source accession or provider, release label, retrieval date, format, or source location. These distinguish origin and version beyond the local filename. (2) Assuming the same algorithm and correct computation, the files have different bytes. The mismatch alone does not show whether the cause was corruption, an intentional update, or a change in serialization. (3) No. The files may encode equivalent parsed data in different byte representations, such as different formats or compression. Checksum equality is about bytes, not parsed matrix equivalence. (4) It records the digest of the bytes received and can support later comparison with that record. It does not establish that the download matches the provider’s intended file or that the file is scientifically correct.