Validation Gates for Single-Cell Workflow Outputs

Why Workflow Outputs Need Validation Gates

A workflow step can finish without producing a usable result. A file may exist but be truncated; a count matrix may have the wrong number of cells; or metadata may refer to cell identifiers absent from the matrix. If the next step proceeds anyway, an error introduced at one boundary can become a plausible-looking but invalid downstream result.

A validation gate is a set of pre-specified checks applied at a workflow boundary before an output is accepted as input to dependent steps. It turns assumptions about the data into executable conditions. A gate does not establish that an analysis is scientifically correct, nor does it determine whether a cell is biologically high quality. It tests whether the artifact meets the structural and integrity requirements defined for that point in the workflow.

Definition: A validation contract specifies the expected properties of an artifact and the action required when any property is violated. A gate passes only when all required checks pass. If a required check fails, the artifact is rejected for downstream use until the cause is investigated and the output is regenerated or explicitly corrected.

The contract should be specific enough that two runs can be judged by the same rules. “The matrix should look right” is not executable. “The matrix has 18,000 rows and 4,200 columns, row names are unique gene identifiers, column names are unique cell identifiers, and all entries are finite nonnegative integers” is testable. These requirements must match the intended data representation; not every workflow uses the same orientation or permits the same values.

Specify What Each Boundary Promises

For examples here, let \(Y\) be a count matrix with genes in rows and cells in columns, and let \(y_{gc}\) denote the count for gene \(g\) in cell \(c\). The orientation is part of the contract, not a universal property of all single-cell software. If a tool expects cells in rows, its boundary should state that instead. Likewise, the expected dimensions should come from the analysis configuration or a verified upstream artifact, rather than an informal guess.

A useful contract addresses four distinct questions:

Dimensions: Are the number of genes and cells within the required range or exactly as expected? Is the object genuinely two-dimensional?
Identifiers: Are gene and cell identifiers present, nonempty, unique where required, and consistent across the matrix and associated tables?
Missing and invalid values: Are missing values allowed in each field? Are entries finite and in the permitted domain? For raw counts, a typical requirement is that every entry be a nonnegative integer.
File integrity: Is the file complete and unchanged relative to the expected artifact? Can it be read, and does its checksum match the expected checksum when one is specified?

The requirements can vary by object. A missing count is usually invalid for a raw count matrix, whereas a missing value in an optional clinical metadata field may be allowed and recorded. A contract should say which columns permit missingness and under what conditions; a blanket rule that either rejects every missing value or silently fills them in is often inappropriate.

Worked Examples: Applying the Checks

Worked Example: Checking Dimensions and Identifier Alignment

Suppose a hypothetical workflow expects a gene-by-cell matrix with 3 genes and 4 cells. Its expected gene identifiers are G1, G2, G3, and its expected cell identifiers are C1, C2, C3, C4. The matrix has 3 rows and 4 columns, row names G1, G2, G3, and column names C1, C2, C3, C4. A metadata table has one row per cell, in the order C1, C2, C3, C4.

The dimension check is \(3=3\) genes and \(4=4\) cells, so it passes. The identifier checks also pass: each list has the expected length, no identifier is empty or repeated, and the matrix column identifiers match the metadata cell identifiers in both membership and order. This output satisfies these parts of the contract.

Now suppose the metadata rows are C1, C2, C4, C5. The row count is still 4, but equal counts alone do not establish alignment. Comparing the identifier sets reveals that C3 is missing and C5 is unexpected. The gate must fail; otherwise values recorded for C5 could be attached to counts for C3 or a later join could silently discard a cell. If reordering is permitted, the workflow may reorder metadata by an explicit identifier match, then check that the aligned identifiers are exactly C1, C2, C3, C4. It should not assume positional correspondence when identifiers disagree.

Worked Example: Finding Missing and Invalid Count Values

Consider the hypothetical count matrix \(Y=\begin{pmatrix}0&2&1\\3&1&0\end{pmatrix}\), with genes in rows and cells in columns. Its six entries are 0, 2, 1, 3, 1, and 0. If the contract requires finite, nonnegative integer counts, all six entries satisfy it: none is missing, each is at least zero, and each has zero fractional part. This check passes.

Suppose instead the entry in row 1, column 2 is missing. The matrix has 6 positions, of which 5 contain observed values and 1 is missing; the missing-value check fails. If that entry were 2.5 instead, the missing-value check would pass but the integer-count check would fail, because \(2.5\) is not an integer. A negative value such as \(-1\) would fail the nonnegative requirement, and an infinite value would fail the finite-value requirement.

These checks do not dictate how to repair the data. Replacing a missing count with zero, rounding 2.5, or removing a cell changes the data and requires a justified, explicit operation. The gate should expose the violation rather than silently transform the matrix.

Worked Example: Checking File Integrity Before Reading

A hypothetical workflow expects a compressed matrix file and stores its expected SHA-256 checksum as a 64-character hexadecimal string. After the file is written, the workflow computes its checksum from the file bytes and compares the result with the stored value. If the computed string exactly matches the expected string, the checksum check passes; if even one character differs, it fails.

For example, let the expected checksum be E and the computed checksum be C. The comparison is the exact test \(C=E\). If \(C\ne E\), the workflow must not treat the file as the expected artifact: it may be incomplete, corrupted, or different from the intended file. The checksum comparison does not prove that the matrix contains the right genes or that its counts are valid. Those require separate checks after successful reading. Conversely, a file that can be read is not necessarily the intended file, so readability alone does not replace the checksum test.

Why a Failed Check Must Stop Dependent Analysis

A gate is most useful when its result controls execution, rather than merely printing a warning that can be overlooked. In a dependency graph, a downstream task is allowed to run only when its required inputs have passed their contracts. This is a fail-closed rule: uncertainty about a required condition is treated as a failure, not as permission to proceed.

Key result: If an output is accepted only when every required predicate \(V_1,\ldots,V_k\) is true, then the pass condition is \(V_1\land V_2\land\cdots\land V_k\). If any required predicate is false, the contract is not satisfied, so dependent steps must not consume that output as validated input.

The justification is direct. Each predicate represents a required property of the artifact. If even one is false, the artifact does not meet the stated contract. A downstream result computed from it would therefore lack a required input guarantee. Stopping does not assert that every failed artifact is useless; it prevents the workflow from presenting an unchecked or known-nonconforming artifact as if it had passed. The failure message should identify the artifact, failed condition, observed value, and expected condition, so the problem can be diagnosed.

Define the contract.
State the expected dimensions, identifier rules, permitted missingness and value constraints, and any required file-integrity checks.

Run checks at the boundary.
Validate the artifact after it is produced and before dependent tasks read it. Check the actual artifact, not merely the parameters that were intended to produce it.

Record and act on the result.
On failure, give an actionable diagnostic and stop dependent tasks. On success, allow the validated artifact to proceed.

Diagnostics, Alternatives, and Limitations

Checks should be designed to catch meaningful failures without confusing distinct issues. An exact dimension check is appropriate when the number of cells and genes is fixed by the contract. If the workflow legitimately permits a range of cells, specify that range instead of hard-coding one count. Identifier checks should distinguish duplicates, empty strings, unexpected identifiers, and missing identifiers; the statement “IDs do not match” is less useful than reporting which condition failed.

There are two common implementation choices. Strict rejection stops at the first failure and provides a focused error. Aggregated reporting evaluates several independent checks and reports all detected failures together before stopping. The latter can reduce repeated runs to fix separate issues, but checks must be safe to perform after earlier failures. For example, a metadata alignment test may not be meaningful if the matrix cannot be read at all. Either approach is valid if a failed required check prevents dependent analysis.

Checksums, dimensions, identifiers, and value constraints answer different questions. A matching checksum supports the claim that the file bytes match a specified expected file. It does not establish semantic correctness. A valid dimension does not show that identifiers are correct. Correct identifiers do not show that counts are finite. Therefore, a single “file exists” test or a single checksum is not a substitute for the complete contract.

Validation gates also have limits. They detect violations of conditions that have been specified and implemented; they cannot detect every possible error. A matrix may pass all structural checks while containing a scientifically inappropriate normalization, a mislabeled assay, or a batch effect. Biological quality control and statistical diagnostics address different questions and should not be conflated with boundary validation. The contract itself must also be reviewed: an incorrect expected cell count can cause a correct output to fail, while an overly permissive rule can allow a faulty one through.

Common pitfall: “The output file exists and has the expected number of columns, so it is safe to analyze.” This reasoning ignores identifier mismatches, invalid values, and corruption. Check each required property independently; accept the output only when all required checks pass.

Check Your Understanding

Answer each question using the validation-contract principles above. Full solutions follow the questions.

A workflow expects 8,000 genes and 1,200 cells, but receives a matrix with 8,000 rows and 1,200 columns whose cell names include a duplicate. Does the dimension check pass, and should the gate pass overall?
A metadata table has four rows with cell IDs C1, C2, C4, C5, while the matrix columns are C1, C2, C3, C4. Why is matching row count insufficient?
A count matrix has values 0, 1, 2.5, and 4, with no missing values. Which checks pass and which fail under a finite, nonnegative integer-count contract?
A file is readable, but its computed checksum differs from the expected checksum. What should happen, and what does the mismatch establish?
Why is it incorrect to describe a structurally valid count matrix as necessarily biologically high quality?

Solutions. (1) The dimension check passes because the matrix has the expected 8,000 rows and 1,200 columns. The identifier check fails because cell identifiers are required to be unique, so the gate must not pass overall. (2) Equal row and column counts do not ensure the same cells are represented. C3 is missing from the metadata and C5 is unexpected; alignment must be checked by identifiers, not just position or length. (3) The missing-value check passes, and all values are finite and nonnegative, but the integer-count check fails because 2.5 is not an integer. (4) Stop dependent analysis and report the mismatch. It establishes that the file bytes do not match the expected checksum; it does not by itself identify whether the cause is corruption, truncation, or a different file. (5) Structural checks test properties such as dimensions, identifiers, and valid values. They do not assess biological quality or establish that the data are suitable for a particular scientific analysis.