UMI Deduplication as Molecule Counting

Why Deduplicate Reads?

Sequencing produces read records, but the quantity of interest in a UMI-based assay is often the number of captured RNA molecules represented by those reads. During library preparation, one original molecule can be copied many times. Those copies can generate multiple reads even though they came from just one captured molecule. If every read were counted separately, sequencing depth and amplification could make that molecule appear to be many molecules.

A unique molecular identifier (UMI) provides a label associated with an original molecule. Once reads have cell barcodes and gene assignments, records with the same cell barcode, assigned gene, and UMI are treated as repeated observations of the same labeled molecule. Deduplication collapses those repeated records to one count. The rule is simple, but its interpretation depends on how labels are assigned and how accurately they are read.

Definition: In exact UMI deduplication, records are grouped by the combination of cell barcode, gene assignment, and UMI sequence. Within each cell-and-gene group, each distinct UMI sequence contributes one molecule count, regardless of how many reads carry that sequence. The resulting count is a UMI-based estimate of the number of captured molecules assigned to that gene in that cell.

Assumptions Behind the Counting Rule

The rule starts after read records have been assigned cell barcodes, genes, and UMI sequences. It does not itself decide which barcode is a real cell or resolve uncertain gene assignments. Those upstream assignments are treated as given. For the basic exact procedure, assume that each record being counted has a usable cell barcode, a gene assignment, and a UMI sequence.

Interpreting one distinct UMI as one captured molecule relies on two main ideas. First, reads from copies of the same original molecule are expected to retain its cell barcode and UMI, and to receive the same gene assignment for this counting step. Second, different original molecules from the same cell and gene are expected, with high probability, to receive different UMIs. The second condition can fail when two molecules happen to receive the same UMI, an event called a UMI collision. In that case, exact deduplication merges them and undercounts molecules.

A sequencing error in a UMI can have the opposite effect. Reads from one original molecule may appear to have different UMI sequences if an error changes a base in some reads. Exact deduplication then counts those sequences separately and may overcount molecules. Thus, the exact rule is a transparent starting point, not a guarantee that the count equals the true number of physical molecules.

Formalizing the Deduplication Rule

Let \(R\) be the set of read records retained for counting. For each record \(r\), write its assigned cell barcode as \(b(r)\), its assigned gene as \(g(r)\), and its UMI sequence as \(u(r)\). For a particular cell barcode \(b\) and gene \(g\), collect the UMI sequences observed among records in that group:

U_{bg}=\{u(r): r\in R,\ b(r)=b,\ g(r)=g\}.

Because \(U_{bg}\) is a set, a sequence appearing on one read or on many reads appears only once in the set. The exact deduplicated count is its size:

C_{bg}=|U_{bg}|.

Equivalently, for each possible UMI sequence \(u\), define an indicator that equals 1 if at least one record in the cell-and-gene group has that UMI, and 0 otherwise. Summing these indicators gives the same count. This makes clear that the count depends on whether a UMI was observed at least once, not on how many reads carried it.

C_{bg}=\sum_{u} I\{\text{at least one record has }(b,g,u)\}.

Worked Examples

Worked Example: Repeated Reads From the Same Labeled Molecule

Suppose six hypothetical read records are assigned to cell barcode BC01 and gene GeneA. Their UMI sequences are AAA, AAA, AAA, AAC, AAC, and AAG. The first UMI appears on three records, the second on two, and the third on one.

The set of distinct UMIs is \(U_{\mathrm{BC01,GeneA}}=\{\mathrm{AAA},\mathrm{AAC},\mathrm{AAG}\}\). Therefore,

C_{\mathrm{BC01,GeneA}}=|U_{\mathrm{BC01,GeneA}}|=3.

There are six read records but three deduplicated molecule counts for this cell-and-gene group. Under the counting assumptions, the three distinct labels represent three captured molecules. The three records carrying AAA do not count as three molecules: they are repeated reads carrying the same cell, gene, and UMI labels.

Worked Example: The Same UMI in Different Groups

Consider these hypothetical records. Each row gives a cell barcode, gene assignment, and UMI; repeated rows represent separate reads.

Cell barcode	Gene	UMI	Number of reads
BC01	GeneA	TTG	2
BC01	GeneA	TTA	1
BC01	GeneB	TTG	3
BC02	GeneA	TTG	2

For BC01 and GeneA, the observed UMI set is \(\{\mathrm{TTG},\mathrm{TTA}\}\), so the count is \(2\). For BC01 and GeneB, the set is \(\{\mathrm{TTG}\}\), so the count is \(1\). For BC02 and GeneA, it is also \(\{\mathrm{TTG}\}\), so the count is \(1\). The total deduplicated count across these three groups is \(2+1+1=4\).

Although TTG occurs in all three groups, it contributes once within each group. The UMI is not a globally unique molecule name: the counting key is the complete combination of cell barcode, gene, and UMI. Grouping only by UMI would incorrectly merge records from different cells or different genes.

Worked Example: How a UMI Error Can Split a Molecule

Suppose four hypothetical records have the same cell barcode, BC03, and gene assignment, GeneC. Their observed UMIs are CCGT, CCGT, CCGT, and CCAT. Exact deduplication produces the set \(\{\mathrm{CCGT},\mathrm{CCAT}\}\), and hence

C_{\mathrm{BC03,GeneC}}=2.

The three identical CCGT records contribute one count, and CCAT contributes another. If CCAT arose from a sequencing error in a copy of the original molecule labeled CCGT, the exact count of 2 would be an overcount. If there were instead two original molecules and they really had those two UMIs, the count of 2 would be appropriate. The records alone do not prove which explanation is correct.

An error-aware method may use sequence similarity and read support to decide whether a low-support UMI is likely an error-derived variant of another UMI. If independent evidence justifies merging CCAT into CCGT, the corrected count would be \(1\), not \(2\). That merge is a modeling decision, not part of exact deduplication; blindly merging similar sequences could also combine labels from genuinely different molecules.

Why Distinct UMIs Approximate Molecule Counts

The connection between unique labels and molecules can be justified with a simple labeling model. Suppose \(m\) molecules from one cell-and-gene group each receive a UMI independently and uniformly from a set of \(M\) possible sequences. If no two molecules receive the same UMI, the number of distinct observed labels equals \(m\). If labels collide, the distinct-label count is smaller.

For any one of the \(M\) possible UMIs, the probability that a particular molecule does not receive it is \(1-1/M\). The probability that none of the \(m\) molecules receives it is \((1-1/M)^m\). Thus, the probability that this UMI is used at least once is \(1-(1-1/M)^m\). Summing that probability over the \(M\) possible UMIs gives the expected number of distinct labels:

E(C_{bg})=M\left[1-\left(1-\frac{1}{M}\right)^m\right].

This expectation is no greater than \(m\), because every observed distinct UMI can represent at most one distinct label, while collisions make multiple molecules share a label. The uniform, independent assignment assumption is an idealization: actual UMI sequence frequencies and assay details may depart from it. The calculation nevertheless shows why a finite label space can lead to undercounting when many molecules are labeled in the same group.

Worked Example: A Small UMI Space and Collisions

For illustration, imagine a toy assay with only \(M=4\) possible UMIs and \(m=3\) molecules in one cell-and-gene group. The expected number of distinct UMIs is

E(C_{bg})=4\left[1-\left(1-\frac{1}{4}\right)^3\right] =4\left[1-\left(\frac{3}{4}\right)^3\right] =4\left(\frac{37}{64}\right) =\frac{37}{16} =2.3125.

There are three molecules, but the expected distinct-label count is only 2.3125 in this toy setting. For comparison, the probability that all three labels are different is \((4/4)(3/4)(2/4)=0.375\), so the probability of at least one collision is \(1-0.375=0.625\). The toy numbers make collisions common; real assays use larger UMI spaces, but the principle remains.

Diagnostics, Alternatives, and Limitations

A useful basic diagnostic is to compare the number of read records with the number of distinct UMIs in each cell-and-gene group. A large difference is consistent with repeated sequencing of molecules or amplification, which is precisely why deduplication is useful. It is not, by itself, evidence that the deduplicated count is correct. Inspecting UMI sequence patterns can help identify possible errors, such as many low-support UMIs differing by one base from a high-support UMI, but sequence similarity alone does not establish that two labels came from the same molecule.

Exact deduplication is reproducible and easy to audit: identical grouping labels are collapsed, and no additional sequence-based decision is made. Error-aware UMI correction can reduce overcounting from sequencing errors, but its result depends on the correction rule and its assumptions. Counting every read instead avoids the UMI-merging decision but measures read records rather than distinct UMI labels; amplified copies can therefore contribute repeatedly. Assays without UMIs do not provide these molecule labels, so read totals have a different interpretation.

Neither exact deduplication nor error-aware correction can recover information that the labels do not distinguish. A collision can make two true molecules look identical, and a sequencing error can make one molecule look like multiple UMIs. Counts also depend on the barcode and gene assignments supplied to the procedure: if those assignments are wrong, deduplication does not repair them. Keep the counting unit explicit when interpreting results: a deduplicated count is a count of observed distinct UMI labels within a specified cell-and-gene group, used as a proxy for captured molecules.

Common Mistake

A common mistake is to say, “This UMI occurs four times, so it represents four molecules.” That reasoning confuses read multiplicity with molecule multiplicity. Under the deduplication rule, four records with the same cell barcode, gene, and UMI contribute one count. The correction is to first define the group, then count its distinct UMI sequences. The reverse mistake is to deduplicate by UMI alone; this can merge records that belong to different cells or genes and therefore uses the wrong grouping key.

Key takeaway: For each cell barcode and gene assignment, count each distinct UMI once. This estimates captured molecule counts when labels are correctly recorded and collisions and UMI errors are limited; the read total itself is not the deduplicated molecule count.

Check Your Understanding

Answer each question, then compare with the solutions below.

Within one cell-and-gene group, the UMI list is GGA, GGA, GTT, GGA, and GTT. What is the exact deduplicated count, and how many read records are there?
Why should the same UMI sequence in two different cell barcodes not automatically be counted as one molecule?
In the uniform-label model, what is the expected number of distinct UMIs when \(m=2\) molecules are labeled from \(M=4\) possible UMIs?
One group has UMIs ACGT, ACGT, and ACAT. What is the exact count? Give one reason an error-aware procedure might produce a different count, and explain why that difference is not certain from these records alone.

Solutions. (1) The distinct set is \(\{\mathrm{GGA},\mathrm{GTT}\}\), so the deduplicated count is \(2\); there are \(5\) read records. (2) The cell barcode is part of the grouping key, so the same sequence in two cells belongs to two separate cell-and-gene groups and can label different molecules. (3) \(E(C)=4[1-(3/4)^2]=4(7/16)=1.75\) distinct UMIs on average. (4) The exact count is \(2\), from ACGT and ACAT. An error-aware method might merge ACAT into ACGT if it judges ACAT to be an error-derived label, but the records alone do not rule out two genuinely distinct molecules receiving those labels.