Reporting Cell-Type Labels With Uncertainty

Why Reporting Is Part of Annotation

A cell-type annotation is not just a name attached to a barcode. A useful report shows what label was assigned, what evidence or uncertainty accompanies it, how unresolved cells were handled, and which reference and decision rules shaped the result. Without those details, readers cannot tell whether a confident-looking label came from strong evidence, a forced choice among unsuitable alternatives, or a decision to report only a broad parent category.

This tutorial focuses on communicating annotation results rather than choosing an annotation method. The earlier material on selective classification, prediction sets, reference mismatch, and adjudicating disagreements provides ways to assess or resolve uncertainty. Here the task is to make the resulting choices interpretable and auditable. In particular, a reported score should not be called a probability unless its meaning and calibration support that description.

A report should define the interpretation of every uncertainty quantity. For example, \(q_{ik}\) might be a normalized classifier output, a similarity score, or a calibrated estimate of the probability of label \(k\) conditional on the model and its inputs. These quantities are not interchangeable. A normalized vector sums to one, but that alone does not establish that its entries are calibrated probabilities. If the method returns scores rather than calibrated probabilities, call them scores and state how they were produced. If it returns calibrated probabilities, state the calibration procedure and the validation setting relevant to interpreting them.

What a Reproducible Report Contains

At minimum, report the unit being annotated, the label vocabulary and its hierarchy, the method and reference used, the rule that converts method output into reported labels, and the handling of unresolved cases. For a cell-level data release, these details can be paired with a table containing one row per cell. For a manuscript or summary, provide aggregate counts as well, while making clear which cells form each denominator.

• Label: Give the assigned label, or an explicit value such as “abstain” or “unresolved.” Do not silently drop cells without a label.
• Uncertainty: Specify whether the field is a score, calibrated probability, prediction set, or other summary. State the candidate labels and any threshold or decision rule used.
• Reference provenance: Identify the reference dataset or atlas, its version or release, relevant tissue and species scope, and any filtering or mapping that affects the candidate labels.
• Granularity: State the hierarchy and the rule for reporting a parent category instead of a child label. Explain whether labels were assigned directly at that level or rolled up from finer predictions.
• Scope: Report cell counts, the number of abstentions, and the denominator for each proportion. If relevant, distinguish a cell-level description from population-level inference.

Provenance should be detailed enough that another analyst can identify the reference and reconstruct the relevant label vocabulary. “Annotated using a public atlas” is not enough if multiple releases, tissues, or label mappings exist. Conversely, provenance should not imply that the reference is a ground truth for every query cell: a reference defines the comparison context, not an independent guarantee that every possible cell type is represented or correctly labeled.

Reporting Uncertainty Without Overstatement

The form of the uncertainty report should match the method output. For a classifier that returns a score for each candidate label, a cell-level record might include the assigned label and the leading scores, along with a note that they are uncalibrated scores if that is their status. For a method returning a prediction set \(A_i\), report the set or a clearly defined summary of its size; do not present one member as uniquely established unless a separate decision rule selects it. If an abstention rule is used, report its criterion and the number of cells affected.

A compact report can summarize distributions of uncertainty, but it should not replace the cell-level information needed to inspect ambiguous cases. For example, report the median and range of a defined uncertainty score across cells, or the fraction of cells whose prediction set contains more than one label. State whether those summaries include abstentions and how missing or undefined scores were handled. A mean score alone can conceal a small group of highly uncertain cells, so useful summaries often include a distribution or thresholded counts.

Where probability estimates are reported, the interpretation is conditional on the model, reference, and validation context. A probability of 0.8 should not automatically be paraphrased as “there is an 80% chance this cell is truly this type.” That reading is justified only under an appropriate probabilistic interpretation and calibration evidence for a sufficiently relevant setting. Even calibrated performance in one validation population may not carry over unchanged under tissue, donor, batch, or reference distribution differences.

Making Label Granularity Explicit

Cell-type labels often form a hierarchy: a broad category can have several more specific child labels. Reporting at fine granularity can be informative when evidence supports it, but it can imply distinctions the data or reference do not resolve. Reporting only broad categories can avoid unsupported specificity while hiding distinctions that are well supported. A transparent report states the available hierarchy and the rule that determines the level shown.

A granularity rule might require a minimum score, a minimum margin between the top two child labels, or a specified validation criterion before reporting a child. These rules should be fixed and described rather than chosen after inspecting which labels produce the preferred narrative. If a child-level assignment fails the criterion, a workflow may report its parent if parent-level evidence supports that category; otherwise it may abstain. The report should distinguish a parent label selected directly from evidence from a parent label produced by rolling up child predictions.

Derivation: Why Denominators Must Be Shown

The distinction between all-cell proportions and assigned-cell proportions follows directly from the counts. Let \(n_k\) be the number of cells assigned label \(k\), and let \(n_A\) be the total number of labeled cells. When \(n_A>0\), the assigned-cell proportion is \(n_k/n_A\), while coverage is \(n_A/n\). Multiplying gives:

The cancellation is valid because \(n_A>0\). This identity does not make the two proportions interchangeable: \(n_k/n_A\) conditions on receiving a label, whereas \(n_k/n\) describes the share of the entire dataset assigned that label. If no cells are labeled, the assigned-cell proportions are undefined; the report should give zero coverage and the abstention count, not a proportion with a zero denominator.

Alternatives, Diagnostics, and Limitations

These forms can be combined: a report may give an assigned label for some cells, prediction sets for others, and abstentions where the protocol calls for them. A single-label table is easy to use downstream, but should not be the only record when uncertainty matters. Conversely, a large collection of scores is not automatically informative if its scale, interpretation, and decision rule are absent.

Several checks help detect reporting problems. Confirm that category counts plus abstentions equal the total number of cells; that proportions use stated denominators; and that per-cell labels match the declared thresholds and hierarchy. Check for missing reference versions, undocumented label mappings, and labels finer than the reference or evidence can support. If uncertainty summaries are unexpectedly uniform, verify that the reported quantity was not inadvertently reduced to the chosen label alone. Compare summaries across relevant batches or donors when those groupings could reveal systematic differences.

Cell-level counts are descriptive of the analyzed dataset. Cells from the same donor are not generally independent biological replicates, so treating thousands of cells as thousands of independent replicates can overstate precision in population-level claims. If uncertainty intervals or inferential comparisons are added, their sampling and resampling units must reflect the study design, often donors rather than individual cells. Annotation reporting alone does not establish that a cell type is present in a broader population or that an observed difference is causal.

Common Reporting Mistakes

• Calling every normalized output a probability. Values summing to one do not guarantee calibration. Name the quantity according to what the method actually provides.
• Hiding abstentions. Removing unlabeled cells changes the denominator and can make coverage appear complete. Report their count and the rule that produced them.
• Reporting a broad label as a fine-grained result. A parent assignment does not resolve which child label applies. State the level and how it was chosen.
• Omitting reference provenance. A label depends partly on the candidate vocabulary and reference context. Give enough identifying information to reproduce that context.
• Using proportions without denominators. “Twenty percent B cells” is ambiguous if it is unclear whether abstained cells are included.

The practical standard is simple: let readers distinguish the assigned decision from the supporting uncertainty information, and let them reconstruct how the reference, thresholds, abstentions, and hierarchy shaped that decision.