Tutorials › Statistical Methods for Single-Cell RNA Sequencing › Using Negative Markers to Distinguish Related Cell Types

Cell-Type Annotation and Its Uncertainty · Tutorial 327 of 700

Using Negative Markers to Distinguish Related Cell Types

Use positive and negative marker evidence together, and judge a missing signal by how likely it was to be detected in the first place.

Intermediate 10 min read

What You'll Learn

  • Define positive and negative marker evidence relative to a candidate cell type.
  • Calculate likelihood-ratio contributions from detected and undetected genes.
  • Explain why an absent signal can be weak evidence when detection is unlikely.
  • Combine concordant and contradictory markers without treating one as a veto.
  • Diagnose dependence, technical sensitivity, and panel-selection problems that can distort scores.

Why Negative Markers Need Careful Interpretation

Closely related cell types can share many positive markers. A marker that is expected in one candidate but not another may help distinguish them, but the absence of a transcript is not automatically evidence against a label. A gene can be biologically low, missed by sequencing, or undetected because of the cell’s library depth. The useful question is therefore not simply whether a marker is absent, but how likely it would have been detected if the candidate label were correct.

This tutorial treats detection as a binary observation and compares two candidate labels using marker-specific detection probabilities. The approach makes the evidence from expected positive markers and expected negative markers explicit. It is a way to organize evidence, not a guarantee that the candidate with the largest score is correct.

Definition: For cell \(i\) and gene \(g\), let \(Y_{ig}=1\) if the gene is detected and \(Y_{ig}=0\) otherwise. For candidate cell type \(k\), let \(q_{kg}=P(Y_{ig}=1\mid T_i=k)\) be the probability that gene \(g\) is detected in a cell of type \(k\), under the assay and conditions being considered. A positive marker for \(k\) is expected to have relatively high \(q_{kg}\). A negative marker for \(k\) is expected to have relatively low \(q_{kg}\). “Negative” means low expected detection, not guaranteed biological absence.

A Detection-Based Comparison

Suppose the candidates are cell types A and B. For a measured marker \(g\), the probability of the observed detection outcome under candidate \(k\) is \(q_{kg}\) if \(Y_{ig}=1\), and \(1-q_{kg}\) if \(Y_{ig}=0\). Comparing A with B gives a likelihood ratio: the probability of the observed outcome under A divided by its probability under B. A ratio above 1 supports A relative to B; a ratio below 1 supports B relative to A.

$$ LR_g(A:B)= \left(\frac{q_{Ag}}{q_{Bg}}\right)^{Y_{ig}} \left(\frac{1-q_{Ag}}{1-q_{Bg}}\right)^{1-Y_{ig}}. $$

The log likelihood ratio is easier to add across genes. A detected gene contributes \(\log(q_{Ag}/q_{Bg})\); an undetected gene contributes \(\log((1-q_{Ag})/(1-q_{Bg}))\). If the panel contains several markers, a simple model assumes their detection outcomes are conditionally independent given the cell type. Under that assumption, the likelihood ratios multiply and their logarithms add:

$$ \log LR(A:B)= \sum_{g\in G} \left[ Y_{ig}\log\left(\frac{q_{Ag}}{q_{Bg}}\right) +(1-Y_{ig})\log\left(\frac{1-q_{Ag}}{1-q_{Bg}}\right) \right]. $$

This comparison requires probabilities appropriate to the assay, sequencing depth, and biological context. They might come from well-characterized cells or other independent evidence. It also assumes the cell is reasonably represented by one of the candidates and that the chosen marker panel is relevant. Conditional independence is an approximation: correlated genes can repeat the same biological signal, making the sum look stronger than the distinct evidence warrants. These assumptions should be considered before interpreting the score.

Interpretation: A log likelihood ratio of 0 means the observed marker outcomes are equally likely under A and B in this model. A positive value favors A; a negative value favors B. The likelihood ratio measures relative evidence under the specified model. It is not, by itself, the probability that either label is true.

Why an Undetected Marker May Say Very Little

For an undetected gene, the comparison is between \(1-q_{Ag}\) and \(1-q_{Bg}\). If both candidates have low detection probabilities, non-detection is expected under both and contributes little. If the gene is usually detected under A but seldom detected under B, non-detection is comparatively surprising under A and can weigh against it. This is the formal reason that “the marker was not seen” is incomplete reasoning.

$$ LR_g(A:B\mid Y_{ig}=0)=\frac{1-q_{Ag}}{1-q_{Bg}}. $$

In particular, a negative marker for A should usually have low \(q_{Ag}\). Its detection can count against A when the competing type is expected to express it more often. But if the marker is rarely detected under both candidates, either its detection or non-detection may be weakly discriminating. The biological label “negative marker” alone does not determine the strength or direction of the evidence; the two candidate-specific detection probabilities do.

Worked Example: A Detected Negative Marker Offsets Positive Evidence

Consider one hypothetical cell and two related candidate types, A and B. The panel has four markers. The probabilities below are assumed to be appropriate for this hypothetical assay and are treated as fixed for the calculation.

GeneRole relative to A\(q_{Ag}\)\(q_{Bg}\)Observed \(Y_{ig}\)
G1Positive0.800.201
G2Positive0.600.301
G3Negative0.050.701
G4Negative0.100.500

G1 is detected, so its contribution is \(\log(0.80/0.20)=\log(4)=1.386\). G2 is also detected and contributes \(\log(0.60/0.30)=\log(2)=0.693\). G3 is a negative marker for A but is detected; its contribution is \(\log(0.05/0.70)=\log(0.07143)=-2.639\). G4 is not detected, so its contribution is \(\log((1-0.10)/(1-0.50))=\log(0.90/0.50)=\log(1.8)=0.588\).

Adding the four contributions gives \(1.386+0.693-2.639+0.588=0.028\). Thus \(LR(A:B)=\exp(0.028)\approx1.028\). Although the two positive markers support A, the detected negative marker largely offsets them. The combined evidence is close to neutral; it does not justify a confident distinction between A and B.

A Short Justification for Combining Markers

The sum in the comparison follows from a probability factorization, not from an arbitrary rule that each marker should receive equal weight. Under conditional independence, the probability of the observed vector of detections under type \(k\) is the product of the gene-specific probabilities:

$$ P(\{Y_{ig}\}_{g\in G}\mid T_i=k) =\prod_{g\in G}q_{kg}^{Y_{ig}}(1-q_{kg})^{1-Y_{ig}}. $$

Taking the ratio of this expression for A and B gives a product of per-gene likelihood ratios. Taking the logarithm turns that product into the sum shown earlier. Each marker’s contribution therefore depends on the observed outcome and on how differently the two candidates predict it. A gene with similar detection probabilities under A and B contributes little, even if it is commonly called a marker.

The factorization also shows where the model can fail. If two markers are strongly correlated, the product treats them as if they supplied separate evidence. The calculated score can then be too extreme. The log-additive calculation is most defensible for a modest panel of informative, nonredundant markers, or when dependence has been addressed by an appropriate model.

Worked Example: Non-Detection of a Weakly Detectable Marker

Suppose gene M is considered a positive marker for A, but its detection probability is only \(q_{AM}=0.25\), compared with \(q_{BM}=0.20\) for B. In the hypothetical cell, M is not detected. Its likelihood ratio for A versus B is

$$ LR_M(A:B)=\frac{1-0.25}{1-0.20} =\frac{0.75}{0.80}=0.9375, \qquad \log LR_M(A:B)\approx-0.0645. $$

This is only slight evidence against A: non-detection is quite likely under both candidates. Now suppose instead that M is detected with probability \(0.90\) under A and \(0.20\) under B. The same non-detection gives

$$ LR_M(A:B)=\frac{1-0.90}{1-0.20} =\frac{0.10}{0.80}=0.125. $$

Here the observed absence is eight times as likely under B as under A, because \(1/0.125=8\). The strength of the evidence comes from the difference in expected detectability, not simply from calling M a positive marker.

Combining Concordant and Contradictory Evidence

A practical comparison should allow evidence to point in different directions. A cell may show several markers expected for A and also show one marker that is unusual for A. Treating a single negative marker as an automatic veto discards useful information; ignoring it entirely discards potentially important contrary evidence. The likelihood-ratio calculation gives each observation a contribution determined by how probable it is under both candidates.

Worked Example: A Panel With One Contradictory Detection

For a second hypothetical comparison, assume equal candidate-specific calibration quality and conditional independence for four genes. G1 and G2 are positive markers for A; G3 and G4 are negative markers for A. The observed outcomes are G1 detected, G2 detected, G3 undetected, and G4 detected. Use the following assumed detection probabilities:

Gene\(q_{Ag}\)\(q_{Bg}\)OutcomeContribution to \(\log LR(A:B)\)
G10.900.20Detected\(\log(0.90/0.20)=1.504\)
G20.800.30Detected\(\log(0.80/0.30)=0.981\)
G30.050.60Undetected\(\log(0.95/0.40)=0.865\)
G40.100.50Detected\(\log(0.10/0.50)=-1.609\)

The total is \(1.504+0.981+0.865-1.609=1.741\), so \(LR(A:B)=\exp(1.741)\approx5.70\). The observed panel is about 5.70 times as likely under A as under B according to these assumed probabilities. G4 is detected despite being a negative marker for A and substantially reduces the evidence for A, but it does not erase the other evidence. This ratio is not a 5.70-fold probability that A is true; it compares the likelihood of the observations under the two specified candidates.

For comparison, omitting G4 would give a log ratio of \(3.350\), or a likelihood ratio of about \(28.5\). Dropping a contradictory marker can therefore make the evidence seem much stronger than the full panel supports.

Diagnostics, Alternatives, and Limitations

Before using a negative marker in an annotation decision, check whether the observation and its expected detection probability are comparable. A marker measured in a shallow library may be undetected even when it is expressed. Likewise, a probability estimated in a different tissue, developmental state, or assay may not represent the cell being considered. If detection varies markedly with library depth, one shared \(q_{kg}\) may be inadequate; comparisons should account for that variation or be restricted to cells with comparable measurement conditions.

Several warning signs merit particular attention:

  • Many zero or one detection probabilities. A probability estimated as exactly 0 or 1 can produce an infinite log contribution. Such estimates are often unstable, especially with few reference cells. Use adequately supported estimates and avoid treating a small observed sample as proof of impossible detection.
  • Redundant markers. Several genes in the same tightly coordinated program may provide overlapping evidence. Counting each as independent can exaggerate support.
  • Contradictory markers concentrated in one cell. This may reflect technical noise, a state not represented in the candidate profiles, or an incorrect candidate set. Do not automatically interpret every contradiction as a decisive exclusion.
  • Unclear marker status. A marker expected to be low in one setting may be detectable in another. If its context-specific behavior is uncertain, its absence or presence should be given little weight.

A positive-only score is simpler and can be useful when the available negative-marker expectations are unreliable. Its limitation is that it cannot penalize a candidate for a strong, well-supported contradiction. A hard exclusion rule—for example, rejecting A whenever a negative marker is detected—is easy to explain but ignores detection uncertainty and can be brittle. A likelihood comparison is more graded, but depends on credible probabilities and assumptions about dependence. None of these approaches repairs a poorly chosen panel or proves a cell’s identity.

Marker probabilities may also be estimated with uncertainty. If the calibration data are limited, plugging a noisy estimate into the formula can give misleadingly precise contributions. Report the evidence as conditional on the assumed probabilities, and assess whether the conclusion changes under plausible alternatives. An annotation should remain tentative when its support depends on one uncertain marker or on an unverified detection rate.

Key takeaway: A missing marker is strong evidence only when it would usually have been detected under the candidate being questioned and its expected detection differs meaningfully between candidates. Combine positive and negative evidence, but interpret the result in light of assay sensitivity, marker dependence, and uncertainty in the expected profiles.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions below.

  1. For a marker with \(q_A=0.30\) and \(q_B=0.25\), what likelihood ratio for A versus B results when the gene is undetected? Is this strong evidence?
  2. A gene has \(q_A=0.05\) and \(q_B=0.60\). Explain why its non-detection supports A relative to B.
  3. Why can counting five highly correlated markers as five independent observations exaggerate the evidence?
  4. What does a likelihood ratio of 4 for A versus B mean, and what does it not mean?

Solutions. (1) The ratio is \((1-0.30)/(1-0.25)=0.70/0.75=0.9333\), only weak evidence against A because non-detection is common under both. (2) The ratio for non-detection is \(0.95/0.40=2.375\), so this outcome is 2.375 times as likely under A as under B. (3) Conditional independence would multiply all five contributions, even though correlated genes may repeat the same underlying signal; the resulting evidence can be overstated. (4) Under the specified model, the observed data are four times as likely under A as under B. It is not a claim that A has four times the probability of being true or that A is certainly the correct label.

Clinical Biostats