Tutorials › Statistical Methods for Single-Cell RNA Sequencing › Marker Genes As Descriptive And Inferential Claims

Marker Genes and the Double-Dipping Problem · Tutorial 293 of 700

Marker Genes As Descriptive And Inferential Claims

Learn to report marker rankings as summaries of observed clusters and to recognize when inferential claims require a valid design that accounts for how those clusters were formed.

Advanced 12 min read

What You'll Learn

  • Define marker rankings and expression contrasts as descriptive summaries of an observed partition.
  • Distinguish a difference among observed cells from a claim about a broader population.
  • Identify the assumptions needed for a gene-level inferential test with fixed group labels.
  • Recognize why data-adaptive cluster labels can undermine ordinary gene-level p-values.
  • Report marker effects, uncertainty, and inferential limitations without overstating the evidence.

Two Different Questions About Marker Genes

A marker-gene table can answer a useful descriptive question: which genes differ most across the partition of cells we observed? It does not automatically answer the inferential question: which genes differ in a target population, with uncertainty calibrated to the process that produced the groups? These questions are related, but their evidence requirements are not the same.

The distinction matters in single-cell analysis because a partition is often discovered from expression measurements. A gene may rank highly because its observed values differ between the groups, because it helped determine which cells were grouped together, or both. A ranking can still be informative for describing an exploratory result. The difficulty arises when that ranking is treated as if it were an independently planned test of a gene-level population difference.

Definition: Let \(Y_{ig}\) be the measured, normalized expression value for gene \(g\) in cell \(i\), and let \(C_i\) denote that cell's assigned cluster. A descriptive marker summary is a statistic calculated from the observed \(Y_{ig}\) values and observed cluster assignments, such as a difference in average expression or in the fraction of cells expressing the gene. A gene-level inferential claim uses data to make a statement about a target population or data-generating process, with uncertainty quantified under an appropriate sampling and analysis model.

For example, a mean difference between cluster \(k\) and the other observed cells can be written as

$$ \widehat{\Delta}_{g,k} = \frac{1}{n_k}\sum_{i:C_i=k}Y_{ig} - \frac{1}{n_{-k}}\sum_{i:C_i\ne k}Y_{ig}, $$

where \(n_k\) and \(n_{-k}\) are the numbers of observed cells in the cluster and its complement. Once the measurements and assignments are fixed, this is a numerical summary of those cells. It has no sampling uncertainty unless a target population and a sampling model are specified. A ranking based on \(\widehat{\Delta}_{g,k}\), a fold change, or another score therefore describes the observed partition; it is not, by itself, a test of a population-level null hypothesis.

What a Descriptive Ranking Can and Cannot Say

Marker rankings depend on the chosen summary. A mean difference emphasizes expression magnitude on the scale being analyzed. A fraction-expressing summary emphasizes how often values meet a chosen detection criterion. A ranking based on one measure need not match a ranking based on another. In all cases, state the scale, the group contrast, and the ranking criterion so that readers know what the table summarizes.

A descriptive statement can be precise without being inferential. For instance: “In this observed partition, gene G had the largest mean normalized-expression difference between cluster A and the remaining cells.” That statement concerns the data at hand. By contrast, “gene G is differentially expressed in the population from which these cells were sampled” goes beyond that finite-data summary. It needs a design and analysis that justify the population-level uncertainty being reported.

Worked Example: Calculating a Descriptive Marker Difference

Hypothetical data. Suppose a small observed partition contains three cells in cluster A and three in cluster B. For gene G, their normalized log-expression values are \(8, 9, 10\) in A and \(2, 3, 4\) in B. Calculate the difference in observed mean expression.

The mean in A is \((8+9+10)/3=27/3=9\) normalized log-expression units. The mean in B is \((2+3+4)/3=9/3=3\) normalized log-expression units. Thus the observed difference, A minus B, is \(9-3=6\) normalized log-expression units.

This calculation supports the descriptive statement that G has a six-unit mean difference between these observed groups on this scale. It does not, on its own, establish that the difference generalizes to other cells, donors, or experiments. Nor does its size establish that G is a useful or biologically specific marker; those are separate judgments.

When a Gene-Level Test Has a Defensible Interpretation

A familiar two-sample test illustrates what inferential claims require. Suppose group labels are fixed independently of the gene values being tested, observations are independent sampling units, and the values in each group are normally distributed with a common variance. Under the null hypothesis of equal group means, the pooled two-sample \(t\)-statistic has a \(t\) distribution with \(n_1+n_2-2\) degrees of freedom. This result also requires that the test and contrast were specified without selecting them because they showed a large difference in these same observations.

In single-cell data, independence must be considered at the level of sampling. Cells from one donor are generally not independent biological replicates. If the target claim concerns differences across donors, the analysis should use a defensible donor-level design, rather than treating many cells from a few donors as if they were independent donors. A valid test also does not by itself address testing many genes; a broad set of gene-level claims requires an appropriate approach to multiplicity.

Worked Example: A Prespecified Difference Across Donors

Hypothetical data. Assume four independent donors in each of two prespecified groups. For one gene, donor-level cell-type summaries on a normalized log-expression scale are \(4, 5, 4, 5\) in group A and \(2, 3, 2, 3\) in group B. Suppose group membership was defined independently of this gene, donors are the sampling units, and the normality and equal-variance assumptions are reasonable for this illustration. Test the prespecified two-sided hypothesis of equal group means at \(\alpha=0.05\).

The sample means are \(\bar{x}_A=(4+5+4+5)/4=4.5\) and \(\bar{x}_B=(2+3+2+3)/4=2.5\), so the estimated difference is \(2.0\) normalized log-expression units. Each group has sample variance \(s^2=1/3\). The pooled variance is therefore

$$ s_p^2 = \frac{(4-1)(1/3)+(4-1)(1/3)}{4+4-2} = \frac{2}{6} = \frac{1}{3}. $$

The standard error is \(SE=\sqrt{s_p^2(1/4+1/4)}=\sqrt{1/6}\approx0.4082\). The test statistic is \(t=(4.5-2.5)/\sqrt{1/6}\approx4.899\), with \(df=4+4-2=6\). The two-sided p-value is approximately \(0.0027\), below \(0.05\); under the stated model and design, the data provide evidence against equal group means. Using \(t_{0.975,6}\approx2.447\), a 95% confidence interval for the difference is \(2.0\pm2.447\sqrt{1/6}\), or approximately \([1.001,\ 2.999]\) normalized log-expression units. The interval is a procedure-based uncertainty statement under the model, not a probability that the fixed difference lies in this particular interval.

This example's inferential interpretation depends on its setup: independent donors, a prespecified comparison, and plausible distributional assumptions. Replacing donors with cells as the independent observations, or selecting the groups because this gene had a large observed difference, would not be justified by the calculation shown.

Why Cluster Discovery Changes the Claim

If the same expression data are used both to form a partition and to rank genes that separate that partition, the assignments are data-adaptive: the observed values helped determine the groups. Ordinary tests that treat those groups as fixed in advance do not account for that selection. As a result, their usual p-values or confidence intervals may not have their advertised calibration for the post-discovery claim.

This does not mean that the descriptive contrast is false or that a discovered cluster is useless. It means that the statistic and the inferential interpretation must not be conflated. The contrast remains a correct summary of the assigned cells. What is not justified by that calculation alone is a claim that the gene would show the same evidence under repeated sampling and the same discovery process. The appropriate inferential strategy depends on the target claim and on how clustering and marker selection were performed.

Worked Example: Reporting a Selected Gene Without Overclaiming

Hypothetical data. Suppose a clustering procedure used expression of gene G along with other genes to create two groups. The observed values for G are \(8, 9, 10\) in cluster A and \(2, 3, 4\) in cluster B. As calculated earlier, the group means are 9 and 3, giving a six-unit observed difference.

The defensible report is: “Gene G showed a six-unit mean normalized log-expression difference between the two groups in this observed partition; G also contributed to the expression information used to form the partition.” This reports both the numerical contrast and its discovery context.

It would be overconfident to attach an ordinary fixed-group p-value and describe it as calibrated evidence of a population difference without addressing the data-adaptive grouping. The arithmetic of the mean difference is unchanged, but the test's reference distribution may no longer represent the process that produced the selected groups. The correct conclusion is not “there is no difference”; it is that this analysis supports a descriptive observation, while the inferential claim requires additional justification.

Justifying the Distinction

The distinction follows from what is held fixed. For a descriptive summary, fix the observed expression values and assignments. The group means, their difference, and the resulting rank are then determined calculations. There is no random variable left in that finite-data calculation.

Inference instead considers what could vary under repeated sampling. For a prespecified contrast with independent observations, the sampling distribution of the test statistic under \(H_0\) supplies the reference for a p-value. The calculation in the preceding donor example uses the pooled two-sample model, so its calibration is conditional on the groups having been specified independently of the tested gene. If the tested gene helped determine the labels or was selected because it ranked highly, the observed statistic has also passed through a selection process. A reference distribution that ignores that process need not give the intended error rate. This is why a descriptive marker table and an inferential result should be labeled and interpreted separately.

Diagnostics, Alternatives, and Reporting

Before making an inferential claim from marker analysis, check the link between the data and the proposed target. Ask whether group labels were externally defined or discovered from the tested expression values; whether the independent units are cells, donors, or another sampling unit; whether the assumed model is plausible; and whether the analysis involves many gene tests. Large cell counts do not substitute for independent donors. Strongly skewed values, outliers, unequal variances, or donor-level dependence can also make a simple pooled \(t\)-test inappropriate.

The main alternatives serve different purposes. A descriptive ranking is appropriate for summarizing and exploring an observed partition, and should be reported with its effect measure and scale. A conventional inferential test can address a prespecified comparison when its sampling and model assumptions are defensible. When labels or genes were selected from the same data, an inferential analysis must additionally account for the discovery process; independent confirmation or a design that separates discovery from testing are examples of broad strategies, not automatic guarantees. The specific method must match the claim and its assumptions.

A useful report makes the status of the result explicit:

  • For a descriptive result, name the observed groups, expression scale, summary statistic, and observed effect size.
  • For an inferential result, identify the target population, independent sampling unit, model, contrast, uncertainty measure, and multiplicity handling when many genes are assessed.
  • If the same data helped discover the groups or select the gene, state that fact and do not present an unadjusted fixed-group p-value as if the discovery step had not occurred.

The limitation is substantive: a strong observed marker contrast can be valuable for describing a dataset, but its magnitude alone does not determine reproducibility, generalizability, or causal relevance. Conversely, failure to justify a particular p-value does not erase the observed contrast. Keeping those conclusions separate makes marker reports both informative and appropriately cautious.

Key takeaway: A marker ranking describes how genes vary across the observed partition. A population-level gene claim requires uncertainty calibrated to the sampling units, model assumptions, and process that produced the groups and selected the gene.

Check Your Understanding

Answer each question, then compare with the solutions.

  1. A gene has an observed mean difference of 1.8 normalized expression units between two clusters. What can be stated descriptively, and what additional basis is needed for a population-level claim?
  2. Why can a large number of cells from two donors not automatically be treated as a large independent sample of donors?
  3. In the donor example, what was the estimated mean difference and the approximate 95% confidence interval?
  4. A gene was used to help form clusters and then received an ordinary fixed-group p-value. What is the central inferential concern?

Solutions. (1) One may report that the observed cluster means differ by 1.8 units on the stated scale. A population-level claim additionally needs a defensible sampling model and uncertainty calculation that accounts for how groups and genes were selected. (2) Cells from the same donor may share biological and technical influences, so they are not independent donor-level replicates; treating them as such can understate uncertainty. (3) The estimated difference was \(4.5-2.5=2.0\) normalized log-expression units, with an approximate 95% interval of \([1.001,\ 2.999]\) units under the stated pooled two-sample assumptions. (4) The usual fixed-group reference distribution ignores that expression helped determine the groups, so the reported p-value may not have its advertised calibration for the selected result.

Clinical Biostats