Interpreting and Diagnosing Learned Topics

Interpreting a Topic Requires More Than Naming It

A learned topic is a statistical pattern in gene counts, not automatically a cell type, a causal mechanism, or a validated biological program. Its interpretation depends on several features considered together: which genes receive high weight, whether that weight is concentrated in a few genes, how broadly the topic is represented across cells, and whether the fitted model can reproduce relevant features of the observed data.

This tutorial uses the standard topic-model notation in which \(X_{ig}\) is the observed count for gene \(g\) in cell \(i\), \(\theta_{ik}\) is the proportion of cell \(i\)'s topic mixture assigned to topic \(k\), and \(\beta_{kg}\) is the weight of gene \(g\) in topic \(k\). The weights for a topic sum to one across the modeled genes, as do the topic proportions for a cell. In a count-based mixture, the expected count for gene \(g\) in cell \(i\), conditional on the cell's total \(N_i\), is \(N_i\sum_k\theta_{ik}\beta_{kg}\). The topic weights describe gene composition; the cell proportions describe where and how strongly the topic is used.

Interpretation assumes that the count matrix, selected genes, and cell inclusion criteria are appropriate for the question; that the fitted model is a reasonable representation of those counts; and that the topic parameters have been estimated well enough to inspect. Topic labels are arbitrary: exchanging topic indices leaves the model unchanged. Therefore, compare topics by their gene and cell patterns, not by index. Stability across initializations is useful context, but even a stable topic may capture a technical pattern rather than biology.

Top Genes and Gene-Weight Concentration

A ranked gene list is a useful starting point because the largest \(\beta_{kg}\) values identify what most contributes to the topic's gene distribution. Consider several top genes together. Do they form a recognizable, plausibly related set? Do they include genes associated with a technical process, such as mitochondrial or ribosomal content, hemoglobin contamination, stress, or dissociation response? A coherent-looking list supports an interpretation, but it does not establish one: several unrelated genes can have high weights, and familiar marker genes can appear because of ambient RNA or other artifacts.

One simple concentration summary is the Herfindahl concentration \(C_k=\sum_g\beta_{kg}^2\). It is small when weight is spread across many genes and larger when a few genes dominate. Its reciprocal, \(G_{\mathrm{eff},k}=1/C_k\), is an effective number of equally weighted genes: if a topic has equal weight on exactly \(m\) genes and zero elsewhere, then \(C_k=1/m\) and \(G_{\mathrm{eff},k}=m\). This is a compact summary, not a measure of biological quality. A broad coherent program can have a large effective number, while a topic dominated by a single technical gene can have a small one.

The reciprocal interpretation follows directly: for \(m\) equally weighted genes, each weight is \(1/m\), so \(C_k=m(1/m)^2=1/m\), and \(1/C_k=m\). With unequal weights, \(G_{\mathrm{eff},k}\) gives the number of equal-weight genes that would have the same concentration. Always examine the ranked list as well; the summary alone hides which genes dominate.

Cell Prevalence Is a Thresholded Description

A topic can have a recognizable gene distribution but appear in few cells, or be distributed broadly with small proportions in many cells. To summarize prevalence, choose a threshold \(\tau\) and calculate the fraction of cells with \(\theta_{ik}>\tau\). State the threshold and the population of cells used in the denominator. There is no universal threshold: because topic proportions are continuous mixture weights, a cell is not intrinsically “in” or “out” of a topic.

Here \(I(\cdot)\) equals 1 when the condition is true and 0 otherwise. Prevalence changes with \(\tau\), with the set of analyzed cells, and with estimation uncertainty in \(\theta_{ik}\). It is helpful to inspect the distribution of topic proportions, not only one thresholded value. If posterior draws are available, uncertainty in the proportions can also be reflected in the prevalence summary rather than treating point estimates as exact.

Posterior Predictive Behavior

A topic's gene list and cell prevalence summarize fitted parameters; neither guarantees that the model captures the observed count patterns. A posterior predictive check compares a feature calculated from observed data with the same feature calculated from replicate data generated under the fitted model. For a Bayesian model, generate a replicate using posterior draws of its parameters; in an approximate or variational fit, draws are based on that approximation. For a count model conditioned on each observed library size \(N_i\), generate replicate counts at those same library sizes. This isolates discrepancies in gene allocation from differences in total counts.

Choose a discrepancy statistic \(T(X)\) that targets a question relevant to interpretation, such as the mitochondrial count fraction in each cell, the number of cells with a high topic proportion, or the distribution of a selected gene set's counts. Compute \(T(X)\) on the observed matrix and on each replicate \(X^{\mathrm{rep}}\). If the observed value is extreme relative to the replicated values, the model does not reproduce that feature well. A posterior predictive tail fraction is a diagnostic summary, not a conventional null-hypothesis p-value: it is not the probability that the model is true or false.

Posterior predictive checks should be targeted rather than reduced to one global score. A model can reproduce total counts while failing to reproduce gene-specific structure, or reproduce an overall gene fraction while failing in a particular batch or cell subset. Conditioning choices matter: fixing library sizes checks allocation of counts given depth, while simulating library sizes as well checks the model's treatment of depth. State which version was used.

Putting the Evidence Together

A practical interpretation uses several checks jointly. First inspect the top genes and their concentration. Then examine the distribution of \(\theta_{ik}\) and a clearly defined prevalence summary. Finally, ask whether model-generated data reproduce relevant observed summaries. For example, a plausible set of related genes, moderate concentration, non-negligible prevalence, and no major mismatch on relevant predictive checks provide converging descriptive support for a coherent topic. None of these findings proves a biological mechanism or a cell identity.

These views complement rather than replace other approaches. A ranked gene list is direct and interpretable but can overemphasize a small number of high-weight genes. Gene-set enrichment can summarize whether a list overlaps known annotations, but results depend on the reference sets, background genes, and selection rule; enrichment is not independent validation. Predictive fit evaluates reproduction of data features but does not guarantee interpretable or biologically meaningful topics. Stability across initializations can indicate reproducibility of a fitted pattern, yet a stable technical artifact remains an artifact. Use each approach to answer its own question.

Diagnostics have limitations. Estimated weights and proportions depend on model assumptions, gene filtering, preprocessing, the number of topics, and prior choices. A topic may combine multiple programs or split one program across topics. Sparse priors can increase concentration, and low counts can make cell-level proportions uncertain. Posterior predictive agreement is only agreement on the selected statistics; it cannot establish that all important structure is captured. If an interpretation is important, seek converging evidence from appropriate external knowledge or independent measurements, while keeping the claim proportional to that evidence.

Common Mistakes

• Calling a topic a cell type from its top genes alone. A gene list suggests an interpretation; the cell distribution and possible technical explanations still need examination.
• Reporting prevalence without a threshold. A fraction of cells above 0.1 is not interchangeable with a fraction above 0.2. State the threshold and denominator.
• Calling concentration “quality.” A large concentration can reflect a narrow, interpretable signal or a single dominating artifact. Inspect the genes themselves.
• Treating a posterior predictive tail fraction as a null-hypothesis p-value. It is a model-checking diagnostic, conditional on the fitted model and statistic, not a probability that the model is correct.
• Assuming a stable topic must be biological. Repeated recovery shows reproducibility under the fitting procedure, not biological validation.