Latent Dirichlet Allocation for Cell Gene Counts

From Cell Gene Counts to Topics

A single cell can express genes associated with several biological programs at once. A topic model represents that pattern as a mixture: each cell has proportions across a set of latent topics, and each topic assigns probabilities across genes. For example, one topic might place relatively high probability on genes involved in a particular process, while a cell can combine that topic with others. The model describes counts probabilistically; it does not require every cell to belong to exactly one topic.

Latent Dirichlet allocation (LDA) adapts a document-topic model to a count matrix. Here, a cell plays the role of a document, a gene identity plays the role of a word, and a gene count records how many times that gene appears among the cell’s count tokens. The word “topic” refers to a probability distribution over genes, not a pre-labelled biological category. Biological interpretation requires separate evidence.

Definition: Let \(C\) be the number of cells, \(G\) the number of genes, and \(K\) the number of topics. The observed count matrix is \(X=(x_{cg})\), where \(x_{cg}\) is the nonnegative integer count for gene \(g\) in cell \(c\). The library size is \(N_c=\sum_{g=1}^{G}x_{cg}\). Cell \(c\) has topic proportions \(\theta_c=(\theta_{c1},\ldots,\theta_{cK})\), with nonnegative entries summing to one. Topic \(k\) has gene probabilities \(\phi_k=(\phi_{k1},\ldots,\phi_{kG})\), also nonnegative and summing to one.

The probability of a gene for a cell is a mixture of the topic-specific gene probabilities. In particular, the probability assigned to gene \(g\) in cell \(c\) is the sum of that gene’s probabilities under each topic, weighted by the cell’s topic proportions. Thus, the model can produce a cell with counts spread across genes from several topics.

p_{cg}=\sum_{k=1}^{K}\theta_{ck}\phi_{kg}

The symbol \(p_{cg}\) denotes a probability, not an expected count. If the cell has \(N_c\) count tokens, its expected count for gene \(g\), conditional on the mixture and topic distributions, is \(N_cp_{cg}\). The count is random, so its realized value need not equal that expectation.

The Generative Construction

A convenient way to define LDA is to describe how hypothetical data could be generated. A Dirichlet distribution supplies probability vectors: it places its probability mass on vectors with nonnegative entries that sum to one. Its positive parameter vector controls the distribution’s concentration and relative weighting. For a vector \(\boldsymbol{a}=(a_1,\ldots,a_J)\), the expectation of component \(j\) under \(\operatorname{Dir}(\boldsymbol{a})\) is \(a_j/\sum_{\ell=1}^{J}a_\ell\).

Definition: In LDA, draw each cell’s topic proportions as \(\theta_c\sim\operatorname{Dir}(\boldsymbol{\alpha})\), where \(\boldsymbol{\alpha}\) has \(K\) positive entries. Draw each topic’s gene probabilities as \(\phi_k\sim\operatorname{Dir}(\boldsymbol{\eta})\), where \(\boldsymbol{\eta}\) has \(G\) positive entries. For each of the \(N_c\) tokens in cell \(c\), draw a topic \(z_{cn}\) from the categorical distribution with probabilities \(\theta_c\), then draw a gene \(w_{cn}\) from the categorical distribution with probabilities \(\phi_{z_{cn}}\). The observed \(x_{cg}\) is the number of tokens in cell \(c\) whose gene is \(g\).

The topic distributions are shared across cells, while the topic proportions vary from cell to cell. The topic assignment \(z_{cn}\) and gene identity \(w_{cn}\) are latent in an observed count matrix: the matrix records gene totals but does not say which topic generated any particular count token. The order of tokens within a cell is not observed or used; only the resulting counts matter.

This construction makes several assumptions explicit. Conditional on a cell’s topic proportions and the shared topic gene distributions, its tokens are generated independently by the stated categorical draws. The model conditions on each cell’s total \(N_c\); it describes how that fixed number of tokens is allocated across genes, rather than providing a separate model for how many total tokens were captured. The gene universe is fixed across cells, and the input counts are nonnegative integers on a consistent scale.

Why the Mixture Gives a Multinomial Count Model

For one token in cell \(c\), the probability of gene \(g\) can be obtained by considering each possible topic assignment. The token has probability \(\theta_{ck}\) of coming from topic \(k\), and then probability \(\phi_{kg}\) of being gene \(g\). Summing these mutually exclusive possibilities gives \(p_{cg}\). Because the tokens are conditionally independent and share these same gene probabilities, their gene counts have a multinomial distribution.

P(w_{cn}=g\mid\theta_c,\phi_1,\ldots,\phi_K) =\sum_{k=1}^{K}\theta_{ck}\phi_{kg}=p_{cg}, \qquad (x_{c1},\ldots,x_{cG})\mid\theta_c,\phi_1,\ldots,\phi_K \sim\operatorname{Multinomial}(N_c,p_{c1},\ldots,p_{cG})

One useful consequence follows directly from writing a gene count as a sum of token indicators. For each token \(n\), let \(I_{cng}=1\) if that token is gene \(g\), and \(0\) otherwise. Then \(x_{cg}=\sum_{n=1}^{N_c} I_{cng}\), with \(E(I_{cng})=p_{cg}\). Independence across tokens gives the mean and variance below. Different gene counts in the same cell are negatively associated under this fixed-total model: a token counted for one gene cannot also be counted for another.

E(x_{cg}\mid\theta_c,\phi)=N_cp_{cg}, \qquad \operatorname{Var}(x_{cg}\mid\theta_c,\phi)=N_cp_{cg}(1-p_{cg}), \qquad \operatorname{Cov}(x_{cg},x_{ch}\mid\theta_c,\phi)=-N_cp_{cg}p_{ch}\quad(g\ne h)

For example, the variance formula follows because each indicator has variance \(p_{cg}(1-p_{cg})\), and the \(N_c\) indicators are independent conditional on the parameters. The covariance formula reflects the competition among gene categories for a fixed number of tokens. These statements are conditional on the cell mixture and topic distributions; they do not say that counts from different cells are independent after uncertainty about shared topics is considered.

Worked Example: Calculating a Cell’s Mixture Probabilities

Suppose there are two topics and three genes. Let the gene distributions be \(\phi_1=(0.70,0.20,0.10)\) and \(\phi_2=(0.10,0.30,0.60)\). A hypothetical cell has topic proportions \(\theta_c=(0.80,0.20)\) and library size \(N_c=20\). Compute its gene probabilities and expected counts.

For gene 1, the mixture probability is \(p_{c1}=(0.80)(0.70)+(0.20)(0.10)=0.56+0.02=0.58\). For gene 2, \(p_{c2}=(0.80)(0.20)+(0.20)(0.30)=0.16+0.06=0.22\). For gene 3, \(p_{c3}=(0.80)(0.10)+(0.20)(0.60)=0.08+0.12=0.20\). These probabilities sum to \(0.58+0.22+0.20=1.00\), as required.

The expected counts are \(20(0.58)=11.6\), \(20(0.22)=4.4\), and \(20(0.20)=4.0\). These are conditional expectations, not a claim that a realized count vector must be \((11.6,4.4,4.0)\); realized counts are integers and fluctuate around their expected values.

Generating a Small Synthetic Count Matrix

The following hypothetical construction makes the latent stages visible. Use two topics and three genes, labelled \(G_1,G_2,G_3\), with the same topic gene distributions as in the preceding example. Set three cell mixtures to \(\theta_A=(0.80,0.20)\), \(\theta_B=(0.25,0.75)\), and \(\theta_C=(0.50,0.50)\). These vectors and the topic distributions are treated as fixed for this demonstration; in a full generative account, they could be draws from their Dirichlet priors.

Choose a cell and its token total.
Set hypothetical library sizes to \(N_A=10\), \(N_B=8\), and \(N_C=10\). Each total is fixed for the subsequent allocation of that cell’s tokens.

Assign each token a topic.
For each cell, draw a topic for every token using that cell’s mixture. One possible random outcome is 8 topic-1 and 2 topic-2 tokens for A; 2 topic-1 and 6 topic-2 tokens for B; and 5 tokens from each topic for C.

Draw a gene within each assigned topic.
For each token, draw its gene using the gene probabilities of its assigned topic. The table below gives one possible set of gene-count outcomes within each cell-topic group.

Sum over topics to form the observed matrix.
Add the topic-specific gene counts within each cell. The topic allocation is used to explain the construction, but the resulting count matrix contains only the cell-by-gene totals.

For clarity, the possible topic-specific outcomes are shown as gene-count triples in \(G_1,G_2,G_3\) order. Cell A has \((6,1,1)\) from topic 1 and \((0,1,1)\) from topic 2. Cell B has \((1,1,0)\) from topic 1 and \((1,1,4)\) from topic 2. Cell C has \((3,1,1)\) from topic 1 and \((1,1,3)\) from topic 2. Each triple sums to its topic-assigned token total. Each is a possible categorical sampling outcome, not the expected allocation.

Cell	Topic 1 gene counts	Topic 2 gene counts	Observed gene counts	Library size
A	(6, 1, 1)	(0, 1, 1)	(6, 2, 2)	10
B	(1, 1, 0)	(1, 1, 4)	(2, 2, 4)	8
C	(3, 1, 1)	(1, 1, 3)	(4, 2, 4)	10

For cell A, for instance, adding the two topic-specific triples gives \((6+0,1+1,1+1)=(6,2,2)\), whose entries sum to 10. The full synthetic matrix is therefore

X= \begin{pmatrix} 6&2&2\\ 2&2&4\\ 4&2&4 \end{pmatrix}

This example separates two objects that are easy to conflate: the observed cell-by-gene counts and the unobserved topic assignment of each token. Many different token-level allocations could produce the same observed row. The count matrix alone does not reveal a unique allocation.

Worked Example: Checking a Synthetic Cell Against Its Mixture

For cell A above, the probabilities are \((0.58,0.22,0.20)\), and the observed hypothetical row is \((6,2,2)\). Verify the library size and calculate the conditional probability of this count vector under the multinomial model.

The row total is \(6+2+2=10=N_A\), so it is consistent with the specified library size. For a multinomial count vector \(x\), the probability is \(N_A!\prod_g p_{Ag}^{x_{Ag}}/x_{Ag}!\). Substituting the values gives

P(X_A=(6,2,2)\mid\theta_A,\phi) =\frac{10!}{6!\,2!\,2!}(0.58)^6(0.22)^2(0.20)^2 =1260(0.58)^6(0.22)^2(0.20)^2 \approx 0.0929

The coefficient is \(10!/(6!2!2!)=1260\), and the resulting probability is approximately 0.0929. This is the probability of the count vector under the specified fixed mixture and topic distributions, not the probability that the model or its parameters are correct. It also sums over the possible token orders represented by that count vector.

Worked Example: Distinguishing Expected Counts From a Draw

Cell B has mixture \((0.25,0.75)\). Using the two topic distributions above, calculate its gene probabilities and expected counts for \(N_B=8\), then compare them with its realized synthetic row \((2,2,4)\).

The probabilities are \(p_{B1}=(0.25)(0.70)+(0.75)(0.10)=0.175+0.075=0.25\), \(p_{B2}=(0.25)(0.20)+(0.75)(0.30)=0.05+0.225=0.275\), and \(p_{B3}=(0.25)(0.10)+(0.75)(0.60)=0.025+0.45=0.475\). They sum to 1.000. Multiplying by eight gives expected counts \((2.00,2.20,3.80)\).

The realized row \((2,2,4)\) differs from those expectations by \((0,-0.20,0.20)\), while still summing to eight. Such a small difference is ordinary sampling variation. It would be incorrect to interpret the expectation as a required observed count or to infer the cell’s topic proportions exactly from one row.

Assumptions, Diagnostics, and Limits

Before applying this construction, check that the input is suitable for a count likelihood. LDA’s usual count representation requires nonnegative integer entries and a common gene definition across cells. A zero-library cell has no tokens to allocate and provides no information about its mixture through this conditional model; it should be investigated rather than treated as an ordinary cell with an inferred composition. Normalized or log-transformed expression values are not token counts and do not meet the multinomial count assumption.

The fixed-total multinomial allocation is a simplifying sampling assumption. Conditional on the probabilities, it ties the total count to \(N_c\) and makes gene counts compete for that total. It does not itself model library size, gene-specific dispersion beyond that multinomial variation, or a special process producing excess zeros. The earlier material on denoised expression versus observed counts is relevant here: an LDA count likelihood is defined for observed count data, not for denoised values substituted as if they were observed tokens.

Useful checks before fitting or interpreting the model include:

Confirm that each row total equals the intended library size and that all entries are nonnegative integers.
Verify that each \(\theta_c\) and \(\phi_k\) is a probability vector: its entries are nonnegative and sum to one.
Check that the implied \(p_{cg}\) values sum to one for every cell; a failure usually signals an indexing or normalization error.
Compare observed gene-count variability and zero patterns with what a fixed-total multinomial allocation can plausibly produce. Systematic discrepancies can indicate that its sampling assumptions are inadequate.

LDA is not the only way to represent cell-level gene counts. A count VAE, as studied in earlier material on count VAEs, uses a different latent-variable construction and can incorporate a richer observation model; it is not simply LDA with different names for its parameters. LDA’s distinctive construction here is the sequence of a cell-level topic mixture, a token-level topic choice, and a topic-specific gene draw. The model’s simplicity makes this structure transparent, but it also means that conclusions depend on its assumptions and on how the number of topics and priors are specified.

Common Mistakes

Treating a topic as a cell type. A topic is a probability distribution over genes. It is not automatically a known cell identity or a biological mechanism.
Calling \(\theta_{ck}\) a gene probability. It is the proportion assigned to topic \(k\) in cell \(c\). Gene probabilities require combining those proportions with the topic distributions.
Equating expected counts with observed counts. \(N_cp_{cg}\) can be non-integer; a realized count is an integer random outcome.
Using transformed expression as token counts. Fractional normalized values and logged values do not satisfy the standard multinomial count construction.
Assuming the observed matrix identifies token topics. The matrix contains gene totals, not the latent topic label of each token. Multiple allocations can yield the same row.

Key takeaway: LDA represents each cell by a probability mixture over topics and each topic by a probability distribution over genes. Topic choices and gene draws for count tokens produce the observed row totals, conditional on a fixed library size.

Check Your Understanding

Work through the questions, then compare your reasoning with the solutions.

A cell has two topics with proportions \((0.60,0.40)\). At a particular gene, the two topics assign probabilities \(0.50\) and \(0.20\). What is the cell’s probability for that gene?
For the cell in question 1, suppose there are 10 tokens. What is the expected count for that gene? Is that count guaranteed to be observed?
A row of a count matrix is \((3,0,5)\). What is its library size, and what does the row total mean in the conditional LDA construction?
Why can the observed gene counts fail to reveal which topic generated each individual token?

Solutions. (1) The mixture probability is \((0.60)(0.50)+(0.40)(0.20)=0.30+0.08=0.38\). (2) The expected count is \(10(0.38)=3.8\). It is not guaranteed to be observed: the realized count is an integer random outcome. (3) The library size is \(3+0+5=8\). It is the fixed total number of tokens allocated across genes for that cell in the conditional model. (4) The observed row records only gene totals. Different assignments of its tokens to topics can produce the same totals, so the row does not uniquely encode token-level topic labels.