Why Model Counts as Generated Data?
A single-cell count matrix records what an assay measured, not every biological feature that produced those measurements. Cells may differ in cell state, type, or activity, while counts also vary because the measurement process is imperfect. A generative latent-variable model gives a probability-based account of both: it describes an unobserved state for each cell and a process that could produce the observed gene counts given that state.
The distinction matters because a pattern in counts is not itself a direct observation of a biological state. A model can represent a pattern as evidence about a latent state, but that interpretation depends on its assumptions. A latent variable is a model component, not a label that the experiment directly measured.
A latent variable may be discrete, such as a modeled cell group, or continuous, such as a coordinate in a latent space. Either way, it is not a gene count or a directly measured covariate. The observed counts are the data; the latent state is inferred from them under a model.
The Generative Factorization
Write \(\theta\) for the model’s parameters. In the simplest cell-level setup, the model first assigns a distribution to each cell’s latent state and then specifies the distribution of that cell’s count vector conditional on the state:
The factorization is the probability chain rule together with the model’s choice to describe the observation conditionally on the latent variable. The first factor, \(p_\theta(z_i)\), is the prior distribution over latent states. The second, \(p_\theta(x_i\mid z_i)\), is the observation model: it assigns probabilities to count vectors that could be observed when the latent state is \(z_i\).
For a collection of cells, a commonly used simplifying setup treats cells as independent conditional on their latent states and shared parameters. Under that assumption, the joint distribution factors as:
This is a modeling assumption, not a fact guaranteed by the format of a count matrix. For example, if cells share an unmodeled source of variation, they may remain dependent even after accounting for their individual latent states. A more elaborate model could include shared latent quantities or measured covariates. The factorization above is useful as a starting point, but it should not be mistaken for a universal description of single-cell data.
Likewise, the factorization \(p_\theta(x_i\mid z_i)=\prod_g p_\theta(x_{ig}\mid z_i)\) requires an additional assumption: conditional independence of the genes given the latent state and parameters. The joint observation model need not factor this way. It is important to state this assumption rather than silently treating every gene count as independent.
From Latent States to Observed Counts
When the latent state is not observed, the probability of the count vector alone is obtained by averaging over possible states. For a discrete latent variable, this means summing; for a continuous latent variable, it means integrating.
This follows because the possible latent states partition the events that can give rise to \(x_i\). The probability of observing \(x_i\) is therefore the sum (or integral) of the joint probabilities across those states. The result is called the marginal distribution of the observed counts.
After observing \(x_i\), the corresponding conditional distribution of the latent state is:
This posterior distribution expresses uncertainty about the latent state after taking the observed counts into account. It does not establish that a state is a directly observed fact, nor does it guarantee that the model’s latent categories correspond to distinct biological types.
Worked Example: Reading a Small Cell-by-Gene Matrix
Consider this hypothetical matrix of recorded counts for four cells and three genes. Counts are in recorded molecules per cell.
| Cell | Gene 1 | Gene 2 | Gene 3 | Total counts |
|---|---|---|---|---|
| A | 12 | 2 | 1 | 15 |
| B | 10 | 3 | 0 | 13 |
| C | 2 | 11 | 1 | 14 |
| D | 1 | 9 | 2 | 12 |
For cell A, the observed vector is \(x_A=(12,2,1)\). Its total is \(12+2+1=15\) recorded counts. Cells A and B have relatively larger counts for Gene 1, while cells C and D have relatively larger counts for Gene 2. A latent-variable model could represent a pattern of this kind with a cell-level state \(z_i\), and then model the observed count vector conditional on that state.
The matrix alone does not prove that A and B share a biological cell type, or that the two patterns have a particular biological cause. Differences could also reflect measurement variation, other biological variation, or both. A generative model represents such uncertainty through its probability distributions and assumptions; it does not make the source of a pattern known merely by assigning a latent variable.
The row totals are also observed summaries, not latent states. For example, cell C’s total is \(2+11+1=14\) counts. A model could account for such totals in a particular way, but that choice would be an additional modeling decision, not a consequence of the factorization alone.
Worked Example: Computing a Posterior for a Discrete Latent State
Suppose a hypothetical model assigns each cell one of two latent states, \(z_i=A\) or \(z_i=B\), with prior probabilities \(p(z_i=A)=0.5\) and \(p(z_i=B)=0.5\). For one observed count vector \(x_i\), suppose the model assigns probabilities \(p(x_i\mid z_i=A)=0.06\) and \(p(x_i\mid z_i=B)=0.01\). These are illustrative probabilities for the exact count vector, not probabilities for an individual gene.
First calculate the marginal probability of the count vector by summing over both possible states:
Then apply Bayes’ rule to find the posterior probability of state A:
Under these specified model probabilities, the posterior probability of state A is about 0.857, or 85.7%. It is not a probability that state A is true in some model-free sense. It is conditional on the stated prior, the observation probabilities, and the model itself.
What the Observation Model Means
The conditional distribution \(p_\theta(x_i\mid z_i)\) describes variation in the recorded counts that remains possible when the latent state is specified. It is useful to think of this as an observation process: given the model’s account of a cell’s latent structure, the observation model assigns probabilities to possible measured count vectors.
The term “measurement noise” should be used carefully. The residual variation under the observation model may reflect technical variation, but it may also absorb biological differences not represented in \(z_i\), or shortcomings of the model. The factorization does not, on its own, identify which source produced any particular discrepancy. Claims about technical versus biological sources require additional information or assumptions.
Similarly, choosing a latent variable does not ensure that it captures all relevant biological structure. If two cells have the same modeled \(z_i\), their observed counts can still differ. Conversely, cells with similar count vectors need not have identical latent states. The posterior distribution summarizes how the model connects counts to states, including uncertainty in that connection.
Worked Example: Checking a Conditional-Independence Assumption
Suppose an observation model for a particular latent state assigns \(p(x_{i1}=8\mid z_i)=0.10\) and \(p(x_{i2}=1\mid z_i)=0.20\). If it assumes that the two gene counts are conditionally independent given \(z_i\), then the probability of observing both specified counts is:
This product is justified only under the conditional-independence assumption. If the model instead specified a joint probability of \(0.050\) for those two counts at the same latent state, the independent-gene product would not reproduce the model’s joint distribution. The difference, \(0.050\) versus \(0.020\), shows why a factorization across genes cannot be inferred just from the fact that the data are arranged in gene columns.
In practice, the assumption should be evaluated in relation to the model’s purpose and checked against patterns the model is expected to reproduce. This example does not establish that a particular dependence is present in real data; it illustrates what the independence assumption entails mathematically.
Assumptions, Diagnostics, and Alternatives
A generative factorization makes its assumptions visible. Before using it, check what it assumes about the units being modeled, the latent-state distribution, the observation process, and dependence among cells or genes. In the simple product over cells, the cell-level latent states and count vectors are independent across cells conditional on the shared parameters. If shared effects or sample structure matter, this simple factorization may be inadequate.
Useful warning signs include systematic count patterns that the fitted model repeatedly fails to reproduce, strong residual relationships between genes when the observation model assumes conditional independence, or latent assignments that change substantially under modest modeling choices. These are reasons to question the model or its assumptions, not automatic evidence for a single alternative explanation. A probability model is a simplified account; diagnostics assess whether its implications are plausible for the data and intended use.
The main alternative at this level is not one particular distribution, but a different way to organize the analysis. A deterministic summary such as a normalized matrix or a low-dimensional embedding can describe patterns without specifying a joint probability distribution for latent states and counts. A generative model instead states how latent states and observations are related probabilistically, enabling marginal and posterior probabilities to be defined. These approaches answer different questions: a summary can be convenient for visualization or downstream analysis, while a generative model makes explicit assumptions about how data could arise.
The generative formulation also has limitations. Its latent states may not have a unique biological interpretation; its conclusions depend on the prior and observation model; and a simple factorization may fail to represent dependence or variation that matters. The model’s probability statements are conditional on these choices. A good fit does not prove that the proposed latent structure is the true biological mechanism.
A Common Mistake
Wrong reasoning: “The model found a latent state for this cell, so that state is a measured cell identity, and all remaining variation is technical noise.”
Correction: The state is unobserved and inferred conditionally on the model. The remaining variation is whatever the observation model leaves unexplained by that state; it may include technical variation, unmodeled biological variation, and model misspecification. Neither the latent label nor the residual source is established by the factorization alone.
Check Your Understanding
Answer each question using the definitions and calculations above. Solutions follow the questions.
- For a cell with three genes, what does \(x_i\) represent, and what does \(z_i\) represent?
- If a discrete latent state has two possible values, how is \(p_\theta(x_i)\) obtained from \(p_\theta(z_i)\) and \(p_\theta(x_i\mid z_i)\)?
- Using prior probabilities 0.4 and 0.6 for states A and B, and likelihoods 0.05 and 0.02 for an observed count vector, calculate \(p(z_i=A\mid x_i)\).
- What additional assumption is required to write the conditional count probability as a product over genes?
- Why is it incorrect to interpret every discrepancy between observed counts and a latent state as technical noise?
Solutions. (1) \(x_i\) is the observed vector of counts for the cell across the three genes; \(z_i\) is its unobserved modeled state. (2) Sum the joint probabilities over possible states: \(p_\theta(x_i)=\sum_{z_i}p_\theta(z_i)p_\theta(x_i\mid z_i)\). (3) The denominator is \((0.4)(0.05)+(0.6)(0.02)=0.032\). Thus \(p(z_i=A\mid x_i)=(0.4)(0.05)/0.032=0.020/0.032=0.625\). (4) Conditional independence of the gene counts given the latent state and parameters. (5) The unexplained variation can also include biological differences omitted from the latent state or model misspecification; the factorization alone does not identify its source.