Sparse Priors for Interpretable Gene Programs

Why Put Sparsity in the Prior?

A topic model represents each cell as a mixture of programs and each program as a distribution over genes. In many applications, a useful summary would have a small number of active programs per cell and a short, interpretable set of genes per program. Sparsity priors can encourage that structure, but they do not establish that a program is biologically real: they express a modeling preference whose influence must be evaluated against the counts.

The introductory material on cell mixtures and gene programs describes how these two simplexes are interpreted. Here, the focus is how prior choice changes their estimates. The examples hold the number of topics fixed; they do not address how that number should be chosen.

In the basic count model, each observed gene count is associated with a latent topic assignment. Conditional on the assignments, let \(n_{ik}\) be the number of cell \(i\)'s counts assigned to topic \(k\), and \(m_{kg}\) the number assigned to gene \(g\) in topic \(k\). The assignments are drawn using the cell's topic proportions, and a topic's assigned counts use its gene probabilities. The Dirichlet prior is conjugate to these categorical probabilities. This gives a simple exact conditional update, even though the assignments themselves are unknown.

What the Concentration Parameters Do

For a symmetric prior on a \(J\)-component simplex, every component has prior mean \(1/J\), regardless of the common concentration \(c\). But changing \(c\) changes how much prior probability lies near the center versus near the boundaries. When \(c<1\), the Dirichlet density favors boundary regions and draws tend to have a few large components and many small ones. When \(c=1\), it is uniform over the simplex. When \(c>1\), it favors values closer to the center. These statements describe the prior distribution, not a guarantee that the posterior will be sparse or diffuse.

The total concentration \(Jc\) matters as well as the per-component value \(c\). For fixed \(J\), lowering \(c\) reduces total concentration. Across models with different numbers of components, keeping \(c\) fixed changes the total concentration; comparisons that aim to hold total concentration constant instead set \(c\) equal to the chosen total divided by \(J\). An asymmetric Dirichlet prior can also express unequal prior expectations: its component means are proportional to its component parameters.

A Dirichlet draw has strictly positive components with probability one when its parameters are positive. Thus “sparse Dirichlet” means near-zero components are encouraged, not that the prior assigns positive probability to exact zeros. Moreover, a posterior mean can look less sparse than typical posterior draws: averaging across uncertainty about which component is active spreads mass among components.

The same conjugate calculation applies to topic gene profiles. With a small \(\eta\), genes receiving no assigned counts get little posterior mean probability; with a larger \(\eta\), the prior contributes more probability to those genes. This can make the inferred gene programs broader, but the effect depends on the amount and pattern of assigned data.

Regularized Factor Priors: Shrinkage Instead of a Simplex Prior

A related approach uses a factorization of the expected counts and places shrinkage priors directly on nonnegative factor loadings. For example, with a fixed size factor \(s_i\), one may model \(Y_{ig}\) as Poisson with mean \(s_i\sum_k\theta_{ik}\beta_{kg}\). Here \(\theta_i\) is constrained to the topic simplex, while \(\beta_{kg}\geq 0\) is an unnormalized loading for gene \(g\) in topic \(k\). A common regularization is an exponential prior on each loading, \(\beta_{kg}\sim\mathrm{Exponential}(\lambda)\), with density \(\lambda e^{-\lambda\beta_{kg}}\) for \(\beta_{kg}\geq0\). In a maximum a posteriori fit, its log density contributes a penalty proportional to \(\lambda\beta_{kg}\).

This is not the same parameterization as LDA's simplex-valued gene profile. Exponential shrinkage acts on the size of an unnormalized loading, whereas a Dirichlet prior redistributes probability across a normalized profile. Scale constraints and size-factor conventions therefore matter when comparing fitted values. In particular, applying an ordinary \(L_1\) penalty to a nonnegative simplex vector has no effect on its composition: its \(L_1\) norm is always one. A regularizer must act on parameters that can vary in scale, or use a different penalty that genuinely varies over the simplex.

Derivation and Practical Comparison

The conditional Dirichlet update follows directly from multiplying the prior density by the assignment likelihood. For a simplex vector \(q=(q_1,\ldots,q_J)\), the Dirichlet density is proportional to \(\prod_j q_j^{a_j-1}\). If the assignments contribute counts \(c_j\), their likelihood is proportional to \(\prod_j q_j^{c_j}\). Multiplication adds the exponents:

The resulting kernel is the density of a Dirichlet distribution with parameters \(a_j+c_j\), which justifies the conditional update used above. This proof assumes categorical assignments and a Dirichlet prior; it does not say that the same update is exact after integrating out uncertain assignments or in a different count factorization.

For LDA, the concentration parameters \(\alpha\) and \(\eta\) have a direct interpretation on normalized mixtures and gene probabilities. In a regularized factor model, a penalty such as the exponential prior instead controls loading magnitude and may create boundary MAP estimates. These approaches differ in their likelihoods, parameter scales, and inferential summaries, so a numerical loading from one is not automatically comparable with a gene probability from the other.

Diagnostics, Limitations, and a Common Pitfall

Prior sensitivity is a central diagnostic. If modest changes to \(\alpha\), \(\eta\), or the shrinkage strength substantially change which topics dominate cells or which genes define a program, the data may not strongly distinguish those alternatives under the fitted model. Also inspect whether the fit systematically misses high-count genes, cell-specific patterns, or observed zero frequencies. A prior that produces visually concise programs can still yield poor predictive count behavior.

These priors do not solve all identifiability problems. Topics can be permuted without changing the model, and correlated programs may share genes or exchange counts. Sparse priors can favor simpler representations, but the resulting decomposition can remain dependent on initialization and modeling assumptions. Likewise, zero inflation, batch effects, or an inadequate count likelihood are not automatically repaired by stronger sparsity. The prior should not be used to make a weakly supported program appear definitive.

Common mistake: “The prior concentration is below one, so every cell has exact zeros for inactive topics.” The correction is that a Dirichlet with positive parameters has support only on simplex interiors: it favors draws near boundaries but does not assign mass to exact zeros. A reported zero may instead be rounding, a thresholding rule, or a boundary estimate from another procedure. State which one was used.