Decision Rules for Choosing an Abundance Test

Start With the Question, Not the Test

An abundance analysis begins with counts of cells assigned to categories, such as cell types, across samples. A method that is appropriate for asking whether one specified cell type changes need not be the best choice for asking whether the whole composition changes or whether the ratio of two types shifts. The decision is therefore not simply “which model handles counts?” It is which model represents the target comparison and the variation among samples?

Let \(Y_{ij}\) be the number of cells of type \(j\) in sample \(i\), and let \(N_i=\sum_{j=1}^K Y_{ij}\) be the total number of cells assigned to the \(K\) types in that sample. The observed proportion is \(Y_{ij}/N_i\). These are relative abundances: they describe each type as a share of the counted total, not an absolute cell number.

These methods rely on counts being defined consistently across samples and on the sample-level observations being appropriate for the model’s independence and replication assumptions. The model also needs a design that can distinguish the condition comparison from other included effects. A sophisticated count distribution cannot correct a comparison that is confounded or lacks independent replication.

What Each Method Targets

For one cell type, a binomial model starts with \(Y_{ij}\mid N_i\sim\text{Binomial}(N_i,\pi_{ij})\), where \(\pi_{ij}\) is the type’s relative abundance for sample \(i\). A regression model can relate \(\pi_{ij}\) to condition and other covariates. Its key sampling assumption is that, conditional on the model and the sample total, the count has binomial variation. A component-wise analysis is most direct when the target is a particular type’s proportion or condition effect. Separate tests across many types require a plan for multiple testing.

The binomial model can be too restrictive if samples vary more than its assumptions allow. A beta-binomial model allows the underlying probability to vary between samples, representing extra-binomial variation. With mean abundance \(\pi_{ij}\) and concentration \(\phi\), its marginal variance for the sample proportion is

A finite \(\phi>0\) permits more variation than a binomial model, but the inflation depends on the sample total. A beta-binomial model remains component-wise: it does not, by itself, represent all dependence among the different cell-type counts in a sample.

A DM model is designed for the full count vector, with \(Y_i=(Y_{i1},\ldots,Y_{iK})\), total \(N_i\), mean composition \(\boldsymbol{\pi}_i\), and concentration \(\phi\). It accommodates overdispersion relative to a multinomial model, with a shared concentration controlling the overall degree of extra variation. Its dependence structure is constrained by the composition: the categories are parts of a fixed total. This is useful for an omnibus question about whether the composition differs, but it is not a flexible model for every possible pattern of category-specific dependence.

A log-ratio analysis targets relative change between components. For two types, the target can be \(\log(\pi_{ij}/\pi_{ik})\); more complex contrasts can compare groups of types using a balance. The result is interpreted as a change in a ratio, not as an absolute increase in either type. Log-ratio methods require care with zeros because the logarithm of zero is undefined. A zero-handling rule must be justified and assessed for sensitivity rather than treated as a harmless numerical detail.

A Variance Check That Helps Distinguish Binomial From Overdispersed Counts

The beta-binomial variance formula shows why model choice cannot be made from the number of cells alone. Comparing it with the binomial proportion variance gives

For \(N_i=1\), the difference is zero. For \(N_i>1\), \(0<\pi_{ij}<1\), and finite \(\phi>0\), the difference is strictly positive. Thus the beta-binomial variance is at least the binomial variance, and is strictly greater under those latter conditions. The formula also shows that extra variation concerns variation among sample-level proportions; a large number of cells within one sample does not substitute for independent samples.

Decision Rules in Practice

Use a component-wise binomial model when the target is a particular type and binomial variation is adequate. Use a beta-binomial component model when that target remains the focus but sample-level variation exceeds the binomial assumption. Use a DM model when the joint composition is the target and its shared overdispersion structure is plausible. Use a log-ratio analysis when a relative contrast is the scientific target and zeros can be handled transparently.

These choices are not a ranking from “simple” to “best.” A DM model may be unnecessarily restrictive for a single prespecified ratio; a collection of component-wise tests may miss the fact that the scientific question concerns the composition jointly. A log-ratio analysis does not avoid compositional interpretation—it makes the relative contrast explicit. Method choice should be set by the hypothesis and design before examining which analysis gives the smallest p-value.

Diagnostics, Failure Modes, and Limitations

Diagnostics should ask whether the fitted model captures the variation and structure that matter for the stated target. For component-wise and beta-binomial fits, inspect sample-level proportions and residuals against fitted values and covariates. Look for persistent overdispersion, influential samples, and groups whose counts are almost always zero. For a DM fit, check convergence, fitted versus observed compositions, and whether a single shared concentration adequately describes variation across categories. For a log-ratio analysis, inspect the distribution of the chosen log-ratio and test whether conclusions depend strongly on the zero-handling rule.

Sparsity is a warning, not an automatic command to use one particular model. A rare type may produce many zeros even when a model is appropriate, but very few nonzero observations can leave its condition effect weakly identified. Structural zeros—types genuinely absent from a condition—pose a different problem from sampling zeros. No method can recover a well-supported contrast from information that is not present. Likewise, a model that converges is not necessarily well specified.

A common mistake is to choose a DM model whenever there are multiple cell types, or a beta-binomial model whenever there are many zeros. The correction is to define the target first, then ask whether the candidate’s assumptions and dependence structure fit the data. Another mistake is to treat the number of cells as the number of independent replicates. Cells contribute to estimating each sample’s composition; inference about condition differences also depends on independent sample-level replication.

All four approaches concern observed relative abundance unless additional information supports an absolute-abundance interpretation. Capture bias, annotation uncertainty, and missing cell types can also change the observed counts. A model choice does not remove those measurement limitations. If several candidate approaches are plausible, report the prespecified primary analysis and use alternatives as sensitivity checks, explaining what each alternative changes in the estimand or assumptions.