Expected Zero Counts Under Poisson And Negative Binomial Models

Why Compare Expected Zero Counts?

A recorded zero is a common feature of single-cell count data, especially for genes with low average counts. But a zero can arise under an ordinary count model even when the model has no special mechanism for producing zeros: a count-generating process may simply yield no captured molecules for that observation. The expected number of zeros therefore depends on the full count distribution, not just its mean.

This tutorial compares Poisson and negative binomial models at the same mean. The comparison isolates one consequence of overdispersion: under the standard negative binomial parameterization, a positive dispersion parameter gives a higher probability of zero than the matched-mean Poisson model. An expected zero count is a model-based benchmark, not by itself a test of model fit or an explanation for why any particular observation is zero.

Zero Probabilities at a Matched Mean

Definition: For a Poisson count \(Y\) with mean \(\mu\), \(Y\sim\operatorname{Poisson}(\mu)\), and \(P(Y=0)=e^{-\mu}\). For a negative binomial count with mean \(\mu\) and dispersion \(\alpha>0\), use the parameterization \(\operatorname{Var}(Y)=\mu+\alpha\mu^2\). Its zero probability is \(P(Y=0)=(1+\alpha\mu)^{-1/\alpha}\). The Poisson model is recovered as \(\alpha\) tends to zero at fixed \(\mu\).

Here, \(\mu\) is the mean count for the observation or group being modeled, and \(\alpha\) controls extra variation relative to a Poisson distribution. The negative binomial variance exceeds the mean when \(\alpha>0\) and \(\mu>0\). A matched-mean comparison uses the same \(\mu\) in both models; it does not match their variances.

The negative binomial zero-probability formula can be justified using the Gamma–Poisson mixture. Let the Poisson rate \(\Lambda\) vary across observations according to a Gamma distribution with shape \(1/\alpha\) and scale \(\alpha\mu\), and let \(Y\mid\Lambda\sim\operatorname{Poisson}(\Lambda)\). This mixture has mean \(\mu\) and variance \(\mu+\alpha\mu^2\). Given the rate, the chance of zero is \(e^{-\Lambda}\), so averaging over rates gives the marginal zero probability.

\begin{aligned} P(Y=0) &=E\!\left[P(Y=0\mid\Lambda)\right]\\ &=E(e^{-\Lambda})\\ &=(1+\alpha\mu)^{-1/\alpha}. \end{aligned}

The last equality is the Laplace transform of a Gamma random variable with shape \(1/\alpha\) and scale \(\alpha\mu\). This mixing interpretation explains why the negative binomial can assign more probability to zero: some observations have relatively low rates, while others have higher rates, even though the overall mean remains \(\mu\).

From a Zero Probability to an Expected Number

Suppose \(n\) observations have specified means \(\mu_1,\ldots,\mu_n\), and let \(I_i\) equal 1 if observation \(i\) is zero and 0 otherwise. The total number of zeros is \(Z=\sum_{i=1}^n I_i\). Since the expectation of a sum is the sum of expectations, the expected total is the sum of the observation-specific zero probabilities.

E(Z)=\sum_{i=1}^{n}E(I_i) =\sum_{i=1}^{n}P(Y_i=0).

This identity does not require the counts to be independent. Independence would matter for calculating the variance of \(Z\), but not its expectation. If all \(n\) observations share one mean and one model, it simplifies to \(E(Z)=nP(Y=0)\). If their means differ, calculate each probability at its own mean and add them; in general, \(n\) times the zero probability at the average mean is not the right calculation.

Worked Example: Matched Means for One Gene

Suppose a hypothetical gene has mean count \(\mu=1.2\) in each of 100 cells. Compare the expected zeros under a Poisson model and a negative binomial model with \(\alpha=0.5\). Assume these marginal models are appropriate for the cells being compared and that the mean is the same in every cell.

Under the Poisson model, the zero probability is \(e^{-1.2}=0.3011942\). Thus the expected number of zeros is \(100(0.3011942)=30.11942\), or about 30.12 cells.

Under the negative binomial model, the variance is \(1.2+0.5(1.2)^2=1.92\), greater than the mean of 1.2, as required for positive overdispersion in this parameterization. The zero probability and expected number of zeros are

P(Y=0)=(1+0.5(1.2))^{-1/0.5} =1.6^{-2} =\frac{1}{2.56} =0.390625, \qquad E(Z)=100(0.390625)=39.0625.

The negative binomial model therefore predicts about \(39.06-30.12=8.94\) more zero-count cells in expectation. This is a comparison of model expectations, not a claim that the particular data set must contain exactly that many additional zeros.

Why Overdispersion Raises the Zero Probability

For \(\mu>0\) and \(\alpha>0\), compare the logarithm of the negative binomial zero probability with the logarithm of the Poisson zero probability. The standard inequality \(\log(1+x)<x\) for \(x>0\) shows that the negative binomial log probability is greater.

\log\!\left((1+\alpha\mu)^{-1/\alpha}\right) =-\frac{\log(1+\alpha\mu)}{\alpha} >-\mu =\log(e^{-\mu}).

Exponentiating preserves this inequality, so \((1+\alpha\mu)^{-1/\alpha}>e^{-\mu}\). Thus, at any fixed positive mean, a negative binomial model with positive dispersion predicts a greater zero probability than its matched-mean Poisson model. If \(\mu=0\), both models assign probability 1 to zero. As \(\alpha\) tends to zero, the negative binomial zero probability approaches the Poisson probability.

The inequality compares these two particular models under their stated parameterizations. It does not say that every overdispersed count distribution must have more zeros than a Poisson distribution, nor that an excess of observed zeros establishes overdispersion as the cause.

Worked Example: Different Means Across Cells

Consider a hypothetical group of 70 cells: 40 have modeled mean count \(0.5\), and 30 have modeled mean count \(2\). Use a Poisson model and, separately, a negative binomial model with the same cell-specific means and common dispersion \(\alpha=0.25\). Assume these means and the selected model apply to the two groups.

For the Poisson model, add the expected zeros within each group:

\begin{aligned} E(Z_{\mathrm{Pois}}) &=40e^{-0.5}+30e^{-2}\\ &=40(0.6065306597)+30(0.1353352832)\\ &=24.26122639+4.060058496\\ &=28.32128489\approx28.32. \end{aligned}

For the negative binomial model, the exponent \(-1/\alpha\) is \(-4\). Apply its zero-probability formula separately at each mean:

\begin{aligned} E(Z_{\mathrm{NB}}) &=40(1+0.25(0.5))^{-4} +30(1+0.25(2))^{-4}\\ &=40(1.125)^{-4}+30(1.5)^{-4}\\ &=40(0.6242950766)+30(0.1975308642)\\ &=24.97180306+5.925925926\\ &=30.89772899\approx30.90. \end{aligned}

The matched-mean negative binomial expectation is about \(30.90-28.32=2.58\) zeros higher. The group with mean 0.5 contributes more expected zeros than the group with mean 2 under both models; averaging the means first would conceal this difference. The calculation assumes a common dispersion for the negative binomial groups, but not a common mean.

Checking a Model-Based Expectation Against Data

Worked Example: An Observed Count Is Not an Expected Count

In a hypothetical set of 60 cells, suppose every cell has modeled mean \(\mu=2\). The observed data contain 23 zeros. Compare this observation with the expected zero counts under a Poisson model and a negative binomial model with \(\alpha=1\). Assume the stated means and marginal models are reasonable for this comparison.

The Poisson expectation is

60e^{-2}=60(0.1353352832)=8.120116992\approx8.12.

For the negative binomial model, the zero probability is \((1+1(2))^{-1}=1/3\), so its expected number of zeros is \(60(1/3)=20\). The observed count of 23 is closer to the negative binomial expectation of 20 than to the Poisson expectation of about 8.12. It is nevertheless only a descriptive comparison: it does not quantify uncertainty, prove either model correct, or identify a mechanism behind the zeros.

Assumptions, Diagnostics, and Alternatives

A zero-count expectation is meaningful only relative to the model and its inputs. Check that the unit of analysis is clear, that the mean refers to the same count and exposure in both models, and that the negative binomial dispersion uses the convention \(\operatorname{Var}(Y)=\mu+\alpha\mu^2\). For a group of cells with different means, use each cell's or subgroup's modeled mean. If \(\mu\) or \(\alpha\) is estimated, the displayed expectation treats those estimates as fixed; uncertainty in the estimates is not represented by the calculation.

A useful diagnostic is to compare the observed number or proportion of zeros with the expectation under a fitted count model, while also checking whether the model's means and dispersion are plausible. A large discrepancy can indicate that the model does not describe the data adequately, but it does not tell which assumption failed. Conversely, close agreement in the total number of zeros does not guarantee that the model describes positive counts, cell-to-cell patterns, or gene-to-gene variation well.

Approach	What it provides	Main limitation
Matched-mean Poisson	A baseline zero expectation when variance equals the mean.	May understate zeros when counts are overdispersed.
Matched-mean negative binomial	A zero expectation that incorporates the specified dispersion.	Depends on the dispersion estimate and the negative binomial model being adequate.
Observed zero count	The number of zeros actually present in the data.	It is descriptive and does not provide a model-based explanation on its own.

The Poisson and negative binomial models are alternatives for this calculation, not proof that one is universally preferable. The Poisson model is a useful baseline; the negative binomial allows variance above the mean through \(\alpha\). Neither expected zero count, alone, establishes why zeros occur. In particular, a discrepancy from a fitted count model is a reason to investigate fit and assumptions, not sufficient evidence for a distinct zero-generating mechanism.

Common Mistakes

Using only the mean to claim the zero probability is fixed. The Poisson and negative binomial models can have the same mean but different zero probabilities because their variances differ.
Multiplying by the number of cells before finding the probability. First compute the zero probability under the chosen model; then multiply by the number of cells when they share that probability.
Substituting an average mean for varying means. Add the cell-specific or subgroup-specific expected zeros; do not generally calculate \(n\) times a probability evaluated at the average mean.
Calling an expected count a guaranteed count. An expectation is a long-run average over repetitions under the model, not a prediction that the realized count must equal it.
Treating excess observed zeros as proof of a special zero mechanism. The comparison may flag lack of fit, but by itself it does not identify its source.

Key takeaway: At a fixed positive mean, the negative binomial model with positive dispersion assigns more probability to zero than the matched-mean Poisson model. For multiple observations, sum the appropriate zero probabilities to obtain the expected total; treat the result as a model-based benchmark, not a diagnosis by itself.

Check Your Understanding

Try each problem, then compare your reasoning with the solutions below.

For a Poisson count with mean 1, calculate the zero probability and the expected number of zeros among 50 observations with that same mean.
For a negative binomial count with mean 1 and dispersion \(\alpha=1\), calculate the zero probability. How does it compare with the matched-mean Poisson probability?
There are 20 observations with mean 0.5 and 20 with mean 2, all following Poisson models. Write and evaluate the expected total number of zeros.
Why does calculating \(n\) times the zero probability at the average mean generally fail when cell means differ?
A data set has more zeros than the matched-mean Poisson expectation. What can this comparison establish, and what can it not establish on its own?

Solutions. (1) \(P(Y=0)=e^{-1}=0.3679\), so the expected total is \(50e^{-1}=18.39\) zeros. (2) \(P(Y=0)=(1+1(1))^{-1}=0.5\). The matched-mean Poisson probability is \(e^{-1}=0.3679\), so the negative binomial probability is higher. (3) \(E(Z)=20e^{-0.5}+20e^{-2}=20(0.6065306597)+20(0.1353352832)=12.13061319+2.706705664=14.83731886\), or about 14.84 zeros. (4) Each mean has its own zero probability, and the total expectation is the sum of those probabilities; evaluating one probability at the average mean generally does not equal that sum. (5) The comparison can show a discrepancy with the specified Poisson expectation and motivate checking model fit. It cannot, by itself, prove which alternative model is appropriate or explain the origin of the zeros.