Deriving The Negative Binomial From Gamma-Poisson Mixing

Why Mix Poisson Rates?

For a particular cell and gene, a Poisson model treats the expected count as fixed: conditional on rate \(\lambda\), the count varies through sampling alone. But cells may differ in their underlying rates, even when they are drawn from the same population or share measured covariates. If those rates vary across cells, a single fixed-rate Poisson distribution can be too restrictive. One way to represent that variation is to let the rate itself be random, then average the Poisson probabilities over its distribution.

This construction is called gamma-Poisson mixing. It produces a negative binomial count distribution. The derivation matters because it shows what the model means: the negative binomial is not just a different formula for counts, but a marginal distribution arising when Poisson rates vary according to a gamma distribution.

The parameter \(b\) is the gamma rate, not its scale: the gamma density has exponential factor \(e^{-b\lambda}\). This distinction is important when substituting values. The mixing assumption describes a population distribution of latent rates; it is not, by itself, a claim that a particular cell’s rate changes randomly from one molecule to the next.

Deriving the Marginal Probability

For a specified count \(y\), the law of total probability gives the marginal probability as an integral over all possible rates. Multiplying the conditional Poisson probability by the gamma density gives an integrand whose power of \(\lambda\) and exponential factor can be combined.

To evaluate the remaining integral, use the gamma-integral identity \(\int_0^\infty \lambda^{c-1}e^{-d\lambda}\,d\lambda=\Gamma(c)/d^c\), for \(c>0\) and \(d>0\). Here \(c=y+a\) and \(d=b+1\), both positive under the model assumptions. Substitution yields

This is a negative binomial probability mass function in a shape-and-probability form: its shape is \(a\), and the probability factor attached to the shape is \(b/(b+1)\). These labels are stated here only to identify the result of the derivation; the same distribution can be written using other parameter conventions. The integral itself establishes the probability law without requiring a choice among those conventions.

The resulting probabilities sum to one. In fact, each conditional Poisson distribution sums to one for every \(\lambda>0\), and the gamma density also integrates to one. Therefore, summing the mixture probabilities over the nonnegative integer counts gives \(\sum_y P(Y=y)=\int_0^\infty \sum_y P(Y=y\mid\Lambda=\lambda)f_\Lambda(\lambda)\,d\lambda=1\). The interchange is valid because all terms are nonnegative.

Interpreting the Mixture

The conditional model describes sampling variation at a given rate. The gamma distribution adds variation among rates. Consequently, the marginal count distribution reflects both sources of variation. Using the law of total variance, which separates conditional variation from variation in conditional means, gives \(\operatorname{Var}(Y)=E(\Lambda)+\operatorname{Var}(\Lambda)\). For the stated gamma shape-rate convention, \(E(\Lambda)=a/b\) and \(\operatorname{Var}(\Lambda)=a/b^2\). Thus, in this mixture, \(\operatorname{Var}(Y)=a/b+a/b^2\), whereas \(E(Y)=a/b\). The additional term comes from heterogeneity in the latent rates. This illustrates how mixing can produce greater count variability than a Poisson model with one fixed rate.

That interpretation depends on treating rates as varying across the units represented by the mixture. In single-cell work, the units might be cells for a given gene, after accounting for the chosen exposure or covariates. The model does not establish why their rates vary, nor does a good fit prove that the true biological mechanism is gamma-distributed.

Assumptions, Diagnostics, and Limits

The gamma-Poisson derivation is exact under its assumptions. In particular, the conditional distribution must be Poisson with rate proportional to the modeled exposure, and the rates across the units being mixed must follow the specified gamma distribution. The rate is positive, and the shape and rate parameters satisfy \(a>0\) and \(b>0\). If counts are dependent across units in ways not represented by their rates, or if the rate distribution is poorly described by a gamma density, the marginal negative binomial model need not describe the data well.

A useful first diagnostic is to compare observed count variability with what a fixed-rate Poisson model would predict, as in the earlier material on diagnosing the Poisson mean-variance assumption. Evidence of excess variability motivates considering rate heterogeneity; it does not identify the gamma distribution as its cause. Model checks should also examine whether fitted probabilities describe the observed count distribution across its range, and whether fit varies systematically with cell-level features or exposure. A mismatch can indicate an inadequate rate distribution, omitted structure, dependence, or other features the simple mixture does not capture.

The main alternative in this comparison is a fixed-rate Poisson model. It is simpler and appropriate when a common conditional rate is a defensible approximation. Gamma-Poisson mixing is useful when variation in rates across units is part of the model, but it adds assumptions and parameters. Neither model automatically accounts for every source of variability. In particular, a negative binomial fit alone cannot determine whether the variation is biological, technical, or due to unmodeled covariates.