Poisson–Gamma Mixing Derives the Negative-Binomial Law

Why Mix a Poisson Rate?

A Poisson model describes counts when events occur at a fixed rate over a specified interval. In applications, however, units can differ in their underlying event rates: patients may have different unobserved risks, sites may have different levels of activity, or regions may experience different baseline incidence. If the rate varies across units, even when each unit’s count is Poisson conditional on its own rate, the count distribution across units need not be Poisson. A gamma distribution for the latent rate produces the negative-binomial distribution exactly.

The distinction is between conditional variation, the randomness of a count at a given rate, and between-unit variation, the differences in rates across units. Integrating over the latter gives a marginal count law that accounts for both sources.

Poisson–Gamma Mixture Definition

Here \(\beta\) is the rate parameter of the gamma distribution, not the probability parameter sometimes used to write a negative-binomial mass function. The gamma function is \(\Gamma(r)=\int_0^\infty t^{r-1}e^{-t}\,dt\) for \(r>0\). For a positive integer \(y\), \(y!=1\cdot2\cdots y\), with \(0!=1\).

Deriving the Negative-Binomial Mass Function

Proof. Since \(\Lambda\) has a density, the marginal probability is found by integrating the conditional probability against that density. The integrand is nonnegative, and direct substitution gives

For \(a>0\), substituting \(u=a\lambda\) in the gamma integral yields \(\int_0^\infty \lambda^{s-1}e^{-a\lambda}\,d\lambda=\Gamma(s)/a^s\) for \(s>0\). Apply this identity with \(s=y+r\) and \(a=\beta+1\):

This is a probability mass function because it was obtained by integrating a conditional probability mass function over a probability density. With \(p=\beta/(\beta+1)\), we have \(1-p=1/(\beta+1)\), giving the equivalent expression. \(\square\)

This is the negative-binomial convention in which \(Y\) counts failures before \(r\) successes when \(r\) is an integer and \(p\) is the success probability. The gamma-mixture derivation also applies for any real \(r>0\), using the gamma function in place of factorials. The mixture provides a direct probabilistic explanation for why this count law arises; it is not merely an algebraic alternative to a Poisson model.

Moments and the Role of Rate Variation

The gamma density has mean \(r/\beta\) and variance \(r/\beta^2\). These follow from the gamma integral and the recurrence \(\Gamma(s+1)=s\Gamma(s)\), obtained by integration by parts. In particular, integrating \(\lambda\) and \(\lambda^2\) against the gamma density gives \(E[\Lambda]=r/\beta\) and \(E[\Lambda^2]=r(r+1)/\beta^2\), hence the stated variance.

Proof. Conditional on \(\Lambda\), the Poisson mean and variance are both \(\Lambda\). The law of total expectation therefore gives

The law of total variance states that \(\operatorname{Var}(Y)=E\{\operatorname{Var}(Y\mid\Lambda)\}+\operatorname{Var}\{E[Y\mid\Lambda]\}\). Substituting the conditional Poisson moments gives

The first term is the average conditional Poisson variation. The second is the additional variation due to different latent rates. It is positive whenever the gamma rate is nondegenerate, so the marginal variance exceeds the marginal mean. \(\square\)

Shared Latent Rates and Association

Rate heterogeneity can also create dependence among counts. Consider two counts observed over fixed exposures \(a>0\) and \(b>0\). Suppose they are conditionally independent given the same latent rate \(\Lambda\), with \(Y_1\mid\Lambda\sim\operatorname{Poisson}(a\Lambda)\) and \(Y_2\mid\Lambda\sim\operatorname{Poisson}(b\Lambda)\). Conditional independence means that, once the shared rate is known, observing one count provides no additional information about the other.

Proof. The law of total covariance gives

Conditional independence makes the first term zero. The two conditional expectations are \(a\Lambda\) and \(b\Lambda\), so the second term is \(\operatorname{Cov}(a\Lambda,b\Lambda)=ab\operatorname{Var}(\Lambda)\). For the gamma rate, this is \(ab r/\beta^2\). \(\square\)

This result gives a useful interpretation of latent heterogeneity: units with a high shared rate tend to have high counts in both observations, while units with a low rate tend to have low counts in both. The counts can therefore be positively associated marginally, even though they are independent conditional on the rate.

Common Mistake: Treating the Marginal Count as Poisson

• Incorrect reasoning: “The count is Poisson conditional on its rate, so the observed counts must have a Poisson distribution.” This ignores that different units have different values of the unobserved rate.
• Correction: A conditional Poisson model does not imply a marginal Poisson model when its rate is random. Integrating over a gamma-distributed rate gives the negative-binomial mass function, with variance \(r/\beta+r/\beta^2\), which exceeds its mean \(r/\beta\).
• Another distinction: Counts sharing a latent rate are conditionally independent, not necessarily marginally independent. Integrating out their shared rate induces positive covariance.

The mixture is useful when variation among units is substantively plausible and a Poisson model’s equal-mean-and-variance structure is too restrictive. It connects an overdispersed count distribution to a specific mechanism: Poisson sampling within units, together with gamma-distributed rate heterogeneity between units. That interpretation is an assumption about the data-generating process, not something established merely by observing a variance larger than a mean.