Deriving the Gamma-Poisson Negative Binomial

Why a Gamma-Poisson Mixture Produces Overdispersion

A Poisson count has equal conditional mean and variance. Yet counts can vary more than that model allows, even when the Poisson assumption is reasonable after conditioning on an unobserved rate. One way to represent this extra variation is to let each observation have its own latent rate multiplier, with multipliers drawn from a gamma distribution. Integrating out those multipliers gives a negative binomial distribution.

The construction makes the source of overdispersion explicit: counts are Poisson conditional on their latent rates, while variation in those rates adds variance to the marginal count distribution. This tutorial derives that result and focuses on the Type 2 variance form \( \operatorname{Var}(Y_i)=\mu_i+\alpha\mu_i^2 \).

Definition: For observation \(i\), let \(Y_i\) be a count and let \(\mu_i>0\) be its marginal mean. Suppose \(Y_i\mid U_i\sim\operatorname{Poisson}(\mu_iU_i)\), where \(U_i\sim\operatorname{Gamma}(k,k)\) uses the shape-rate convention. Thus \(E(U_i)=1\) and \(\operatorname{Var}(U_i)=1/k\). If the multipliers are independent across observations, integrating them out gives a Gamma-Poisson negative binomial model with dispersion \( \alpha=1/k \).

The mean \( \mu_i \) may depend on predictors and exposure. For example, a log-link model can specify \( \mu_i=e_i\exp(\mathbf{x}_i^\mathsf{T}\boldsymbol{\beta}) \), where \(e_i\) is exposure and \( \mathbf{x}_i^\mathsf{T}\boldsymbol{\beta} \) is the linear predictor. In the mixture, \(U_i\) multiplies this mean for observation \(i\). The standard construction assumes the \(U_i\)'s are independent, gamma distributed with a common shape-rate parameterization, and independent of the predictors and exposures. Conditional on \(U_i\), the count has a Poisson distribution with mean \( \mu_iU_i \).

Integrating Out the Latent Rate

With shape \(k\) and rate \(k\), the density of \(U_i\) is \( f(u)=k^ku^{k-1}e^{-ku}/\Gamma(k) \) for \(u>0\). Here \( \Gamma(\cdot) \) is the gamma function. The conditional Poisson probability of observing \(y\), for a nonnegative integer \(y\), is \( e^{-\mu_i u}(\mu_i u)^y/y! \). The marginal probability is the integral of their product:

\begin{aligned} P(Y_i=y) &=\int_0^\infty \frac{e^{-\mu_i u}(\mu_i u)^y}{y!} \frac{k^k u^{k-1}e^{-ku}}{\Gamma(k)}\,du\\ &=\frac{k^k\mu_i^y}{y!\,\Gamma(k)} \int_0^\infty u^{y+k-1}e^{-(k+\mu_i)u}\,du\\ &=\frac{\Gamma(y+k)}{\Gamma(k)y!} \left(\frac{k}{k+\mu_i}\right)^k \left(\frac{\mu_i}{k+\mu_i}\right)^y. \end{aligned}

For the last equality, use the gamma-integral identity \( \int_0^\infty u^{a-1}e^{-bu}\,du=\Gamma(a)/b^a \), valid when \(a>0\) and \(b>0\). Here \(a=y+k>0\) and \(b=k+\mu_i>0\). This probability mass function is the negative binomial distribution obtained by the Gamma-Poisson mixture. Its mean-variance relationship follows directly from the conditional model, as shown next.

Why the Variance Exceeds the Mean

Conditional on \(U_i\), a Poisson variable has mean and variance \( \mu_iU_i \). The law of total expectation gives \(E(Y_i)=E\{E(Y_i\mid U_i)\}\), and the law of total variance separates the expected conditional variance from the variance of the conditional mean:

\begin{aligned} E(Y_i)&=E(\mu_iU_i)=\mu_i,\\ \operatorname{Var}(Y_i) &=E\{\operatorname{Var}(Y_i\mid U_i)\} +\operatorname{Var}\{E(Y_i\mid U_i)\}\\ &=E(\mu_iU_i)+\operatorname{Var}(\mu_iU_i)\\ &=\mu_i+\frac{\mu_i^2}{k} =\mu_i+\alpha\mu_i^2,\qquad \alpha=\frac{1}{k}. \end{aligned}

The first variance term, \( \mu_i \), is ordinary Poisson sampling variation. The second, \( \alpha\mu_i^2 \), is the contribution from variation in the latent rate. Thus \( \alpha \) is not an extra variance added in the same units as the count; it scales a squared-mean term. The latent multiplier has coefficient of variation \(1/\sqrt{k}=\sqrt{\alpha}\). Larger \( \alpha \) means more heterogeneity in the multipliers and more marginal overdispersion. As \( \alpha\) approaches zero, \(k\) grows, the multiplier concentrates at 1, and the mixture approaches a Poisson distribution.

Key result: Under independent, mean-one gamma mixing of observation-specific Poisson rates, the marginal count is negative binomial with mean \( \mu_i \) and variance \( \mu_i+\alpha\mu_i^2 \), where \( \alpha \) is the variance of the latent multiplier.

Worked Consequences of the Mixture

Worked Example: Computing Probabilities From the Mixture

Suppose a hypothetical count has marginal mean \( \mu=2 \) and latent multiplier \(U\sim\operatorname{Gamma}(2,2)\). Then \(k=2\) and \( \alpha=1/2 \). The assumptions are a Poisson count conditional on \(U\), a gamma multiplier with mean 1, and no additional shared multiplier affecting other observations.

The marginal probability of zero counts is \(P(Y=0)=\left(\frac{k}{k+\mu}\right)^k=(\frac{2}{4})^2=0.25\). For one count, substitute \(y=1\) into the derived mass function:

P(Y=1)= \frac{\Gamma(3)}{\Gamma(2)1!} \left(\frac{2}{4}\right)^2 \left(\frac{2}{4}\right) =2(0.25)(0.5)=0.25.

For two counts, \(P(Y=2)=\frac{\Gamma(4)}{\Gamma(2)2!}(0.5)^2(0.5)^2 =3(0.0625)=0.1875\). These are probabilities under the marginal distribution, not probabilities conditional on a particular value of the latent rate.

Worked Example: Quantifying the Extra-Poisson Variance

Consider a hypothetical observation with marginal mean \( \mu=4 \) and \( \alpha=0.25 \). Equivalently, \(k=1/\alpha=4\), so \(U\sim\operatorname{Gamma}(4,4)\). The mean-one multiplier assumption is satisfied, and conditional on that multiplier the count is Poisson with mean \(4U\).

The marginal variance is \( \operatorname{Var}(Y)=\mu+\alpha\mu^2=4+0.25(16)=8 \). The corresponding standard deviation is \( \sqrt{8}\approx2.828 \) counts. A Poisson distribution with the same mean would have variance 4 and standard deviation 2 counts. In this mixture, the extra variance is \(8-4=4\) squared counts, specifically the rate-heterogeneity component \( \alpha\mu^2 \). The multiplier's coefficient of variation is \( \sqrt{\alpha}=0.5 \), or 50 percent.

This calculation describes the model-implied variance. It does not establish that the data require this mixture; that requires assessing the model against observed counts and plausible alternatives.

Worked Example: Incorporating Exposure

Suppose a hypothetical count has exposure \(e=2\) units and a modeled rate of 1.5 counts per exposure unit, giving \( \mu=e(1.5)=3 \). Let \( \alpha=0.2 \), so \(k=5\). Assume the gamma multiplier is observation-specific, has mean 1 and variance \(0.2\), and the count is Poisson conditional on that multiplier.

The marginal probability of no counts during the two exposure units is \(P(Y=0)=(\frac{5}{5+3})^5=(\frac{5}{8})^5=0.09537\), to five significant figures. The variance is \(3+0.2(3^2)=4.8\) squared counts. Exposure raises the modeled mean from 1.5 to 3 before the mixing; it does not change the multiplier's mean or dispersion parameter in this specification.

The model's assumptions should still be checked: the chosen mean structure must adequately represent how exposure and predictors relate to the count, and the gamma variation must plausibly act independently at the observation level. A zero probability alone is not enough to establish that these assumptions hold.

Mixing Structure, Diagnostics, and Alternatives

The level at which the latent rate varies matters. Independent \(U_i\)'s give the marginal negative binomial distribution for each observation and do not, by themselves, induce dependence between different observations. If instead two conditionally independent counts share a gamma multiplier \(U\), with conditional means \( \mu_1U \) and \( \mu_2U \), their marginal covariance is \( \operatorname{Cov}(Y_1,Y_2)=\mu_1\mu_2\operatorname{Var}(U)=\mu_1\mu_2/k \). A shared multiplier therefore induces positive dependence. The marginal distribution of each count may look negative binomial in either construction, but the joint model is different. A mixing level should reflect the data structure rather than be chosen only to obtain a larger variance.

Useful checks include comparing observed and model-predicted count frequencies across the range of counts, checking whether the model captures the frequency of zeros and the upper tail, and examining residual patterns against fitted means, predictors, exposure, and grouping. Persistent discrepancies can indicate a misspecified mean structure, a poor dispersion assumption, dependence not represented by the model, or a small number of influential observations. Excess zeros may be one visible discrepancy, but do not by themselves prove that a separate structural-zero process is present.

The Poisson model is the limiting comparison: it has variance equal to the mean and is recovered as \( \alpha\to0 \). The Gamma-Poisson model instead makes the extra variation arise from gamma-distributed rate heterogeneity. Other overdispersion models can imply different relationships between the mean and variance or use a different latent distribution. Model choice should be based on the scientific structure and on fit, not merely on which model produces a more flexible variance.

The mixture has limitations. It imposes a particular mean-variance relationship, \( \mu+\alpha\mu^2 \), and a particular marginal count distribution; it does not accommodate every pattern of overdispersion. A single dispersion parameter may be inadequate if heterogeneity changes in ways not represented by the model. Nor does the mixture automatically model dependence between observations: that depends on whether latent rates are independent or shared. In a GLMM, this distinction is especially important because other random effects may already induce dependence.

A Practical Derivation Checklist

Specify the conditional count model.
Write \(Y_i\mid U_i\sim\operatorname{Poisson}(\mu_iU_i)\) and identify what determines \(\mu_i\).

Specify the latent-rate distribution.
Use a mean-one gamma multiplier with shape and rate \(k\), so its variance is \(1/k\).

Integrate or use conditional moments.
Integrate the conditional Poisson mass function to obtain the marginal probabilities; use total expectation and variance to identify the mean and dispersion.

Check the mixing level and fit.
Decide whether rates vary independently by observation or are shared, then inspect whether the implied marginal distribution and dependence are plausible.

Common Pitfall

A frequent mistake is to say that the negative binomial variance is larger because “the Poisson variance is \( \mu+\alpha\mu^2 \).” Conditional on the latent multiplier, the Poisson variance is \( \mu U \), not \( \mu+\alpha\mu^2 \). The latter is the marginal variance after the multiplier is integrated out. The correction is to apply total variance: average the conditional Poisson variance, then add the variance of the conditional mean. Another mistake is to infer that a negative binomial marginal distribution establishes independent observations; shared latent rates or other random effects can still create dependence.

Key takeaway: Gamma mixing preserves the Poisson rate on average but adds variability between latent rates. With \( \alpha=1/k \), this yields a negative binomial marginal variance of \( \mu+\alpha\mu^2 \). The interpretation is valid only with the specified gamma mixing and the appropriate assumptions about which observations share latent rates.

Check Your Understanding

Use the Gamma-Poisson construction above. Solutions follow the questions so that each calculation can be checked directly.

If \(k=10\), what are the dispersion parameter and the variance of the mean-one multiplier?
For \( \mu=5 \) and \( \alpha=0.1 \), calculate the marginal variance and compare it with the Poisson variance at the same mean.
For \( \mu=2 \) and \(k=3\), calculate the probability of zero counts from the mixture.
Two counts with means \( \mu_1=2 \) and \( \mu_2=3 \) share one multiplier with \(k=4\). Find their marginal covariance, assuming conditional independence given that multiplier.

Solutions. (1) \( \alpha=1/k=0.1 \), and \( \operatorname{Var}(U)=1/k=0.1 \). (2) \( \operatorname{Var}(Y)=5+0.1(25)=7.5 \) squared counts; the Poisson variance is 5 squared counts, so the mixture adds 2.5 squared counts. (3) \(P(Y=0)=(\frac{k}{k+\mu})^k=(\frac{3}{5})^3=0.216\). (4) \( \operatorname{Cov}(Y_1,Y_2)=\mu_1\mu_2/k=(2)(3)/4=1.5 \) squared counts. The positive covariance arises because the shared multiplier raises or lowers both conditional means together.