Deriving The Negative Binomial As A Poisson Mixture

Why Mix Poisson Rates?

A Poisson model describes claim counts when a risk has a fixed underlying rate over the period being modeled. In a portfolio, however, risks may have different underlying rates even after they have been grouped into a segment. If the rate varies across risks, a model that treats every risk as having exactly the same rate can understate the variation in observed counts. A gamma mixture provides a tractable way to represent that risk-to-risk variation.

The key distinction is between a risk's underlying rate and the random count generated at that rate. In the model below, each risk has a rate $\Lambda$, which varies across risks according to a gamma distribution. Conditional on that rate, the claim count is Poisson. When the rate is not observed and is averaged over, the marginal count follows a negative binomial distribution.

Definition: For a risk observed over exposure $t>0$, let $\Lambda$ be its latent claim rate per unit exposure. Assume $\Lambda\sim\operatorname{Gamma}(\alpha,\beta)$ with shape $\alpha>0$ and rate $\beta>0$, having density $$ f_\Lambda(\lambda)=\frac{\beta^\alpha}{\Gamma(\alpha)}\lambda^{\alpha-1}e^{-\beta\lambda},\qquad \lambda>0. $$ Conditional on $\Lambda=\lambda$, assume $N\mid\Lambda=\lambda\sim\operatorname{Poisson}(t\lambda)$. Here $\Gamma(\alpha)$ is the gamma function, and the second gamma parameter $\beta$ is a rate, not a scale.

Under this parameterization, $E[\Lambda]=\alpha/\beta$ and $\operatorname{Var}(\Lambda)=\alpha/\beta^2$. The exposure $t$ scales the conditional Poisson mean: doubling exposure doubles the expected count for a given risk rate. It does not change the risk's underlying rate $\Lambda$.

Deriving the Negative Binomial Probability

To obtain the marginal distribution of $N$, average its conditional Poisson probability over all possible values of $\Lambda$. For an integer $n\geq 0$, the conditional probability is $e^{-t\lambda}(t\lambda)^n/n!$. Multiplying by the gamma density and integrating gives

\begin{aligned} P(N=n) &=\int_0^\infty P(N=n\mid\Lambda=\lambda)f_\Lambda(\lambda)\,d\lambda\\ &=\int_0^\infty \frac{e^{-t\lambda}(t\lambda)^n}{n!} \frac{\beta^\alpha}{\Gamma(\alpha)}\lambda^{\alpha-1}e^{-\beta\lambda}\,d\lambda\\ &=\frac{\beta^\alpha t^n}{n!\,\Gamma(\alpha)} \int_0^\infty \lambda^{n+\alpha-1}e^{-(\beta+t)\lambda}\,d\lambda. \end{aligned}

The gamma integral identity is $\int_0^\infty x^{a-1}e^{-cx}\,dx=\Gamma(a)/c^a$, for $a>0$ and $c>0$. Applying it with $a=n+\alpha$ and $c=\beta+t$ yields the marginal probability:

P(N=n)= \frac{\Gamma(n+\alpha)}{\Gamma(\alpha)n!} \left(\frac{\beta}{\beta+t}\right)^\alpha \left(\frac{t}{\beta+t}\right)^n,\qquad n=0,1,2,\ldots.

This is the negative binomial probability mass function in a parameterization that makes the mixture connection explicit. Define $p=\beta/(\beta+t)$, so $1-p=t/(\beta+t)$. Then the probability of $n$ counts is $\Gamma(n+\alpha)/(\Gamma(\alpha)n!)\,p^\alpha(1-p)^n$. The shape $\alpha$ need not be an integer; the gamma-function expression applies for every $\alpha>0$.

Result: If $\Lambda\sim\operatorname{Gamma}(\alpha,\beta)$ in shape-rate form and $N\mid\Lambda\sim\operatorname{Poisson}(t\Lambda)$, then $N$ has a negative binomial distribution with probability mass function shown above. Its mean is $\alpha t/\beta$, and its variance is $\alpha t/\beta+\alpha t^2/\beta^2$.

Why the Mixture Has Extra Variability

The mean and variance follow from conditioning on the latent rate. Given $\Lambda$, a Poisson count has conditional mean and variance $t\Lambda$. The law of total expectation therefore gives

E[N]=E[E[N\mid\Lambda]]=E[t\Lambda]=\frac{\alpha t}{\beta}.

For the variance, the law of total variance separates the average conditional variance from the variation in conditional means:

\begin{aligned} \operatorname{Var}(N) &=E[\operatorname{Var}(N\mid\Lambda)] +\operatorname{Var}(E[N\mid\Lambda])\\ &=E[t\Lambda]+\operatorname{Var}(t\Lambda)\\ &=\frac{\alpha t}{\beta}+\frac{\alpha t^2}{\beta^2}. \end{aligned}

The first term is the usual Poisson variation, averaged across risks. The second term is the additional variation caused by differences in their underlying rates. In terms of the marginal mean $\mu_N=\alpha t/\beta$, the variance can also be written

\operatorname{Var}(N)=\mu_N+\frac{\mu_N^2}{\alpha}.

For finite $\alpha$, this variance exceeds the mean. As $\alpha$ increases while the mean is held fixed, the gamma rates become less variable relative to their mean and the extra variance shrinks. In the limiting case where rate heterogeneity vanishes, the mixed model approaches an ordinary Poisson model. The negative binomial mixture is not a claim that an individual risk's count is conditionally non-Poisson; the extra marginal variability comes from averaging over different risk rates.

Worked Example: Deriving Probabilities for a One-Period Mixture

Method: Use the gamma-Poisson mixture with shape-rate parameters. In a hypothetical short-term equipment policy segment, suppose each risk's latent claim rate per year satisfies $\Lambda\sim\operatorname{Gamma}(2,4)$, and each risk is observed for $t=1$ year. Conditional on $\Lambda=\lambda$, the annual count is Poisson with mean $\lambda$. Find $P(N=0)$, $P(N=1)$, $P(N=2)$, and the mean and variance.

The mixture parameters are $\alpha=2$, $\beta=4$, and $t=1$. Thus $p=\beta/(\beta+t)=4/5$, while $1-p=t/(\beta+t)=1/5$. Apply the marginal mass function:

P(N=n)=\frac{\Gamma(n+2)}{\Gamma(2)n!} \left(\frac45\right)^2\left(\frac15\right)^n.

For zero claims, the coefficient is 1:

P(N=0)=\left(\frac45\right)^2=\frac{16}{25}=0.6400.

For one claim, the coefficient is $\Gamma(3)/(\Gamma(2)1!)=2$:

P(N=1)=2\left(\frac45\right)^2\left(\frac15\right) =\frac{32}{125}=0.2560.

For two claims, the coefficient is $\Gamma(4)/(\Gamma(2)2!)=6/2=3$:

P(N=2)=3\left(\frac45\right)^2\left(\frac15\right)^2 =\frac{48}{625}=0.0768.

The mean is $\alpha t/\beta=2/4=0.5$. The variance is $0.5+2(1)^2/4^2=0.5+0.125=0.625$. As a check on the probabilities, the chance of at least three claims is $1-0.6400-0.2560-0.0768=0.0272$, which is nonnegative. Common mistake avoided: using the gamma scale $1/\beta$ as though it were the rate parameter would give the wrong mixture probabilities.

Exposure Changes the Mixture Probability

For a fixed gamma distribution of annual rates, changing the observation exposure changes the negative binomial probability parameter as well as the mean. In particular, $p=\beta/(\beta+t)$, so $p$ is not generally $\beta/(\beta+1)$ unless exposure is exactly one unit. This distinction matters when a risk is observed for a partial or extended period.

Worked Example: A Longer Observation Period

Method: Substitute the actual exposure into the mixed probability mass function. In a hypothetical policy study, let $\Lambda\sim\operatorname{Gamma}(3,6)$ be the annual claim rate and observe each risk for $t=2$ years. Find $P(N=0)$, $P(N=1)$, $P(N=2)$, $P(N\geq3)$, the mean, and the variance.

Here $\alpha=3$, $\beta=6$, and $t=2$. Therefore $p=6/(6+2)=3/4$ and $1-p=2/8=1/4$. The probabilities are

\begin{aligned} P(N=0)&=\left(\frac34\right)^3=\frac{27}{64}=0.421875,\\ P(N=1)&=3\left(\frac34\right)^3\left(\frac14\right) =\frac{81}{256}=0.31640625,\\ P(N=2)&=6\left(\frac34\right)^3\left(\frac14\right)^2 =\frac{81}{512}=0.158203125. \end{aligned}

The probability of three or more claims is the remaining probability:

P(N\geq3)=1-\left(\frac{216}{512}+\frac{162}{512}+\frac{81}{512}\right) =\frac{53}{512}=0.103515625.

The mean is $\alpha t/\beta=3(2)/6=1$. The variance is $1+3(2)^2/6^2=1+1/3=4/3$. The four-decimal probabilities are $0.4219$, $0.3164$, $0.1582$, and $0.1035$; they sum to $1.0000$ at that precision. Common mistake avoided: using $t=1$ in the probability parameter would incorrectly treat a two-year observation as a one-year observation.

Comparing Heterogeneous and Homogeneous Rates

A comparison with an ordinary Poisson model helps isolate what the mixture changes. If the two models have the same marginal mean, their expected counts agree, but the mixed model has a larger variance when the gamma rate has positive variance. This extra variability can shift probability away from the center of the count distribution and increase the chance of larger counts. It does not mean every individual probability must be larger under the mixture.

Worked Example: Same Mean, Different Probability of Multiple Claims

Method: Compare the mixed-model tail probability directly with the Poisson probability at the same mean. Suppose, hypothetically, that annual risk rates follow $\Lambda\sim\operatorname{Gamma}(4,8)$, with one year of exposure. Find the mixed-model mean, variance, and probability of at least two claims, then compare the latter probability with that from a Poisson model with the same mean.

For the mixture, $\alpha=4$, $\beta=8$, and $t=1$, so $p=8/9$ and $1-p=1/9$. The mean and variance are

E[N]=\frac{4}{8}=0.5,\qquad \operatorname{Var}(N)=0.5+\frac{4}{8^2}=0.5625.

The probabilities of zero and one claim are

P(N=0)=\left(\frac89\right)^4=\frac{4096}{6561}, \qquad P(N=1)=4\left(\frac89\right)^4\left(\frac19\right) =\frac{16384}{59049}.

Writing the zero-claim probability with denominator $59049$ gives $36864/59049$. Therefore

P(N\geq2)=1-\frac{36864+16384}{59049} =\frac{5801}{59049}=0.0982

to four decimal places. A Poisson model with the same mean $0.5$ gives

P(N\geq2)=1-P(N=0)-P(N=1) =1-e^{-0.5}(1+0.5)=0.0902.

Thus the mixed-model probability of at least two claims is higher in this example, even though both models have mean $0.5$. The mixed variance is also greater: $0.5625$ rather than $0.5$. Common mistake avoided: comparing models with different means would confound the effect of heterogeneity with a difference in expected claim frequency.

Interpretation and Modeling Conditions

The mixture interpretation relies on a specific hierarchy: rates vary across risks according to a gamma distribution, and conditional counts follow a Poisson distribution with mean equal to rate times exposure. The resulting negative binomial distribution describes the count after the latent rate has been averaged out. It does not establish that a gamma distribution is appropriate for every portfolio; that is a modeling choice requiring support from the data and the purpose of the analysis.

A useful implication is that repeated periods for the same risk can be related through the shared latent rate. If counts $N_1$ and $N_2$ in disjoint periods of exposures $t_1$ and $t_2$ are conditionally independent given the same $\Lambda$, their unconditional covariance is

\operatorname{Cov}(N_1,N_2) =\operatorname{Cov}(t_1\Lambda,t_2\Lambda) =t_1t_2\operatorname{Var}(\Lambda) =\frac{\alpha t_1t_2}{\beta^2}.

The covariance is positive because a risk with a higher latent rate tends to have more claims in both periods. This interpretation differs from treating all claim counts as independent Poisson observations with one common fixed rate.

When applying the derivation, keep four points explicit:

State the gamma parameterization. Here $\beta$ is the rate, so the mean latent rate is $\alpha/\beta$.
Include exposure in the conditional mean. The Poisson mean is $t\Lambda$, not just $\Lambda$ unless $t=1$.
Distinguish conditional from marginal variation. Conditional variance is $t\Lambda$; marginal variance includes the additional variation in $t\Lambda$.
Use the mixture probability consistently. In this form, $p=\beta/(\beta+t)$ and the exponent on $1-p$ is the observed count $n$.

The derivation explains both the negative binomial form and its actuarial interpretation: an unobserved spread of risk-specific claim rates creates a count distribution that is more variable than a Poisson distribution with the same mean. This relationship is aligned with the CAS Exam 5 (Basic Ratemaking and Estimated Claim Liabilities) syllabus.

Check Your Understanding

For each problem, use a gamma shape-rate model and the conditional Poisson assumption stated in the question.

A hypothetical portfolio has $\Lambda\sim\operatorname{Gamma}(2,3)$ and one unit of exposure. What are the mixed count's mean and variance? (Answer: mean $=2/3$; variance $=8/9$.)
For $\Lambda\sim\operatorname{Gamma}(3,6)$ and exposure $t=1.5$, calculate $P(N=0)$, the mean, and the variance. (Answer: $P(N=0)=0.5120$; mean $=0.7500$; variance $=0.9375$.)
For $\Lambda\sim\operatorname{Gamma}(2,4)$ and exposure $t=2$, find $P(N=0)$, the mean, and the variance. (Answer: $P(N=0)=4/9=0.4444$; mean $=1$; variance $=1.5$.)
A one-unit exposure mixture has mean $0.8$ and variance $1.12$. Assuming a gamma-Poisson model, determine $\alpha$ and $\beta$. (Answer: $\alpha=2$, $\beta=2.5$.)
One risk has $\Lambda\sim\operatorname{Gamma}(2,5)$. Conditional on $\Lambda$, counts in two disjoint periods of exposures $1$ and $2$ are independent Poisson variables. Find their unconditional covariance. (Answer: $4/25=0.16$.)

What You'll Learn