Poisson And Negative Binomial Comparisons

Why Compare Count Models at the Same Mean?

A claim-count model is used to describe how many claims occur over a stated period or exposure. Two models can have the same expected claim count but differ in how variable the counts are and in the probability of observing no claims. Those differences matter when a rating analysis needs to represent the full range of possible counts, rather than only their average.

The Poisson distribution is a standard starting point for claim counts. The negative binomial distribution provides a way to represent greater variance than the Poisson distribution while retaining the same mean. In this comparison, matched means means that both models use the same expected claim count \(\mu\). The negative binomial shape parameter \(r\) controls how much additional variance it has.

Definition: Let \(N\) be a nonnegative integer-valued claim count with mean \(\mu>0\). Under a Poisson model, \(N\sim\operatorname{Poisson}(\mu)\), so \(P(N=k)=e^{-\mu}\mu^k/k!\), \(E[N]=\mu\), and \(\operatorname{Var}(N)=\mu\). Under the negative binomial convention used here, \(N\) has shape \(r>0\) and success probability \(p=r/(r+\mu)\). Writing \(q=1-p=\mu/(r+\mu)\), its probability mass function is \(P(N=k)=\frac{\Gamma(k+r)}{\Gamma(r)k!}p^r q^k\), for \(k=0,1,2,\ldots\). This parameterization gives \(E[N]=\mu\) and \(\operatorname{Var}(N)=\mu+\mu^2/r\).

For positive integer \(r\), the negative binomial can be interpreted as the number of failures before the \(r\)th success in a sequence of independent Bernoulli trials, each with success probability \(p\). The formula also allows noninteger \(r>0\), which is useful as a statistical model. In this tutorial, the \(p\) in the negative binomial formula is the success probability, not the probability of a claim.

What the Shape Parameter Changes

At a fixed mean \(\mu\), the Poisson variance is \(\mu\). The negative binomial variance is \(\mu+\mu^2/r\), so its variance exceeds the Poisson variance whenever \(0<r<\infty\) and \(\mu>0\). A smaller \(r\) means a larger extra term \(\mu^2/r\); a larger \(r\) brings the variance closer to the Poisson variance. In the limit as \(r\) increases without bound, the negative binomial approaches the Poisson distribution with mean \(\mu\).

This comparison is a variance comparison, not a claim that the negative binomial assigns greater probability to every possible count. Probability mass must sum to one. A negative binomial model with a greater variance can assign more probability to zero and to some high counts while assigning less probability to other counts.

\operatorname{Var}(N_{\mathrm{NB}})-\operatorname{Var}(N_{\mathrm{Pois}}) =\left(\mu+\frac{\mu^2}{r}\right)-\mu =\frac{\mu^2}{r}>0

The probability of zero claims gives a particularly useful comparison. Substituting \(k=0\) into the negative binomial mass function leaves \(p^r\), because \(\Gamma(r)/\Gamma(r)=1\) and \(0!=1\). For a matched-mean Poisson model, the zero probability is \(e^{-\mu}\).

P_{\mathrm{NB}}(N=0) =p^r =\left(\frac{r}{r+\mu}\right)^r =\left(1+\frac{\mu}{r}\right)^{-r}, \qquad P_{\mathrm{Pois}}(N=0)=e^{-\mu}

Why the Negative Binomial Has More Zero Counts

For finite \(r>0\) and \(\mu>0\), the negative binomial zero probability is greater than the Poisson zero probability. The inequality follows from the standard logarithm inequality \(\ln(1+x)<x\) for \(x>0\). Taking logarithms of the negative binomial zero probability gives the comparison directly.

\ln P_{\mathrm{NB}}(N=0) =-r\ln\left(1+\frac{\mu}{r}\right) > -r\left(\frac{\mu}{r}\right) =-\mu =\ln P_{\mathrm{Pois}}(N=0)

Exponentiation preserves the inequality, so \(P_{\mathrm{NB}}(N=0)>P_{\mathrm{Pois}}(N=0)\). As \(r\) grows, the difference decreases and the negative binomial zero probability approaches \(e^{-\mu}\). This result is about the probability of exactly zero claims; it does not by itself establish which model has a larger probability of exactly one claim, exactly two claims, or any other individual count.

Worked Example: Comparing Zero-Claim Probabilities at a Mean of Two

A hypothetical portfolio segment has an expected claim count of 2 over the period being modeled. Compare a Poisson model with a negative binomial model having shape \(r=4\). Calculate both variances and both zero-claim probabilities. Round probabilities to four decimals.

Method and parameter setup. Both models have \(\mu=2\). For the negative binomial, \(p=r/(r+\mu)=4/(4+2)=2/3\), and \(q=1-p=1/3\).

Variance comparison. The Poisson variance is \(\mu=2\). The negative binomial variance is \(2+2^2/4=2+4/4=3\). Thus the negative binomial variance is 3, one greater than the Poisson variance of 2.

Zero probabilities. The negative binomial probability of zero claims is \(p^r=(2/3)^4=16/81=0.197530\ldots\), which rounds to 0.1975. The Poisson probability is \(e^{-2}=0.135335\ldots\), which rounds to 0.1353.

Check. The negative binomial zero probability exceeds the Poisson zero probability, as the inequality predicts, and the negative binomial variance exceeds the Poisson variance. The method avoids comparing models with different means: both calculations use the same expected count of 2.

Worked Example: Comparing Several Count Probabilities

For a hypothetical book of small equipment policies, suppose the expected count over one policy period is 3. Compare Poisson and negative binomial models with \(r=2\) for counts of zero, one, and two claims. Also compare the variances. Round probabilities to four decimals.

Method and parameter setup. The common mean is \(\mu=3\), and the negative binomial shape is \(r=2\). Therefore \(p=2/(2+3)=0.4\) and \(q=0.6\).

Variances. The Poisson variance is 3. The negative binomial variance is \(3+3^2/2=3+9/2=7.5\).

Zero claims. The negative binomial probability is \(p^r=0.4^2=0.16\). The Poisson probability is \(e^{-3}=0.049787\ldots\), or 0.0498 to four decimals.

Exactly one claim. The negative binomial probability is \(\frac{\Gamma(3)}{\Gamma(2)1!}p^2q=2(0.16)(0.6)=0.192\). The Poisson probability is \(e^{-3}3^1/1!=3e^{-3}=0.149361\ldots\), or 0.1494.

Exactly two claims. The negative binomial probability is \(\frac{\Gamma(4)}{\Gamma(2)2!}p^2q^2=3(0.16)(0.36)=0.1728\). The Poisson probability is \(e^{-3}3^2/2!=4.5e^{-3}=0.224042\ldots\), or 0.2240.

Check. The negative binomial assigns more probability to zero and one claim here, but less probability to exactly two claims. The probabilities for each model are calculated from its own mass function; the larger negative binomial variance does not mean that every individual count probability is larger.

A Short Derivation of the Negative Binomial Moments

The probability generating function provides a compact justification for the stated negative binomial mean and variance. For a nonnegative integer-valued count, its probability generating function is \(G(z)=E[z^N]\). Using the negative binomial mass function and the binomial series gives \(G(z)=[p/(1-qz)]^r\), for \(z\) in a region where the series converges. Differentiating this function gives the first two factorial moments.

G'(z)=rp^r q(1-qz)^{-r-1}, \qquad G''(z)=r(r+1)p^r q^2(1-qz)^{-r-2}

At \(z=1\), \(1-q=p\). Hence \(G'(1)=rq/p=E[N]\). Also \(G''(1)=r(r+1)q^2/p^2=E[N(N-1)]\). Since \(E[N^2]=E[N(N-1)]+E[N]\), subtracting the squared mean yields the variance.

E[N]=\frac{rq}{p}=\mu, \qquad \operatorname{Var}(N) =G''(1)+G'(1)-[G'(1)]^2 =\frac{rq}{p^2} =\mu+\frac{\mu^2}{r}

The last equality uses \(q=\mu/(r+\mu)\) and \(p=r/(r+\mu)\). This derivation also shows why the shape parameter matters: at a fixed mean, decreasing \(r\) increases the variance contribution \(\mu^2/r\).

Worked Example: Reading the Effect of a Larger Shape

A hypothetical specialty-insurance segment has a mean of 1.5 claims per policy period. An analyst compares negative binomial shapes \(r=3\) and \(r=6\), each matched to a Poisson model with mean 1.5. Calculate the two negative binomial variances and zero probabilities, then compare them with the Poisson values. Round probabilities to four decimals.

Method and parameter setup. For each negative binomial model, use the same \(\mu=1.5\), but its own specified shape. The Poisson variance is 1.5 and its zero probability is \(e^{-1.5}=0.223130\ldots\), or 0.2231.

Shape \(r=3\). The variance is \(1.5+(1.5)^2/3=1.5+2.25/3=2.25\). Here \(p=3/(3+1.5)=2/3\), so the zero probability is \((2/3)^3=8/27=0.296296\ldots\), or 0.2963.

Shape \(r=6\). The variance is \(1.5+2.25/6=1.5+0.375=1.875\). Here \(p=6/(6+1.5)=6/7.5=0.8\), so the zero probability is \(0.8^6=0.262144\), or 0.2621.

Check. Both negative binomial zero probabilities exceed the Poisson value of 0.2231. Increasing \(r\) from 3 to 6 reduces both the variance and the zero probability toward the Poisson values. The method avoids treating \(r\) as a second mean: the mean remains 1.5 in all three models.

Common Exam Traps and Practical Interpretation

A frequent mistake is to reverse the negative binomial probability convention. In the parameterization here, \(p=r/(r+\mu)\) is the success probability, while \(q=\mu/(r+\mu)\) is the complementary probability. Zero counts have probability \(p^r\), not \(q^r\). For example, with \(\mu=2\) and \(r=4\), using \(q=1/3\) as if it were \(p\) would produce the incorrect zero probability \((1/3)^4=1/81\). The correct success probability is \(p=2/3\), giving \(P(N=0)=(2/3)^4=16/81\).

Another trap is to infer individual count probabilities from a variance comparison. Greater variance means the distribution is more spread out in aggregate; it does not mean that every probability away from the mean increases. When a question asks for \(P(N=k)\), calculate it from the appropriate probability mass function. When it asks for zero claims, use the zero-count formula directly.

For ratemaking, the mean describes the expected count, while the variance and selected probabilities describe other aspects of the count distribution. A negative binomial model may be considered when a Poisson model’s variance assumption is too restrictive. Matching the means isolates the effect of the distributional choice: it makes clear that differences in zero probability or dispersion are not caused by changing the expected count. Model selection still depends on the data and the purpose of the analysis.

Key takeaway: For matched mean \(\mu\), a Poisson count has variance \(\mu\) and zero probability \(e^{-\mu}\). A negative binomial count with finite shape \(r\) has variance \(\mu+\mu^2/r\) and zero probability \((1+\mu/r)^{-r}\), which is greater than \(e^{-\mu}\). Its shape controls the size of the difference.

Check Your Understanding

For each problem, compare models at the stated common mean. Round probabilities to four decimals unless an exact fraction is requested.

A hypothetical segment has mean count \(\mu=4\) and negative binomial shape \(r=4\). Find the negative binomial variance and zero probability, then compare the zero probability with the Poisson value. (Answer: variance \(8\); negative binomial \(P(N=0)=1/16=0.0625\), greater than Poisson \(e^{-4}=0.0183\).)
For matched mean \(\mu=2\), a negative binomial model has shape \(r=1\). Calculate its variance and zero probability. (Answer: variance \(6\); \(P(N=0)=1/3=0.3333\), compared with Poisson \(e^{-2}=0.1353\).)
A count has mean \(\mu=5\), and the negative binomial shape is \(r=10\). Calculate the negative binomial variance and its zero probability, and give the matched Poisson zero probability. (Answer: variance \(7.5\); negative binomial \(P(N=0)=(2/3)^{10}=0.0173\), compared with Poisson \(e^{-5}=0.0067\).)
For \(\mu=1\), compare the variance and zero probabilities for a Poisson model and a negative binomial model with \(r=2\). (Answer: Poisson variance \(1\) and zero probability \(e^{-1}=0.3679\); negative binomial variance \(1.5\) and zero probability \(4/9=0.4444\).)
For matched mean \(\mu=2\) and negative binomial shape \(r=2\), calculate the negative binomial probability of exactly one claim and compare it with the Poisson probability of exactly one claim. (Answer: negative binomial \(P(N=1)=0.25\); Poisson \(P(N=1)=2e^{-2}=0.2707\), so the negative binomial probability is smaller.)

What You'll Learn