Negative Binomial Estimation and Overdispersion

Why Use a Negative Binomial Frequency Model?

A Poisson frequency model has the defining property that its variance equals its mean. In claim-count data, the observed variance may be larger: some policies have few claims while others have several. A negative binomial model provides a parametric way to represent this overdispersion, meaning variance greater than the mean. It is useful to know both how to estimate its parameters and what the model can—and cannot—explain.

The parameterization matters. Here, the count is the number of failures before a specified number of successes. The success probability is $p$, and the positive dispersion parameter is $r$. The parameter $r$ need not be an integer. Assume throughout that observed counts are independent and have the same distribution and exposure, unless stated otherwise. The likelihood formulas below do not apply unchanged to unequal exposures, dependent observations, or counts selected by truncation.

Definition: For $r>0$ and $0<p<1$, a negative binomial count $N$ has probability function $$ P(N=y)=\frac{\Gamma(y+r)}{\Gamma(r)\,y!}p^r(1-p)^y,\qquad y=0,1,2,\ldots $$ Its mean and variance are $$ E[N]=\frac{r(1-p)}{p},\qquad \operatorname{Var}(N)=\frac{r(1-p)}{p^2} =E[N]+\frac{E[N]^2}{r}. $$

The last expression separates the variance into a Poisson-like mean component and an additional positive component. Small $r$ means greater extra variability for a fixed mean; as $r$ increases without bound while the mean is held fixed, the negative binomial approaches a Poisson distribution.

Method of Moments

For a sample $y_1,\ldots,y_n$, write $\bar y$ for the sample mean and use the empirical second central moment with divisor $n$: $m_2=n^{-1}\sum_{i=1}^n(y_i-\bar y)^2$. The method of moments equates these two sample quantities to the model mean and variance. If $m_2>\bar y>0$, solving the equations gives estimates $\widehat r$ and $\widehat p$. If $m_2\leq\bar y$, there is no positive finite $r$ that matches both moments. This is a limitation of the moment fit, not proof that every other fitting method must fail.

\widehat r_{\mathrm{MM}}=\frac{\bar y^2}{m_2-\bar y}, \qquad \widehat p_{\mathrm{MM}}=\frac{\widehat r_{\mathrm{MM}}}{\widehat r_{\mathrm{MM}}+\bar y}.

The divisor convention is important: the method equates theoretical moments to empirical moments calculated with divisor $n$, not the unbiased sample variance with divisor $n-1$. Using the latter without adjusting it changes the estimate.

Worked Example: Moment Estimates from Claim Counts

A hypothetical group of six equal-exposure policies has claim counts $0,0,0,1,3,5$. Estimate the negative binomial parameters by moments, then check the fitted mean and variance.

Method. Calculate the empirical mean and central moment using divisor $n=6$, then solve the moment equations. This avoids treating the unbiased sample variance as the moment-matching quantity.

The total count is $0+0+0+1+3+5=9$, so $\bar y=9/6=1.5$. The mean of the squared counts is $(0+0+0+1+9+25)/6=35/6$. Therefore, $m_2=35/6-(1.5)^2=35/6-9/4=43/12$, or approximately $3.5833$.

Because $m_2-\bar y=43/12-18/12=25/12>0$, a finite positive moment estimate exists. Thus $\widehat r_{\mathrm{MM}}=(1.5)^2/(25/12)=27/25=1.08$. The corresponding probability estimate is $\widehat p_{\mathrm{MM}}=1.08/(1.08+1.5)=54/129=18/43\approx0.4186$.

Check. The fitted mean is $1.08(1-18/43)/(18/43)=1.5$. Its variance is $1.5+(1.5)^2/1.08=1.5+2.0833\ldots=3.5833\ldots$, equal to $43/12$ apart from rounding. The fitted variance exceeds the mean, as required for a finite negative binomial dispersion parameter.

Likelihood Estimation

For independent observations with the stated common distribution, the likelihood is the product of their probability functions. For positive total count, maximizing it with respect to $p$ at a fixed $r$ gives a simple expression in terms of $\bar y$. Substituting that expression into the likelihood leaves a one-dimensional profile likelihood for $r$, which can generally be maximized numerically.

Likelihood result: If $\sum_i y_i>0$, the likelihood estimate of $p$ for fixed $r$ is $$ \widehat p(r)=\frac{r}{r+\bar y}. $$ The profile score equation for a finite interior estimate of $r$ is $$ \sum_{i=1}^n\sum_{j=0}^{y_i-1}\frac{1}{r+j} =n\log\!\left(1+\frac{\bar y}{r}\right), $$ where the inner sum is zero when $y_i=0$. A solution must be checked against the profile likelihood and any relevant boundary limit; a score root alone does not establish a maximum.

To see why the fixed-$r$ result holds, the terms in the log likelihood that depend on $p$ are $nr\log p+(\sum_i y_i)\log(1-p)$. Differentiating and setting the derivative to zero gives $nr/p-(\sum_i y_i)/(1-p)=0$. Since $\sum_i y_i=n\bar y$, solving yields $p=r/(r+\bar y)$. The second derivative is negative for $0<p<1$, so this critical point maximizes the likelihood in $p$ for fixed $r$.

If all observations are zero, $\bar y=0$, and that interior calculation does not apply: it would give $p=1$, outside the parameter space. In this case the likelihood is $p^{nr}$, which increases toward 1 as $p$ approaches 1. For $0<p<1$, its supremum is not attained, so there is no maximum likelihood estimate in the stated parameter space.

Worked Example: Profiling the Likelihood for the Dispersion

Four hypothetical equal-exposure policies have counts $0,0,2,4$. Find the maximum likelihood estimates, using numerical solution for the dispersion parameter.

Method. Profile out $p$, then solve the score equation in $r$ and check that the score changes from positive to negative at the root. The common mistake is to assume that the method-of-moments estimate and the likelihood estimate of $r$ must coincide.

The total is $6$, so $n=4$ and $\bar y=6/4=1.5$. For these observations, the left side of the profile score equation is $2/r+2/(r+1)+1/(r+2)+1/(r+3)$. The right side is $4\log(1+1.5/r)$. At $r=0.807$, the two sides are approximately $4.20405$ and $4.20152$, respectively, so the score is still positive. Numerical solution gives $\widehat r\approx0.811$.

Substitute this value into the profiled estimate: $\widehat p=0.811/(0.811+1.5)=0.811/2.311\approx0.3509$. The fitted mean is $0.811(1-0.3509)/0.3509\approx1.50$, consistent with the profile formula. The fitted variance is $1.50+(1.50)^2/0.811\approx4.274$, or $4.27$ to two decimal places.

At the root, the profile score changes from positive to negative, indicating a local maximum; checking the profile likelihood confirms the maximizing solution for this sample. The fitted variance is much larger than the mean, which is consistent with the observed concentration of two zero counts alongside counts of two and four. The estimate is approximate because the score equation is solved numerically.

Interpreting Overdispersion

For any finite $r>0$, the negative binomial variance exceeds its mean. Consequently, it can match a sample whose empirical variance is greater than its mean, with a smaller fitted $r$ corresponding to greater excess variability. It cannot match an empirical variance below the mean by a finite positive $r$ using the method of moments. Equality is the Poisson limiting case, not a finite-$r$ moment solution.

Overdispersion describes a feature of the counts; it does not identify its cause. Unobserved differences between risks, dependence among counts, or an unsuitable exposure definition may contribute to extra variation. A negative binomial fit does not establish which explanation is appropriate. Assessing the fit requires more than noting that the model can have variance above the mean; the later material on frequency-fit diagnostics addresses comparison of fitted and observed count patterns.

Worked Example: Interpreting an Overdispersed Summary

A hypothetical equal-exposure portfolio has a sample mean claim count of $2$ and an empirical variance of $6$, calculated with divisor $n$. Use moments to estimate the negative binomial parameters and find the fitted probability of at least four claims.

Method. Match the two empirical moments, then use the negative binomial tail for the special case $r=1$. The trap is to read $r$ itself as the mean; the mean depends on both $r$ and $p$.

Here, $m_2-\bar y=6-2=4$, so $\widehat r_{\mathrm{MM}}=2^2/4=1$. Then $\widehat p_{\mathrm{MM}}=1/(1+2)=1/3$. The fitted mean is $1(1-1/3)/(1/3)=2$, and the fitted variance is $2+2^2/1=6$. Thus the fit matches both stated moments.

When $r=1$, the probability function simplifies to $P(N=y)=p(1-p)^y$. Summing its geometric tail gives $P(N\geq4)=(1-p)^4$. Hence the fitted probability is $(2/3)^4=16/81\approx0.1975$, to four decimal places. This is a probability under the fitted model, not a guarantee that future observed counts will follow the fitted frequency exactly.

Common Mistakes and Model Checks

Switching parameterizations. Some sources define a negative binomial as the number of successes before a fixed number of failures. Check the probability function before using a mean or variance formula; the formulas here use failures before $r$ successes.
Using the wrong variance convention. Moment matching here uses divisor $n$. Replacing it with the unbiased variance changes the method-of-moments estimate.
Ignoring the all-zero boundary. The fixed-$r$ formula for $\widehat p$ assumes a positive sample total. An all-zero sample has a likelihood supremum at the excluded boundary $p=1$, not an interior MLE.
Interpreting a moment failure as a fitted negative $r$. If $m_2\leq\bar y$, the moment equation gives no finite positive dispersion estimate. Do not report a negative value as a valid negative binomial parameter.
Equating overdispersion with a proven cause. Variance above the mean motivates considering a negative binomial model, but does not by itself show that the model describes the full count distribution or explain why variability is high.

Key takeaway: In this parameterization, the negative binomial mean is $r(1-p)/p$, and its variance is the mean plus the squared mean divided by $r$. Moments provide a direct fit only when the empirical variance exceeds the mean; likelihood estimation profiles out $p$ and generally requires numerical work for $r$. These estimation and interpretation skills are aligned with the SOA Exam FAM (Fundamentals of Actuarial Mathematics) syllabus.

Check Your Understanding

For each problem, assume independent, equal-exposure counts unless the question states otherwise.

A hypothetical sample has empirical mean $1$ and empirical variance $2$, both calculated with divisor $n$. Find the negative binomial method-of-moments estimates of $r$ and $p$. (Answer: $\widehat r=1,\ \widehat p=0.5000$)
A count sample has empirical mean $3$ and variance $6$, calculated with divisor $n$. Find the method-of-moments estimates and verify the fitted variance. (Answer: $\widehat r=3,\ \widehat p=0.5000$; fitted variance $=6$)
Every observed count in a sample is zero. Does the likelihood have an MLE with $0<p<1$ for a fixed $r>0$? (Answer: No; the likelihood supremum occurs as $p$ approaches $1$, and is not attained in the stated parameter space.)
A hypothetical portfolio has empirical mean $2$ and empirical variance $1.5$, calculated with divisor $n$. Can a finite positive $r$ match both moments? (Answer: No; the variance is below the mean, so there is no finite positive method-of-moments estimate.)
Suppose a fitted negative binomial model has $r=2$ and $p=0.4$. Find its mean, variance, and probability of at least two claims. (Answer: Mean $=3$, variance $=7.5$, and $P(N\geq2)=0.6480$)

What You'll Learn