Overdispersion and Negative Binomial Standards

Why Overdispersion Changes the Standard

A limited-fluctuation standard asks how much exposure is needed for observed claim counts to stay sufficiently close to their expected value with a specified probability. Under a Poisson model, the mean and variance of the count are equal. A negative binomial model allows the variance to exceed the mean, representing extra variability across otherwise comparable exposure units. That extra variability generally requires more exposure to achieve the same precision and confidence.

The key is to use the variance of the count model, not just its mean. This tutorial derives the negative binomial variance and the corresponding full-credibility requirement for a relative-error criterion. The formulas use a normal approximation to the aggregate count, so they are standards for planning exposure rather than exact discrete probability calculations.

Definition: A negative binomial count $N$ with mean $\lambda>0$ and size parameter $k>0$ has probability mass function $$ P(N=m)=\frac{\Gamma(m+k)}{\Gamma(k)m!} \left(\frac{k}{k+\lambda}\right)^k \left(\frac{\lambda}{k+\lambda}\right)^m,\qquad m=0,1,2,\ldots. $$ Under this parameterization, $E[N]=\lambda$ and $\operatorname{Var}(N)=\lambda+\lambda^2/k$. Smaller $k$ means greater overdispersion; as $k$ tends to infinity, the variance approaches the Poisson variance $\lambda$.

Suppose there are $n$ independent, comparable exposure units, each with a negative binomial count having mean $\lambda$ and size $k$. Let $N_n$ be the total count across those units. The expected total is $n\lambda$, while the total variance is $n(\lambda+\lambda^2/k)$. The independence assumption matters: it is what permits the individual variances to add without covariance terms.

Deriving the Negative Binomial Full Standard

For a two-sided relative-error requirement, the observed total is required to lie within a fraction $r$ of its expected value. If the desired confidence level is $p$, let $z$ be the standard normal quantile satisfying $P(-z\le Z\le z)=p$, where $Z\sim N(0,1)$. For example, $z=1.96$ is commonly used for a 95% two-sided standard.

P\left(\left|N_n-n\lambda\right|\le r n\lambda\right)\ge p.

Using the normal approximation, a centered interval of half-width $r n\lambda$ contains approximately a proportion $p$ of the aggregate-count distribution when that half-width is at least $z$ standard deviations. Since the standard deviation is $\sqrt{n(\lambda+\lambda^2/k)}$, the requirement becomes

r n\lambda\ge z\sqrt{n\left(\lambda+\frac{\lambda^2}{k}\right)}.

Both sides are nonnegative. Squaring and dividing by $n r^2\lambda^2$, which is positive, gives the minimum exposure requirement before rounding:

n\ge \frac{z^2}{r^2\lambda}\left(1+\frac{\lambda}{k}\right).

The expression $1+\lambda/k$ is the overdispersion multiplier relative to the Poisson requirement. The Poisson standard has the same mean $\lambda$, but variance $\lambda$, giving $n_{\text{Pois}}\ge z^2/(r^2\lambda)$. Thus, for the same confidence and relative tolerance, the negative binomial requirement is larger by the factor $1+\lambda/k$. When exposure is a count of indivisible units, round the calculated minimum upward to the next whole unit.

Result: For independent negative binomial counts per exposure unit with mean $\lambda$ and size $k$, a normal-approximation two-sided relative-error standard requires $$ n_{\text{NB}}=\left\lceil \frac{z^2}{r^2\lambda}\left(1+\frac{\lambda}{k}\right) \right\rceil. $$ This formula does not apply unchanged if the exposure-unit counts are dependent, have different parameters, or if the required standard is an exact discrete probability calculation rather than a normal approximation.

A useful justification for the variance formula comes from the Poisson-gamma mixture representation. Let the Poisson rate $\Theta$ vary across exposure units, with $N\mid\Theta\sim\operatorname{Poisson}(\Theta)$, $E[\Theta]=\lambda$, and $\operatorname{Var}(\Theta)=\lambda^2/k$. The law of total variance gives

\operatorname{Var}(N) =E[\operatorname{Var}(N\mid\Theta)] +\operatorname{Var}(E[N\mid\Theta]) =E[\Theta]+\operatorname{Var}(\Theta) =\lambda+\frac{\lambda^2}{k}.

The first term is the ordinary Poisson variation conditional on the rate. The second term is additional variation because the underlying rate itself varies. This explains why using only $\lambda$ in a negative binomial standard understates the required exposure.

Worked Requirements

Worked Example: Comparing Negative Binomial and Poisson Requirements

A hypothetical portfolio has independent negative binomial claim counts per exposure unit with mean $\lambda=2$ and size $k=4$. Find the full-credibility requirements for a 95% two-sided relative tolerance of $r=0.10$, using $z=1.96$. Also calculate the Poisson requirement with the same mean.

Method. Use the negative binomial formula and compare it with the Poisson formula. The confidence level is two-sided, so the supplied quantile is $z=1.96$.

First calculate $z^2=1.96^2=3.8416$ and $r^2=0.10^2=0.01$. The Poisson requirement before rounding is

\frac{3.8416}{0.01(2)} =\frac{3.8416}{0.02} =192.08.

For the negative binomial count, the overdispersion multiplier is \[ 1+\frac{\lambda}{k}=1+\frac{2}{4}=1.5. \] Therefore, its requirement before rounding is

$$ 192.08(1.5)=288.12. $$

Rounding each minimum upward gives 193 Poisson exposure units and 289 negative binomial exposure units. The negative binomial requirement is higher because its per-unit variance is $2+2^2/4=3$, rather than the Poisson variance of 2. The calculation avoids the mistake of treating the size parameter $k$ as the variance.

Check. The ratio of the unrounded requirements is $288.12/192.08=1.5$, exactly the overdispersion multiplier.

Worked Example: A Small Mean Does Not Remove the Multiplier

A hypothetical book has independent negative binomial claim counts per exposure unit, with $\lambda=1$ and $k=4$. How many exposure units are required for a 95% two-sided standard with relative tolerance $r=0.10$, using $z=1.96$?

Method. Calculate the Poisson component first, then multiply by the negative binomial overdispersion factor. The calculation uses $r^2=0.01$, not $r=0.10$ in the denominator.

The Poisson component is

\frac{1.96^2}{0.10^2(1)} =\frac{3.8416}{0.01} =384.16.

The overdispersion multiplier is $1+1/4=1.25$. Thus the negative binomial requirement before rounding is

$$ 384.16(1.25)=480.20. $$

The minimum whole-number requirement is therefore $\lceil480.20\rceil=481$ exposure units. This is not 49: that result would be inconsistent with the displayed multiplication and would fail to meet the calculated minimum. As a check, 480 units are below 480.20, while 481 units meet or exceed it.

Common mistake avoided. Losing a factor of ten when squaring the 10% tolerance changes the answer substantially. The denominator is $r^2\lambda=0.10^2(1)=0.01$, so the unrounded result is 480.20.

Worked Example: A Different Confidence Level and Tolerance

For a hypothetical portfolio, independent negative binomial counts per exposure unit have mean $\lambda=0.5$ and size $k=2$. Find the exposure required for a 90% two-sided relative tolerance of $r=0.20$, using $z=1.645$. Compare with the Poisson requirement.

Method. Apply the same standard, using the quantile for the stated 90% two-sided confidence level and the stated relative tolerance.

The squared quantile is $1.645^2=2.706025$, and $r^2=0.20^2=0.04$. The Poisson requirement before rounding is

\frac{2.706025}{0.04(0.5)} =\frac{2.706025}{0.02} =135.30125.

The negative binomial multiplier is \[ 1+\frac{0.5}{2}=1.25. \] Its unrounded requirement is therefore

$$ 135.30125(1.25)=169.1265625. $$

Rounding upward gives 136 units under the Poisson model and 170 units under the negative binomial model. For a variance check, the negative binomial per-unit variance is $0.5+0.5^2/2=0.625$, which is 1.25 times the Poisson variance $0.5$. The unrounded exposure requirements have that same ratio.

Interpreting and Applying the Standard

The size parameter $k$ is a dispersion parameter, not an exposure count and not the variance. For fixed $\lambda$, decreasing $k$ increases $\lambda^2/k$ and therefore increases the required exposure. As $k$ grows without bound, the negative binomial model approaches the Poisson model and the additional multiplier approaches one.

Count model per unit	Mean	Variance	Relative-error requirement before rounding
Poisson	$\lambda$	$\lambda$	$z^2/(r^2\lambda)$
Negative binomial	$\lambda$	$\lambda+\lambda^2/k$	$[z^2/(r^2\lambda)](1+\lambda/k)$

The normal approximation is most useful as an exposure-planning standard when aggregate counts are not too small or too concentrated near zero. Because counts are discrete, an exact probability calculation can differ from the approximation, particularly for small expected totals or stringent tolerances. If a problem explicitly requests an exact probability, evaluate the count probabilities rather than treating the normal-approximation standard as exact.

A frequent exam trap is to calculate a Poisson standard and stop, even though the problem specifies negative binomial counts. Another is to round down a noninteger requirement. The correct sequence is to identify the model and its variance, substitute into the standard with $r$ squared, and round the final minimum exposure upward. This comparison is aligned with the SOA Exam ASTAM (Advanced Short-Term Actuarial Mathematics) syllabus.

Key takeaway: For negative binomial counts, the variance-to-mean ratio is $1+\lambda/k$. Under the same normal-approximation confidence and relative precision standard, multiply the Poisson full-credibility requirement by this ratio, then round the exposure requirement up.

Check Your Understanding

Use the normal-approximation standard. Unless stated otherwise, round each minimum exposure requirement upward to a whole number.

Independent negative binomial counts have $\lambda=3$ and $k=6$ per exposure unit. Find the requirement for a 95% two-sided standard with $r=0.15$, using $z=1.96$. (Answer: 86 exposure units)
Independent negative binomial counts have $\lambda=2$ and $k=2$. Find the requirement for a 95% two-sided standard with $r=0.10$, using $z=1.96$. (Answer: 385 exposure units)
Independent negative binomial counts have $\lambda=0.8$ and $k=4$. Find the requirement for a 90% two-sided standard with $r=0.10$, using $z=1.645$, and compare with the Poisson requirement. (Answer: 406 negative binomial units; 339 Poisson units)
A portfolio has $n=300$ independent negative binomial exposure units, with $\lambda=1.5$ and $k=3$. Using $z=1.96$, what relative tolerance is supported by the normal-approximation two-sided 95% standard? Give the result as a percentage to two decimal places. (Answer: 11.32%)

Count model per unit	Mean	Variance	Relative-error requirement before rounding
Poisson	\(\lambda\)	\(\lambda\)	\(z^2/(r^2\lambda)\)
Negative binomial	\(\lambda\)	\(\lambda+\lambda^2/k\)	\([z^2/(r^2\lambda)](1+\lambda/k)\)

What You'll Learn