Comparing Poisson And Negative Binomial Fits

Why Compare Frequency Distributions?

Poisson frequency models provide a natural starting point for claim counts: conditional on modeled characteristics and exposure, the count has a mean equal to its variance. Claim counts may vary more than that assumption allows. A negative binomial model can represent this extra variation, but its additional flexibility comes at a cost: it estimates a dispersion parameter as well as the frequency-model coefficients.

A useful comparison must therefore consider both fit and complexity. Log-likelihood measures how well a specified model assigns probability to the observed data. Deviance measures lack of fit relative to a saturated model. AIC and BIC balance likelihood against the number of estimated parameters. These measures answer related but different questions, and they must be calculated using comparable data and likelihood conventions.

Definition: In the NB2 negative binomial parameterization, a count $Y_i$ has mean $\mu_i$ and variance $\mu_i+\alpha\mu_i^2$, where $\alpha>0$ is the dispersion parameter. Its probability mass function is $$ P(Y_i=y_i)= \frac{\Gamma(y_i+1/\alpha)} {\Gamma(1/\alpha)y_i!} \left(\frac{1}{1+\alpha\mu_i}\right)^{1/\alpha} \left(\frac{\alpha\mu_i}{1+\alpha\mu_i}\right)^{y_i}. $$ As $\alpha$ approaches zero, this distribution approaches the Poisson distribution with mean $\mu_i$. In a frequency GLM, the mean may be modeled with a log link and an exposure offset; the comparison requires the same observations and the same mean-model structure unless the purpose is to compare structures as well as distributions.

Likelihood and Deviance

For independent Poisson counts with fitted means $\mu_i$, the log-likelihood is

\ell_P=\sum_i\left[y_i\log(\mu_i)-\mu_i-\log(y_i!)\right].

For the NB2 model, the log-likelihood is the sum of the logarithms of the probability mass function in the definition. In addition to the mean-model coefficients, an estimated NB2 model generally has an estimated $\alpha$. The log-likelihood used for information criteria should be the maximized likelihood under each model, not merely a likelihood evaluated at arbitrary parameter values.

For either distribution, deviance is defined as twice the difference between the saturated-model log-likelihood and the fitted-model log-likelihood. A saturated model assigns a separate mean to each observation and therefore reproduces each observed count. The Poisson deviance is

D_P=2\sum_i\left[y_i\log\left(\frac{y_i}{\mu_i}\right)-(y_i-\mu_i)\right],

where a term with $y_i=0$ uses $0\log(0/\mu_i)=0$. For NB2 with a common dispersion parameter $\alpha$, the corresponding deviance is

D_{NB}=2\sum_i\left[ y_i\log\left(\frac{y_i}{\mu_i}\right) -\left(y_i+\frac{1}{\alpha}\right) \log\left(\frac{1+\alpha y_i}{1+\alpha\mu_i}\right) \right].

These formulas compare a fitted model with a saturated model under the same distributional family. Thus, a smaller deviance within a family indicates better fit relative to that family’s saturated benchmark. It is not generally valid to select between Poisson and negative binomial models simply by comparing their raw deviances: the saturated log-likelihoods, and hence the reference scales, differ.

Justification: By definition, $D=2(\ell_{\text{sat}}-\ell_{\text{fit}})$. For Poisson data, the saturated mean for observation $i$ is $y_i$. Subtracting the fitted log-likelihood from the saturated log-likelihood gives $$ 2\left\{y_i\log(y_i)-y_i-\log(y_i!) -\left[y_i\log(\mu_i)-\mu_i-\log(y_i!)\right]\right\} =2\left[y_i\log\left(\frac{y_i}{\mu_i}\right)-(y_i-\mu_i)\right]. $$ Summing over observations yields the Poisson deviance. The NB2 expression follows the same saturated-minus-fitted construction, holding $\alpha$ fixed in the saturated and fitted likelihoods.

Worked Example: Comparing Candidate Log-Likelihoods

A hypothetical portfolio has four independent policy-period counts, $0,1,2,3$, each with one unit of exposure. Compare the log-likelihoods of two candidate specifications: Poisson means of $1.5$ for all four observations, and NB2 means of $1.5$ with $\alpha=0.5$. Treat these as candidate parameter values for likelihood scoring, not as a claim that either parameter set is the maximum-likelihood fit.

Method. Calculate each model’s log-likelihood on the identical observations. A higher log-likelihood means the specified parameters assign greater likelihood to these observed counts. This comparison alone does not account for parameter uncertainty or model complexity.

For the Poisson model, the sum of the counts is $6$, and the sum of their log-factorials is $\log(0!)+\log(1!)+\log(2!)+\log(3!)=\log(12)$. Therefore,

\ell_P=6\log(1.5)-4(1.5)-\log(12) =2.432790649-6-2.484906650 =-6.052116001.

For the NB2 model, $1/\alpha=2$, and its probability mass function has factors $4/7$ and $3/7$ for the zero-count and count-dependent terms, respectively. The coefficient $\Gamma(y+2)/[\Gamma(2)y!]$ equals $1,2,3,4$ for counts $0,1,2,3$. The product of those coefficients is $24$, and the counts sum to $6$. Thus,

\ell_{NB} =8\log(4/7)+6\log(3/7)+\log(24) \approx -6.382660.

Here the specified Poisson parameters have the higher log-likelihood: $-6.052116>-6.382660$. This does not establish that a fitted Poisson model is preferable; it compares only the stated parameter values. In a model-selection comparison, both likelihoods should be maximized under their respective models, and complexity should then be considered.

Common mistake avoided: Choosing a distribution because it is more flexible. Flexibility does not guarantee that every set of candidate parameters has a higher likelihood, and a fitted model’s likelihood improvement must be weighed against its extra parameter.

Information Criteria and Parameter Counts

For maximized log-likelihood $\ell$, $k$ estimated parameters, and $n$ independent likelihood contributions, the Akaike information criterion and Bayesian information criterion are

\mathrm{AIC}=-2\ell+2k, \qquad \mathrm{BIC}=-2\ell+k\log(n).

A lower value is preferred when comparing models fitted to the same response data using comparable likelihoods. Count every estimated model parameter: an intercept and each estimated slope count, and an estimated NB2 dispersion parameter counts as one additional parameter. A fixed offset coefficient is not an estimated parameter. The value of $n$ should reflect the independent observations contributing to the likelihood; do not substitute total exposure for $n$.

Suppose the NB2 model has one additional estimated parameter and improves the maximized log-likelihood by $\Delta\ell=\ell_{NB}-\ell_P$. Then

\mathrm{AIC}_{NB}-\mathrm{AIC}_P=-2\Delta\ell+2, \qquad \mathrm{BIC}_{NB}-\mathrm{BIC}_P=-2\Delta\ell+\log(n).

Consequently, the NB2 model has lower AIC if its log-likelihood gain exceeds $1$. It has lower BIC if the gain exceeds $\tfrac12\log(n)$. BIC’s penalty for the extra parameter depends on sample size and can be larger than AIC’s.

Worked Example: Comparing AIC and BIC With an Estimated Dispersion

Two models are fitted to the same $n=40$ independent policy-period counts. The Poisson model has maximized log-likelihood $-121.4$ and $k_P=5$ estimated parameters. The NB2 model has maximized log-likelihood $-117.0$ and $k_{NB}=6$, including its estimated dispersion parameter. Calculate both criteria and select the lower value for each.

Method. Use the maximized log-likelihoods, count the NB2 dispersion parameter, and use $n=40$ in BIC. The improvement is $\Delta\ell=-117.0-(-121.4)=4.4$.

For Poisson,

\mathrm{AIC}_P=-2(-121.4)+2(5)=242.8+10=252.8, $$ $$ \mathrm{BIC}_P=242.8+5\log(40) =242.8+18.444397 =261.244397.

For NB2,

\mathrm{AIC}_{NB}=-2(-117.0)+2(6)=234+12=246.0, $$ $$ \mathrm{BIC}_{NB}=234+6\log(40) =234+22.133276 =256.133276.

To four decimal places, the BIC values are $261.2444$ and $256.1333$. The differences, NB2 minus Poisson, are $-6.8$ for AIC and $-5.1111$ for BIC. Both criteria favor NB2. As a check, the BIC threshold for one extra parameter is $\tfrac12\log(40)=1.8444$; the likelihood gain $4.4$ exceeds it.

Common mistake avoided: Omitting the dispersion parameter would incorrectly set $k_{NB}=5$. That would under-penalize NB2 by $2$ AIC units and by $\log(40)$ BIC units. The dispersion is estimated, so it must be counted.

Deviance Is a Diagnostic, Not a Universal Score

The next calculation shows how deviance is evaluated within each distribution. It also illustrates why a lower value in one family cannot by itself be interpreted as a direct cross-family selection rule.

Worked Example: Computing Poisson and NB2 Deviance

For three hypothetical policy groups, observed counts are $0,1,3$, and fitted means are $1,1,2$ under both a Poisson and an NB2 model. The NB2 model has $\alpha=0.5$. Compute each deviance to four decimal places.

Method. Apply each model’s own saturated-model deviance formula. Keep the zero-count term’s convention explicit; do not attempt to evaluate $\log(0)$ separately.

For Poisson, the count-zero contribution inside the sum is $1$, so its deviance contribution is $2$. The count-one observation has fitted mean $1$, giving contribution $0$. For the count-three observation,

2\left[3\log(3/2)-(3-2)\right] =2\left[1.216395325-1\right] =0.432790649.

Therefore $D_P=2+0+0.432790649=2.432790649$, or $2.4328$ to four decimal places.

For NB2, $1/\alpha=2$. The count-zero observation contributes

2\left[-2\log\left(\frac{1}{1.5}\right)\right] =1.621860432.

The count-one observation has $y=\mu=1$, so its contribution is zero. For the count-three observation,

2\left[3\log(3/2)-5\log(2.5/2)\right] =2\left[1.216395325-1.115717756\right] =0.201355138.

Thus $D_{NB}=1.621860432+0+0.201355138=1.823215570$, or $1.8232$ to four decimal places. The arithmetic is checked by summing the three contributions in each model. Although the NB2 deviance is smaller here, the two deviances use different saturated likelihoods; compare likelihood-based criteria on the same data to make a cross-distribution selection.

Common mistake avoided: Treating the smaller raw deviance as decisive evidence for NB2. Deviance is defined relative to a family-specific saturated model, so this numerical comparison is not the same as comparing AIC or BIC.

Interpretation and Practical Checks

AIC and BIC are relative selection tools, not proofs that a selected model is true. Their comparison is meaningful only when the models use the same response observations and a compatible likelihood basis. If missing data cause the models to be fitted to different records, a direct comparison can reward a model merely because it was evaluated on an easier subset. Likewise, counts aggregated into different groupings may not represent the same likelihood contributions.

The Poisson model is the limiting case of NB2 as $\alpha$ approaches zero. Because the null value lies on the boundary of the allowed parameter space $\alpha\geq0$, a likelihood-ratio comparison between Poisson and NB2 does not generally use the ordinary one-degree-of-freedom chi-square reference without qualification. Under regular conditions for this boundary test, the asymptotic reference can be a mixture of a point mass at zero and a one-degree-of-freedom chi-square distribution. For model choice in this tutorial, AIC and BIC offer explicit complexity penalties; they do not remove the need to assess whether the model and data assumptions are appropriate.

A disciplined comparison can be carried out in this order:

Confirm comparability.
Verify that both fits use the same claim counts, exposure basis, and likelihood contributions.

Compare maximized likelihoods.
Record each model’s maximized log-likelihood, not a score at an unoptimized parameter choice.

Count parameters correctly.
Include the NB2 dispersion parameter if estimated; do not count fixed offsets as fitted coefficients.

Calculate and interpret criteria.
Use the same $n$ convention in BIC and prefer the lower AIC or BIC only as a relative model-selection result.

A final exam check is to write the difference between criteria before substituting numbers. For one extra NB2 parameter, AIC’s penalty increase is exactly $2$, whereas BIC’s is $\log(n)$. This makes it easier to catch a missing dispersion parameter or a mistaken use of exposure as the BIC sample size.

Key takeaway: Compare Poisson and negative binomial fits using the same observations and maximized likelihoods. Include every estimated parameter, especially NB2 dispersion, in AIC and BIC; interpret deviance relative to its own distribution’s saturated model rather than as a universal cross-family score.

Check Your Understanding

For each original problem, calculate the requested comparison and check the final answer.

On the same data, an NB2 fit improves maximized log-likelihood over Poisson by $1.6$ and estimates one additional parameter. Find the AIC difference, NB2 minus Poisson, and state which model has lower AIC. (Answer: $-1.2$; NB2 has lower AIC.)
For $n=25$, an NB2 model improves maximized log-likelihood by $2.0$ and has one extra parameter. Find the BIC difference, NB2 minus Poisson, to four decimals, and select the lower BIC. (Answer: $-0.7811$; NB2 has lower BIC.)
A Poisson model has observed counts $0,2$ and fitted means $1,1$. Calculate its deviance to four decimals. (Answer: $2.7726$.)
For one NB2 observation, $y=0$, $\mu=1$, and $\alpha=0.25$. Calculate its deviance contribution to four decimals. (Answer: $1.7851$.)
With $n=64$, NB2 improves maximized log-likelihood by $2.1$ and estimates one extra parameter. Calculate the BIC difference, NB2 minus Poisson, to four decimals and identify the lower BIC. (Answer: $-0.0411$; NB2 has lower BIC.)

What You'll Learn