Testing Equidispersion In Claim Counts

Why Test Equidispersion?

A Poisson model is a common starting point for claim counts: when the expected count is \(\mu\), its variance is also \(\mu\). This equality is called equidispersion. In a portfolio with more variation between policies or observation periods than a Poisson model allows, a negative binomial model can represent the extra variation. A likelihood-ratio comparison asks whether the improvement in fit is large enough to warrant the more flexible model.

The comparison has an important qualification. The Poisson model is obtained at a boundary value of the negative binomial dispersion parameter. Consequently, counting one additional parameter does not, by itself, justify using the ordinary chi-square likelihood-ratio calibration. The statistic can still be calculated and reported, but its reference distribution must be considered separately.

Definition: Let \(N_1,\ldots,N_n\) be independent claim counts observed over equal exposures. Under the Poisson model, \(N_i\sim\operatorname{Poisson}(\mu)\), so \(E[N_i]=\operatorname{Var}(N_i)=\mu\). Under the negative binomial model used here, \(E[N_i]=\mu\) and \(\operatorname{Var}(N_i)=\mu+\alpha\mu^2\), where \(\alpha\geq 0\) is the dispersion parameter. Equidispersion is the restriction \(\alpha=0\); the negative binomial model with \(\alpha>0\) allows overdispersion.

Parameterization and the Likelihood-Ratio Statistic

For clarity, use the negative binomial parameterization in which the probability of a count \(y=0,1,2,\ldots\) is

P(N_i=y)= \frac{\Gamma(y+1/\alpha)}{\Gamma(1/\alpha)y!} \left(\frac{\alpha\mu}{1+\alpha\mu}\right)^y \left(\frac{1}{1+\alpha\mu}\right)^{1/\alpha}, \qquad \alpha>0.

Here \(\Gamma(\cdot)\) is the gamma function. As \(\alpha\) tends to zero, this probability tends to the Poisson probability \(e^{-\mu}\mu^y/y!\). The null and alternative hypotheses are therefore \(H_0:\alpha=0\) (the Poisson model) and \(H_1:\alpha>0\) (the overdispersed negative binomial model). In each model, estimate any other unknown parameters by maximum likelihood.

Let \(\ell_P\) be the maximized log-likelihood under the Poisson restriction, and let \(\ell_{NB}\) be the maximized log-likelihood under the negative binomial model, using the same observed counts. The likelihood-ratio statistic is

D=2\left(\ell_{NB}-\ell_P\right).

Because the Poisson model is nested in the negative binomial model, the maximized alternative likelihood cannot be smaller: \(\ell_{NB}\geq\ell_P\), so \(D\geq0\). If the constrained negative binomial estimate is \(\widehat{\alpha}=0\), the improvement is zero and \(D=0\).

Why the Negative Binomial Allows Extra Variation

One justification for this model comes from a Poisson mixture. Suppose that, conditional on a latent claim intensity \(\Lambda_i\), the count is Poisson: \(N_i\mid\Lambda_i\sim\operatorname{Poisson}(\Lambda_i)\). Let \(\Lambda_i\) have a gamma distribution with mean \(\mu\), shape \(1/\alpha\), and scale \(\alpha\mu\). Integrating over \(\Lambda_i\) gives the negative binomial probability above.

The variance follows directly from conditioning. The conditional mean and variance of a Poisson count are both \(\Lambda_i\). Applying the laws of total expectation and total variance gives

E[N_i]=E[E[N_i\mid\Lambda_i]]=E[\Lambda_i]=\mu,

\operatorname{Var}(N_i) =E[\operatorname{Var}(N_i\mid\Lambda_i)] +\operatorname{Var}(E[N_i\mid\Lambda_i]) =E[\Lambda_i]+\operatorname{Var}(\Lambda_i) =\mu+\alpha\mu^2.

Thus \(\alpha\) measures variation in the latent intensity in this mixture interpretation. At \(\alpha=0\), the gamma mixing distribution collapses to a constant intensity, leaving the Poisson model. This explains both the equidispersion restriction and why the null sits at the edge, rather than in the interior, of the allowed parameter values.

Worked Likelihood Comparisons

Worked Example: Testing Equidispersion in Four Claim Counts

A hypothetical insurer records independent counts for four equal-exposure policy groups: \(0,0,1,3\). Compare the Poisson model with the negative binomial model above. Estimate the common mean in both models, calculate the likelihood-ratio statistic, and identify the calibration issue.

Step 1: Estimate the Poisson mean. The total count is \(0+0+1+3=4\), so the sample mean and Poisson maximum-likelihood estimate are \(\widehat{\mu}=4/4=1\). The Poisson log-likelihood is

\ell_P=\sum_{i=1}^{4}\left(y_i\log\widehat{\mu}-\widehat{\mu}-\log(y_i!)\right) =0-4-\log(1!)-\log(3!) =-4-\log 6 \approx -5.791759.

Step 2: Profile the negative binomial likelihood. For equal exposures, the maximum-likelihood estimate of \(\mu\) for each fixed \(\alpha\) is also the sample mean, \(1\). At \(\mu=1\), the probabilities satisfy \(p_0=(1+\alpha)^{-1/\alpha}\), \(p_1=p_0/(1+\alpha)\), and \(p_3=p_0(1+2\alpha)/[6(1+\alpha)^2]\). The observed counts contain two zeros, one one, and one three, so

\ell_{NB}(\alpha) =4\log p_0+\log\left(\frac{1}{1+\alpha}\right) +\log\left(\frac{1+2\alpha}{6(1+\alpha)^2}\right) =-\frac{4\log(1+\alpha)}{\alpha} +\log(1+2\alpha)-3\log(1+\alpha)-\log 6.

Numerically maximizing this one-parameter profile over \(\alpha\geq0\) gives \(\widehat{\alpha}\approx0.797\). Substitution gives \(\log(1.797)\approx0.586119\), \(\log(2.594)\approx0.953201\), and \(\ell_{NB}\approx-5.5385\).

Step 3: Calculate the statistic. The log-likelihood improvement is \(-5.5385-(-5.791759)=0.2533\), to the displayed precision. Hence \(D\approx2(0.2533)=0.5065\). The alternative fits better, but this small statistic alone does not determine a formally calibrated significance level.

Calibration. A routine parameter-count argument might suggest comparing \(D\) with a chi-square distribution having one degree of freedom. That ordinary Wilks calibration is not automatically justified: under \(H_0\), \(\alpha=0\) is the boundary of the permitted range \(\alpha\geq0\). Report the statistic and recognize the boundary issue rather than asserting that the standard chi-square test applies.

Method and trap avoided. This is a profile likelihood-ratio comparison, with \(\mu\) re-estimated under each fixed dispersion value. The common error is to treat \(\widehat{\mu}\) as fixed at an externally chosen value or to regard one extra parameter as sufficient proof of ordinary chi-square calibration.

Worked Example: When the Negative Binomial Estimate Is on the Boundary

A hypothetical set of four equal-exposure groups has counts \(0,0,2,2\). Find the Poisson log-likelihood and determine whether fitting a nonnegative negative binomial dispersion improves the maximized likelihood.

The total is \(4\), so \(\widehat{\mu}=4/4=1\). The sum of the log-factorials is \(2\log(2!)=\log4\). Therefore

\ell_P=-4-\log4\approx-5.386294.

At \(\mu=1\), \(p_2=p_0/[2(1+\alpha)]\). There are two zeros and two twos, so the negative binomial profile log-likelihood is

\ell_{NB}(\alpha) =4\log p_0-2\log 2-2\log(1+\alpha) =-\frac{4\log(1+\alpha)}{\alpha}-2\log(1+\alpha)-\log4.

At \(\alpha=0\), use \(\lim_{\alpha\to0}\log(1+\alpha)/\alpha=1\), giving \(\ell_{NB}(0)=-4-\log4=\ell_P\). Maximizing the profile over the permitted values \(\alpha\geq0\) places the estimate at \(\widehat{\alpha}=0\); there is no positive improvement. Thus \(\ell_{NB}=\ell_P\) and \(D=2(0)=0\).

The check is consistent with nesting: the null model is included in the alternative, and the alternative can match it exactly. A negative likelihood-ratio statistic would indicate an optimization or model-comparison error. The trap here is to force \(\widehat{\alpha}>0\) merely because the alternative is called negative binomial.

Worked Example: Interpreting a Nominal Chi-Square Output

For a hypothetical portfolio of 24 independent, equal-exposure groups, software reports maximized log-likelihoods of \(-31.42\) for Poisson and \(-29.87\) for negative binomial, with the dispersion constrained to be nonnegative. Calculate the likelihood-ratio statistic and explain what a reported one-degree-of-freedom p-value would mean.

The difference in maximized log-likelihoods is \(-29.87-(-31.42)=1.55\). Therefore \(D=2(1.55)=3.10\). If one mechanically used a chi-square distribution with one degree of freedom, the nominal upper-tail probability would be approximately \(0.0783\), since \(P(\chi^2_1\geq3.10)\approx0.0783\).

That value is a nominal result under the ordinary chi-square reference, not a justification for using that reference. The null fixes \(\alpha\) at zero, its lower boundary. A valid inferential conclusion needs a calibration appropriate to this boundary setting and the model assumptions; the ordinary parameter-count rule alone does not establish it.

Method and trap avoided. The statistic is computed from the two maximized log-likelihoods on the same data. The common trap is to interpret a software p-value as automatically valid without checking how its reference distribution was selected.

What to Report and What Not to Assume

A clear comparison states the models, the restriction, the fitted likelihoods, and the statistic. It then distinguishes the numerical result from its calibration. For independent equal-exposure counts, the null is Poisson equidispersion, while the alternative permits \(\alpha>0\). For unequal exposures, the expected count generally depends on exposure; the likelihood must represent those exposure-specific means consistently under both models. The equal-exposure formulas above should not be applied unchanged to unequal exposures.

The likelihood-ratio statistic measures relative fit, not the practical importance of extra dispersion. Nor does a larger negative binomial likelihood by itself establish that the model is adequate in other respects. A sound exam response identifies the nested restriction, calculates \(D\) correctly, and flags the calibration condition rather than treating the degrees-of-freedom difference as a complete argument.

Key takeaway: In the negative binomial model \(\operatorname{Var}(N)=\mu+\alpha\mu^2\), Poisson equidispersion is the restriction \(\alpha=0\). The likelihood-ratio statistic is twice the difference in maximized log-likelihoods, but the null is on the boundary \(\alpha\geq0\), so ordinary chi-square calibration is not automatic.

Check Your Understanding

For each problem, identify the restriction or calculation requested before checking the final answer.

A hypothetical sample has counts \(1,0,2,1\) across four equal-exposure groups. Find the Poisson estimate of the common mean and its maximized log-likelihood. (Answer: \(\widehat{\mu}=1\), \(\ell_P=-4-\log2\approx-4.6931\).)
For a fitted comparison, \(\ell_P=-22.6\) and \(\ell_{NB}=-21.1\). Calculate \(D\), and state whether the ordinary one-degree-of-freedom chi-square calibration follows automatically. (Answer: \(D=3.0\); no, because the null dispersion value is on the boundary.)
A negative binomial count model has mean \(\mu=1.5\) and dispersion \(\alpha=0.4\). What is its variance, and is it equidispersed? (Answer: \(\operatorname{Var}(N)=1.5+0.4(1.5)^2=2.4\); no, because \(\alpha>0\).)
For a nested comparison, constrained optimization gives \(\widehat{\alpha}=0\). What are the relationship between the maximized likelihoods and the likelihood-ratio statistic? (Answer: \(\ell_{NB}=\ell_P\) and \(D=0\).)
A report gives \(D=3.10\) and a one-degree-of-freedom chi-square p-value of about \(0.0783\) for a Poisson-versus-negative-binomial comparison. What qualification should accompany that p-value? (Answer: It is nominal only unless that chi-square calibration is justified; the Poisson restriction \(\alpha=0\) lies on the boundary of \(\alpha\geq0\).)

What You'll Learn