Common Errors In Poisson And Negative Binomial Frequency Models

Why Small Specification Errors Matter

A frequency model can be fitted correctly by software and still be used incorrectly. A rate expressed per policy-year cannot be combined with exposure measured in months without conversion; a negative binomial dispersion parameter is not itself a variance; and a log-link coefficient is not the percentage change in a rate. These errors can affect expected claim counts, indicated relativities, and the apparent precision of results.

This tutorial focuses on checking those interpretations in Poisson and negative binomial claim-count models. The model forms below assume independent observations conditional on the predictors and exposure. They are useful only when the data and assumptions are appropriate: for example, an exposure offset accounts for time at risk but does not by itself correct dependence between observations or omitted risk characteristics. These practical checks are aligned with the CAS Exam 8 (Advanced Ratemaking) syllabus.

Definition: In a Poisson frequency model, the count \(N_i\) for observation \(i\), conditional on predictors \(x_i\) and positive exposure \(e_i\), has distribution \(N_i\sim\operatorname{Poisson}(\mu_i)\), with mean and variance \(\mu_i=e_i\lambda_i\). A log-link model for the rate is \(\log(\lambda_i)=x_i^\mathsf{T}\beta\), so \(\log(\mu_i)=\log(e_i)+x_i^\mathsf{T}\beta\). The term \(\log(e_i)\) is an offset: its coefficient is fixed at 1.

Check the Exposure Unit Before Calculating

The exposure and rate must use consistent units. If \(\lambda_i\) is an annual expected count per policy-year, then \(e_i\) must be measured in policy-years. If exposure is instead recorded in months, either convert it to years or express the modeled rate per month. Mixing units changes the mean count, even though the fitted predictor coefficients may look plausible.

The offset follows directly from the mean structure. Since \(\mu_i=e_i\lambda_i\), taking logarithms gives \(\log(\mu_i)=\log(e_i)+\log(\lambda_i)\). Substituting \(\log(\lambda_i)=x_i^\mathsf{T}\beta\) yields the model with offset \(\log(e_i)\). The coefficient is fixed at 1 because doubling exposure, with the rate held constant, doubles expected count. Estimating a different coefficient would instead let the model learn an exposure elasticity other than 1, which is not the standard frequency specification.

Worked Example: A Quarterly Policy and an Annual Rate

A hypothetical insurer models claim frequency as Poisson with an annual rate of \(0.12\) claims per policy-year for a particular risk class. A policy is covered for one quarter, recorded as \(0.25\) policy-years. Find the expected count and the probability of at least one claim. No interest or mortality assumptions apply; the claim count is Poisson with the stated mean.

Method. Multiply the annual rate by exposure measured in policy-years:

\mu=e\lambda=0.25(0.12)=0.03.

For a Poisson count, \(P(N=0)=e^{-\mu}\). Therefore,

P(N\geq 1)=1-P(N=0)=1-e^{-0.03}=0.029554\ldots.

The probability is \(0.0296\), or about \(2.96\%\), to four decimal places. The expected count is \(0.03\) claims. As a check, the zero-claim probability is about \(0.9704\), and the two probabilities sum to 1.

Common mistake and correction. Treating \(0.25\) as \(0.25\) months, or using 3 months directly with an annual rate, mixes units. In particular, the incorrect calculation \(3(0.12)=0.36\) gives \(1-e^{-0.36}=0.3023\), a substantially different probability. If exposure is recorded in months, the compatible rate is \(0.12/12=0.01\) per month, giving \(3(0.01)=0.03\), the same correct mean.

An offset is also not an exposure weight. It enters the linear predictor for the mean, whereas a weight changes how an observation contributes to estimation. Using one in place of the other does not preserve the model just because both involve exposure.

Do Not Confuse Negative Binomial Dispersion With Variance

A common negative binomial frequency specification is the NB2 form. It allows the variance to exceed the mean through a positive parameter \(\alpha\). The parameterization matters: software and references can use different negative binomial conventions, so check the stated variance formula rather than relying on a parameter name alone.

Definition: In the NB2 model, \(E[N_i]=\mu_i\) and \(\operatorname{Var}(N_i)=\mu_i+\alpha\mu_i^2\), where \(\alpha>0\). Equivalently, the negative binomial size parameter is \(r=1/\alpha\). Under the convention \(P(N=n)=\frac{\Gamma(n+r)}{\Gamma(r)n!}p^r(1-p)^n\), the mean is \(r(1-p)/p\) and the variance is \(r(1-p)/p^2\). Thus \(p=r/(r+\mu)\). As \(\alpha\) approaches zero, the NB2 variance approaches the Poisson variance \(\mu\).

Worked Example: Reading an NB2 Dispersion Parameter

A hypothetical portfolio model gives a mean claim count of \(0.8\) per policy-year and uses NB2 dispersion \(\alpha=0.5\). Find the variance, the equivalent size and probability parameters, and the probabilities of zero and one claim. No interest or mortality assumptions apply; the assumed count distribution is NB2 with the stated mean and dispersion.

Method. First use the NB2 variance formula:

\operatorname{Var}(N)=\mu+\alpha\mu^2 =0.8+0.5(0.8)^2 =0.8+0.32 =1.12.

The equivalent size is \(r=1/\alpha=1/0.5=2\), and

p=\frac{r}{r+\mu}=\frac{2}{2+0.8}=\frac{5}{7}.

For \(n=0\), the probability formula gives \(P(N=0)=p^r=(5/7)^2=25/49=0.510204\ldots\). For \(n=1\),

P(N=1)=r p^r(1-p) =2\left(\frac{5}{7}\right)^2\left(\frac{2}{7}\right) =\frac{100}{343} =0.291545\ldots.

To four decimal places, \(P(N=0)=0.5102\) and \(P(N=1)=0.2915\). The mean and variance are \(0.8\) and \(1.12\), respectively, so the variance exceeds the mean as expected for this overdispersed NB2 model.

Common mistake and correction. Calling \(\alpha=0.5\) the variance, or using it as a standard-error multiplier, is incorrect. The variance is \(1.12\), calculated from \(\mu+\alpha\mu^2\). Also, the size is \(1/\alpha=2\), not \(0.5\). Always identify the parameterization before computing probabilities or uncertainty.

Interpret Relativities on the Scale Where They Are Defined

With a log link, a coefficient describes a change in log mean, not a direct additive change in claim frequency. For a binary rating variable with coefficient \(\beta_j\), holding other predictors and exposure fixed, the rate ratio between the category coded 1 and the reference category coded 0 is \(\exp(\beta_j)\). The percentage change is \((\exp(\beta_j)-1)\times100\%\).

Uncertainty needs the same care. A standard error reported for \(\hat\beta_j\) is on the log scale. A confidence interval for the relativity is obtained by forming the interval for \(\beta_j\) and exponentiating both endpoints. The standard error of the coefficient is not the standard error of the relativity.

Worked Example: Converting a Log Coefficient to a Relativity

A hypothetical Poisson model uses a log link for annual claim frequency. For a binary vehicle-use category, the fitted coefficient is \(\hat\beta=\log(1.5)=0.405465\), with standard error \(0.20\). Find the relativity, its percentage interpretation, and a 95% Wald confidence interval. No interest or mortality assumptions apply; the count distribution is Poisson and the stated standard error is for the log coefficient.

Method. Exponentiating the coefficient gives the rate ratio:

\exp(\hat\beta)=\exp(\log(1.5))=1.5.

Thus, holding other modeled factors fixed, the category has an estimated rate 1.5 times the reference rate, or a \( (1.5-1)\times100\%=50\% \) higher rate.

The 95% Wald interval on the coefficient scale is \(\hat\beta\pm1.96\,SE(\hat\beta)\):

0.405465\pm1.96(0.20) =0.405465\pm0.392 =[0.013465,\ 0.797465].

Exponentiating the endpoints gives

[\exp(0.013465),\exp(0.797465)] \approx[1.0136,\ 2.2200].

The interval is approximately \([1.0136,2.2200]\), to four decimal places. It is centered on the log scale, not on the relativity scale; its endpoints need not be equally distant from 1.5.

Common mistake and correction. Reporting \(0.405465\) as a 40.55% increase confuses a log coefficient with a percentage change. The correct calculation is to exponentiate first: \(\exp(0.405465)=1.5\), which corresponds to a 50% increase. Similarly, the \(0.20\) standard error belongs to the coefficient; do not report it as the standard error of the factor.

Interpret Standard Errors Under the Model Actually Used

A Poisson model assumes conditional variance equal to the conditional mean. If the data are overdispersed, a model-based Poisson standard error can be too small. One simple adjustment is the quasi-Poisson scalar dispersion approach, under which \(\operatorname{Var}(N_i)=\phi\mu_i\). In the usual large-sample covariance approximation, the coefficient covariance matrix is multiplied by \(\phi\); therefore, standard errors are multiplied by \(\sqrt{\phi}\).

This scalar adjustment is not the same calculation as a sandwich (robust) standard error. A sandwich estimate uses the fitted residuals and model design in a different covariance calculation; it is not generally obtained by multiplying every Poisson standard error by one common scalar. State which uncertainty estimate is being used before interpreting it.

Worked Example: Scaling a Quasi-Poisson Standard Error

A hypothetical Poisson fit has 40 residual degrees of freedom and a Pearson statistic of 67.6. The model-based standard error of one fitted coefficient is \(0.08\). Under a quasi-Poisson scalar dispersion adjustment, calculate the adjusted standard error. No interest or mortality assumptions apply; the adjustment uses \(\phi\) estimated as Pearson statistic divided by residual degrees of freedom.

Method. Estimate dispersion:

\hat\phi=\frac{67.6}{40}=1.69.

The standard-error multiplier is the square root of the dispersion estimate:

\sqrt{\hat\phi}=\sqrt{1.69}=1.3.

Therefore, the adjusted standard error is \(0.08(1.3)=0.104\). The answer is \(0.1040\) to four decimal places. The scale of the adjustment is sensible: because the estimated variance is 1.69 times the Poisson variance, the standard deviation scale is 1.3 times as large.

Common mistake and correction. Multiplying the standard error by \(1.69\) gives \(0.1352\), which applies the variance multiplier directly to a standard deviation. The correct operation is multiplication by \(\sqrt{1.69}=1.3\). This calculation describes a scalar quasi-Poisson adjustment; it does not claim that a robust sandwich standard error must have the same value.

A Practical Error-Checking Sequence

Before interpreting a fitted frequency model, check the following in order. Each check targets a different quantity; passing one does not establish that the others are correct.

Verify the observation and exposure units.
Confirm that each count corresponds to its stated exposure and that the rate and exposure use compatible units.

Verify the mean specification.
For a log-link frequency model, check that \(\log(e_i)\) enters as an offset with coefficient fixed at 1 when the standard proportional-exposure assumption is intended.

Identify the variance convention.
For a Poisson model the variance is \(\mu_i\); for NB2 it is \(\mu_i+\alpha\mu_i^2\). Do not transfer a parameter interpretation from another convention.

Translate coefficients and uncertainty.
Exponentiate log-link coefficients to obtain multiplicative relativities, and keep coefficient standard errors on their original scale until transforming an interval or uncertainty measure appropriately.

The distinctions matter in practice. A unit error changes the expected count, a mistaken dispersion interpretation changes the claimed amount of variation, and a coefficient-scale error changes the indicated rate factor. A standard error that is too small can also make a weakly supported relativity appear more certain than the data warrant. Read the mean model, variance assumption, parameterization, and uncertainty method together rather than treating a model output as a collection of interchangeable numbers.

Key takeaway: Match exposure to the rate unit; use the stated negative binomial variance formula; exponentiate log-link coefficients to obtain relativities; and adjust standard errors according to the specified variance and covariance method.

Check Your Understanding

For each hypothetical frequency-model calculation, identify the relevant specification and check the units or scale before using the formula.

A Poisson model has an annual rate of \(0.075\) claims per policy-year. A policy has \(0.4\) policy-years of exposure. Find its expected count and probability of at least one claim. (Answer: \(\mu=0.030\); \(P(N\geq1)=0.0296\).)
A policy is exposed for 8 months, and its modeled annual claim rate is \(0.09\). Find the expected count and probability of at least one claim under a Poisson model. (Answer: \(\mu=(8/12)(0.09)=0.06\); \(P(N\geq1)=0.0582\).)
An NB2 model has mean \(\mu=1.2\) and dispersion \(\alpha=0.25\). Find its variance, size \(r\), and probability of zero claims. (Answer: variance \(=1.56\), \(r=4\), \(p=10/13\), and \(P(N=0)=(10/13)^4=0.3501\).)
A log-link coefficient is \(\hat\beta=\log(1.25)\), with standard error \(0.15\). Find the relativity and 95% Wald interval on the relativity scale. (Answer: relativity \(=1.25\); coefficient interval \([ -0.0709,\ 0.5171]\); relativity interval \([0.9316,\ 1.6772]\), approximately.)
A Poisson fit has Pearson statistic 54 on 30 residual degrees of freedom, and a coefficient's model-based standard error is \(0.06\). Under a quasi-Poisson scalar adjustment, what is the adjusted standard error? (Answer: \(\hat\phi=54/30=1.8\); adjusted standard error \(=0.06\sqrt{1.8}\approx0.0805\).)

What You'll Learn