Why Choose Between These Count Models?
A frequency model should reflect how claims are generated, not simply provide a convenient formula. A Poisson model is a natural candidate when claims arise at a stable rate over a stated exposure. A negative binomial model can be more suitable when claim propensity varies across the risks being counted, creating more variability in the counts than a Poisson model allows.
The central diagnostic is the relationship between the count’s mean and variance. It is a useful guide, not a mechanical rule: observed sample moments fluctuate, and a variance above the mean does not by itself prove that the negative binomial model is correct. The assumptions about exposure and claim-generating behavior matter as well.
The negative binomial convention used here counts the number \(N\) of failures before the \(r\)th success in independent trials, where each trial has success probability \(p\). Here \(r\) is a positive integer and \(0<p<1\). This model has support \(0,1,2,\ldots\). Its probability at \(k\) is \(\binom{k+r-1}{k}p^r(1-p)^k\), and its moments are:
Unlike the Poisson model, a negative binomial model has variance greater than its mean for finite \(r\), since \(\operatorname{Var}(N)=E[N]/p\) and \(p<1\). Equivalently, \(\operatorname{Var}(N)=E[N]+E[N]^2/r\). This extra variance is called overdispersion relative to a Poisson model.
What the Claim-Generating Process Tells You
Consider a hypothetical portfolio in which each risk has the same claim rate during the observation period, and claims occur independently at that rate. A Poisson count may be reasonable: its conditional rate is stable, and it does not include an additional source of variation in claim propensity.
Now suppose risks have different underlying claim propensities, or a risk’s propensity varies over time. Even if each individual count is Poisson conditional on its rate, pooling counts with different rates can increase the overall variance. A negative binomial model is one standard way to represent such extra variability. It is an exact model for a Poisson count whose rate follows a gamma distribution; in other situations, it can serve as a useful overdispersed model without being the exact data-generating distribution.
| Feature | Poisson model | Negative binomial model |
|---|---|---|
| Support | 0, 1, 2, ... | 0, 1, 2, ... |
| Mean | \(\lambda\) | \(r(1-p)/p\) |
| Variance | \(\lambda\), equal to the mean | \(r(1-p)/p^2\), greater than the mean |
| Claim-rate interpretation | A stable rate is a plausible description | Variation in propensity can account for extra variability |
| Key caution | Unmodeled rate variation can make it too restrictive | Overdispersion alone does not establish that this model is correct |
Why Rate Variation Creates Extra Variance
The gamma-mixture construction gives a direct justification for the negative binomial’s larger variance. Suppose the claim count \(N\), conditional on a random rate \(\Lambda\), is Poisson with parameter \(\Lambda\). Assume \(\Lambda\) has a gamma distribution with shape \(r\) and rate \(\beta\), so \(E[\Lambda]=r/\beta\) and \(\operatorname{Var}(\Lambda)=r/\beta^2\). The conditional Poisson moments are \(E[N\mid\Lambda]=\Lambda\) and \(\operatorname{Var}(N\mid\Lambda)=\Lambda\).
The law of total expectation gives \(E[N]=E[\Lambda]=r/\beta\). The law of total variance separates the average Poisson variation from variation in the underlying rate:
To match the failure-count parameterization, set \(p=\beta/(\beta+1)\). Then \((1-p)/p=1/\beta\), and \(r/\beta=r(1-p)/p\). The variance becomes \(r(1-p)/p+r(1-p)^2/p^2=r(1-p)/p^2\). Thus, the extra variance arises from variation in the latent rate in addition to the ordinary Poisson variation. If the rate is constant instead, this additional term is zero and the conditional count is Poisson.
Worked Example: A Stable Rate and a Poisson Candidate
A hypothetical insurer reviews counts for six equal-exposure policy groups. The observed claim counts are 0, 1, 1, 2, 2, and 3. Assume the groups have comparable risk characteristics and that claims arise independently at a stable rate. Calculate the sample mean and sample variance, assess the Poisson model, and calculate the model probability of exactly two claims using the fitted Poisson parameter.
Method and common mistake to avoid. Compare the empirical mean and variance while treating the stated stability and equal exposure as part of the model assessment. Do not require a small sample variance to equal the mean exactly: sample moments fluctuate around population moments.
The sum of the counts is \(0+1+1+2+2+3=9\), so the sample mean is \(9/6=1.5\). For the sample variance, the squared deviations from 1.5 are \(2.25, 0.25, 0.25, 0.25, 0.25,\) and \(2.25\). Their sum is 5.5, giving the unbiased sample variance \(5.5/(6-1)=1.1\).
The observed variance is below the observed mean, not above it. With only six groups, that difference is not proof that the true variance is below the mean. Given the problem’s stable-rate assumption, a Poisson model is a reasonable candidate; the fitted parameter is \(\lambda=1.5\). The probability of exactly two claims is:
The probability is rounded to four decimal places. As a check, the fitted Poisson model has both mean and variance 1.5. The empirical variance of 1.1 is a sample statistic, not the fitted model’s theoretical variance.
Worked Example: A Negative Binomial Model for Variable Propensity
In a hypothetical portfolio, policyholders have different underlying claim propensities. Suppose each policyholder’s rate \(\Lambda\) has a gamma distribution with shape \(r=4\) and rate \(\beta=2\), and the claim count conditional on that rate is Poisson. Find the unconditional mean and variance, identify the corresponding negative binomial parameters, and compare the probability of zero claims with a Poisson model having the same mean.
Method and common mistake to avoid. Use the gamma-Poisson mixture and translate its parameters into the stated failure-count convention. Do not use the success probability as though it were the gamma rate.
The mean and variance of the random rate are \(E[\Lambda]=r/\beta=4/2=2\) and \(\operatorname{Var}(\Lambda)=r/\beta^2=4/4=1\). Therefore the unconditional count has mean 2 and variance \(2+1=3\), by the total-variance calculation above.
The negative binomial success probability is \(p=\beta/(\beta+1)=2/(2+1)=2/3\). Its parameters are thus \(r=4\) and \(p=2/3\). Directly checking with the failure-count formulas gives:
For a negative binomial count, zero failures means the first \(r\) trials are all successes, so \(P(N=0)=p^r=(2/3)^4=16/81=0.1975\), rounded to four decimal places. A Poisson count with the same mean \(\lambda=2\) has \(P(N=0)=e^{-2}=0.1353\), also rounded to four decimal places. The probabilities differ because the negative binomial model allows the rate to vary across policyholders.
Worked Example: Using Sample Moments to Fit a Negative Binomial Model
A hypothetical set of five equal-exposure accounts has observed counts 0, 1, 1, 3, and 5. Use the sample mean and unbiased sample variance to assess whether a Poisson model’s mean-variance relationship is plausible. If matching moments to a negative binomial model, calculate \(r\) and \(p\), then verify its theoretical moments.
Method and common mistake to avoid. Match the empirical moments to the negative binomial moments. Do not report a negative binomial parameter until checking that the sample variance exceeds the mean; and do not confuse the variance formula with the mean formula.
The total count is \(0+1+1+3+5=10\), so the sample mean is \(10/5=2\). The squared deviations from 2 are \(4,1,1,1,\) and \(9\), with sum 16. The unbiased sample variance is \(16/(5-1)=4\), which exceeds the sample mean of 2. This is evidence of overdispersion in these observations, although five counts alone provide limited evidence about the underlying distribution.
For the negative binomial model, \(\operatorname{Var}(N)=\mu+\mu^2/r\), where \(\mu=E[N]\). Matching \(\mu=2\) and variance 4 gives \(4=2+2^2/r\), so \(2=4/r\) and \(r=2\). From \(\mu=r(1-p)/p\), rearrange to \(p=r/(r+\mu)\). Thus \(p=2/(2+2)=0.5\).
Check the parameters in the original formulas: \(E[N]=2(1-0.5)/0.5=2\), and \(\operatorname{Var}(N)=2(1-0.5)/(0.5)^2=1/0.25=4\). The fitted moments agree with the values used in the calculation. A Poisson model fitted to the same mean would have variance 2, so it would not reproduce this sample’s elevated variance.
Common Exam Traps
- Using only the average. A mean can be matched by either model. Compare variability as well, and consider how the claims are generated.
- Ignoring exposure and risk differences. Counts from unequal exposure periods or substantially different risks can appear more variable when pooled. Check whether rates are comparable before interpreting a variance-to-mean comparison.
- Confusing the negative binomial mean and variance. The mean is \(r(1-p)/p\); the variance is \(r(1-p)/p^2\). The latter has one additional factor of \(1/p\).
- Treating overdispersion as proof. A sample variance greater than the sample mean suggests that a Poisson model may be too restrictive, but it does not establish that the negative binomial distribution is the correct model.
- Mixing parameter conventions. Here \(p\) is the success probability and \(N\) counts failures before \(r\) successes. State that convention before applying a formula.
Check Your Understanding
For each hypothetical situation, use the stated assumptions and parameterization to select or assess the model. Answers are rounded only where indicated.
- A homogeneous portfolio has a Poisson claim count with parameter \(\lambda=1.2\) per policy-year. State the model’s mean and variance, and say whether this model has overdispersion relative to its own mean. (Answer: Poisson; mean \(=1.2\), variance \(=1.2\), no overdispersion.)
- A count follows the negative binomial failure-count model with \(r=3\) and \(p=0.6\). Calculate its mean and variance, then state whether its variance exceeds its mean. (Answer: mean \(=2\), variance \(=10/3\approx3.3333\); yes.)
- Five equal-exposure accounts have counts 0, 0, 2, 2, and 4. Calculate the sample mean and unbiased sample variance. Does the observed variance suggest overdispersion relative to a Poisson model? (Answer: mean \(=1.6\), sample variance \(=2.8\); yes, it suggests overdispersion.)
- Moment estimates for a count are mean 3 and variance 6. Match these moments to a negative binomial failure-count model and verify the resulting \(r\) and \(p\). (Answer: \(r=3\), \(p=0.5\); the model mean is 3 and variance is 6.)
- Claims for each risk are assumed to arise independently at a common, stable rate, with equal exposure. The observed variance is slightly greater than the observed mean in a small sample. Must a negative binomial model be selected? Explain the role of the sample size and assumptions. (Answer: No. The small-sample difference may be sampling fluctuation; assess the stable-rate assumptions and the strength of evidence before selecting a model.)