Why Compare the Mean and Variance?
A claim-frequency model describes the number of claims \(N\) in a specified period or exposure unit. Its mean describes the average count, while its variance describes how much counts vary around that average. Comparing these two quantities is a quick way to decide which common frequency models are compatible with a set of assumptions or observed data.
This comparison is a diagnostic, not a complete model-selection procedure. Two distributions can share the same mean and variance while having different probabilities, and empirical moments fluctuate from sample to sample. The useful first question is narrower: does the variance indicate underdispersion, equidispersion, or overdispersion relative to the mean?
The Mean-Variance Signatures
For a binomial count \(N\sim\operatorname{Binomial}(n,p)\), \(N\) is the number of successes in \(n\) independent trials, each with the same success probability \(p\). Its mean and variance are \(np\) and \(np(1-p)\). For a Poisson count \(N\sim\operatorname{Poisson}(\lambda)\), the mean and variance are both \(\lambda\). For a negative binomial count, this tutorial uses the convention that \(N\) is the number of failures before the \(r\)th success, where each independent trial has success probability \(p\). Under this convention, \(E[N]=r(1-p)/p\) and \(\operatorname{Var}(N)=r(1-p)/p^2\).
| Model | Mean | Variance | Typical pattern |
|---|---|---|---|
| Binomial \((n,p)\) | \(np\) | \(np(1-p)\) | Underdispersed when \(0<p<1\) |
| Poisson \((\lambda)\) | \(\lambda\) | \(\lambda\) | Equidispersed |
| Negative binomial \((r,p)\), failures before \(r\) successes | \(r(1-p)/p\) | \(r(1-p)/p^2\) | Overdispersed when \(0<p<1\) |
The binomial model has a fixed maximum count \(n\). Its variance is strictly below its mean for \(0<p<1\), because each trial contributes a variable that can only be 0 or 1. The Poisson model has no fixed upper bound and has equal mean and variance. The stated negative binomial model also has no fixed upper bound; its variance exceeds its mean when \(0<p<1\).
Deriving the Signatures and Recovering Parameters
The mean-variance comparison follows directly from the model formulas. For a binomial count, put \(\mu=E[N]=np\). Since \(p=\mu/n\),
For \(0<p<1\), this gives \(\operatorname{Var}(N)<\mu\). This also shows why the binomial variance is bounded above by its mean. If a positive mean \(\mu\) and a variance \(V\) are known exactly and \(0\leq V<\mu\), the moment equations imply
These are compatible with a binomial model only if the resulting \(n\) is a positive integer, \(0<p\leq1\), and the count has the required fixed-trial interpretation. Matching two moments alone does not establish that such an interpretation is appropriate.
For the negative binomial convention above, write \(\mu=r(1-p)/p\). Dividing the variance by the mean gives \(\operatorname{Var}(N)/\mu=1/p\). Thus, if \(V>\mu>0\), the moment equations can be solved as
These parameters satisfy \(0<p<1\) and \(r>0\) when \(V>\mu>0\). An equivalent form of the variance is \(\mu+\mu^2/r\), which makes the extra variation over the mean explicit. Under the chosen convention, the probability of zero failures is \(P(N=0)=p^r\); this fact can help check a parameterization, but the mean-variance comparison does not by itself determine the full distribution.
Define what is counted and the period or exposure unit. For empirical comparisons, use counts from comparable, independent units where possible.
A variance below the mean is compatible with a nondegenerate binomial model; equality is the Poisson signature; a variance above the mean is compatible with a negative binomial model.
For a binomial fit, require a valid integer \(n\) and \(p\). For a negative binomial fit, require \(r>0\) and \(0<p<1\). Then check whether the model’s support and assumptions fit the situation.
Worked Examples
Worked Example: An Exact Underdispersed Frequency
A hypothetical program records the number \(N\) of covered equipment breakdowns per contract-year. Assume the exact population mean is \(\mu=4\) and the exact population variance is \(V=3\). If a binomial model is appropriate, determine its parameters and verify the moments. State why Poisson and negative binomial models do not match these exact moments.
Method. The variance is below the mean, so begin with the binomial moment equations. Substitution gives
The parameter checks pass: \(n=16\) is a positive integer and \(0<p<1\). Verify the mean:
Verify the variance:
The calculation is internally consistent. A Poisson model would require variance \(4\), not \(3\), and a negative binomial model with \(0<p<1\) would require variance greater than its mean. Common mistake avoided: setting \(n=\mu\) or treating the binomial variance as equal to its mean. The inferred number of trials comes from both moments, not from the mean alone.
Worked Example: Estimating a Negative Binomial Model
For a hypothetical portfolio with a fixed exposure unit, suppose the exact mean claim count is \(\mu=6\) and the exact variance is \(V=10\). Under the convention that \(N\) counts failures before the \(r\)th success, find \(r\) and \(p\), then verify both moments.
Method. Since \(10>6\), the moments are overdispersed and are compatible with a negative binomial model. Use the moment equations:
The parameter conditions hold: \(r=9>0\) and \(0<p=0.6<1\). Under this parameterization, the mean is
The variance is
Thus the fitted moments are verified. As an additional parameterization check, \(P(N=0)=p^r=0.6^9=0.010077696\), or \(0.0101\) to four decimals. Common mistake avoided: using a different negative binomial convention without translating its parameters. Here \(p\) is the success probability, while \(N\) counts failures before \(r\) successes.
Worked Example: Reading Overdispersion in a Small Sample
A hypothetical analyst observes claim counts from eight independent, equal-exposure contract-years: \(0,0,1,1,2,2,4,6\). Calculate the sample mean and sample variance, use their pattern as a diagnostic, and find the negative binomial method-of-moments parameter estimates if that model is used. Do not treat the sample pattern as proof of the model.
Method. There are \(m=8\) observations. Their sum and sum of squares are
The sample mean is
Using the usual sample-variance denominator \(m-1\),
Since \(s^2\approx4.2857\) exceeds \(\bar{N}=2\), these observations show a sample overdispersion pattern. Treating the sample moments as method-of-moments estimates gives
Check the fitted variance using \(\hat{r}=7/4\) and \(\hat{p}=7/15\). Since \(1-\hat{p}=8/15\),
The fitted mean equals the sample mean, and the fitted variance equals the sample variance \(30/7\). This is a moment fit, not evidence that the true counts must be negative binomial: eight observations provide noisy estimates, and other mechanisms can also produce a sample variance greater than the sample mean. Common mistake avoided: dividing by \(m\) instead of \(m-1\) when calculating the usual sample variance.
Worked Example: Equal Moments Do Not Prove a Poisson Model
A hypothetical frequency specification states that the exact mean and variance of a nondegenerate count are both \(2.5\). Identify the model among the three considered here whose moments match, and explain what the moment comparison does and does not establish.
Method. For a Poisson model, set \(\lambda=\mu=2.5\). Its variance is also \(\lambda=2.5\), so its moments match exactly. A nondegenerate binomial model has \(np(1-p)<np\) for \(0<p<1\), so it cannot have equal positive mean and variance. A negative binomial model with \(0<p<1\) has variance greater than its mean, so it also cannot match these moments.
Therefore, of these three specified model families, the Poisson model is the one with the matching nondegenerate mean-variance signature, with \(\lambda=2.5\). This does not prove that the count is Poisson: moments do not specify every probability, and a different distribution could also have mean and variance \(2.5\). Common mistake avoided: concluding that equality of empirical mean and variance establishes a Poisson model. The conclusion here depends on the moments being specified as exact and concerns only the three model families being compared.
Interpreting Observed Overdispersion
In practice, a sample mean and sample variance are estimates, not the exact population moments. A sample variance above the sample mean can occur even when the underlying model is Poisson, especially with limited data. It is evidence to investigate, rather than a definitive test of a negative binomial model. Conversely, a sample variance below the mean does not prove a binomial model: a binomial interpretation also requires a meaningful fixed number of independent trials.
Overdispersion can arise when observations differ in their underlying risk, when counts are dependent, when exposure units are not comparable, or when the chosen model is otherwise inadequate. Before fitting a more dispersed model, check that counts use the same time or exposure basis and that the data have been grouped appropriately. A model’s support also matters: a binomial count cannot exceed \(n\), while Poisson and negative binomial counts have no fixed upper limit.
A useful exam procedure is to write the model’s mean and variance before comparing them. Then check the direction of the inequality and solve for parameters only when the conditions permit it. This prevents a common reversal: a variance less than the mean points toward binomial compatibility, whereas a variance greater than the mean points toward negative binomial compatibility under the convention used here. A Poisson model sits at equality.
Check Your Understanding
For each item, compare the stated moments and use only the model conventions defined above. Treat exact moments as exact and empirical moments as estimates.
- A hypothetical fixed-trial count has exact mean \(8\) and variance \(6\). Find the compatible binomial parameters \(n\) and \(p\), and verify that the resulting \(n\) is an integer. (Answer: \(n=32,\ p=0.25\).)
- A negative binomial count of failures before \(r\) successes has exact mean \(4\) and variance \(7\). Find \(r\) and \(p\), and state the direction of dispersion. (Answer: \(r=16/3,\ p=4/7\); overdispersed.)
- A nondegenerate count has exact mean \(2.5\) and variance \(2.5\). Which of the three model families has the matching signature, and what is its parameter? (Answer: Poisson with \(\lambda=2.5\); the moment match alone does not prove the model.)
- A count has exact mean \(3\) and variance \(4.5\). Fit a negative binomial model by moments and verify the implied success probability is valid. (Answer: \(r=6,\ p=2/3\), which is valid since \(r>0\) and \(0<p<1\).)
- A hypothetical data set has sample mean \(5\) and sample variance \(2.5\). Find the binomial method-of-moments values and state one additional condition required for a binomial interpretation. (Answer: \(n=10,\ p=0.5\); a fixed integer number of independent trials with a common success probability is required.)