Introduction
A claim-count model describes both the expected number of claims and the variation around that expectation. Comparing variation across groups can help assess whether a Poisson model is a reasonable starting point. The comparison must account for exposure: a group covering more policies or more time should generally have more claims and more claim-count variance, even when every group has the same underlying frequency rate.
The central question is therefore not whether two groups have the same raw count variance. It is whether their observed variation is broadly consistent with the variation expected from their Poisson means. A pattern beyond that benchmark may signal that the model or its grouping needs further investigation; it does not, by itself, establish why the pattern occurred.
What to Compare
For groups with the same exposure, a common Poisson rate implies the same expected count and the same count variance. If several comparable subgroups are observed, their sample mean and sample variance provide descriptive estimates of those quantities. For observations \(N_1,\ldots,N_k\), the sample mean is \(\bar N=\sum_j N_j/k\), and the unbiased sample variance is \(s^2=\sum_j(N_j-\bar N)^2/(k-1)\). The variance-to-mean ratio \(s^2/\bar N\) is a useful summary: a value near 1 is consistent with the Poisson equality of mean and variance, while a notably larger value suggests more observed variation than that benchmark. A ratio below 1 indicates less observed variation, which can also occur by chance.
These are sample summaries, not exact tests of a model. In a small sample, the observed variance can differ substantially from the model variance simply through sampling variation. A ratio noticeably above 1 is a prompt for scrutiny, not proof that a particular alternative model is correct.
For unequal exposures, compare observed counts with their exposure-specific means, not with one common count mean. If the common rate is unknown, a natural estimate from several groups is the total number of claims divided by total exposure:
The Pearson statistic summarizes the squared differences between observed and expected counts relative to their expected counts. It is \(Q=\sum_j (N_j-\hat{\lambda}e_j)^2/(\hat{\lambda}e_j)\), provided each expected count in the denominator is positive. A large contribution identifies a group whose count is relatively far from its expected value. When the rate is estimated from the same \(k\) independent Poisson observations, \(Q\) is often compared approximately with a chi-square distribution having \(k-1\) degrees of freedom, subject to adequate expected counts and the model assumptions. For this tutorial, the main use is to compare the size and pattern of the contributions with the Poisson benchmark.
Why Exposure Differences Matter
The Poisson mean-variance equality applies to a count with a specified mean. Counts with different exposures have different means and therefore different variances. Pooling them into one raw sample and comparing its variance with its overall mean can make a correctly specified Poisson process appear unusually variable.
One way to see the distinction is to imagine selecting a group at random from several groups with different fixed exposures. Conditional on selecting a group with exposure \(e\), the count has mean and variance \(\lambda e\). Across the mixture of groups, the variation includes both the conditional Poisson variation and the differences in expected counts due to exposure:
The second term is present because exposures differ. It is not evidence, by itself, of extra variation beyond the Poisson model conditional on exposure. This is why groups should be compared at a common exposure or assessed against their own exposure-adjusted expected counts.
Worked Example: Comparing Equal-Exposure Groups
Two hypothetical portfolios each contain four equal-exposure subgroups. Their observed claim counts are \(4,5,5,6\) in Portfolio A and \(14,15,15,16\) in Portfolio B. Compare their sample means, sample variances, and variance-to-mean ratios. State what can and cannot be concluded.
Method. Since the subgroup exposures are equal within and between the portfolios, calculate the count summaries on the same basis. Avoid comparing the variances without considering that the group means differ.
Portfolio A. The total is \(4+5+5+6=20\), so \(\bar N_A=20/4=5\). The squared deviations from 5 are \(1,0,0,1\), with sum 2. Therefore \(s_A^2=2/(4-1)=2/3\), and \(s_A^2/\bar N_A=(2/3)/5=2/15\approx0.1333\).
Portfolio B. The total is \(14+15+15+16=60\), so \(\bar N_B=60/4=15\). The squared deviations from 15 are \(1,0,0,1\), again with sum 2. Therefore \(s_B^2=2/(4-1)=2/3\), and \(s_B^2/\bar N_B=(2/3)/15=2/45\approx0.0444\).
Interpretation and check. The sample variances are equal, but the sample means are not; the variance-to-mean ratios therefore differ. Both ratios are below 1, so these four observations per portfolio show less variation than the corresponding Poisson benchmark estimates. The samples are too small to conclude that either portfolio violates a Poisson model. The totals, 20 and 60, also check the means of 5 and 15.
Common mistake avoided: The second total is 60, not 50. In addition, comparing raw variances alone would miss that the portfolios have different average counts.
Worked Example: Adjusting for Unequal Exposure
A hypothetical insurer observes four regional claim counts with exposures \(1,2,3,4\) units, respectively. The counts are \(1,4,5,10\). Estimate a common Poisson rate, calculate each expected count, and find the Pearson statistic. Explain why the raw count variance alone could mislead.
Method. Estimate the rate by total claims divided by total exposure. Then use each region’s own expected count in the Pearson contribution.
Total claims are \(1+4+5+10=20\), and total exposure is \(1+2+3+4=10\). Thus \(\hat{\lambda}=20/10=2\) claims per exposure unit. The expected counts are \(2(1)=2\), \(2(2)=4\), \(2(3)=6\), and \(2(4)=8\).
The four Pearson contributions are \((1-2)^2/2=1/2\), \((4-4)^2/4=0\), \((5-6)^2/6=1/6\), and \((10-8)^2/8=4/8=1/2\). Therefore \(Q=1/2+0+1/6+1/2=7/6\approx1.1667\).
The raw count mean is \(20/4=5\). Deviations from 5 are \(-4,-1,0,5\); their squared values sum to \(16+1+0+25=42\). The sample variance is \(42/3=14\), which is greater than the raw mean of 5. That comparison ignores the unequal exposures: the model itself expects larger counts and larger variances in higher-exposure regions. The exposure-adjusted statistic, rather than the raw variance-to-mean ratio, is the relevant descriptive comparison here.
Interpretation and check. The contributions sum to \(7/6\), and the observed counts sum to their expected counts, 20. The statistic does not show a large aggregate departure from the exposure-adjusted expectations. It does not prove the common-rate model is correct; the data comprise only four groups.
Common mistake avoided: Treating the four regional counts as equally exposed and interpreting their raw sample variance as a direct Poisson dispersion measure.
Reading Variation Across Groups
For equal-exposure groups, compare their average counts and their observed variances on a consistent basis. If the average count differs, the Poisson benchmark variance differs too. For unequal-exposure groups, use exposure-specific expected counts or organize the comparison among groups with similar exposures. A high observed variance-to-mean ratio can point to a pattern needing investigation, but it is not a diagnosis of the cause.
A useful adjustment for unequal exposures is the Pearson statistic. If the rate is estimated from the same data, its reference degrees of freedom are reduced by one because one rate parameter has been estimated. The approximate chi-square comparison becomes less reliable when expected counts are small, and no statistic removes the need to check whether the observations are independent and the exposure definition is appropriate.
In practical work, a single aggregate statistic can conceal which groups drive the result. Review the individual expected counts and contributions as well as the total. A large contribution could result from an unusually high or low count in one group; a repeated pattern across groups may be more informative than one isolated result. Neither pattern alone establishes an underlying explanation.
Worked Example: Comparing Observed Dispersion at a Common Mean
Two hypothetical books of business each have eight equal-exposure cells. Each book has an observed average of four claims per cell. Book A’s counts are \(2,3,3,4,4,5,5,6\), while Book B’s counts are \(0,1,2,3,5,6,7,8\). Calculate the sample variances and variance-to-mean ratios, then compare them with the Poisson benchmark.
Method. With equal exposures and the same observed mean, calculate squared deviations from four. Under a Poisson benchmark with mean four, the variance is also four.
For Book A, the total count is \(32\), giving mean \(32/8=4\). The squared deviations are \(4,1,1,0,0,1,1,4\), which sum to 12. Thus \(s_A^2=12/7\approx1.7143\), and \(s_A^2/\bar N_A=(12/7)/4=3/7\approx0.4286\).
For Book B, the total is also \(32\), so the mean is again 4. The squared deviations are \(16,9,4,1,1,4,9,16\), which sum to 60. Thus \(s_B^2=60/7\approx8.5714\), and \(s_B^2/\bar N_B=(60/7)/4=15/7\approx2.1429\).
Interpretation and check. Book A’s observed variance is below the Poisson benchmark of 4, while Book B’s is above it. Book B therefore shows more variation than the benchmark in this sample; further assessment would be needed before drawing a conclusion about its frequency model. Both totals and means agree, while the squared-deviation totals, 12 and 60, confirm the variance contrast.
Common mistake avoided: Concluding that equal means imply equal variation. The books have the same mean and exposure but markedly different observed sample variances.
Common Exam Traps
- Comparing raw count variances across different exposures. Larger exposure raises the Poisson mean and variance. Standardize the comparison using exposure-specific expected counts.
- Using the wrong variance denominator. The unbiased sample variance divides the sum of squared deviations by \(k-1\), not \(k\). If using the empirical variance with divisor \(k\), state that convention and do not quietly compare it with an unbiased estimate.
- Treating a sample ratio as an exact model test. A ratio above 1 can occur by chance, particularly with few groups. It signals a descriptive departure, not a certain conclusion.
- Forgetting rate estimation. When estimating one common rate from \(k\) groups, the usual approximate Pearson reference has \(k-1\), not \(k\), degrees of freedom.
- Ignoring the pattern behind a total. Inspect the group-level counts and expected values; the same overall statistic can arise from different group contributions.
Check Your Understanding
For each problem, compare observed variation with the appropriate Poisson benchmark. Use the supplied convention and round only where requested.
- Five equal-exposure cells have counts \(2,3,4,4,7\). Find the sample mean, unbiased sample variance, and variance-to-mean ratio. (Answer: mean \(=4\), sample variance \(=3.5\), ratio \(=0.8750\).)
- Three equal-exposure cells have counts \(3,5,7\). Find the sample mean and sample variance. Is the variance-to-mean ratio above or below 1? (Answer: mean \(=5\), sample variance \(=4\), ratio \(=0.8000\), below 1.)
- Two regions have exposures 2 and 5 units, and observed counts 3 and 9. Estimate a common Poisson rate and state the expected counts. (Answer: \(\hat{\lambda}=12/7\approx1.7143\) per unit; expected counts \(=24/7\approx3.4286\) and \(60/7\approx8.5714\).)
- Two equal-exposure groups each contain four cells. Group A has counts \(1,2,2,3\), and Group B has counts \(0,1,3,4\). Find each sample mean and sample variance, then compare the variance-to-mean ratios. (Answer: both means \(=2\); Group A variance \(=2/3\), ratio \(=1/3\); Group B variance \(=10/3\), ratio \(=5/3\).)
- Three regions have exposures \(1,2,3\) and counts \(1,2,6\). Estimate the common rate, find the expected counts, and calculate \(Q=\sum_j(N_j-\hat{\lambda}e_j)^2/(\hat{\lambda}e_j)\). (Answer: \(\hat{\lambda}=9/6=1.5\); expected counts \(1.5,3,4.5\); \(Q=1/6+1/3+1/2=1.0000\).)