Why Fit and Diagnose Frequency Models?
A frequency distribution is useful only when its parameters and assumptions are credible for the event units and exposure being modeled. For operational losses, event counts may differ across periods because exposure differs, because the underlying event rate changes, or because a chosen distribution does not describe the observed variation. A fitted average rate alone cannot distinguish these explanations.
The fitting step estimates parameters from observed counts and exposure. Diagnosis then asks whether the fitted model reproduces important features of the data, such as the count level, differences among groups, and behavior across time. These checks are evidence about model adequacy, not automatic proof that a model is correct or incorrect. The methods below are aligned with the CAS Exam 9 (Risk Management for Actuaries) syllabus.
Maximum Likelihood with Exposure
For independent Poisson counts, the likelihood is the product of each period's Poisson probability. Its logarithm, after omitting terms that do not depend on \(\lambda\), is especially simple:
Differentiating with respect to \(\lambda\), setting the derivative to zero, and solving gives the maximum-likelihood estimate:
Thus the estimate is total events divided by total exposure, not the unweighted average of period-specific rates. Exposure belongs in the expected count, \(\widehat{\mu}_j=\widehat{\lambda}e_j\). If exposure is measured in a different unit, the numerical rate changes accordingly; it is essential to state the unit.
Worked Example: Fit a Poisson Rate Across Unequal Exposures
A hypothetical operational unit records independent event counts of 1, 0, and 2 over exposures of 0.5, 1, and 1.5 system-years, respectively. Fit the exposure-adjusted Poisson rate and calculate the Pearson goodness-of-fit statistic. Use the Pearson statistic as a descriptive diagnostic, not as a definitive test in this small sample.
Method and estimate. The total count is \(1+0+2=3\), and total exposure is \(0.5+1+1.5=3\) system-years. Therefore, \(\widehat{\lambda}=3/3=1.0000\) event per system-year. The fitted expected counts are \(0.5, 1,\) and \(1.5\), respectively.
Pearson calculation. For observed count \(N_j\) and fitted mean \(\widehat{\mu}_j\), the contribution is \((N_j-\widehat{\mu}_j)^2/\widehat{\mu}_j\). The three contributions are \((1-0.5)^2/0.5=0.5\), \((0-1)^2/1=1\), and \((2-1.5)^2/1.5=0.1667\). Their sum is \(X^2=0.5+1+0.1667=1.6667\), rounded to four decimal places.
Check and trap avoided. The fitted means sum to \(0.5+1+1.5=3\), matching the total observed count because the rate was estimated from the same exposures. The zero-count period contributes 1, not zero: its observed count differs from its fitted mean of 1. Omitting that contribution would give an incorrect statistic of 0.6667.
Comparing Nested Frequency Models
A likelihood comparison can assess whether a more flexible model improves fit enough to justify its additional parameters. For nested models, the likelihood-ratio statistic is twice the difference between the maximized log-likelihoods. When regularity conditions hold and the simpler model is true, this statistic is approximately chi-squared with degrees of freedom equal to the difference in the number of free parameters. This approximation can be unreliable with sparse counts, parameters on boundaries, or very small samples.
For the Poisson model, the deviance against a saturated model (one that assigns a separate fitted mean to each observation) is \(D=2\sum_j[N_j\log(N_j/\widehat{\mu}_j)-(N_j-\widehat{\mu}_j)]\), with the \(N_j\log(N_j/\widehat{\mu}_j)\) term defined as zero when \(N_j=0\). A larger deviance indicates greater discrepancy from the saturated fit, but its reference distribution is an approximation and should be used cautiously when expected counts are small.
Worked Example: Test Whether Two Groups Need Different Rates
In a hypothetical control-process study, group A has 8 events over 4 system-years and group B has 3 events over 6 system-years. Compare a common-rate Poisson model with a model allowing a separate rate for each group. Assume independent Poisson counts and fixed exposures.
Method. Under the common-rate model, total events are \(8+3=11\), and total exposure is \(4+6=10\) system-years. The estimate is \(\widehat{\lambda}=11/10=1.1\). The fitted means are \(4(1.1)=4.4\) for A and \(6(1.1)=6.6\) for B.
Under the separate-rate model, the estimates are \(\widehat{\lambda}_A=8/4=2\) and \(\widehat{\lambda}_B=3/6=0.5\). The separate-rate model has one additional parameter. The likelihood-ratio statistic simplifies to twice the sum of observed counts times the log of the separate fitted mean divided by the common fitted mean:
The log terms are \(\log(2/1.1)=0.5978\) and \(\log(0.5/1.1)=-0.7885\). Hence \(G^2=2[8(0.5978)+3(-0.7885)] =2[4.7824-2.3655]=4.8338\) using the displayed rounded log values. Using unrounded logs gives \(G^2=4.8347\), or 4.835 to three decimals.
Interpretation and check. The approximate reference distribution is chi-squared with one degree of freedom, giving a p-value of about 0.0279. This is evidence against a common rate under the stated model, not proof that the groups' underlying rates differ for a causal reason. The fitted total expected count under the common model is \(4.4+6.6=11\), as required. A common mistake is to compare raw counts, 8 versus 3, without accounting for the unequal exposures; the estimated rates, 2 versus 0.5 per system-year, are the relevant comparison.
Residuals and Dispersion
Residuals show how each observed count departs from its fitted value. For a Poisson fit, a Pearson residual is \(r_j=(N_j-\widehat{\mu}_j)/\sqrt{\widehat{\mu}_j}\), so its square is that observation's contribution to \(X^2\). A deviance residual is the signed square root of the corresponding deviance contribution. The sign indicates whether the observed count is above or below its fitted mean; the magnitude indicates the size of the discrepancy relative to the model's scale.
Plotting residuals against time, exposure, or a relevant group can reveal structure that a single overall statistic conceals. Runs of positive residuals may indicate a rising rate or an omitted factor. Large residuals in high-exposure periods may point to a changing event process, data-quality issue, or inadequate distributional assumption. A pattern is a prompt for investigation, not a standalone diagnosis: operational event counts are noisy, and a few extreme counts can occur by chance.
The Pearson dispersion estimate is often calculated as \(X^2/(m-p)\), where \(m\) is the number of independent observations and \(p\) is the number of fitted parameters. This is a rough diagnostic, particularly when counts are small or fitted means vary widely. A value materially above 1 can indicate overdispersion relative to the Poisson assumption; a value materially below 1 can indicate underdispersion. Neither result identifies the cause. Dependence, omitted rate variation, aggregation choices, or data problems can affect the statistic.
Worked Example: Diagnose Dispersion and Check a Later Period
A hypothetical operation observes counts \(0,1,0,5,1,5\) in six consecutive one-system-year periods. Fit a constant-rate Poisson model to all six periods, calculate the Pearson dispersion estimate, and then perform a simple time-based check using the first four periods as training data and the final two as holdout data.
Full-period fit. The total count is \(0+1+0+5+1+5=12\), so the fitted rate is \(12/6=2\) events per system-year. Every fitted mean is 2. The Pearson contributions for counts 0, 1, and 5 are, respectively, \((0-2)^2/2=2\), \((1-2)^2/2=0.5\), and \((5-2)^2/2=4.5\). Each occurs twice, so \(X^2=2(2+0.5+4.5)=14\).
One rate parameter was estimated, so the nominal residual degrees of freedom are \(6-1=5\). The dispersion estimate is \(14/5=2.8\). This is a substantial indication of more variation than the fitted Poisson model expects, but with only six periods it is not conclusive evidence. The counts also show a visible pattern of low and high observations; temporal structure merits investigation rather than being summarized away.
Time-based check. The first four periods contain \(0+1+0+5=6\) events over 4 system-years, giving a training estimate of \(1.5\) events per system-year. The last two periods have 2 system-years of exposure, so the model predicts an expected total of \(1.5(2)=3\) events. The holdout count is \(1+5=6\). Under the fixed training estimate, the holdout total has a Poisson distribution with mean 3; its upper-tail probability is \(P(N\ge6)=1-e^{-3}\sum_{k=0}^{5}3^k/k!=0.0839\).
Interpretation and check. The holdout total is high relative to the forecast but not particularly rare at a conventional 5% threshold. It is one check, not a reason by itself to accept the model. The fit uses the first four periods only for forecasting, so the holdout observations have not leaked into the estimate. Refitting on all six periods before judging those same six periods would not be a genuine out-of-sample validation.
Validation Choices and Common Traps
Out-of-sample validation evaluates predictions on observations not used to estimate the model. A time-based split is often more relevant than a random split when the intended use is forecasting future operational losses. The training period should precede the validation period, exposure definitions should be consistent, and any model selection should be completed without using holdout results. If many alternatives are repeatedly tested against the same holdout, that holdout gradually becomes part of the modeling process.
A sound diagnosis combines several views rather than relying on one p-value. Likelihood comparisons address whether additional model structure improves fit; residuals help locate where discrepancies arise; dispersion checks summarize extra variability; and validation examines predictive performance on new or later data. These methods do not replace judgment about event-unit definitions, exposure measurement, or changes in business processes. When those foundations shift, a statistical fit to the combined history can be misleading.
- Ignoring exposure. Compare rates or use exposure offsets, not counts alone, when observation periods differ in size.
- Treating a small-sample chi-squared approximation as exact. Sparse expected counts can make likelihood-ratio and goodness-of-fit reference distributions inaccurate.
- Reading overdispersion as a unique diagnosis. A high Pearson ratio flags a mismatch with Poisson variance; it does not establish whether dependence, omitted heterogeneity, or data issues caused it.
- Validating on fitted data. In-sample fit measures describe the data used to estimate parameters and can understate forecast error.
Check Your Understanding
For each problem, assume independent Poisson counts with fixed exposure unless stated otherwise. Round rates and statistics to four decimal places and probabilities to four decimal places.
- Two periods have counts 2 and 3, with exposures of 1 and 2 system-years. Find the maximum-likelihood rate and the Pearson statistic using fitted means. (Answer: \(\widehat{\lambda}=1.6667\) per system-year; \(X^2=0.1000\).)
- Two periods have counts 0 and 4, with exposures 1 and 3 system-years. The common-rate estimate is 1 event per system-year. Calculate the Poisson deviance against the saturated model. (Answer: \(D=2.3015\).)
- Group A has 6 events over 3 system-years and group B has 2 events over 4 system-years. Compare a common rate with separate group rates using a likelihood-ratio statistic. Give its degrees of freedom and approximate p-value. (Answer: \(G^2=3.4087\), 1 degree of freedom, \(p\approx0.0648\).)
- Four equal-exposure periods have counts \(1,2,3,2\). Fit a common Poisson rate, calculate the Pearson statistic, and calculate its dispersion estimate using \(m-p\) degrees of freedom. (Answer: \(\widehat{\lambda}=2.0000\) per period-exposure unit; \(X^2=1.0000\); dispersion estimate \(=0.3333\).)
- A rate of 2 events per year is estimated from training data and held fixed. For a two-year holdout period, 8 events are observed. Under the fitted model, calculate the predicted mean and the probability of at least 8 events. (Answer: predicted mean \(=4\); \(P(N\ge8)\approx0.0511\).)