Why Check a Claim-Count Model?
A claim-count model can produce a plausible average while missing important features of the observed data. For example, it may predict the total number of claims reasonably well but predict too few zero-claim risks, too little variation between risks, or no counts as large as those observed. A posterior predictive check asks whether data generated under the fitted model resemble the observed data in features that matter for the intended application.
The check does not establish that a model is true. It can identify mismatches between the model and the data, conditional on the model, prior, and chosen diagnostic. This tutorial uses a Poisson model with a common uncertain claim rate to show the calculations. The same checking framework can be applied to richer count models, provided the replicated data are generated under the fitted model and the relevant exposures and covariates are held consistent.
Poisson-Gamma Model and Replicated Counts
Suppose risks have exposures \(e_1,\ldots,e_n\), measured in consistent exposure units. Conditional on a common rate \(\lambda\), assume the counts are independent and \(N_i\mid\lambda\sim\operatorname{Poisson}(e_i\lambda)\). The Poisson assumption makes the conditional mean and variance of \(N_i\) both equal to \(e_i\lambda\). A Gamma prior with shape \(a\) and rate \(b\), written \(\lambda\sim\operatorname{Gamma}(a,b)\), has density proportional to \(\lambda^{a-1}e^{-b\lambda}\). In this parameterization, \(E[\lambda]=a/b\).
Given observed counts \(n_1,\ldots,n_n\), the posterior is \(\operatorname{Gamma}(a',b')\), where \(a'=a+\sum_i n_i\) and \(b'=b+\sum_i e_i\). A replicated data set uses the same exposures as the observed data, but its counts are newly generated from the posterior predictive distribution. For a single future count at exposure \(e\), integrating over the posterior gives the predictive probability
In particular, the predictive probability of zero claims is \((b'/(b'+e))^{a'}\). For several risks, the posterior predictive counts are generally dependent: they share the same uncertain rate drawn from the posterior. Therefore, multiplying marginal predictive probabilities as if each risk had an independently drawn rate is usually incorrect.
How a Posterior Predictive Check Works
Choose a discrepancy that targets a feature of concern, calculate it for the observed data, and compare it with the same discrepancy calculated for replicated data. Conceptually, the posterior predictive distribution for the discrepancy is obtained by repeating the following steps:
Use the observed counts, exposures, and prior to obtain the posterior distribution of the model parameters.
Draw a parameter value from its posterior, then generate a complete replicated data set at the observed exposures.
Calculate \(T(y)\) for the observed data and \(T(Y^{\mathrm{rep}})\) for each replicate. Examine the reference distribution and where the observed value falls.
A posterior predictive tail probability can summarize one comparison, for example \(P(T(Y^{\mathrm{rep}})\ge T(y)\mid y)\). It is not an ordinary frequentist p-value: the data are used both to fit the posterior and to define the observed discrepancy. Values near zero or one can flag a mismatch, but no universal cutoff establishes model adequacy. Graphs and several substantively chosen discrepancies are often more informative than a single number.
Worked Example: Checking the Number of Zero-Claim Risks
A hypothetical portfolio has two risks with exposures \(0.5\) and \(2\), and observed counts \(0\) and \(2\). Assume conditional independent Poisson counts with common rate \(\lambda\), and prior \(\lambda\sim\operatorname{Gamma}(1,1)\). Find the posterior and the posterior predictive expected number of zero-count risks when replicating these two risks at the same exposures.
Method. Use the zero-count indicator as the discrepancy. The observed data contain one zero, so \(T(y)=1\). The posterior shape is \(1+(0+2)=3\), and its rate is \(1+(0.5+2)=3.5\). Thus \(\lambda\mid y\sim\operatorname{Gamma}(3,3.5)\).
For exposure \(e\), the predictive probability of zero is \(E[e^{-e\lambda}\mid y]=(3.5/(3.5+e))^3\). At exposure \(0.5\), this is \((3.5/4)^3=(7/8)^3=343/512\approx0.6699\). At exposure \(2\), it is \((3.5/5.5)^3=(7/11)^3=343/1331\approx0.2577\).
The number of zeros is the sum of the two zero-count indicators. By linearity of expectation, its predictive expectation is the sum of the individual zero probabilities: \(343/512+343/1331\approx0.9276\) zero-count risks. The expected count is slightly below the observed count of one.
Check and common mistake. The posterior rate is \(3.5\), not \(1\): the total exposure, \(2.5\), must be added to the prior rate. The posterior predictive zero probabilities are therefore \((7/8)^3\) and \((7/11)^3\), not probabilities calculated from the prior. The result, about \(0.9276\), is an expectation; it is not a posterior predictive tail probability for observing at least one zero.
Zero Frequency, Variance, and Maximum Count
The zero-count frequency is \(Z(y)=\sum_i \mathbf{1}(n_i=0)\). It can reveal a mismatch when a model produces too many or too few claim-free risks, even if its predicted total is reasonable. Its expected replicated value is the sum of the predictive zero probabilities across risks. The indicators need not be independent for this expectation to hold.
The sample variance can reveal whether the model reproduces the spread of counts. With equal exposures and one common Poisson rate, the replicated counts are identically distributed conditional on \(\lambda\), and their conditional sample variance has expectation \(e\lambda\). Across posterior replicates, the expected sample variance is therefore \(eE[\lambda\mid y]\). This expectation is a useful reference point, but it is not a substitute for examining the full replicated distribution.
The maximum count, \(M(y)=\max_i n_i\), focuses on the largest observed count. It can identify a model that fails to allow sufficiently large counts. Unlike a calculation based on independent marginal predictive distributions, an exact maximum-count calculation must account for the shared posterior rate. For a fixed posterior draw, replicated counts are conditionally independent; after integrating over that common draw, they are dependent.
Worked Example: Assessing the Maximum Count
A hypothetical two-risk dataset has exposures \(0.5\) and \(0.5\), and observed counts \(1\) and \(0\). With a \(\operatorname{Gamma}(1,1)\) prior and the common-rate Poisson model, find the posterior predictive probability that the replicated maximum is at least the observed maximum.
Method. The observed total count is \(1\) and total exposure is \(1\), so the posterior is \(\operatorname{Gamma}(2,2)\). The observed maximum is \(M(y)=1\). The event \(M(Y^{\mathrm{rep}})\ge1\) is the complement of both replicated counts being zero.
Conditional on \(\lambda\), the probability both counts are zero is \(e^{-0.5\lambda}e^{-0.5\lambda}=e^{-\lambda}\). Integrating over the posterior gives \(P(M(Y^{\mathrm{rep}})=0\mid y)=E[e^{-\lambda}\mid y]=(2/(2+1))^2=4/9\). Hence \(P(M(Y^{\mathrm{rep}})\ge1\mid y)=1-4/9=5/9\approx0.5556\).
As an additional check, the probability that the replicated maximum is at least \(2\) is \(5/27\approx0.1852\). To obtain it, the probability that both counts are at most \(1\), conditional on \(\lambda\), is \(e^{-\lambda}(1+\lambda/2)^2\). Its posterior expectation is \(4/9+8/27+2/27=22/27\), so the complementary probability is \(5/27\).
Check and common mistake. The posterior uses the observed total exposure of \(1\), giving rate \(2\). It is incorrect to draw a new independent \(\lambda\) for each risk and multiply marginal probabilities; both risks share one rate in each replicated dataset. The calculation also confirms that a maximum of at least one is not unusual here: its posterior predictive probability is \(0.5556\).
Worked Example: Comparing Observed and Replicated Sample Variance
A hypothetical group has four unit-exposure risks with observed counts \(0,1,5,6\). Assume a common Poisson rate and prior \(\lambda\sim\operatorname{Gamma}(2,3)\). Calculate the posterior expected sample variance of a replicated dataset and compare it with the observed sample variance.
Method. The observed total is \(12\), and the total exposure is \(4\). Thus the posterior is \(\operatorname{Gamma}(2+12,3+4)=\operatorname{Gamma}(14,7)\), so \(E[\lambda\mid y]=14/7=2\).
The observed mean is \((0+1+5+6)/4=3\). The sum of squared deviations is \((0-3)^2+(1-3)^2+(5-3)^2+(6-3)^2=9+4+4+9=26\). Therefore the observed sample variance is \(s^2(y)=26/(4-1)=26/3\approx8.6667\).
For each posterior draw of \(\lambda\), the four replicated counts are independent Poisson\((\lambda)\) variables. Their unbiased sample variance has conditional expectation \(\lambda\). Averaging over the posterior gives \(E[s^2(Y^{\mathrm{rep}})\mid y]=E[\lambda\mid y]=2\). The observed variance, approximately \(8.6667\), is substantially larger than this predictive expectation.
Check and common mistake. The sample variance divides by \(n-1=3\), not by \(n=4\). Also, \(2\) is the expected replicated sample variance, not the variance of an individual count after integrating over \(\lambda\). A formal assessment should compare \(26/3\) with the distribution of replicated sample variances, not conclude from the expectation alone that a particular tail probability has been established.
Interpreting Checks Carefully
The usefulness of a check depends on whether its statistic reflects a meaningful feature of the count data. Zero frequency is relevant when claim-free risks affect pricing or operations; variance matters when the model must represent dispersion; and the maximum can be important when unusually high counts have material consequences. A check can also be performed within exposure or rating groups, provided the replication preserves the design being assessed.
A common exam trap is to confuse a posterior predictive expectation with a probability of a model failure. For instance, an expected zero count of \(0.93\) does not mean there is a \(93\%\) chance the model is adequate. Another trap is to hold the posterior rate fixed at its mean and generate all replicates from that single value. That is a plug-in check, not a posterior predictive check: it omits uncertainty in \(\lambda\), which can materially affect the spread and dependence of replicated counts.
The posterior predictive construction explains why that uncertainty belongs in the check:
Each possible rate is weighted by its posterior probability, and the replicated-data distribution averages over those rates. This is the reason a posterior predictive check evaluates the fitted model together with the uncertainty remaining after observing the data. Checks can motivate revisiting assumptions or considering a different model, but model selection should also consider the intended use, stability, and substantive constraints.
Check Your Understanding
For each problem, use a Gamma distribution parameterized by shape and rate. Give rates in claims per exposure unit.
- A common-rate Poisson model has prior \(\lambda\sim\operatorname{Gamma}(2,1)\). An observed count total of \(3\) arises over total exposure \(2\). Find the posterior predictive expected number of zero counts among two new risks, each with exposure \(0.5\). (Answer: \(2(6/7)^5\approx0.9253\).)
- A fitted model has posterior \(\lambda\mid y\sim\operatorname{Gamma}(3,4)\). For three replicated risks with exposures \(0.5,1,2\), find the expected number of zero-count risks. (Answer: \((8/9)^3+(4/5)^3+(2/3)^3\approx1.5106\).)
- Four equal-exposure replicated counts follow a common-rate Poisson model with unit exposure per risk, and the fitted posterior is \(\operatorname{Gamma}(5,4)\). What is the posterior predictive expected unbiased sample variance? (Answer: \(5/4=1.25\).)
- Two observed risks have exposures \(0.5,0.5\) and counts \(1,0\). Under prior \(\operatorname{Gamma}(1,1)\) and the common-rate Poisson model, what is the posterior predictive probability that the replicated maximum is at least the observed maximum? (Answer: \(5/9\approx0.5556\).)
- A fitted common-rate model has posterior \(\operatorname{Gamma}(3,5)\). For one future risk with exposure \(4\), find the predictive expected count and predictive probability of zero claims. (Answer: expected count \(12/5=2.4\); zero probability \((5/9)^3=125/729\approx0.1715\).)