Diagnosing Prior-Data Conflict in Claim Models

What Prior-Data Conflict Means

A prior distribution describes uncertainty about model parameters before the current experience is observed. The prior predictive distribution describes what data summaries the model expects to see after accounting for that parameter uncertainty. Comparing the observed experience with this distribution can reveal whether the data are surprising under the prior model. This is a diagnostic, not a verdict: an unusual observation may indicate a poorly calibrated prior, an unsuitable likelihood, a data issue, or simply a rare event.

The comparison must respect the data-generating process. For claim counts, that means using the observed exposure; for a sample mean of claim amounts, it means using the sample size and the assumed variability of individual amounts. An observed count should not be compared directly with a rate, or a sample mean with the prior distribution of an individual claim.

Definition: Prior-data conflict is a substantial discrepancy between an observed data summary and the prior predictive distribution of that same summary, under the stated model and exposure. The posterior predictive distribution is different: it describes future or otherwise unobserved data after the prior has been updated using the observed data.

A standardized discrepancy can summarize how far an observation lies from a predictive mean. If $T$ is a statistic with prior predictive mean $E[T]$ and variance $\operatorname{Var}(T)$, its standardized value is

z_T=\frac{T_{\text{obs}}-E[T]}{\sqrt{\operatorname{Var}(T)}}.

This is a scale measure, not automatically a standard Normal test statistic. Predictive distributions can be discrete, skewed, or heavy-tailed. When possible, calculate a tail probability from the actual predictive distribution. A one-sided tail is appropriate when the concern is specifically unusually high or low experience; a two-sided measure is appropriate when departures in either direction matter. There is no universal cutoff at which a prior must be rejected.

Gamma-Poisson Diagnostics for Claim Counts

Suppose a risk has claim rate $\lambda$ per exposure unit, and its count $K$, conditional on $\lambda$, is Poisson with mean $e\lambda$, where $e$ is exposure. Assume the prior is Gamma with shape $\alpha$ and rate $\beta$, so its density is proportional to $\lambda^{\alpha-1}e^{-\beta\lambda}$. In this parameterization, $E[\lambda]=\alpha/\beta$ and $\operatorname{Var}(\lambda)=\alpha/\beta^2$.

Model result: Under $K\mid\lambda\sim\operatorname{Poisson}(e\lambda)$ and $\lambda\sim\operatorname{Gamma}(\alpha,\beta)$, the prior predictive count has mean $e\alpha/\beta$ and variance $e\alpha/\beta+e^2\alpha/\beta^2$. Its probability of zero claims is $(\beta/(\beta+e))^\alpha$. After observing $K=k$, the posterior is $\operatorname{Gamma}(\alpha+k,\beta+e)$.

The extra term in the predictive variance is important. Conditional on $\lambda$, the count variance is $e\lambda$. Across the prior, the law of total variance gives

\operatorname{Var}(K) =E[\operatorname{Var}(K\mid\lambda)] +\operatorname{Var}(E[K\mid\lambda]) =\frac{e\alpha}{\beta}+\frac{e^2\alpha}{\beta^2}.

The second term represents rate uncertainty. Ignoring it by treating the prior mean rate as known generally understates predictive variation and can make ordinary experience look more surprising than it is. The posterior formula follows because the Poisson likelihood contributes a factor proportional to $\lambda^k e^{-e\lambda}$; multiplying by the Gamma prior gives a Gamma kernel with shape $\alpha+k$ and rate $\beta+e$.

Worked Example: An Unexpectedly High Claim Count

For a hypothetical portfolio segment, let the annual claim rate per exposure unit have prior $\lambda\sim\operatorname{Gamma}(2,1)$, using the shape-rate convention. One exposure unit produces $K\mid\lambda\sim\operatorname{Poisson}(\lambda)$. The observed year has $K=8$. Diagnose the observation and calculate the posterior forecast for one additional exposure unit.

Method and prior predictive mean and variance. Here $e=1$, $\alpha=2$, and $\beta=1$. Therefore $E[K]=1(2/1)=2$, and $\operatorname{Var}(K)=2+1^2(2/1^2)=4$. The predictive standard deviation is $\sqrt{4}=2$. The observed count is $(8-2)/2=3$ predictive standard deviations above the mean.

Calculate the upper-tail probability. Integrating the Poisson probability over the Gamma prior gives $P(K=j)=(j+1)/2^{j+2}$ for $j=0,1,\ldots$. For this distribution, $P(K\ge m)=(m+2)/2^{m+1}$. Thus

P(K\ge 8)=\frac{10}{2^9} =\frac{10}{512} =0.01953125\approx 0.0195.

The count is unusual under this prior predictive model, though a tail probability of about 0.0195 is not proof that the prior or likelihood is wrong.

Update and forecast. The posterior is $\operatorname{Gamma}(2+8,1+1)=\operatorname{Gamma}(10,2)$. For one future exposure unit, the posterior predictive mean is $10/2=5$, and its variance is $5+10/2^2=7.5$. Its probability of zero claims is

P(K_{\text{new}}=0\mid K=8) =\left(\frac{2}{2+1}\right)^{10} =\left(\frac{2}{3}\right)^{10} =\frac{1024}{59049} \approx 0.0173.

The posterior expected future count is 5 rather than the prior expected count of 2. The data have shifted the forecast upward, while the predictive variance still includes uncertainty about the rate. A plug-in Poisson forecast using rate 5 would give a zero-count probability of $e^{-5}\approx0.0067$, which is smaller than the integrated predictive probability. The plug-in calculation omits parameter uncertainty. Common mistake avoided: do not use a Poisson distribution with the prior mean rate as if that rate were known when assessing prior predictive surprise.

Worked Example: An Unusually Quiet Exposure Period

For a hypothetical book of small commercial risks, assume $\lambda\sim\operatorname{Gamma}(3,2)$. The observed exposure is $e=4$, and no claims are reported. Calculate the prior predictive probability of this observation, update the rate, and forecast the expected count and probability of zero claims over a further exposure of $e_{\text{new}}=2$.

Prior diagnostic. The prior mean rate is $3/2=1.5$, so the expected count over four exposure units is $4(1.5)=6$. The probability of zero is

P(K=0)=\left(\frac{2}{2+4}\right)^3 =\left(\frac{1}{3}\right)^3 =\frac{1}{27} \approx 0.0370.

Zero claims are therefore in the lower tail of the prior predictive distribution. The observation is noteworthy under the assumed model, but one quiet period does not establish that the underlying rate is permanently low.

Posterior and future forecast. With $k=0$, the posterior is $\operatorname{Gamma}(3,2+4)=\operatorname{Gamma}(3,6)$. For the next two exposure units, the expected count is $2(3/6)=1$. The posterior predictive probability of no claims is

P(K_{\text{new}}=0\mid K=0) =\left(\frac{6}{6+2}\right)^3 =\left(\frac{3}{4}\right)^3 =\frac{27}{64} =0.421875\approx 0.4219.

Before observing the quiet period, the expected count for that future exposure was $2(3/2)=3$; after observing it, the posterior expected count is 1. The posterior predictive variance is $1+2^2(3/6^2)=1+\frac{1}{3}=\frac{4}{3}$. This variance is larger than the variance of a Poisson count with known rate $0.5$, which would be 1. Common mistake avoided: do not confuse the probability of the observed zero under the prior with the probability of a future zero after updating.

Diagnosing Conflict in a Severity Summary

The same reasoning applies to a severity model, provided the observed summary and its predictive distribution are matched. Consider a Normal model with known individual claim variance $\sigma^2$ and uncertain mean $\mu$: $Y_i\mid\mu\sim N(\mu,\sigma^2)$, independently, and $\mu\sim N(m_0,\tau_0^2)$. For $n$ observations, the prior predictive distribution of the sample mean is $N(m_0,\tau_0^2+\sigma^2/n)$. The $1/n$ term is sampling variation; the prior variance accounts for uncertainty in the mean.

Worked Example: Comparing a Claim Sample Mean With Its Prior

A hypothetical severity study observes two claims with sample mean $\bar{Y}=\$3{,}600$. Assume independent Normal claim amounts with known variance $\sigma^2=\$800{,}000$ per claim and prior $\mu\sim N(\$2{,}000,\$100{,}000)$, where the second Normal parameter is a variance. Diagnose the sample mean and calculate the posterior mean and the predictive distribution for one future claim.

Prior predictive comparison. The prior predictive mean of $\bar{Y}$ is $\$2{,}000$, and its variance is $\$100{,}000+\$800{,}000/2=\$500{,}000$. The predictive standard deviation is $\sqrt{\$500{,}000}\approx \$707.11$. Thus $z=(\$3{,}600-\$2{,}000)/\$707.11\approx2.2627$. Under the stated Normal model, the two-sided tail probability is approximately $0.02365$, or $0.0237$ to four decimal places. This is evidence of a relatively unusual sample mean under the prior predictive distribution.

Posterior update. Normal conjugacy gives posterior variance

\tau_1^2 =\left(\frac{1}{100{,}000}+\frac{2}{800{,}000}\right)^{-1} =80{,}000.

The posterior mean is the precision-weighted average

m_1 =80{,}000\left(\frac{2{,}000}{100{,}000} +\frac{2(3{,}600)}{800{,}000}\right) =80{,}000(0.0200+0.0090) =\$2{,}320.

For one future claim, integrating over the posterior uncertainty in $\mu$ gives a Normal predictive distribution with mean $\$2{,}320$ and variance $\$800{,}000+\$80{,}000=\$880{,}000$. Its standard deviation is approximately $\$938.08$. The observation moves the expected severity upward from the prior mean, but the posterior forecast does not simply set the mean equal to the observed sample mean: it accounts for both the prior information and the sampling uncertainty. Common mistake avoided: use the variance of the sample mean, $\sigma^2/n$, when checking $\bar{Y}$; do not compare it with the variability of an individual claim.

A Normal severity model permits negative amounts, so it may be only an approximation for claim sizes. If its support or tail behavior is implausible for the application, the apparent conflict may reflect model misspecification rather than an unsuitable prior alone.

Interpreting the Diagnostic Carefully

A conflict diagnosis concerns the joint model: prior, likelihood, exposure, and data quality. Before changing a prior, check that the exposure unit was applied correctly, claims were counted over the intended period, and the summary was calculated as specified. Then ask whether the likelihood captures relevant features such as changing conditions or differences among risks. A count model that assumes one constant rate may be inadequate when risks have materially different rates.

A tail probability is calculated under the model, not a probability that the prior is false. It is also sensitive to which summary and tail were selected. If many summaries are examined, one may look unusual by chance. Record the diagnostic that was relevant to the modeling question rather than treating every small tail probability as decisive.

The practical consequence of conflict is that posterior forecasts can depend strongly on the observed experience and on how much prior information the model assigns. A high count may raise a future frequency forecast; a quiet period may lower it. The size of the change is governed by the posterior update, not by the tail probability alone. Communicate both the prior predictive comparison and the updated forecast, along with the assumptions that support them.

Match the summary to its predictive distribution.
Use the same statistic, exposure, and sampling process as the observation.

Quantify the discrepancy.
Use a standardized difference for scale and, where appropriate, a tail probability from the full predictive distribution.

Update and interpret the forecast.
Calculate the posterior and the future predictive distribution; do not equate a surprising observation with proof that the prior is wrong.

Key takeaway: Diagnose prior-data conflict by asking how plausible the observed summary was under the prior predictive distribution. Then update the model and report the resulting future forecast with its uncertainty. A surprising observation prompts investigation; it does not identify the cause by itself.

Check Your Understanding

For each problem, identify the relevant predictive quantity before calculating.

A claim rate has prior $\lambda\sim\operatorname{Gamma}(1,1)$, shape-rate parameterization. For exposure $e=2$, the observed count is 5. Find the prior predictive mean, variance, and upper-tail probability $P(K\ge5)$. (Answer: Mean $2$, variance $4$, and $P(K\ge5)=(2/3)^5=0.1317$.)
Assume $\lambda\sim\operatorname{Gamma}(4,2)$, and observe zero claims over exposure $e=1$. Find the prior predictive probability of zero and the posterior expected count over future exposure $e=3$. (Answer: Prior probability $P(K=0)=(2/3)^4=0.1975$; posterior is $\operatorname{Gamma}(4,3)$, giving future expected count $3(4/3)=4$.)
Let $Y_i\mid\mu\sim N(\mu,160{,}000)$ independently and $\mu\sim N(1{,}000,40{,}000)$. For $n=4$ and observed $\bar{Y}=1{,}300$, find the two-sided prior predictive tail probability and posterior mean. (Answer: Predictive variance $80{,}000$, $z\approx1.0607$, two-sided tail probability $\approx0.2888$; posterior mean $1{,}150$.)
For $\lambda\sim\operatorname{Gamma}(2,4)$, observe 3 claims over exposure 2. Find the posterior expected count and predictive variance over a further exposure of 2. (Answer: Posterior $\operatorname{Gamma}(5,6)$; future mean $10/6=1.6667$ and variance $10/6+4(5/36)=20/9\approx2.2222$.)
A prior predictive analysis gives a two-sided tail probability of 0.01 for an observed claim summary. State what this does and does not establish. (Answer: The summary is unusual under the specified prior and likelihood; it does not establish that the prior is false or identify whether the prior, likelihood, data, or chance explains the discrepancy.)

What You'll Learn