Predictive Effects of Latent Risk Heterogeneity

Why Latent Rate Variation Changes a Count Forecast

Two risks can have the same estimated average claim rate and still differ in their predictive count distributions. If each risk has an unobserved underlying rate, then a forecast must account for uncertainty about that rate as well as the randomness of claims conditional on it. Ignoring the rate variation can leave the forecast mean unchanged while understating predictive variance and, for some thresholds, upper-tail probability.

The model below makes this distinction explicit. It is a frequency model: it describes claim counts over a stated exposure period, not claim severities or aggregate losses. The comparison is with a homogeneous-rate Poisson forecast that fixes the rate at the posterior mean. A larger variance is a direct consequence of latent rate uncertainty; the direction of a tail-probability difference must be calculated for the particular threshold.

Definition: Let $\Lambda$ be a risk's latent claim rate per unit of exposure. Assume $\Lambda\sim\operatorname{Gamma}(a,b)$, using shape $a>0$ and rate $b>0$, so $E[\Lambda]=a/b$ and $\operatorname{Var}(\Lambda)=a/b^2$. Conditional on $\Lambda$, the count $N$ over future exposure $t>0$ follows $\operatorname{Poisson}(t\Lambda)$. The rate is assumed constant over this period, and counts over disjoint periods are conditionally independent given $\Lambda$.

The Gamma distribution is a convenient mixing distribution because it is conjugate to the Poisson likelihood. For example, if the prior is $\operatorname{Gamma}(\alpha,\beta)$ and the observed count is $n$ over exposure $e$, then the posterior rate is $\operatorname{Gamma}(\alpha+n,\beta+e)$. Thus the predictive calculation can use $a=\alpha+n$ and $b=\beta+e$. When no individual history is available, $a,b$ can instead describe the population's prior distribution of rates.

The homogeneous comparison fixes $\Lambda$ at its predictive mean $a/b$. Its count forecast is therefore Poisson with mean $ta/b$. This is a useful benchmark, but it does not represent the full predictive distribution when the rate itself varies or remains uncertain.

Deriving the Predictive Count Distribution

To obtain the predictive distribution, average the conditional Poisson probability over the Gamma distribution of $\Lambda$. This integration is what introduces extra count variation: observations in a risk with a high latent rate are more likely to be large, and observations in a risk with a low latent rate are more likely to be small.

\begin{aligned} P(N=n) &=\int_0^\infty P(N=n\mid\Lambda=\lambda)f_\Lambda(\lambda)\,d\lambda\\ &=\int_0^\infty \frac{e^{-t\lambda}(t\lambda)^n}{n!} \frac{b^a}{\Gamma(a)}\lambda^{a-1}e^{-b\lambda}\,d\lambda\\ &=\frac{t^n b^a}{n!\Gamma(a)} \int_0^\infty \lambda^{a+n-1}e^{-(b+t)\lambda}\,d\lambda\\ &=\frac{\Gamma(a+n)}{\Gamma(a)n!} \left(\frac{b}{b+t}\right)^a \left(\frac{t}{b+t}\right)^n,\qquad n=0,1,2,\ldots . \end{aligned}

The last equality uses the Gamma integral $\int_0^\infty x^{c-1}e^{-rx}\,dx=\Gamma(c)/r^c$ for $c,r>0$. This is a negative-binomial distribution in the convention where $n$ counts failures before $a$ successes, with success probability $p=b/(b+t)$. The formula also applies when $a$ is not an integer, although the combinatorial coefficient is then expressed with Gamma functions.

Theorem: Under the stated Poisson-Gamma model, the predictive mean and variance for future count $N$ are $$ E[N]=\frac{ta}{b},\qquad \operatorname{Var}(N)=\frac{ta}{b}+\frac{t^2a}{b^2}. $$ The first variance term is conditional Poisson process variation averaged over the rate distribution. The second is the added variation due to uncertainty or heterogeneity in the latent rate.

The variance result follows from the law of total variance. Conditional on $\Lambda$, both the mean and variance of $N$ are $t\Lambda$. Therefore,

\begin{aligned} E[N]&=E[E[N\mid\Lambda]]=E[t\Lambda]=\frac{ta}{b},\\ \operatorname{Var}(N) &=E[\operatorname{Var}(N\mid\Lambda)] +\operatorname{Var}(E[N\mid\Lambda])\\ &=E[t\Lambda]+\operatorname{Var}(t\Lambda) =\frac{ta}{b}+\frac{t^2a}{b^2}. \end{aligned}

In contrast, the homogeneous Poisson model with the same mean has variance $ta/b$. The difference $t^2a/b^2$ is nonnegative and is positive whenever the latent rate has positive variance and future exposure is positive. As exposure increases, this additional term grows quadratically in $t$, while the process-variance term grows linearly.

Worked Comparisons

Worked Example: Updating a Rate Before Forecasting

A hypothetical portfolio segment has a prior annual claim rate $\Lambda\sim\operatorname{Gamma}(2,4)$, with the second parameter expressed as a rate. It records 1 claim over 3 years of exposure. Find the predictive mean and variance for a further 2 years, then compare the probability of at least 2 claims with the same-mean Poisson forecast.

Method and posterior. For a Poisson count observed over exposure $e$, the likelihood contributes $n$ to the Gamma shape and $e$ to the Gamma rate. The posterior is $\operatorname{Gamma}(2+1,4+3)=\operatorname{Gamma}(3,7)$. Thus $a=3$, $b=7$, and $t=2$.

Mean and variance. The predictive mean is $ta/b=2(3)/7=6/7=0.8571$. The variance is \[ \frac{6}{7}+\frac{2^2(3)}{7^2} =\frac{42}{49}+\frac{12}{49} =\frac{54}{49}=1.1020. \] A homogeneous Poisson forecast with mean $6/7$ has variance $6/7=0.8571$. The extra $12/49$ is the rate-uncertainty component.

Negative-binomial tail. Here $p=7/(7+2)=7/9$ and $1-p=2/9$. The first two probabilities are \[ P(N=0)=(7/9)^3=343/729=3087/6561, \] \[ P(N=1)=3(7/9)^3(2/9)=2058/6561. \] Consequently $P(N\ge2)=1-(3087+2058)/6561=1416/6561=0.2158$.

Same-mean Poisson tail and check. For a Poisson variable with mean $6/7$, \[ P(N\ge2)=1-e^{-6/7}\left(1+\frac67\right)=0.2119. \] The predictive negative-binomial tail is slightly larger, $0.2158$ versus $0.2119$, while both forecasts have mean $0.8571$. The method avoids the common mistake of treating the posterior mean rate as known; doing so drops the additional rate-uncertainty variance and produces the Poisson result.

Worked Example: Computing a Higher-Threshold Tail

For a hypothetical risk, suppose the predictive count distribution is negative binomial with shape $5$ and success probability $p=0.75$. Compare its probability of at least 5 claims with that from a Poisson model having the same mean.

Method and mean. Write $q=1-p=0.25$. For this negative-binomial convention, the mean is $5q/p=5/3=1.6667$. The same-mean homogeneous model is $\operatorname{Poisson}(5/3)$.

Negative-binomial probabilities. For $n=0,\ldots,4$, use $P(N=n)=\binom{4+n}{n}(0.75)^5(0.25)^n$. The calculations are \[ \begin{aligned} P(N=0)&=0.2373046875,\\ P(N=1)&=5(0.75)^5(0.25)=0.296630859375,\\ P(N=2)&=15(0.75)^5(0.25)^2=0.22247314453125,\\ P(N=3)&=35(0.75)^5(0.25)^3=0.1297760009765625,\\ P(N=4)&=70(0.75)^5(0.25)^4=0.06488800048828125. \end{aligned} \] Adding these gives $P(N\le4)=0.95107269287109375$, so $P(N\ge5)=0.04892730712890625$, or $0.0489$ to four decimal places.

Poisson comparison and check. For the Poisson model, \[ P(N\ge5)=1-e^{-5/3}\sum_{k=0}^{4}\frac{(5/3)^k}{k!} =1-e^{-5/3}\left(\frac{10009}{1944}\right) =0.0275. \] Thus the negative-binomial probability is higher at this threshold: $0.0489$ versus $0.0275$. The calculation checks by complementing the explicitly summed probabilities for counts 0 through 4. A common mistake is to substitute a rounded intermediate probability into the sum; retaining the displayed values gives the stated four-decimal result.

Worked Example: Greater Variance Does Not Order Every Tail

A risk has posterior latent rate $\Lambda\sim\operatorname{Gamma}(4,8)$ and a future exposure of $t=2$. Compare the probability of at least 3 claims with a Poisson forecast using the same predictive mean.

Method and moments. The mean is $ta/b=2(4)/8=1$, and the predictive variance is $1+2^2(4)/8^2=1.25$. The Poisson benchmark is therefore $\operatorname{Poisson}(1)$, with variance 1. In the mixture, $p=b/(b+t)=8/10=0.8$.

Mixture tail. The probabilities through count 2 are \[ P(N=0)=0.8^4=0.4096,\quad P(N=1)=4(0.8)^4(0.2)=0.32768, \] \[ P(N=2)=\binom{5}{2}(0.8)^4(0.2)^2=0.16384. \] Their sum is $0.90112$, so $P(N\ge3)=1-0.90112=0.09888$.

Poisson tail and check. The Poisson probability is \[ P(N\ge3)=1-e^{-1}\left(1+1+\frac12\right)=0.0803. \] At the threshold 3, the mixture tail is higher: $0.0989$ versus $0.0803$ when rounded to four decimals. The calculation illustrates a useful pattern, not a universal ordering: greater variance does not by itself prove that the mixture tail exceeds the Poisson tail at every possible threshold.

Interpretation, Scope, and Exam Traps

The model combines two sources of randomness. Conditional on a particular rate, the count varies because claims arrive randomly. Across risks—or across plausible rate values for an individual—the underlying rate also varies. In the predictive distribution, both sources contribute. The Gamma-Poisson mixture is therefore overdispersed relative to a same-mean Poisson distribution, meaning its variance exceeds its mean when the mixing rate has positive variance.

This variance comparison does not guarantee that every probability comparison has the same direction. A distribution with greater variance can have more probability near some counts and less near others. Calculate a requested tail from the negative-binomial probabilities or a suitable complement; do not infer it solely from the variance. In particular, a higher upper-tail probability for one threshold does not imply that all thresholds have higher upper tails.

Keep the parameterization straight. Here $b$ is a Gamma rate, not a scale. The posterior rate is prior rate plus exposure, and the predictive success probability is $b/(b+t)$.
Keep exposure units consistent. If $\Lambda$ is an annual rate, $t$ must be measured in years. For months, either convert exposure to years or define a monthly rate consistently.
Do not equate a latent rate with a known rate. Fixing $\Lambda=a/b$ gives a same-mean Poisson benchmark, but removes the variance term $t^2a/b^2$.
Distinguish a population mixture from individual posterior uncertainty. A prior Gamma distribution describes latent variation among risks before observing their histories. A posterior Gamma distribution describes remaining uncertainty for a risk after its own data are observed. The predictive mixture calculation is the same, but its interpretation and parameters differ.
Do not apply this model without its assumptions. The derivation assumes a Poisson count conditional on a constant rate and a Gamma mixing distribution. Changing the conditional frequency model, allowing the rate to vary over time, or using a different mixing distribution may produce another predictive distribution.

The comparison is useful for evaluating the consequences of pooling rates or making plug-in forecasts. It isolates the effect of unobserved rate variation while holding the expected count fixed. That makes the model a direct way to assess whether a homogeneous forecast may be too narrow for frequency-risk decisions. This treatment is aligned with the CAS Exam 8 (Advanced Ratemaking) syllabus.

Check Your Understanding

For each problem, use the stated Gamma rate parameterization and distinguish the mixture forecast from a same-mean Poisson forecast where requested.

A prior annual rate is $\operatorname{Gamma}(3,6)$. A risk has 2 claims over 2 years, followed by 1 year of forecast exposure. Find the posterior predictive mean and variance. (Answer: posterior $\operatorname{Gamma}(5,8)$; mean $0.625$, variance $0.703125$.)
A predictive rate is $\operatorname{Gamma}(2,5)$, with future exposure $t=3$. Find the predictive mean and variance, and the same-mean Poisson variance. (Answer: mean $1.2$, mixture variance $1.92$, Poisson variance $1.2$.)
A negative-binomial predictive count has shape $3$ and $p=0.6$. Find $P(N\ge2)$. (Answer: $0.5248$.)
Compare $P(N\ge4)$ for a negative-binomial distribution with shape $4$ and $p=2/3$ against a Poisson distribution with the same mean. (Answer: both means are $2$; negative-binomial probability is $379/2187=0.1733$, versus Poisson probability $0.1429$, to four decimals.)
In one or two sentences, explain why a larger predictive variance alone cannot determine whether $P(N\ge k)$ is larger for every integer threshold $k$. (Answer: Variance summarizes spread but does not order probabilities at each threshold. The tail comparison must be calculated for the particular $k$; probability can shift differently across count ranges.)

What You'll Learn