Two Sources of Predictive Uncertainty
A forecast for a risk’s future claims is uncertain for two different reasons. Even if the risk’s expected claim level were known, future claims would vary around it. In addition, the expected claim level itself may be uncertain because it is inferred from limited experience. A useful comparison of Bayesian and classical credibility keeps these sources separate.
The Bayesian calculation is conditional on the observed data: it averages process variation over the posterior distribution of the risk parameter and adds posterior uncertainty about the risk’s mean. Classical Bühlmann credibility instead begins with population-level variance components and gives a linear estimate of the risk mean. Its estimation error is generally an average over the assumed population of risks and possible data, not a posterior variance for a particular observed risk.
The formula requires finite variances and a future observation that is conditionally independent of the observed data given \(\Theta\). If observations have different exposures, their conditional means and variances must reflect those exposures. The formula still applies, but the components must be calculated for the actual future quantity being predicted.
Why the Bayesian Decomposition Holds
This is an application of the law of total variance. Conditional on \(\Theta\) and the data, the future observation has mean \(m(\Theta)\) and variance \(v(\Theta)\). The observed data update our distribution for \(\Theta\), so both quantities are averaged over the posterior.
Given \(\Theta\) and \(D\), the future observation has variance \(v(\Theta)\) and mean \(m(\Theta)\), by conditional independence.
The posterior average process variance is \(E[\operatorname{Var}(X_{n+1}\mid\Theta,D)\mid D]=E[v(\Theta)\mid D]\).
The posterior variation among the possible values of \(m(\Theta)\) is \(\operatorname{Var}(E[X_{n+1}\mid\Theta,D]\mid D)=\operatorname{Var}(m(\Theta)\mid D)\).
The law of total variance gives the displayed predictive variance. The first term is variation around a given risk mean; the second is variation in that mean across posterior possibilities.
This conditional decomposition differs from a population-level decomposition. Under the classical Bühlmann model, define the expected process variance and variance of hypothetical means by
The law of total variance gives \(\operatorname{Var}(X_j)=\mathrm{EPV}+\mathrm{VHM}\). These are prior or population quantities. They do not, by themselves, specify the posterior variance for one particular set of observed claims.
Classical Bühlmann Credibility and Estimation Error
For \(n\) equal-exposure observations satisfying the Bühlmann assumptions, the classical credibility estimate of the risk’s expected claim level is
where \(\bar X\) is the observed average and \(\mu=E[m(\Theta)]\) is the collective mean. This formula assumes finite, positive EPV and VHM, equal exposures, and conditionally independent observations with common conditional mean and variance. If VHM is zero, there is no variation in risk means to learn about; the displayed definition of \(K\) does not apply.
A classical measure of how well the credibility estimate recovers the latent risk mean is its mean squared error across risks and samples. Under these assumptions,
To justify the first equality, use \(\operatorname{Var}(\bar X)=\mathrm{VHM}+\mathrm{EPV}/n\) and \(\operatorname{Cov}(m(\Theta),\bar X)=\mathrm{VHM}\). The credibility estimator is unbiased for \(m(\Theta)\) when averaged over the risk population and data, so its mean squared error is the variance of \(m(\Theta)-\widehat m_n\). Expanding that variance gives the first expression. Substituting \(Z=n/(n+K)\) and \(K=\mathrm{EPV}/\mathrm{VHM}\) gives the second.
For predicting one additional observation with \(\widehat m_n\), the population-averaged squared prediction error is \(\mathrm{EPV}+E[(m(\Theta)-\widehat m_n)^2]\). Conditional independence makes the cross term zero: given \(\Theta\) and the past data, the future process deviation has mean zero. This is a useful classical error measure, but it is not generally the Bayesian predictive variance conditional on the actual observed data.
Worked Example: Matching Bayesian and Classical Results in a Normal Model
Suppose a hypothetical portfolio has risk mean \(\Theta\sim N(10,4)\), where the second parameter is the variance, and annual observations satisfy \(X_j\mid\Theta\sim N(\Theta,9)\), independently. Four observed annual amounts have average \(\bar X=12\). Find the posterior predictive variance for one future annual amount and compare it with the classical Bühlmann quantities.
Method. The normal prior and normal observations give a normal posterior. The posterior precision is prior precision plus data precision: $$ \frac{1}{4}+\frac{4}{9} =\frac{9}{36}+\frac{16}{36} =\frac{25}{36}. $$ Thus the posterior variance of \(\Theta\) is \(36/25=1.44\).
The posterior mean is the precision-weighted average: $$ E[\Theta\mid D] = \frac{10/4+4(12)/9}{25/36} = \frac{5/2+16/3}{25/36} = 11.28. $$
Conditional process variance is 9 for every value of \(\Theta\). Therefore the posterior predictive variance is $$ E[9\mid D]+\operatorname{Var}(\Theta\mid D) =9+1.44 =10.44. $$ The future predictive mean is also 11.28.
For the classical calculation, \(\mathrm{EPV}=9\), \(\mathrm{VHM}=4\), and \(K=9/4=2.25\). Hence \(Z=4/(4+2.25)=0.64\), and $$ \widehat m_4=0.64(12)+0.36(10)=7.68+3.60=11.28. $$ The classical mean squared error for the latent mean is \(9/(4+2.25)=1.44\). Adding EPV gives \(9+1.44=10.44\), the same forecast error variance in this normal model. The agreement occurs because this conjugate normal model’s posterior variance equals the corresponding classical credibility estimation error. It is not a general identity for every observed data set and model.
Check. The posterior mean equals the Bühlmann estimate, and both uncertainty calculations total 10.44. No rounding was needed.
When the Numbers Differ
Classical EPV and VHM describe averages across risks. A Bayesian posterior variance describes uncertainty after a specific set of data. Even when the Bayesian posterior mean agrees with a classical credibility estimate, the uncertainty measures need not agree for the observed data. The posterior distribution may also change the expected process variance, since \(E[v(\Theta)\mid D]\) can depend on the data.
The next example uses Poisson counts. It shows both how uncertainty scales with future exposure and why plugging a posterior mean into a process variance is not enough.
Worked Example: Poisson-Gamma Prediction and Classical Average Error
For a hypothetical risk, let the claim count in one exposure unit satisfy \(N_j\mid\lambda\sim\operatorname{Poisson}(\lambda)\). Assume a prior \(\lambda\sim\operatorname{Gamma}(2,1)\), using shape and rate parameters. Two unit-exposure observations have a total count of 3. Find the posterior predictive variance for a future period with exposure \(h=2\), and compare it with classical average prediction error.
Method. The posterior shape is \(2+3=5\), and its rate is \(1+2=3\). Thus $$ \lambda\mid D\sim\operatorname{Gamma}(5,3), \qquad E[\lambda\mid D]=\frac{5}{3}, \qquad \operatorname{Var}(\lambda\mid D)=\frac{5}{9}. $$ For future exposure \(h=2\), \(N_{\mathrm{new}}\mid\lambda,D\sim\operatorname{Poisson}(2\lambda)\). Its conditional mean and variance are both \(2\lambda\).
The expected process variance is \(E[2\lambda\mid D]=10/3\). The variance of the conditional mean is \(\operatorname{Var}(2\lambda\mid D)=4(5/9)=20/9\). Therefore $$ \operatorname{Var}(N_{\mathrm{new}}\mid D) = \frac{10}{3}+\frac{20}{9} = \frac{30}{9}+\frac{20}{9} = \frac{50}{9} \approx 5.5556. $$ The predictive mean is \(2(5/3)=10/3\).
For the classical comparison, the prior has \(\mathrm{EPV}=E[\lambda]=2\) and \(\mathrm{VHM}=\operatorname{Var}(\lambda)=2\), so \(K=1\). With \(n=2\) unit observations, \(Z=2/3\), and \(\bar N=3/2\). The Bühlmann estimate is $$ \widehat\lambda_2=\frac{2}{3}\left(\frac{3}{2}\right)+\frac{1}{3}(2)=1+\frac{2}{3}=\frac{5}{3}. $$ The classical average squared error for estimating \(\lambda\) is \(\mathrm{EPV}/(n+K)=2/3\). For a future count over exposure 2, scaling the estimated mean by 2 scales this error by \(2^2=4\). The average process variance is \(2\mathrm{EPV}=4\), so the classical average squared prediction error is \(4+4(2/3)=20/3\), or about 6.6667.
The Bayesian result \(50/9\) is conditional on the observed total of 3. The classical value \(20/3\) averages over possible samples under the model. These different conditioning bases explain why the values need not match. Averaging the Bayesian posterior predictive variance over repeated samples recovers the classical average prediction error here.
Common mistake. Using only the Poisson variance at the posterior mean gives \(2(5/3)=10/3\), omitting the parameter-uncertainty term \(20/9\). The correct conditional predictive variance is \(50/9\), not \(10/3\).
Check. The parameter-uncertainty component is \(4\) times the posterior variance of \(\lambda\), because future exposure is 2. The process component is linear in exposure; the parameter component is quadratic.
Reading the Comparison Correctly
| Quantity | What it measures | Conditioning or averaging |
|---|---|---|
| \(E[v(\Theta)\mid D]\) | Future process variation, averaged over plausible risk parameters | Conditional on the observed data |
| \(\operatorname{Var}(m(\Theta)\mid D)\) | Posterior uncertainty about the risk’s expected claim level | Conditional on the observed data |
| EPV | Average conditional process variance in the risk population | Across the population of risks |
| \(\mathrm{EPV}/(n+K)\) | Mean squared error of the classical credibility estimate of the risk mean | Across risks and possible samples |
The comparison is most informative when each quantity is identified by both its source of uncertainty and its conditioning basis. A posterior variance is not an automatic replacement for the Bühlmann estimation error, and EPV is not automatically the process variance for a particular posterior. If \(v(\Theta)\) changes across risks, its Bayesian contribution is \(E[v(\Theta)\mid D]\), not necessarily the population average EPV.
For a future total formed from \(h\) conditionally independent, equal-exposure observations with conditional mean \(m(\Theta)\) and variance \(v(\Theta)\), the conditional total has mean \(h\,m(\Theta)\) and variance \(h\,v(\Theta)\). Its posterior predictive variance is therefore $$ hE[v(\Theta)\mid D]+h^2\operatorname{Var}(m(\Theta)\mid D). $$ The process term grows linearly with exposure; uncertainty about the shared risk mean grows quadratically. This distinction matters whenever a forecast covers multiple future periods for the same risk.
Check Your Understanding
For each problem, identify the process and risk-mean uncertainty contributions before combining them.
- A posterior distribution for a Poisson rate is \(\operatorname{Gamma}(4,2)\), using shape and rate. Find the predictive variance of a count over future exposure \(h=3\). (Answer: process variance \(6\), parameter-uncertainty variance \(9\), total \(15\).)
- Assume \(\Theta\sim N(20,25)\) and \(X_j\mid\Theta\sim N(\Theta,36)\), independently. Four observations have average 23. Find the posterior mean of \(\Theta\) and the predictive variance for one future observation. (Answer: posterior mean \(755/34\approx22.2059\); predictive variance \(1449/34\approx42.6176\).)
- A classical Bühlmann model has \(\mathrm{EPV}=12\), \(\mathrm{VHM}=3\), and \(n=2\). Find \(K\), \(Z\), and the mean squared error of the credibility estimate for the latent risk mean. (Answer: \(K=4\), \(Z=1/3\), mean squared error \(2\).)
- Given data, a Poisson rate has posterior probabilities \(P(\lambda=2\mid D)=0.4\) and \(P(\lambda=6\mid D)=0.6\). Find the predictive variance of a count over exposure 2. (Answer: process variance \(8.8\), parameter-uncertainty variance \(15.36\), total \(24.16\).)
- Given data, a risk mean has posterior mean 3 and variance 4. For each future unit, the conditional claim variance is \(5+2\Theta\); three future observations are conditionally independent given \(\Theta\). Find the predictive variance of their total. (Answer: process variance \(33\), risk-mean uncertainty \(36\), total \(69\).)