The Bühlmann Estimate as a Linear Projection

Why a Linear Projection Leads to Credibility

A risk’s observed experience contains two kinds of variation. Some variation occurs from one observation to the next even for the same risk; other variation reflects genuine differences in expected outcomes among risks. Bühlmann credibility balances these sources when predicting a future observation from a risk’s past experience.

The key question here is not simply how to calculate a weighted average. It is why the weights take the Bühlmann form. The answer is that, under a collective risk model, the Bühlmann estimate is the best linear predictor: among predictors that are linear in the available observations, it minimizes expected squared prediction error.

Definition: Let $\Theta$ denote a risk’s latent parameter, and let $X_1,\ldots,X_n,X_{n+1}$ be conditionally independent and identically distributed given $\Theta$. Define the conditional mean $m(\Theta)=E[X_j\mid\Theta]$. The expected process variance and variance of hypothetical means are $$ \text{EPV}=v=E[\operatorname{Var}(X_j\mid\Theta)], \qquad \text{VHM}=a=\operatorname{Var}(m(\Theta)). $$ The collective mean is $\mu=E[X_j]=E[m(\Theta)]$. Assume these quantities are finite.

The process variance measures variability within a risk after its parameter is fixed. The VHM measures how much the risks’ expected values differ across the collective. Conditional independence is important: given $\Theta$, one observation does not provide additional information about another observation’s random process fluctuation.

Write the class sample mean as $\bar X=(X_1+\cdots+X_n)/n$. The Bühlmann estimate of the future observation is $$ \widehat X_{n+1}=\mu+Z(\bar X-\mu), \qquad Z=\frac{na}{v+na}=\frac{n}{n+K}, \qquad K=\frac{v}{a}, $$ when $a>0$. Thus the credibility factor $Z$ determines how much the estimate moves from the collective mean $\mu$ toward the observed class mean $\bar X$.

Deriving the Projection

Consider the general affine predictor $c+\sum_{j=1}^{n}b_jX_j$. “Best” means that its expected squared error for predicting $X_{n+1}$ is as small as possible among all such affine predictors. The normal equations for least squares require the prediction error to have zero covariance with the constant and with each predictor variable. In other words, the error is orthogonal to every direction in the linear predictor space.

Center the observations.
Since all observations have mean $\mu$, write the predictor as $\mu+\sum_{j=1}^{n}b_j(X_j-\mu)$. The constant is then chosen so the predictor has mean $\mu$.

Use the model covariances.
The law of total variance gives $\operatorname{Var}(X_j)=v+a$. For distinct observations, conditional independence gives $\operatorname{Cov}(X_j,X_k)=a$. Also, $\operatorname{Cov}(X_{n+1},X_j)=a$.

Apply orthogonality.
For each $j$, the covariance of the prediction error with $X_j-\mu$ must be zero. Symmetry makes all the coefficients equal, say $b_j=b$, and the resulting equation determines $b$.

For completeness, the covariance calculations in Step 2 follow directly from conditioning. The law of total variance gives

\operatorname{Var}(X_j) =E[\operatorname{Var}(X_j\mid\Theta)] +\operatorname{Var}(E[X_j\mid\Theta]) =v+a.

For $j\ne k$, conditional independence makes the conditional covariance zero, so

\operatorname{Cov}(X_j,X_k) =E[\operatorname{Cov}(X_j,X_k\mid\Theta)] +\operatorname{Cov}(E[X_j\mid\Theta],E[X_k\mid\Theta]) =a.

The same reasoning applies to the future observation and any observed one, giving $\operatorname{Cov}(X_{n+1},X_j)=a$. Now impose the orthogonality condition for one observed value. With equal coefficients, the covariance between the centered predictor and $X_j-\mu$ is $b(v+a)+b(n-1)a$. It must equal the covariance between $X_{n+1}-\mu$ and $X_j-\mu$, which is $a$. Therefore,

b\{v+a+(n-1)a\}=a \quad\Longrightarrow\quad b=\frac{a}{v+na}.

The predictor uses the sum of $n$ centered observations, so the total coefficient on $\bar X-\mu$ is $nb$. Consequently,

\widehat X_{n+1} =\mu+nb(\bar X-\mu) =\mu+\frac{na}{v+na}(\bar X-\mu).

This establishes the Bühlmann factor $Z$. The estimate can also be written as a weighted average:

\widehat X_{n+1}=Z\bar X+(1-Z)\mu.

Because the two weights sum to one, the estimate remains on the scale of the observed quantities. If $v>0$ and $a>0$, increasing $n$ increases $Z$, while increasing process variance $v$ decreases it. These conclusions follow from the formula: $Z=n/(n+v/a)$.

Worked Applications

Worked Example: Deriving the Estimate from a Two-Point Risk Model

A hypothetical portfolio has risks with latent expected annual costs $\Theta=80$ or $\Theta=120$, each with probability $1/2$. Given $\Theta$, each annual cost is independent and $X_j\mid\Theta\sim N(\Theta,400)$, where the second parameter is the variance. For a risk with observed costs of $90$, $110$, and $130$, find the Bühlmann linear estimate of its next annual cost. There is no discounting or mortality assumption in this problem.

1. Find the collective mean. The latent mean is equally likely to be $80$ or $120$, so $$ \mu=E[\Theta]=\frac{80+120}{2}=100. $$

2. Find EPV and VHM. The conditional variance is $400$ in either class, so $v=400$. The latent means are each $20$ from their collective mean, giving $$ a=\operatorname{Var}(\Theta) =\frac{(80-100)^2+(120-100)^2}{2} =\frac{400+400}{2}=400. $$

3. Calculate the credibility factor. There are $n=3$ observations: $$ K=\frac{v}{a}=\frac{400}{400}=1, \qquad Z=\frac{n}{n+K}=\frac{3}{3+1}=0.75. $$

4. Calculate the sample mean and estimate. $$ \bar X=\frac{90+110+130}{3}=\frac{330}{3}=110, \qquad \widehat X_4=100+0.75(110-100)=107.50. $$

The best linear estimate is $107.50. It is closer to the class mean of $110 than to the collective mean of $100 because the three observations provide credibility $0.75$. This is a linear projection; with this two-point mixing distribution, it should not automatically be interpreted as the exact conditional expectation given the observations.

Worked Example: Obtaining the Factor from the Normal Equations

For a hypothetical risk population, suppose $\Theta\sim N(100,300)$ and, given $\Theta$, observations are independent with $X_j\mid\Theta\sim N(\Theta,900)$. The second parameter in each normal distribution is its variance. A risk has four observed annual costs: $115$, $125$, $135$, and $125$. Use the projection equations to find the estimate of the fifth cost. No interest or mortality assumptions are involved.

1. Identify the variance components. Here $\mu=100$, $v=900$, and $a=300$. The variance of each observation is $v+a=1{,}200$, and the covariance of any two distinct observations is $a=300$.

2. Write one normal equation. By symmetry, give each centered observation coefficient $b$. The covariance of the centered predictor with any one observation is the diagonal covariance times $b$, plus the three off-diagonal covariances times $b$: $$ 1{,}200b+3(300b)=300. $$ The right side is the covariance of the future observation with that observed value. Thus $$ 2{,}100b=300,\qquad b=\frac{300}{2{,}100}=\frac{1}{7}. $$

3. Convert to the sample-mean coefficient. Four observations each receive coefficient $1/7$, so the total coefficient on their mean is $4/7$. Equivalently, $Z=na/(v+na)=4(300)/(900+4(300))=4/7$.

4. Apply the predictor. The sample mean is $$ \bar X=\frac{115+125+135+125}{4}=\frac{500}{4}=125. $$ Therefore, $$ \widehat X_5=100+\frac{4}{7}(125-100) =100+\frac{100}{7} =114.2857\ldots. $$ Rounded only at the final step to the nearest cent, the estimate is $114.29. The direct normal equation and the credibility formula agree.

Worked Example: When the Projection Is Also the Conditional Mean

In a hypothetical model, let $\Theta\sim N(120,400)$, and let $X_j\mid\Theta$ be conditionally independent $N(\Theta,1{,}600)$ observations. A risk has four observed costs $90$, $120$, $150$, and $140$. Find the linear projection estimate of the fifth cost, and state why in this model the conditional expectation is also linear. All normal-distribution second parameters are variances; interest and mortality are not applicable.

1. Calculate the credibility factor. The parameters are $\mu=120$, $a=400$, $v=1{,}600$, and $n=4$. Hence $$ K=\frac{1{,}600}{400}=4,\qquad Z=\frac{4}{4+4}=0.5. $$

2. Calculate the observed mean and projection. $$ \bar X=\frac{90+120+150+140}{4}=\frac{500}{4}=125, \qquad \widehat X_5=120+0.5(125-120)=122.50. $$

3. Check the conditional-mean interpretation. In this normal model, the prior variance of $\Theta$ is $400$. The sample mean has conditional variance $1{,}600/4=400$ around $\Theta$. Combining a normal prior and normal observations gives a posterior mean that weights the observed mean by $$ \frac{400}{400+400}=0.5. $$ Thus $E[\Theta\mid X_1,\ldots,X_4]=120+0.5(125-120)=122.50$. Since $E[X_5\mid\Theta]=\Theta$, the conditional expectation of the future observation is also $122.50$. Normality is the additional condition supporting this exact equality; the Bühlmann linear projection itself does not require normal distributions.

Interpretation, Conditions, and Exam Trap

The projection derivation depends on conditional independence, a common conditional mean and variance structure across observations for a given risk, and finite second moments. The linear predictor minimizes expected squared error among affine functions of the observations. Without an additional distributional assumption, it does not claim to be the unrestricted conditional expectation $E[X_{n+1}\mid X_1,\ldots,X_n]$, which could be nonlinear.

The factor $K=v/a$ is the ratio of process variance to between-risk variance. It is a useful way to express the balance of information: a large $K$ means process noise is large relative to differences in risk means, so more observations are needed before the class experience receives substantial weight. A large $n$ or large $a$, by contrast, increases the weight on the class mean.

A frequent exam error is to reverse EPV and VHM in the factor. For the first worked example, the incorrect expression $v/(v+na)$ gives $400/(400+3(400))=0.25$, producing $100+0.25(110-100)=102.50$. That is not the Bühlmann factor. The correct factor is $na/(v+na)=1{,}200/1{,}600=0.75$, giving $107.50. A quick check is that credibility must increase with the number of observations when the variance components are fixed.

If $a=0$ and $v>0$, all risks have the same expected value, and the projection gives $Z=0$: class-specific deviations do not improve the linear estimate. If $a>0$ and $v=0$, observations reveal the risk mean without process noise, and $Z=1$ for any positive $n$. If both components are zero, observations are constant and the ratio $v/a$ is undefined; the estimate is simply that common constant.

Key takeaway: Under the Bühlmann collective model, the covariance of each observation with a future observation is VHM, while the variance of an observation is EPV plus VHM. Enforcing the least-squares orthogonality equations gives $Z=na/(v+na)$, and therefore a linear blend of the class mean and the collective mean.

Check Your Understanding

For each problem, identify the relevant moments and apply the projection result. Treat every scenario as hypothetical.

A model has $v=250$, $a=500$, and $n=2$. Calculate $K$ and $Z$. (Answer: $K=0.5$, $Z=0.8$.)
A hypothetical portfolio has $\mu=80$, $v=600$, and $a=200$. A risk’s three observations are $70$, $100$, and $110$. Find its Bühlmann linear estimate. (Answer: $\bar X=280/3$, $Z=1/2$, and $\widehat X=260/3\approx86.67$.)
Suppose $v=900$, $a=300$, and $n=3$. What is the coefficient on each centered observation in the best linear predictor, and what is the total credibility on the sample mean? (Answer: each coefficient is $1/6$, and $Z=1/2$.)
In a model with $\mu=150$, $v=1{,}200$, and $a=300$, a class has five observations with mean $138$. Find the linear estimate and explain the direction of shrinkage. (Answer: $Z=5/(5+4)=5/9$, $\widehat X=150+(5/9)(138-150)=143.33$, rounded to cents; it is between the class and collective means.)
For $v=400$, $a=100$, and $n=4$, a candidate uses $Z=v/(v+na)$. Identify the error and give the correct factor. (Answer: the numerator should be $na$, not $v$; $Z=400/(400+400)=0.5$.)

What You'll Learn