Estimating Bühlmann Structural Parameters

Why Estimate Structural Parameters?

Bühlmann credibility combines a risk class’s own observed experience with an overall mean. To use that model in practice, an actuary needs estimates of the model’s structural quantities: the overall mean, the expected process variance, and the variance of hypothetical means. Repeated observations on several risk classes provide information about all three, but the sources of variation must be separated carefully.

This tutorial considers a balanced data set: \(r\) risk classes, each with \(n\) observations on the same basis. The observations might be annual loss costs per exposure, for example, provided they are comparable across classes and periods. The calculations estimate variation between risk classes and variation within each class; they do not establish that the model assumptions are appropriate for every portfolio.

Definition: Let \(\Theta_i\) denote the underlying parameter for risk class \(i\), and let \(X_{ij}\) be its \(j\)th observation. The overall mean is \(\mu=E[E[X_{ij}\mid\Theta_i]]\). The expected process variance (EPV) is \(E[\operatorname{Var}(X_{ij}\mid\Theta_i)]\). The variance of hypothetical means (VHM) is \(\operatorname{Var}(E[X_{ij}\mid\Theta_i])\). EPV describes expected variation within a risk class; VHM describes variation in the underlying class means.

Model and Balanced-Data Estimates

For the estimates below, assume that the risk classes are independent draws from a common population of classes. Within a class, the \(n\) observations are conditionally independent given its risk parameter and have the same conditional mean and variance. All observations use a common measurement basis. These assumptions imply that the class-specific means vary around \(\mu\), while individual observations also vary around their own class mean.

Write \(\bar X_i\) for the average of the \(n\) observations in class \(i\), and \(\bar X_{\cdot\cdot}\) for the average of all \(rn\) observations. In a balanced data set, the grand average is also the unweighted average of the class averages.

\bar X_i=\frac{1}{n}\sum_{j=1}^{n}X_{ij}, \qquad \bar X_{\cdot\cdot}=\frac{1}{rn}\sum_{i=1}^{r}\sum_{j=1}^{n}X_{ij} =\frac{1}{r}\sum_{i=1}^{r}\bar X_i.

The natural estimate of the overall mean is the grand average. Estimate EPV by pooling the within-class sums of squared deviations and dividing by their total degrees of freedom. Estimate VHM by taking the sample variance of the class averages and subtracting the portion of that variance attributable to process variation.

\widehat{\mu}=\bar X_{\cdot\cdot}, \qquad \widehat{\operatorname{EPV}} =\frac{\displaystyle\sum_{i=1}^{r}\sum_{j=1}^{n}(X_{ij}-\bar X_i)^2}{r(n-1)}, \qquad \widehat{\operatorname{VHM}}_{\mathrm{raw}} =\frac{\displaystyle\sum_{i=1}^{r}(\bar X_i-\bar X_{\cdot\cdot})^2}{r-1} -\frac{\widehat{\operatorname{EPV}}}{n}.

The subscript “raw” matters: the last expression can be negative. A variance parameter cannot be negative, but sampling variability can make the observed between-class variance smaller than the estimated process contribution. A common nonnegative constrained estimate is \(\max(0,\widehat{\operatorname{VHM}}_{\mathrm{raw}})\). Report clearly which estimate is being used; replacing a negative raw value by zero is a constraint, not a recalculation of the raw estimate.

Why the VHM Correction Is Needed

The class averages do not reveal only differences in underlying class means. Each class average also contains the average of its process fluctuations. Conditional independence makes the variance of that average’s process component equal to the conditional process variance divided by \(n\). Applying the law of total variance gives the unconditional variance of a class average.

\begin{aligned} \operatorname{Var}(\bar X_i) &=\operatorname{Var}(E[\bar X_i\mid\Theta_i]) +E[\operatorname{Var}(\bar X_i\mid\Theta_i)]\\ &=\operatorname{VHM}+\frac{\operatorname{EPV}}{n}. \end{aligned}

Since the classes are independent draws from the same class population, the expected sample variance of their averages equals \(\operatorname{Var}(\bar X_i)\). Thus, the sample variance of the class averages estimates VHM plus \(\operatorname{EPV}/n\), not VHM alone. Subtracting the estimated process contribution \(\widehat{\operatorname{EPV}}/n\) isolates the raw VHM estimate. Similarly, within each class, centering observations on that class’s own average removes the class mean; the expected within-class sum of squares is \((n-1)\) times the conditional process variance. Pooling over classes and dividing by \(r(n-1)\) therefore estimates EPV.

Calculate each class average.
Use the same number \(n\) of comparable observations in every class.

Calculate the grand average and pooled within-class sum of squares.
The grand average estimates \(\mu\); the pooled within-class variation estimates EPV.

Calculate between-class variation and correct it.
Subtract \(\widehat{\operatorname{EPV}}/n\) from the sample variance of the class averages to obtain raw VHM.

Worked Calculations

Worked Example: Estimating All Three Parameters

A hypothetical portfolio has four risk classes, with two annual loss-cost observations per class, all in thousands of dollars per exposure. The observations are \((3,5)\), \((7,9)\), \((9,11)\), and \((13,15)\). Estimate \(\mu\), EPV, and raw VHM. Use the balanced repeated-observation estimates.

Class averages. The four class averages are \(4,8,10,14\). Their grand average is \((4+8+10+14)/4=9\), so \(\widehat{\mu}=9\) thousand dollars per exposure.

EPV. The within-class squared deviations are \(1+1=2\) in each class. The pooled sum is \(2+2+2+2=8\), and the denominator is \(r(n-1)=4(2-1)=4\). Therefore \(\widehat{\operatorname{EPV}}=8/4=2\) in squared thousands of dollars per exposure.

Raw VHM. The sum of squared deviations of the class averages from 9 is \(25+1+1+25=52\). The between-class sample variance is \(52/(4-1)=52/3\). The process contribution to the variance of a two-observation class average is \(2/2=1\). Hence \(\widehat{\operatorname{VHM}}_{\mathrm{raw}}=52/3-1=49/3\), or approximately \(16.333\) in squared thousands of dollars per exposure.

Check. The class-average variance exceeds the estimated process contribution, so the raw VHM is positive. The calculations use four classes for the between-class denominator and four within-class degrees of freedom for EPV.

Worked Example: A Negative Raw VHM Estimate

A hypothetical insurer observes two loss-cost values per class, in dollars per exposure: \((48,52)\), \((49,51)\), \((49,53)\), and \((50,52)\). Calculate the overall mean, EPV, and raw VHM, then state the nonnegative constrained VHM estimate.

Class averages and overall mean. The averages are \(50,50,51,51\), so the grand average is \((50+50+51+51)/4=50.5\). Thus, \(\widehat{\mu}=50.5\) dollars per exposure.

EPV. The within-class sums of squares are \(8,2,8,2\), respectively, for a pooled sum of \(20\). The denominator is \(4(2-1)=4\), giving \(\widehat{\operatorname{EPV}}=20/4=5\) squared dollars per exposure.

Raw and constrained VHM. The class-average deviations from 50.5 are \(-0.5,-0.5,0.5,0.5\). Their squared deviations sum to \(1\), so the sample variance is \(1/(4-1)=1/3\). The process contribution is \(5/2=2.5\). Therefore, the raw estimate is \(1/3-2.5=2/6-15/6=-13/6\), or approximately \(-2.167\) squared dollars per exposure. The nonnegative constrained estimate is \(\max(0,-13/6)=0\).

Check. The raw estimate is negative because the observed class-average variance, \(1/3\), is below the estimated process contribution, \(2.5\). The data do not imply a negative true variance; the negative value signals that this unconstrained estimate is unstable at this sample size.

Worked Example: Keeping the Variance Units Straight

Five hypothetical risk classes each have three annual observations, measured in hundreds of dollars per exposure. Their class averages are \(12,14,16,18,20\), and the pooled within-class sum of squares is \(40\) in squared units of that measurement. Estimate all three structural parameters.

Overall mean. The class averages sum to \(80\), so the grand average is \(80/5=16\). Thus, \(\widehat{\mu}=16\) hundreds of dollars per exposure.

EPV. There are \(r(n-1)=5(3-1)=10\) within-class degrees of freedom. Hence \(\widehat{\operatorname{EPV}}=40/10=4\) squared hundreds of dollars per exposure.

Raw VHM. Deviations of the class averages from 16 are \(-4,-2,0,2,4\). Their squared deviations sum to \(16+4+0+4+16=40\). The between-class sample variance is \(40/(5-1)=10\). The process contribution to the variance of a class average is \(4/3\). Thus, \(\widehat{\operatorname{VHM}}_{\mathrm{raw}}=10-4/3=26/3\), approximately \(8.667\) squared hundreds of dollars per exposure.

Check. The raw VHM is positive, and its units are squared measurement units, just like EPV. Do not convert only one of the variance components into dollars before subtracting them.

Common Mistakes and Practical Limits

The most consequential error is to treat the sample variance of class averages as VHM without adjustment. That variance contains both structural differences and process noise. A second frequent error is using the wrong denominator: EPV uses \(r(n-1)\), whereas the sample variance of the \(r\) class averages uses \(r-1\). These denominators count different sources of information.

Common exam trap: Suppose the sample variance of class averages is \(7\), and the estimated EPV is \(6\) with \(n=3\). The incorrect calculation sets VHM equal to \(7\). The correct raw estimate is \(7-6/3=5\), because the averages still contain process variation. The correction is based on the number of observations per class, not the number of classes.

These estimates rely on a balanced design and comparable observations. If class sizes differ, the unweighted variance of class averages and pooled within-class formula above do not generally provide the corresponding balanced-data estimates; unequal weights require a different treatment. Likewise, trends, changing exposure bases, or dependence among observations can make within-class variation reflect more than the process variance in the assumed model. Establish a common measurement basis before interpreting the estimates.

This estimation step matters because the structural parameters govern how experience is divided between process noise and genuine differences among risks. An overestimated EPV or underestimated VHM generally changes how much credibility is assigned to class experience. For actual applications, inspect the data and model assumptions, and distinguish a raw estimate from a constrained value used in a calculation.

Check Your Understanding

For each balanced hypothetical data set, apply the formulas above. Unless otherwise stated, calculate the raw VHM estimate; report any constrained value when requested.

Three classes each have two observations. Their class averages are \(4,7,9\), and their pooled within-class sum of squares is \(12\). Find \(\widehat{\mu}\), \(\widehat{\operatorname{EPV}}\), and \(\widehat{\operatorname{VHM}}_{\mathrm{raw}}\). (Answer: \(\widehat{\mu}=20/3\), \(\widehat{\operatorname{EPV}}=4\), \(\widehat{\operatorname{VHM}}_{\mathrm{raw}}=13/3\))
Four classes each have three observations. Their class averages are \(5,7,9,11\), and their pooled within-class sum of squares is \(32\). Find all three estimates. (Answer: \(\widehat{\mu}=8\), \(\widehat{\operatorname{EPV}}=4\), \(\widehat{\operatorname{VHM}}_{\mathrm{raw}}=16/3\))
Four classes each have two observations. Their class averages are \(30,30,32,32\), and their pooled within-class sum of squares is \(40\). Find raw and nonnegative constrained VHM estimates. (Answer: \(\widehat{\operatorname{VHM}}_{\mathrm{raw}}=-11/3\), constrained estimate \(0\))
Five classes each have two observations. The sample variance of their class averages is \(9\), and the pooled within-class sum of squares is \(20\). Find EPV and raw VHM. (Answer: \(\widehat{\operatorname{EPV}}=4\), \(\widehat{\operatorname{VHM}}_{\mathrm{raw}}=7\))
In a balanced data set with \(r=6\) classes and \(n=4\) observations per class, the pooled within-class sum of squares is \(90\), and the sample variance of the class averages is \(8\). Find EPV and raw VHM, explaining the process correction numerically. (Answer: \(\widehat{\operatorname{EPV}}=5\), process correction \(5/4=1.25\), \(\widehat{\operatorname{VHM}}_{\mathrm{raw}}=6.75\))

What You'll Learn