Method-of-Moments Estimation of K

Why Estimate K From Variance Components?

The Bühlmann credibility factor depends on the balance between variation within a risk and variation among risks. Within-risk variation makes an individual risk’s observed experience less informative about its own expected outcome. Between-risk variation makes that risk’s experience more valuable for distinguishing its expected outcome from the collective mean.

The quantity \(K\) summarizes this balance as the ratio of the expected process variance (EPV) to the variance of hypothetical means (VHM). This tutorial develops a method-of-moments estimate of that ratio for a balanced set of repeated observations. The estimates also provide the components needed to calculate the corresponding Bühlmann credibility factor.

Definition: In the Bühlmann model, let \(X_{ij}\) be observation \(j\) for risk \(i\), and let \(\Theta_i\) represent that risk’s risk parameter. The EPV is \(v=E[\operatorname{Var}(X_{ij}\mid\Theta_i)]\), the average conditional variance within risks. The VHM is \(a=\operatorname{Var}(E[X_{ij}\mid\Theta_i])\), the variance across risks’ conditional means. When \(a>0\), \(K=v/a\). For \(n\) observations on a risk, the Bühlmann credibility factor is \(Z=n/(n+K)\).

Balanced Data and the Moment Estimates

Use the following setup for the formulas in this tutorial. There are \(r\) risks, each with \(n\) observations on the same basis and in the same units; \(r\geq 2\) and \(n\geq 2\). Risks are representative draws from the collective. Conditional on a risk’s parameter, its observations are independent and have the same conditional mean and variance. The conditional variance may differ among risks, but its average across the collective is the EPV. Assume the moments used below are finite.

Write \(\bar X_i\) for risk \(i\)’s sample mean, and \(\bar X_{\cdot}\) for the average of the \(r\) risk means. A pooled within-risk sample variance estimates the EPV. The sample variance of the risk means, however, estimates more than the VHM: each risk mean also contains process variation because it is based on only \(n\) observations. The method-of-moments estimate corrects for that sampling variation.

\widehat v = \frac{\displaystyle\sum_{i=1}^{r}\sum_{j=1}^{n}(X_{ij}-\bar X_i)^2} {r(n-1)} \qquad S_{\bar X}^{\,2} = \frac{\displaystyle\sum_{i=1}^{r}(\bar X_i-\bar X_{\cdot})^2}{r-1}

\widehat a=S_{\bar X}^{\,2}-\frac{\widehat v}{n}, \qquad \widehat K=\frac{\widehat v}{\widehat a}, \qquad \widehat Z=\frac{n}{n+\widehat K}

The divisor \(r(n-1)\) in \(\widehat v\) is the total within-risk degrees of freedom in this balanced setup. The divisor \(r-1\) in \(S_{\bar X}^{\,2}\) makes it a sample variance across the \(r\) risk means. The correction \(\widehat v/n\) is essential: without it, the variation of the risk means includes process variation that should not be attributed to the VHM.

Why the Correction Works

Let \(m(\Theta_i)=E[X_{ij}\mid\Theta_i]\), and let \(v(\Theta_i)=\operatorname{Var}(X_{ij}\mid\Theta_i)\). Conditional independence gives the conditional variance of the mean of \(n\) observations as \(v(\Theta_i)/n\). Applying the law of total variance to a risk mean separates variation in its conditional mean from variation in its observations:

\begin{aligned} \operatorname{Var}(\bar X_i) &=\operatorname{Var}\!\left(E[\bar X_i\mid\Theta_i]\right) + E\!\left[\operatorname{Var}(\bar X_i\mid\Theta_i)\right]\\ &=\operatorname{Var}(m(\Theta_i)) + E\!\left[\frac{v(\Theta_i)}{n}\right]\\ &=a+\frac{v}{n}. \end{aligned}

Thus the population variance of risk means is \(a+v/n\), not \(a\). The sample variance \(S_{\bar X}^{\,2}\) estimates this total variance, while the pooled within-risk variance estimates \(v\). Subtracting \(\widehat v/n\) therefore gives the method-of-moments estimate \(\widehat a\). This is a moment-matching argument: each sample quantity is matched to the population moment it estimates.

Once the two components are estimated, the ratio \(\widehat K=\widehat v/\widehat a\) can be used in the Bühlmann factor. A larger estimated EPV relative to VHM produces a larger \(\widehat K\), and hence a smaller factor for a fixed \(n\). This calculation assumes the component estimates are on a consistent observation basis; changing the time period or exposure basis without adjusting the components can make the ratio misleading.

Worked Calculations

Worked Example: Estimating K From Three Risks

A hypothetical portfolio has three risks, each observed for three equal periods. The observations are all annual loss costs in thousands of dollars, with no exposure adjustment needed. Estimate EPV, VHM, \(K\), and the credibility factor for three observations. Use the balanced method of moments and round final decimal values to four decimal places.

Step 1: Find each risk mean and within-risk sum of squares. Risk A has observations \(8,10,12\), so its mean is \(10\), and its within-risk sum of squares is \((8-10)^2+(10-10)^2+(12-10)^2=8\). Risk B has \(12,14,16\), mean \(14\), and within-risk sum of squares \(8\). Risk C has \(16,18,20\), mean \(18\), and within-risk sum of squares \(8\). The total within-risk sum of squares is \(8+8+8=24\).

Step 2: Estimate EPV. The within-risk degrees of freedom are \(r(n-1)=3(3-1)=6\). Therefore, \(\widehat v=24/6=4\), in squared thousands of dollars.

Step 3: Estimate VHM. The average of the risk means is \((10+14+18)/3=14\). Their sample variance is \(\big((10-14)^2+(14-14)^2+(18-14)^2\big)/(3-1)=32/2=16\). The process-variance contribution to this sample variance is \(\widehat v/n=4/3\). Hence \(\widehat a=16-4/3=44/3\).

Step 4: Estimate \(K\) and credibility. The ratio is \(\widehat K=4/(44/3)=12/44=3/11\approx0.2727\). For \(n=3\), \(\widehat Z=3/(3+3/11)=33/36=11/12\approx0.9167\).

Check and common mistake. The risk-mean variance \(16\) equals the estimated VHM \(44/3\) plus the correction \(4/3\), as required. Using \(16\) directly as the VHM would omit the process variation in each three-period mean and would understate \(K\).

Worked Example: Keeping the Within-Risk Degrees of Freedom Straight

A hypothetical insurer observes four risks for two equal periods. The loss-cost observations are \(19,21\) for Risk A, \(23,25\) for Risk B, \(27,29\) for Risk C, and \(31,33\) for Risk D. Estimate the variance components, \(K\), and \(Z\) for two observations. Report final decimal values to four decimal places.

Step 1: Calculate the risk means and within-risk sums. The four means are \(20,24,28,32\). Each pair is one unit below and one unit above its mean, so each within-risk sum of squares is \(1^2+1^2=2\). The pooled within-risk sum is \(4(2)=8\), and the within-risk degrees of freedom are \(4(2-1)=4\). Thus \(\widehat v=8/4=2\).

Step 2: Calculate the sample variance of the risk means. Their average is \((20+24+28+32)/4=26\). The sum of squared deviations is \(36+4+4+36=80\), so \(S_{\bar X}^{\,2}=80/(4-1)=80/3\).

Step 3: Correct for variation in the two-period means. The correction is \(\widehat v/n=2/2=1\). Therefore, \(\widehat a=80/3-1=77/3\approx25.6667\).

Step 4: Obtain \(K\) and \(Z\). \(\widehat K=2/(77/3)=6/77\approx0.0779\). Then \(\widehat Z=2/(2+6/77)=77/80=0.9625\).

Check and common mistake. The estimated variance of a risk mean is \(77/3+1=80/3\). Dividing the pooled within-risk sum of squares by \(rn=8\), rather than \(r(n-1)=4\), would incorrectly count the fitted risk means as within-risk degrees of freedom.

Worked Example: Combining a Pooled Estimate With Risk Means

A hypothetical portfolio has five risks, each with four observations on the same annual cost basis. The pooled within-risk sum of squares is \(120\). The five risk means are \(16,18,20,22,24\). Estimate EPV, VHM, \(K\), and \(Z\) for four observations. Give final decimal values to four decimal places.

Step 1: Estimate EPV from the pooled observations. The within-risk degrees of freedom are \(5(4-1)=15\). Thus \(\widehat v=120/15=8\).

Step 2: Find the sample variance of risk means. Their average is \(20\), and the squared deviations sum to \(16+4+0+4+16=40\). Therefore, \(S_{\bar X}^{\,2}=40/(5-1)=10\).

Step 3: Remove the process contribution. Since each risk mean averages four observations, the correction is \(\widehat v/n=8/4=2\). Thus \(\widehat a=10-2=8\).

Step 4: Calculate the ratio and factor. \(\widehat K=8/8=1\). The credibility factor is \(\widehat Z=4/(4+1)=0.8000\).

Check and common mistake. The sample variance of risk means decomposes here as \(10=8+2\). Subtracting the EPV itself, rather than the EPV divided by \(n\), would give \(10-8=2\), which is not the moment correction for a four-observation mean.

Interpretation, Limits, and Exam Traps

The estimate \(\widehat K\) is a ratio of two estimates with the same squared measurement units, so \(K\) is dimensionless when the observation unit is fixed. It is interpreted in units of observation periods in the formula \(Z=n/(n+K)\): \(n\) must count the same kind of observations used to estimate the components. If the data are quarterly costs, for example, \(n\) counts quarters, not years, unless the model and components have been converted consistently.

The estimate is sensitive to both the portfolio design and the data basis. The simple pooled formula for \(\widehat v\) and the correction \(\widehat v/n\) rely on equal observation counts and comparable observations. If risks have different exposure amounts or different numbers of observations, these balanced formulas cannot simply be applied unchanged. Likewise, pooling observations measured on different scales or time periods does not produce a meaningful common component estimate.

A method-of-moments estimate \(\widehat a\) can be zero or negative in a finite sample, even though the theoretical VHM is nonnegative. In that case, the ordinary positive ratio \(\widehat v/\widehat a\) does not give a usable positive \(K\); do not silently replace a negative estimate by its absolute value or proceed as if it were positive. The key calculation here is the unmodified moment estimate and a clear recognition of when its ratio cannot be used directly.

Do not confuse the two sources of variation. Within-risk residuals estimate EPV; variation among sample means contains both VHM and EPV divided by \(n\).
Use the correct denominators. The within-risk divisor is \(r(n-1)\); the sample variance divisor for risk means is \(r-1\).
Keep the exposure and observation unit consistent. The \(n\) in the correction and credibility factor must match the unit underlying the component estimates.
Check the VHM before forming the ratio. A nonpositive \(\widehat a\) makes the ordinary positive-\(K\) calculation unavailable; it is not a reason to change the sign of the estimate.

Key takeaway: For a balanced portfolio, estimate EPV from pooled within-risk variation, estimate VHM as the sample variance of risk means minus \(\widehat v/n\), and then form \(\widehat K=\widehat v/\widehat a\). A correct credibility factor depends on preserving the model’s observation basis at every step.

Check Your Understanding

For each hypothetical balanced portfolio, estimate the components first, then calculate \(K\) and the credibility factor for the stated \(n\). Round final decimal answers to four decimal places where needed.

Three risks each have two observations. The pooled within-risk sum of squares is \(18\), and the sample variance of the risk means is \(11\). Find \(\widehat v\), \(\widehat a\), \(\widehat K\), and \(\widehat Z\). (Answer: \(\widehat v=6,\ \widehat a=8,\ \widehat K=0.75,\ \widehat Z=0.7273\).)
Four risks each have three observations. The pooled within-risk sum of squares is \(48\), and the sample variance of the risk means is \(7\). Find the variance components and the factor. (Answer: \(\widehat v=6,\ \widehat a=5,\ \widehat K=1.2,\ \widehat Z=0.7143\).)
Five risks each have two observations on the same cost basis. The pooled within-risk sum of squares is \(40\), and the sample variance of their means is \(13\). Calculate \(\widehat a\), \(\widehat K\), and \(Z\) for two observations. (Answer: \(\widehat a=9,\ \widehat K=8/9\approx0.8889,\ \widehat Z=9/13\approx0.6923\).)
Four risks each have four annual observations. Their means are \(12,16,20,24\), and the pooled within-risk sum of squares is \(144\). Estimate both components, \(K\), and the four-observation factor. (Answer: \(\widehat v=12,\ \widehat a=71/3\approx23.6667,\ \widehat K=36/71\approx0.5070,\ \widehat Z=0.8875\).)
A balanced two-period study estimates EPV as \(18\), and the sample variance of its risk means is \(12\). Find the corrected VHM, \(K\), and \(Z\). (Answer: \(\widehat a=3,\ \widehat K=6,\ \widehat Z=0.2500\).)

What You'll Learn