What the Two Components Measure
A balanced claims table records the same number of observations for each risk, such as each insured group observed over the same number of periods. Its variation has two sources: observations for a given risk fluctuate from period to period, and the underlying expected outcomes differ across risks. Bühlmann variance-component estimation separates these sources so they can be interpreted and used consistently.
Let \(X_{ij}\) be the observation for risk \(i\) in period \(j\), where \(i=1,\ldots,m\) and \(j=1,\ldots,n\). The risk parameter \(\Theta_i\) represents the risk’s underlying characteristics. Conditional on \(\Theta_i\), the periods for that risk are assumed to have the same mean and variance and to be independent. Different risks are treated as independent draws from the same portfolio population.
The total-variance identity separates the variance of an observation into its expected conditional variance and the variance of its conditional mean:
This is a population identity, not a claim that the sample variance of every finite claims table must equal the sum of the two component estimates. Estimation uses the table’s within-risk and between-risk patterns to estimate the population quantities.
Estimators for a Balanced Table
For each risk, calculate its mean \(\bar X_i\), and calculate the grand mean \(\bar X\). The within-risk sum of squares measures deviations around each risk’s own mean. The between-risk sum of squares measures how far the risk means lie from the grand mean, with each risk mean representing \(n\) observations.
The corresponding degrees of freedom are \(m(n-1)\) within risks and \(m-1\) between risks. Dividing each sum of squares by its degrees of freedom gives the within and between mean squares. The balanced method-of-moments estimates are:
The estimator for VHM is an unconstrained moment estimate. Sampling variation can make it negative, even though a population variance cannot be negative. The estimation method and any boundary adjustment should be kept distinct; do not silently change the raw estimate.
Why the Mean-Square Formulas Work
A standard balanced random-effects model makes the logic explicit. Write \(X_{ij}=\mu+A_i+\varepsilon_{ij}\), where \(\mu\) is the portfolio mean, \(A_i\) is the deviation in risk \(i\)’s hypothetical mean, and \(\varepsilon_{ij}\) is the period-specific process fluctuation. Assume \(E[A_i]=0\), \(\operatorname{Var}(A_i)=\tau^2\), \(E[\varepsilon_{ij}\mid A_i]=0\), and \(\operatorname{Var}(\varepsilon_{ij}\mid A_i)=\sigma^2\). Assume the process fluctuations are conditionally independent across periods and independent across risks. A normal distribution is not required for these expectation results; finite second moments and the stated variance structure suffice. Under this model, \(EPV=\sigma^2\) and \(VHM=\tau^2\).
Subtracting \(\bar X_i\) from \(X_{ij}\) cancels both \(\mu\) and \(A_i\). The within-risk deviations therefore contain only process fluctuations. Their pooled mean square has expectation \(E[MS_W]=\sigma^2\).
The risk mean is \(\bar X_i=\mu+A_i+\bar\varepsilon_i\). Its variance is \(\tau^2+\sigma^2/n\), since averaging \(n\) independent process fluctuations reduces their variance to \(\sigma^2/n\).
The expected between-risk sum of squares is \((m-1)n(\tau^2+\sigma^2/n)\). Dividing by \(m-1\) gives \(E[MS_B]=\sigma^2+n\tau^2\).
Subtracting the within expectation from the between expectation gives \(E[MS_B-MS_W]=n\tau^2\). Dividing by \(n\) yields the VHM moment estimator.
The division by \(n\) is essential: the between mean square contains \(n\) times the between-risk variance, not just the between-risk variance itself. This result relies on a balanced table and a common process variance. With unequal numbers of periods or a different variance structure, these simple mean-square formulas do not automatically apply.
Worked Estimation
Worked Example: Pooling Within and Between Variation
A hypothetical insurer observes the same annual claim measure for four risk groups over three periods. Treat the observations as independent across periods conditional on each group’s risk parameter, with a common conditional mean and variance structure across groups. No parametric distribution is assumed. Estimate EPV and VHM, then add the estimates to obtain the estimated total variance.
| Risk | Period 1 | Period 2 | Period 3 | Risk mean |
|---|---|---|---|---|
| A | 8 | 10 | 12 | 10 |
| B | 12 | 14 | 16 | 14 |
| C | 16 | 18 | 20 | 18 |
| D | 20 | 22 | 24 | 22 |
Step 1: Find the grand mean. The risk means sum to \(10+14+18+22=64\), so \(\bar X=64/4=16\).
Step 2: Calculate within-risk variation. For risk A, the squared deviations are \((8-10)^2=4\), \((10-10)^2=0\), and \((12-10)^2=4\), giving 8. Each other row has the same pattern and sum of 8. Thus \(SS_W=4(8)=32\), and the within degrees of freedom are \(4(3-1)=8\). Therefore \(MS_W=32/8=4\), so \(\widehat{EPV}=4\).
Step 3: Calculate between-risk variation. The squared deviations of the risk means from 16 are \(36,4,4,36\), summing to 80. Each risk mean represents three periods, so \(SS_B=3(80)=240\). The between degrees of freedom are \(4-1=3\), giving \(MS_B=240/3=80\).
Step 4: Estimate VHM. Here \(n=3\), so \(\widehat{VHM}=(80-4)/3=76/3=25.3333\ldots\). Rounding only at the final step to two decimals gives \(\widehat{VHM}=25.33\).
Step 5: Verify the component sum. The estimated total variance is \(4+76/3=88/3=29.3333\ldots\), or 29.33 to two decimals. This is the sum of the estimated components under the total-variance identity; it is not asserted to equal the sample variance of these twelve observations.
Method and mistake avoided. This is the balanced mean-square method of moments. A common incorrect calculation is to report \(MS_B-MS_W=76\) as VHM. That difference estimates \(n\cdot VHM\); dividing by the three periods gives the correct estimate, 25.33.
Worked Example: Checking Variation in Risk Averages
A hypothetical portfolio has three risk groups, each observed in two periods. Assume conditionally independent periods with the same conditional variance for each risk, and independent risks drawn from one population. Use the table to estimate both components and check whether the observed sample variance of the risk averages is consistent with the estimated decomposition.
| Risk | Period 1 | Period 2 |
|---|---|---|
| A | 4 | 8 |
| B | 8 | 12 |
| C | 12 | 16 |
Step 1: Calculate the risk means and grand mean. The means are \((4+8)/2=6\), \((8+12)/2=10\), and \((12+16)/2=14\). The grand mean is \((6+10+14)/3=10\).
Step 2: Estimate EPV. Each risk has squared deviations 4 and 4 around its own mean, so each within sum is 8 and \(SS_W=24\). The within degrees of freedom are \(3(2-1)=3\). Thus \(MS_W=24/3=8\) and \(\widehat{EPV}=8\).
Step 3: Estimate VHM. The squared deviations of the risk means from 10 are 16, 0, and 16, summing to 32. Since each mean represents two periods, \(SS_B=2(32)=64\). Then \(MS_B=64/(3-1)=32\), and \(\widehat{VHM}=(32-8)/2=12\).
Step 4: Check the risk-average variance. The sample variance of the three risk means is \((16+0+16)/(3-1)=16\). The model says a risk average over two periods has variance \(VHM+EPV/2\). The estimates give \(12+8/2=16\), matching the observed sample variance of the risk means.
Conclusion and mistake avoided. The components are EPV \(=8\) and VHM \(=12\), and their sum estimates the total variance as 20. The check concerns the variance of the risk averages, not the variance of a single observation: the averaging step reduces the process contribution from 8 to \(8/2=4\).
Worked Example: Interpreting Components in Thousands of Dollars
Suppose a hypothetical balanced table records annual claim amounts in thousands of dollars for three risk groups over three periods. Assume the same random-effects conditions as above, with no specified parametric distribution. The rows are \(18,20,22\), \(26,28,30\), and \(22,24,26\). Estimate the components and state their units.
Step 1: Calculate the means. The risk means are 20, 28, and 24, and the grand mean is \((20+28+24)/3=24\).
Step 2: Calculate EPV. Each row has squared deviations \(4,0,4\), so \(SS_W=3(8)=24\). The within degrees of freedom are \(3(3-1)=6\), giving \(MS_W=24/6=4\). Thus \(\widehat{EPV}=4\) thousand-dollars-squared.
Step 3: Calculate VHM. The squared deviations of the risk means from 24 are 16, 16, and 0, summing to 32. Therefore \(SS_B=3(32)=96\), and \(MS_B=96/(3-1)=48\). The VHM estimate is \((48-4)/3=44/3=14.6666\ldots\), or 14.67 thousand-dollars-squared to two decimals.
Step 4: Check the risk-average variance and component sum. The sample variance of the means is \((16+16+0)/2=16\). The estimated variance of a three-period risk average is \(14.6666\ldots+4/3=16\). The estimated variance of a single observation is \(4+14.6666\ldots=18.6666\ldots\), or 18.67 thousand-dollars-squared.
Method and mistake avoided. The calculations use the balanced mean squares, retaining the factor of three in \(SS_B\) because each risk mean represents three periods. Omitting that factor would understate between-risk variation. Squared units are appropriate because both components are variances; the components themselves are not dollar amounts.
Interpretation and Exam Traps
EPV describes period-to-period fluctuation remaining after accounting for the risk’s hypothetical mean. VHM describes how much those hypothetical means differ across risks. A high VHM relative to EPV indicates appreciable variation among risks’ underlying means; a high EPV relative to VHM indicates more fluctuation within a risk. These estimates depend on the observation unit: changing from individual-period claims to averages over several periods changes the process variance of the averaged observations.
- Do not omit the replication factor. The between sum of squares includes \(n\) times the squared deviations of the risk means. The VHM estimator also divides the mean-square difference by \(n\).
- Do not use the wrong degrees of freedom. The within denominator is \(m(n-1)\), and the between denominator is \(m-1\). They serve different sources of variation.
- Do not confuse a sample check with a population identity. The total-variance identity applies to population quantities. Sample estimates need not sum to the ordinary sample variance of all observations exactly.
- Check balance before applying the formulas. The displayed estimators assume equal numbers of comparable observations for every risk. Unequal period counts require a method that accounts for the differing precision of risk means.
- Keep units visible. If observations are measured in thousands of dollars, the variance components are in thousands-of-dollars squared.
Check Your Understanding
For each balanced table, use the stated mean-square method. Keep the period count in the VHM calculation and report final numerical answers to two decimals where needed.
- Three risks are observed twice, with rows \((2,6)\), \((6,10)\), and \((10,14)\). Calculate EPV and VHM. (Answer: EPV \(=8\), VHM \(=12\).)
- Four risks are observed twice, with rows \((10,14)\), \((14,18)\), \((18,22)\), and \((22,26)\). Calculate EPV and VHM. (Answer: EPV \(=8\), VHM \(=22.67\).)
- A balanced table has \(m=5\) risks and \(n=4\) periods, with \(SS_W=180\) and \(SS_B=640\). Calculate EPV, VHM, and the estimated total variance. (Answer: EPV \(=12\), VHM \(=37\), estimated total variance \(=49\).)
- For a balanced table with \(m=4\), \(n=3\), \(SS_W=96\), and \(SS_B=24\), calculate the raw moment estimates and interpret a negative VHM estimate. (Answer: EPV \(=12\), raw VHM \(=-1.33\); sampling variation can produce a negative raw estimate even though a true variance is nonnegative.)
- A risk average is based on \(n=5\) periods. The component estimates are EPV \(=20\) and VHM \(=30\). Calculate the estimated variance of one observation and of a five-period risk average. (Answer: single-observation variance \(=50\); risk-average variance \(=30+20/5=34\).)