Why Zero Variance Components Need Careful Interpretation
In the Bühlmann model, observed outcomes can vary for two different reasons: observations fluctuate within a risk, and risks differ in their expected outcomes. A zero estimate for either source does not mean the same thing as a zero estimate for the other. Distinguishing them is essential when interpreting a fitted model or a small set of repeated-period data.
Let \(\Theta\) denote a risk's underlying parameter, and let \(X\) be an observation for that risk. Conditional on \(\Theta\), the expected observation is the risk's hypothetical mean. The two variance components measure variation around that mean and variation among those means. These definitions concern the mean structure of the model; in particular, zero variance of hypothetical means does not necessarily mean every aspect of every risk's distribution is identical.
What the Two Zero Cases Mean
If \(\operatorname{EPV}=0\), then \(\operatorname{Var}(X\mid\Theta)=0\) for almost every risk parameter value. A random variable with zero variance equals its mean with probability one, so observations for a given risk are deterministic at that risk's hypothetical mean under the model. This does not require different risks to have the same mean: the VHM may be positive.
If \(\operatorname{VHM}=0\), then \(E[X\mid\Theta]\) equals one common value almost surely. Risks therefore do not differ in their expected outcome under this model. The process variance can still be positive, so repeated observations can fluctuate even though the risks have the same hypothetical mean. A zero VHM is not proof that the risks have identical distributions in every respect; it only rules out variation in their conditional means.
The law of total variance gives a concise summary. Provided \(X\) has finite variance, write its variance as the expected conditional variance plus the variance of its conditional expectation:
This identity shows why the zero cases have different implications. With zero EPV, all variation in \(X\) comes from differences in hypothetical means. With zero VHM, all variation comes from process fluctuation around a common mean. If both components are zero, \(\operatorname{Var}(X)=0\), so \(X\) equals its overall mean almost surely.
Worked Example: Zero Process Variance With Different Risk Means
A hypothetical book has two equally likely risk types, represented by \(\Theta=80\) and \(\Theta=120\). For each risk, every period's outcome is deterministically \(X=\Theta\). Find the overall mean, EPV, VHM, and total variance. No discounting or mortality applies; the outcomes are in dollars.
Method. Calculate each component from its conditional definition, then check the total using the law of total variance. The common mistake is to conclude that zero process variance means zero variation across all risks.
For either type, \(\operatorname{Var}(X\mid\Theta)=0\), because \(X\) is fixed once \(\Theta\) is given. Thus \(\operatorname{EPV}=0\).
The overall mean is \(E[X]=\tfrac12(80)+\tfrac12(120)=100\). The conditional means are 80 and 120, each with probability one-half, so \(\operatorname{VHM}=\tfrac12(80-100)^2+\tfrac12(120-100)^2 =\tfrac12(400)+\tfrac12(400)=400\) dollars squared.
The total variance is \(0+400=400\) dollars squared. Directly, \(X\) is 20 dollars below or above 100 with equal probability, giving \(\operatorname{Var}(X)=\tfrac12(20^2)+\tfrac12(20^2)=400\). The check agrees. Observations within each risk do not fluctuate, but the two risk types have different means.
Worked Example: Zero Variance of Hypothetical Means With Process Fluctuation
Suppose every risk has \(X\mid\Theta\sim\operatorname{Poisson}(3)\), independently across periods. The risk label \(\Theta\) may vary, but it does not change the conditional distribution. Find the overall mean, EPV, VHM, and total variance. No discounting or mortality applies; \(X\) is a claim count per period.
Method. Use the conditional mean and variance of a Poisson distribution with parameter 3. The mistake to avoid is interpreting a common mean as meaning that every observed count must be 3.
For a Poisson random variable with parameter \(\lambda\), both the mean and variance are \(\lambda\). Here, for every \(\Theta\), \(E[X\mid\Theta]=3\) and \(\operatorname{Var}(X\mid\Theta)=3\). Therefore, \(\operatorname{EPV}=E[3]=3\) and \(\operatorname{VHM}=\operatorname{Var}(3)=0\).
The overall mean is \(E[X]=3\), and total variance is \(3+0=3\). A count can be 0, 1, 2, 4, or another nonnegative integer even though all risks share the same hypothetical mean. The model permits period-to-period process variation but no variation in expected counts by risk.
Boundary Estimates From Balanced Repeated-Period Data
A population component can equal zero, but an estimate based on a finite portfolio may also land at zero by chance. To see how the boundary arises, consider \(I\) risks observed for the same number \(n\) of periods. Use the balanced random-effects model \(X_{ij}=\mu+A_i+\varepsilon_{ij}\), where \(A_i\) are independent risk effects with mean 0 and variance \(\tau^2\), and \(\varepsilon_{ij}\) are independent process errors with mean 0 and variance \(\sigma^2\), independent of the risk effects. Then the process variance is \(\sigma^2\), and the VHM is \(\tau^2\). Normality is not required for the following expectation formulas; the stated mean, variance, independence, and equal-period assumptions are required.
Let \(\bar X_i\) be the mean for risk \(i\), and let \(\bar X\) be the grand mean. Define the within-risk and between-risk mean squares as follows:
Under the model, \(E[MS_W]=\sigma^2\) and \(E[MS_B]=\sigma^2+n\tau^2\). These expectations motivate the raw method-of-moments estimates \(\widehat{\sigma^2}=MS_W\) and \(\widehat{\tau^2}_{\rm raw}=(MS_B-MS_W)/n\).
The between-risk mean square includes both process variation in each risk average and variation in the true risk means. Subtracting the within-risk mean square removes the process-variance contribution in expectation. Dividing by \(n\) then puts the remaining estimate on the scale of the variance among hypothetical means.
Subtracting each risk's average removes its shared risk effect \(A_i\). The within-risk squared deviations therefore estimate the process variance, giving \(E[MS_W]=\sigma^2\).
Each \(\bar X_i=\mu+A_i+\bar\varepsilon_i\) has variance \(\tau^2+\sigma^2/n\), because the independent terms have variances \(\tau^2\) and \(\sigma^2/n\).
The sample variance of the \(I\) independent risk averages has expectation \(\tau^2+\sigma^2/n\). Multiplying it by \(n\), as in \(MS_B\), gives \(E[MS_B]=n\tau^2+\sigma^2\).
Subtracting the two expectations gives \(E[MS_B-MS_W]=n\tau^2\). Hence the raw contrast divided by \(n\) estimates \(\tau^2\), though its realized value need not be nonnegative.
Worked Example: A Zero VHM Estimate at the Boundary
Three hypothetical risks are observed for two periods. Their values are \((8,12)\), \((10,14)\), and \((12,16)\). Assume the balanced random-effects model just defined, with independent normal risk effects \(A_i\sim N(0,\tau^2)\) and independent normal process errors \(\varepsilon_{ij}\sim N(0,\sigma^2)\); \(\tau^2\) and \(\sigma^2\) are unknown. Calculate the raw component estimates. No discounting or mortality applies; values are in dollars.
Method. Calculate within- and between-risk mean squares, then apply the moment formulas. The mistake to avoid is treating different observed risk averages alone as proof that the raw VHM estimate must be positive.
The three risk means are 10, 12, and 14, and the grand mean is \((10+12+14)/3=12\). Within each pair, the deviations from that risk's mean are \(-2\) and \(2\), so each risk contributes \(4+4=8\) to the within sum of squares. The total is \(24\); its degrees of freedom are \(3(2-1)=3\). Thus \(MS_W=24/3=8\).
The between-risk sum of squares is \((10-12)^2+(12-12)^2+(14-12)^2=4+0+4=8\). Therefore \(MS_B=2(8)/(3-1)=8\). The estimates are \(\widehat{\sigma^2}=8\) and \(\widehat{\tau^2}_{\rm raw}=(8-8)/2=0\).
The observed averages differ, yet the component contrast is exactly zero: the estimated between-risk variation in averages does not exceed the process contribution in this calculation. The result is a boundary estimate, not proof that the true VHM is exactly zero. Substitution into the defining contrast confirms the result: \((MS_B-MS_W)/n=(8-8)/2=0\).
Negative Raw Estimates and Sampling Instability
A raw method-of-moments estimate can be negative when \(MS_B<MS_W\). That outcome is impossible for a true variance, but it is possible for a finite-sample estimate: sampling variation can make the observed between-risk mean square smaller than the observed within-risk mean square. The negative value is a warning that the data do not support a stable positive estimate through this raw contrast; it is not evidence of a negative population variance.
The estimate can be especially unstable when the number of risks is small, the number of periods per risk is limited, or the true VHM is near zero. The difference \(MS_B-MS_W\) may then be small relative to the uncertainty in either mean square. Report and interpret a negative raw estimate as an invalid variance estimate requiring careful treatment, not as a literal model component. Which boundary estimation procedure to use is a separate modeling decision.
Worked Example: Diagnosing a Negative Raw VHM Estimate
Three hypothetical risks are observed twice, with values \((8,12)\), \((8.5,12.5)\), and \((9,13)\). Assume a balanced random-effects model with independent normal risk effects \(A_i\sim N(0,\tau^2)\) and independent normal process errors \(\varepsilon_{ij}\sim N(0,\sigma^2)\), independent of \(A_i\); both variance parameters are unknown. Calculate the raw estimates and interpret them. No discounting or mortality applies; values are in dollars.
Method. Use the same balanced mean-square contrast. The common mistake is to report a negative estimate as though a variance could be negative.
The risk means are 10, 10.5, and 11; the grand mean is 10.5. Each pair has deviations \(-2\) and \(2\), so each within sum of squares is 8 and the total within sum of squares is 24. With \(3(2-1)=3\) within degrees of freedom, \(MS_W=24/3=8\).
The between-risk sum of squares is \((10-10.5)^2+(10.5-10.5)^2+(11-10.5)^2=0.25+0+0.25=0.5\). Thus \(MS_B=2(0.5)/(3-1)=0.5\). The raw estimates are \(\widehat{\sigma^2}=8\) and \(\widehat{\tau^2}_{\rm raw}=(0.5-8)/2=-7.5/2=-3.75\).
The process-variance estimate is 8, but \(-3.75\) is not an admissible VHM. It signals that this sample's between-risk mean square is below its within-risk mean square. It does not establish that the true VHM is negative, nor does it establish that the true VHM equals zero. The calculation is checked by substituting \(MS_B=0.5\) and \(MS_W=8\) into the contrast, which reproduces \(-3.75\).
Common Interpretation Traps
- Equating zero EPV with no differences across risks. Zero EPV removes within-risk fluctuation under the model; risk means may still vary.
- Equating zero VHM with identical observations. A common hypothetical mean can coexist with substantial process variance.
- Treating a zero estimate as a known population fact. A boundary estimate from a finite sample is not, by itself, proof that the population component is exactly zero.
- Accepting a negative raw variance as meaningful. A negative estimate is a sampling outcome of the raw contrast, not a possible value of a true variance.
- Inferring identical risk distributions from equal means. The VHM describes variability in conditional means, not every possible risk-level distributional difference.
These distinctions are aligned with the CAS Exam 8 (Advanced Ratemaking) syllabus. In practice, they help separate what the model says about risk-level means from what finite data can establish about the components.
Check Your Understanding
For each problem, identify the relevant component or calculate the requested estimate before checking the answer.
- Two equally likely risk types have \(\Theta=50\) and \(\Theta=90\). For every risk, \(X=\Theta\) deterministically. Find the overall mean, EPV, and VHM. (Answer: mean \(70\), EPV \(0\), VHM \(400\).)
- Every risk has \(X\mid\Theta\sim\operatorname{Bernoulli}(0.2)\), independently by period, regardless of \(\Theta\). Find EPV and VHM. (Answer: EPV \(0.2(0.8)=0.16\), VHM \(0\).)
- Two risks are each observed twice: risk 1 has values \((4,8)\), and risk 2 has values \((6,10)\). Under the balanced method-of-moments formulas, calculate \(MS_W\), \(MS_B\), and the raw VHM estimate. (Answer: \(MS_W=8\), \(MS_B=4\), raw VHM estimate \((4-8)/2=-2\); the negative estimate is not a valid variance.)
- A model has process variance 5 and VHM 12. What is the variance of an observation, and which source contributes more? (Answer: total variance \(5+12=17\); VHM contributes more.)
- Three equally likely risk types have deterministic outcomes 2, 4, and 6 in every period. Calculate the overall mean, EPV, and VHM. (Answer: mean \(4\), EPV \(0\), VHM \(\{(2-4)^2+(4-4)^2+(6-4)^2\}/3=8/3\).)