Estimating VHM From Between-Risk Variation

Why Between-Risk Variation Needs an Adjustment

Risk-level averages do not differ only because the underlying risks have different expected outcomes. Even if every risk had the same expected outcome, finite experience would produce different averages through random process variation. Estimating the variance of hypothetical risk means therefore requires separating variation between risks from variation within a risk due to random outcomes.

This tutorial focuses on the balanced case: each risk has the same total exposure. The EPV, or expected process variance per unit of exposure, is taken as known or estimated separately from within-risk experience. The target is VHM, the variance of the risk-specific means.

For risk \(i\), write its experience average as \(\bar X_i\). The model assumptions are that risks are independent draws from a portfolio with common mean \(E[\Theta]=\mu\) and variance \(\operatorname{Var}(\Theta)=\tau^2\); conditional on its risk mean, each risk's experience average is unbiased; and process variation is independent across risks. For the balanced case, all risks have the same positive exposure \(w\), and \(\operatorname{Var}(\bar X_i\mid\Theta_i)=\phi/w\). Here \(\tau^2\) is VHM, while \(\phi/w\) is process variance in an experience average.

The Balanced-Portfolio Estimator

With \(m\) risks, calculate the ordinary sample variance of the \(m\) experience averages:

The sample variance measures the total observed spread among risk averages. Part of that spread is expected even when the underlying risk means do not vary: each average has process variance \(\phi/w\). Removing this contribution gives the raw method-of-moments estimate:

The estimate has squared outcome units. For example, if outcomes are measured in dollars, both the sample variance and the EPV contribution are in dollars squared. Exposure \(w\) is measured in the same exposure units used to define \(\phi\).

Why Subtract the Process Variance?

The estimator follows from the law of total variance. A risk average varies because the risk's underlying mean varies and because the observed experience fluctuates around that mean. Applying the law to \(\bar X_i\) gives:

The first equality is the law of total variance. In the second, the model gives \(E[\bar X_i\mid\Theta_i]=\Theta_i\) and conditional variance \(\phi/w\). Thus the population variance of the observed risk averages is VHM plus process variance. Under independent, identically distributed risk averages, the ordinary sample variance \(s_{\bar X}^{\,2}\) estimates that population variance. Subtracting the known process contribution yields the stated method-of-moments estimate.

This argument depends on the assumptions. In particular, the exposure must be the same for every risk, and process variance must scale as \(\phi/w\). If exposures differ, the risk-average variances differ; the balanced formula cannot simply be applied to a weighted sample variance without deriving the appropriate estimator. The formula also does not apply unchanged when risks are dependent or when their EPVs are not represented by the same \(\phi\).

Worked Calculations

Interpreting and Using the Estimate

A raw estimate is a sample-based estimate, not a guarantee about the true parameter. In particular, sampling variation can make \(s_{\bar X}^{\,2}\) smaller than \(\phi/w\), producing a negative raw estimate. Since a variance cannot be negative, such a result does not mean that the true VHM is negative. It indicates that this sample's observed spread was insufficient to distinguish positive between-risk variation from the expected process noise. Report the raw result as such and follow the estimation convention required by the specific problem; do not silently replace it with a different estimator.

The estimate can also be unstable when there are few risks. With only two risks, the sample variance rests on one squared difference. More risks generally provide more information about between-risk variation, although the actual precision depends on the model and the data. Greater common exposure reduces process variance \(\phi/w\), making the observed spread among averages more informative about VHM.

When EPV is estimated rather than supplied, substituting that estimate into \(\phi/w\) is a plug-in method-of-moments calculation. Its uncertainty is not reflected in the formula above. Keep the estimation stages distinct: within-risk variation informs EPV; between-risk variation informs VHM only after accounting for the process variation expected in each risk average.