Mean-Squared Error and Probability Guarantees

Two Different Standards for an Estimate

A credibility estimate can be judged by its average squared error across a portfolio of risks, or by the probability that it falls within a specified distance of the correct value for an individual risk. These are different questions. The first averages squared errors over both risk variation and process variation; the second concerns the distribution of an error and may be conditional on a particular risk.

The distinction matters in ratemaking. A credibility factor selected to minimize portfolio-wide expected squared error does not, by itself, ensure that an individual estimate is within a stated dollar tolerance with a stated probability. A probability guarantee needs information about the shape and tails of the error distribution, as well as a clear statement of whether the guarantee is conditional on a risk or averaged across risks.

Definition: Let $\Theta$ denote a risk’s underlying parameter, let $\mu(\Theta)=E[X\mid\Theta]$ be its expected outcome, and let $X_1,\ldots,X_n$ be conditionally independent observations with the same conditional mean and variance. Write $m=E[\mu(\Theta)]$, $\operatorname{EPV}=E[\operatorname{Var}(X\mid\Theta)]$, and $\operatorname{VHM}=\operatorname{Var}(\mu(\Theta))$. The portfolio-wide mean-squared error of an estimate $\hat{\mu}$ is $E[(\hat{\mu}-\mu(\Theta))^2]$, where the expectation includes both the distribution of risks and the observations generated for them.

The Bühlmann linear credibility estimate is $\hat{\mu}=Z\bar{X}+(1-Z)m$, where $\bar{X}$ is the risk’s sample mean and $Z$ is its credibility factor. For the usual Bühlmann choice, $K=\operatorname{EPV}/\operatorname{VHM}$ and $Z=n/(n+K)$, provided both components are positive and finite. These quantities summarize how process variation compares with variation among risk-level means. They do not specify the full distribution of an estimation error.

Why the Bühlmann Factor Minimizes Expected Squared Error

The MSE calculation uses the conditional mean and variance, not a normality assumption. Assume the observations are conditionally independent given $\Theta$, have conditional mean $\mu(\Theta)$, and have finite conditional variance. Assume also that $m$, EPV, and VHM are finite. Under these conditions, consider the linear estimate $m+Z(\bar{X}-m)$.

Separate the two sources of error.
Subtract the risk-level mean from the estimate: $\hat{\mu}-\mu(\Theta)=Z(\bar{X}-\mu(\Theta))-(1-Z)(\mu(\Theta)-m)$.

Take the expected square.
Given $\Theta$, the sample-mean deviation has mean zero, so its cross-product with the fixed, conditional term has expectation zero. Conditional independence gives $E[(\bar{X}-\mu(\Theta))^2]=\operatorname{EPV}/n$ after averaging over risks.

Minimize over $Z$.
The resulting quadratic is $Z^2\operatorname{EPV}/n+(1-Z)^2\operatorname{VHM}$. Setting its derivative to zero gives the minimizing weight.

\begin{aligned} \operatorname{MSE}(Z) &=E[(\hat{\mu}-\mu(\Theta))^2]\\ &=Z^2\frac{\operatorname{EPV}}{n}+(1-Z)^2\operatorname{VHM},\\ \frac{d}{dZ}\operatorname{MSE}(Z) &=2Z\frac{\operatorname{EPV}}{n}-2(1-Z)\operatorname{VHM}=0,\\ Z &=\frac{\operatorname{VHM}}{\operatorname{VHM}+\operatorname{EPV}/n} =\frac{n}{n+K}. \end{aligned}

The quadratic has positive second derivative $2\operatorname{EPV}/n+2\operatorname{VHM}$, so this stationary value is the minimum when EPV and VHM are positive. Substitution gives $\operatorname{MSE}_{\min}=\operatorname{EPV}/(n+K)$. This is a portfolio-wide average squared error for estimating $\mu(\Theta)$; it is not a statement that the error for every risk is small, nor is it itself a probability.

Worked Example: Minimum Portfolio-Wide Mean-Squared Error

A hypothetical portfolio has $\operatorname{EPV}=\$36{,}000$ and $\operatorname{VHM}=\$9{,}000$, in squared-dollar units. Each risk has $n=6$ conditionally independent observations. Find the Bühlmann factor and the minimum MSE for estimating a risk-level mean. No discounting or mortality model is involved. Only the stated moments are assumed; no distributional form is specified.

Method. First calculate $K$, then calculate $Z$, and use the minimum-MSE formula:

K=\frac{36{,}000}{9{,}000}=4,\qquad Z=\frac{6}{6+4}=0.6.

The minimum MSE is

\operatorname{MSE}_{\min} =\frac{36{,}000}{6+4} =\$3{,}600.

As a scale summary, the square root of this MSE is $\sqrt{\$3{,}600}=\$60$. It is an unconditional root-mean-squared error, not a standard deviation guaranteed for every risk. Check: the two terms in the MSE at $Z=0.6$ are $0.6^2(36{,}000/6)=\$2{,}160$ and $0.4^2(9{,}000)=\$1{,}440$; their sum is $\$3{,}600$. The method avoids interpreting the average MSE as a risk-specific probability statement.

Probability Guarantees Need an Error Distribution

A probability guarantee is a statement such as $P(|\hat{\mu}-\mu(\Theta)|\leq e)\geq p$, where $e$ is an allowed error and $p$ is a target probability. Unlike the MSE formula, this statement depends on the error distribution, including its tail behavior. EPV and VHM alone do not determine that distribution.

For example, suppose that, conditional on a fixed risk mean $\mu$, the observations are independent and $X_j\mid\mu\sim N(\mu,\sigma^2)$, with known $\sigma$. For a credibility estimate with known $m$ and fixed $Z$, the conditional error is normal:

\hat{\mu}-\mu \mid\mu \sim N\left((1-Z)(m-\mu),\,\frac{Z^2\sigma^2}{n}\right).

Here the second parameter of $N$ is the variance. The conditional bias is $(1-Z)(m-\mu)$, and the conditional standard deviation is $Z\sigma/\sqrt{n}$ for $Z\geq0$. If the guarantee is centered on the true mean, the probability is

P(|\hat{\mu}-\mu|\leq e\mid\mu) = \Phi\left(\frac{e-(1-Z)(m-\mu)}{Z\sigma/\sqrt n}\right) - \Phi\left(\frac{-e-(1-Z)(m-\mu)}{Z\sigma/\sqrt n}\right),

where $\Phi$ is the standard normal cumulative distribution function. If $\mu=m$, the conditional bias is zero. For the unshrunk sample mean, $Z=1$, the probability simplifies to $2\Phi(e\sqrt{n}/\sigma)-1$. A guarantee averaged over the portfolio instead requires averaging conditional probabilities over the distribution of $\Theta$; it is not generally the same as the guarantee for a particular risk.

Worked Example: Observation Count for a Normal-Theory Guarantee

In a hypothetical setting, a risk’s mean $\mu$ is fixed and the estimate is the sample mean of conditionally independent observations $X_j\mid\mu\sim N(\mu,\$200^2)$. Find the smallest $n$ for which the probability that the sample mean is within $\$100$ of $\mu$ is at least $90\%$. There is no discounting or mortality assumption; the conditional normal model and its standard deviation are as stated.

For the sample mean, the error has mean zero and standard deviation $\$200/\sqrt n$. Therefore,

P(|\bar{X}-\mu|\leq \$100\mid\mu) =2\Phi\left(\frac{100\sqrt n}{200}\right)-1 =2\Phi\left(\frac{\sqrt n}{2}\right)-1.

The $90\%$ two-sided target requires $\sqrt n/2\geq z_{0.95}$, where $z_{0.95}\approx1.6449$. Thus

n\geq(2\times1.6449)^2=10.823\ldots

Since $n$ must be an integer, the smallest qualifying value is $n=11$. At $n=10$, the standardized tolerance is $\sqrt{10}/2\approx1.5811$, giving $2\Phi(1.5811)-1\approx0.8862$, below the target. At $n=11$, it is $\sqrt{11}/2\approx1.6583$, giving a probability of about $0.9028$. The answer is checked on both sides of the integer threshold. The method avoids rounding $n$ down or treating a result just below $90\%$ as sufficient.

Worked Example: An Asymmetric Individual Error Band

Consider a hypothetical credibility estimate with $n=13$, $K=4$, and conditional observations $X_j\mid\mu\sim N(\mu,\$200^2)$, independent given $\mu$. Take a risk with $\mu=m$, so the conditional bias is zero. Find the probability that the estimation error $E=\hat{\mu}-\mu$ lies between $-1.336$ and $13\sqrt{14}/14$ conditional standard deviations. There is no discounting or mortality assumption.

First, $Z=13/(13+4)=13/17$. The conditional standard deviation of the credibility estimate’s error is

s_E=\frac{Z(200)}{\sqrt{13}} =\frac{13}{17}\frac{200}{\sqrt{13}} =\frac{200\sqrt{13}}{17} \approx \$42.418.

The requested interval is asymmetric about zero. Its dollar endpoints are $-1.336s_E\approx-\$56.67$ and $(13\sqrt{14}/14)s_E\approx\$147.38$. Standardizing the error gives the probability directly:

\begin{aligned} P(-1.336s_E\leq E\leq (13\sqrt{14}/14)s_E\mid\mu=m) &=\Phi\left(\frac{13\sqrt{14}}{14}\right)-\Phi(-1.336)\\ &\approx\Phi(3.4744)-\Phi(-1.336)\\ &\approx0.9090. \end{aligned}

The standard deviation calculation uses $200\sqrt{13}/17\approx\$42.418$, not $\$42.424$. The upper standardized limit is $13\sqrt{14}/14\approx3.4744$, which rounds to $3.474$ at three decimals. The probability, rounded to four decimals, is approximately $0.9090$. Check: the event’s lower endpoint is below zero and its upper endpoint is above zero, and the computed probability is consequently greater than one-half. The method avoids assuming that an asymmetric error band has a symmetric probability.

What the Two Criteria Do—and Do Not—Guarantee

The MSE criterion weights an error of size $2e$ four times as heavily as an error of size $e$, because the loss is squared. Its expectation smooths over risks and observations. A probability criterion instead counts whether the error falls inside a specified band; errors just inside and just outside the boundary may have nearly the same squared size but different outcomes under that criterion.

Even a known MSE does not reveal the probability inside a band. Two error distributions can have the same mean and variance but different tail probabilities. Normal-theory calculations are justified only when the relevant conditional or unconditional error distribution is actually normal under the stated model. A conditional guarantee for one risk also cannot be substituted for a portfolio-average guarantee without considering how the conditional probability varies across risks.

Key distinction: The Bühlmann factor $Z=n/(n+K)$ minimizes the expected squared error among linear estimates of the stated form under the Bühlmann moment structure. A probability guarantee such as $P(|\hat{\mu}-\mu(\Theta)|\leq e)\geq p$ is a separate requirement. It needs a specified error distribution and a clearly defined conditioning basis.

Common Exam Traps

Calling the minimum MSE a probability. An MSE has squared units, such as dollars squared. It cannot be read as a percentage chance of being within a dollar tolerance.
Using a normal distribution without a normal assumption. Knowing EPV and VHM is enough for the Bühlmann MSE derivation, but not enough to calculate a tail probability.
Ignoring conditional bias. For a credibility estimate, when $\mu\neq m$, the conditional error mean is $(1-Z)(m-\mu)$. Applying a centered normal formula to that risk gives the wrong probability.
Confusing conditional and portfolio-average guarantees. A probability conditional on one fixed risk mean answers a different question from a probability averaged over the portfolio’s risk distribution.
Rounding a required volume down. A minimum integer observation count must satisfy the probability target, not merely be close to the continuous threshold.

When a problem specifies only EPV, VHM, and observation volume, calculate the MSE or credibility factor that those moments support. If it asks for a probability, first identify the error distribution, the tolerance event, and whether the probability is conditional or unconditional. Keeping these criteria separate is aligned with the CAS Exam 8 (Advanced Ratemaking) syllabus.

Check Your Understanding

For each problem, identify whether the requested result is an MSE calculation or a probability guarantee, and check the assumptions before calculating.

A hypothetical portfolio has EPV $=\$25{,}000$, VHM $=\$5{,}000$, and $n=5$. Find $K$, $Z$, the minimum MSE, and its square root. (Answer: $K=5$, $Z=0.5000$, minimum MSE $=\$2{,}500$, square root $=\$50$.)
For a fixed risk mean, independent observations are normal with standard deviation $\$150$, and the estimate is their sample mean based on $n=9$. Find the probability that the estimate is within $\$75$ of the true mean. (Answer: $2\Phi(1.5)-1\approx0.8664$.)
Under the same fixed-mean normal model, the observation standard deviation is $\$120$. What is the smallest $n$ for which the sample mean is within $\$60$ of the true mean with probability at least $90\%$? (Answer: $n=\lceil(1.6449\times120/60)^2\rceil=11$.)
A hypothetical risk has $n=8$, EPV $=\$18{,}000$, and VHM $=\$9{,}000$. Find its Bühlmann factor and the minimum MSE. (Answer: $K=2$, $Z=8/(8+2)=0.8000$, minimum MSE $=\$18{,}000/(8+2)=\$1{,}800$.)
A problem supplies only the mean and variance of an estimation error and asks for the probability that its absolute value is at most a stated tolerance. Is the probability determined? State what additional information would make a calculation possible. (Answer: No; the mean and variance alone do not determine the error distribution or its tail probability. A distributional assumption, plus the relevant conditional or unconditional basis, is needed.)

What You'll Learn