Fitting Heavy-Tailed Operational Loss Severities

Why Tail Fitting Needs More Than a Distribution Choice

Operational loss data often contain many modest events and only a few very large ones. A model that describes the body of the severity distribution can therefore fit ordinary losses well while misrepresenting the upper tail that drives large-loss estimates. Tail fitting focuses on losses above a threshold and asks whether a simpler tail model is defensible there. The methods here are aligned with the CAS Exam 9 (Risk Management for Actuaries) syllabus.

The central practical difficulty is choosing the threshold. A low threshold leaves more observations but may include losses that do not follow the proposed tail model. A high threshold may produce a more credible tail approximation but leave very few observations, making estimates unstable. A fitted parameter is not persuasive by itself: it should be considered alongside plots, stability across thresholds, the number of exceedances, and uncertainty.

Definition: Let $X$ be a loss severity and let $u>0$ be a threshold. An exceedance is an observed loss $X>u$. The threshold-excess variable is $Y=X-u$, conditional on $X>u$. A tail model describes the distribution of losses above $u$, or equivalently the distribution of $Y$ among exceedances.

Pareto Tails Above a Threshold

A Pareto tail assumes that, conditional on exceeding $u$, the survival probability for a loss $x\geq u$ is a power function. Write $\alpha>0$ for the tail index. A smaller $\alpha$ means a heavier tail: conditional loss probabilities decline more slowly as $x$ increases.

Definition: Under the threshold Pareto model, $$ P(X>x\mid X>u)=\left(\frac{x}{u}\right)^{-\alpha},\qquad x\geq u. $$ The corresponding conditional density is $f(x\mid X>u)=\alpha u^\alpha x^{-\alpha-1}$. This is a model for the conditional tail, not for the probability of exceeding $u$ in the full loss population.

Suppose $x_1,\ldots,x_n$ are the observed losses above $u$, treated as independent observations from this conditional tail. The maximum-likelihood estimate is

\widehat{\alpha}=\frac{n}{\displaystyle\sum_{i=1}^{n}\ln(x_i/u)}.

To see why, set $Z_i=\ln(X_i/u)$. Under the Pareto tail, $P(Z_i>z)=e^{-\alpha z}$ for $z\geq0$, so $Z_i$ has an exponential distribution with rate $\alpha$. Its likelihood is proportional to $\alpha^n e^{-\alpha\sum_i Z_i}$. The log-likelihood is $n\ln(\alpha)-\alpha\sum_i Z_i$; setting its derivative $n/\alpha-\sum_i Z_i$ to zero gives the estimate above.

Worked Example: Estimating a Pareto Tail and Its Uncertainty

In a hypothetical operational loss study, 100 losses exceed $u=\$100{,}000$. The sum of their log ratios, $\sum_{i=1}^{100}\ln(x_i/u)$, is 50. Estimate the Pareto tail index, give an approximate standard error and 95% interval, and calculate the conditional probability that a loss exceeds $\$500{,}000$, given that it exceeds the threshold.

Method. Use the conditional Pareto maximum-likelihood estimate. For this model, the approximate standard error of $\widehat{\alpha}$ is $\widehat{\alpha}/\sqrt{n}$, based on the large-sample likelihood approximation.

Estimate. $$ \widehat{\alpha}=\frac{100}{50}=2. $$ The standard error is $2/\sqrt{100}=0.20$. The approximate normal 95% interval is $$ 2\pm1.96(0.20)=2\pm0.392, $$ or $(1.608,2.392)$. This interval is an approximation; it does not account for uncertainty from choosing the threshold or for possible model misspecification.

Conditional tail probability. Since $\$500{,}000/u=5$, $$ P(X>\$500{,}000\mid X>\$100{,}000)=5^{-2}=0.04. $$ Thus the estimated probability is 4% among losses already above $\$100{,}000$. As a check, if a separate estimate says 2% of all losses exceed $\$100{,}000$, the implied unconditional probability of exceeding $\$500{,}000$ is $0.02(0.04)=0.0008$, or 0.08%.

Common mistake avoided. Applying 4% directly to all losses confuses a conditional tail probability with an unconditional one. The multiplication by the probability of exceeding $u$ is required to obtain the unconditional probability.

The Pareto model also implies a finite conditional mean only when $\alpha>1$, in which case $E[X\mid X>u]=\alpha u/(\alpha-1)$. It implies a finite conditional variance only when $\alpha>2$. These conditions matter if a tail estimate is used to summarize expected severity: a numerical fit does not make a theoretically infinite moment finite.

Generalized Pareto Tails

A generalized Pareto distribution (GPD) allows the tail shape to vary beyond the pure power-tail case. For threshold excess $Y=X-u$, let $\beta>0$ be the scale parameter and $\xi$ the shape parameter. Positive $\xi$ gives a heavy, unbounded tail; $\xi=0$ gives the exponential limit; and negative $\xi$ gives a finite upper endpoint.

Definition: The GPD excess survival function is $$ P(Y>y\mid X>u)= \begin{cases} (1+\xi y/\beta)^{-1/\xi}, & \xi\ne0,\\ e^{-y/\beta}, & \xi=0, \end{cases} $$ for $y\geq0$ and $1+\xi y/\beta>0$. When $\xi<0$, the support ends at $y=-\beta/\xi$. For $\xi<1$, the excess mean is $\beta/(1-\xi)$; for $\xi<1/2$, its variance is $\beta^2/((1-\xi)^2(1-2\xi))$.

GPD parameters are commonly estimated by maximum likelihood, usually with numerical optimization. A useful hand-calculation illustration is the method of moments, which matches the empirical mean and variance of the excesses to the GPD moments. It requires a finite empirical variance and gives a feasible moment fit only if the resulting shape and support are compatible with the observations. It is not a substitute for checking fit or for a likelihood-based analysis when that is required.

Result: If the empirical excess mean is $m$ and the empirical variance is $s^2$, the method-of-moments estimates are $$ \widehat{\xi}=\frac{1}{2}\left(1-\frac{m^2}{s^2}\right), \qquad \widehat{\beta}=m(1-\widehat{\xi}). $$ These follow by dividing the GPD variance by the square of its mean, giving $s^2/m^2=1/(1-2\xi)$, and then solving the mean equation for $\beta$.

Worked Example: A Method-of-Moments GPD Fit

A hypothetical portfolio has four losses above a threshold. Their excesses, in thousands of dollars, are $1,3,7,9$. Fit a GPD by moments using the empirical variance with divisor $n$, and check that the fitted support includes the observed excesses.

Method. Calculate the empirical mean and variance, then substitute them into the moment formulas.

The mean excess is $m=(1+3+7+9)/4=20/4=5$. The squared deviations from 5 are $16,4,4,16$, so the empirical variance is $s^2=(16+4+4+16)/4=40/4=10$. Therefore $m^2/s^2=25/10=2.5$, and $$ \widehat{\xi}=\frac{1}{2}(1-2.5)=-0.75,\qquad \widehat{\beta}=5(1-(-0.75))=8.75. $$

Support and moment check. The estimated upper endpoint for the excess is $-\widehat{\beta}/\widehat{\xi}=8.75/0.75=11.6667$, which exceeds the largest observed excess, 9. The fitted mean is $8.75/(1.75)=5$. The fitted variance is $8.75^2/(1.75^2(2.5))=76.5625/7.65625=10$. Both match the empirical moments, as required.

Common mistake avoided. Using the sample variance with divisor $n-1$ would give a different moment fit. Here the method explicitly uses divisor $n$; the variance convention must be consistent with the moment equations.

Threshold Choice and Tail Diagnostics

A threshold should not be selected merely because it produces a convenient estimate. Several diagnostics provide complementary evidence. An empirical exceedance plot displays observed tail behavior above candidate thresholds. For a Pareto model, plotting the logarithm of the empirical conditional exceedance probability against $\ln(x/u)$ should produce an approximately straight line with slope $-\alpha$. Marking the threshold and the number of exceedances makes clear how much data supports the apparent straightness.

A mean-excess plot displays the average of $X-u$ among observations exceeding each candidate threshold $u$. For a GPD tail with $\xi<1$, the theoretical mean excess at threshold $u$ is $\beta_u/(1-\xi)$. As the threshold rises by $t$, the GPD scale changes to $\beta_{u+t}=\beta_u+\xi t$, while $\xi$ remains unchanged. Consequently, the theoretical mean excess is linear in the threshold where the GPD approximation applies. Real plots are noisy, especially when few observations remain.

This threshold-stability property gives a practical check. Estimates of $\xi$ should be reasonably stable over a range of plausible thresholds. If $\xi$ is stable, the corresponding scale estimates should change in a way broadly consistent with $\beta_v=\beta_u+\xi(v-u)$ when moving from threshold $u$ to a higher threshold $v$. A threshold is more credible when plots and stability checks support it, while enough exceedances remain for useful estimation.

Worked Example: Using Stability to Select a Threshold

A hypothetical loss sample yields the following GPD fits. Plot inspection shows curvature in the empirical exceedance plot below $u=\$80{,}000$, and approximately linear behavior above that level.

Threshold $u$ (thousands of dollars)	Exceedances	Estimated $\xi$	Estimated $\beta_u$ (thousands)
80	80	0.12	25
100	50	0.25	30
120	31	0.27	35
140	19	0.24	40

Method. Compare the shape estimates and check scale changes against the GPD threshold-stability relation. At $u=100$, $\widehat{\xi}=0.25$ and $\widehat{\beta}_{100}=30$. Raising the threshold by 20 predicts $$ \widehat{\beta}_{120}=30+0.25(120-100)=35. $$ Raising it by 40 predicts \(\widehat{\beta}_{140}=30+0.25(140-100)=40. $$ Both agree with the table. The shape estimates at 100, 120, and 140 are also close, while the estimate at 80 differs.

A threshold around 100 is a defensible candidate: behavior below 80 appears inconsistent with the tail approximation, and the estimates from 100 upward are broadly stable. The choice is not automatic. At 140 only 19 exceedances remain, so the apparently stable estimate has greater sampling uncertainty. Additional diagnostics and sensitivity analysis would be appropriate before relying on the fit.

Common mistake avoided. Choosing the largest threshold because it is furthest into the tail ignores the loss of information from having fewer exceedances. Threshold selection balances bias from including non-tail observations against variance from fitting too few observations.

Uncertainty, Limitations, and Interpretation

For the Pareto maximum-likelihood estimate, the approximate standard error $\widehat{\alpha}/\sqrt{n}$ relies on a correctly specified conditional Pareto model and a sufficiently large number of independent exceedances. A normal interval can be unreliable with very few observations, and its lower endpoint should not be interpreted as a valid negative tail index. In that situation, use a suitable positive-parameter interval or likelihood-based limits rather than treating the rough normal approximation as exact.

For a GPD fit, uncertainty is commonly assessed from the likelihood’s curvature or by resampling exceedances and refitting the model. A bootstrap should preserve the analysis being evaluated: if threshold selection is part of the procedure, repeat that selection within each resample to reflect some of its uncertainty. An interval conditional on a fixed threshold does not capture uncertainty about whether that threshold is appropriate.

No diagnostic proves that the far tail follows the fitted model. Exceedance plots can look linear over a limited range by chance, and estimates can appear stable simply because the data are sparse. Report the threshold, exceedance count, model, parameter estimates, uncertainty method, and meaningful sensitivity checks together. Keep conditional tail probabilities distinct from probabilities for all losses; the latter also depend on how often losses exceed the threshold.

Set candidate thresholds.
Use the loss data and modeling context to identify plausible levels; retain the count of exceedances at each.

Inspect tail behavior.
Use empirical exceedance and mean-excess plots to identify ranges where a tail approximation is plausible.

Fit and compare.
Estimate Pareto or GPD parameters across candidate thresholds and check shape and scale stability.

Quantify uncertainty.
Use an appropriate likelihood or resampling method, and state whether threshold uncertainty is included.

Key takeaway: A tail estimate is credible only when the assumed tail model is plausible above the chosen threshold, the parameter behavior is reasonably stable, and the remaining exceedance data support the claimed precision.

Check Your Understanding

For each problem, state any conditional interpretation clearly.

Forty losses exceed a threshold $u$, and their log ratios have sum 20. Under a Pareto model, find the maximum-likelihood estimate of $\alpha$ and estimate $P(X>4u\mid X>u)$. (Answer: $\widehat{\alpha}=2$; probability $=0.0625$.)
A GPD fit has $\xi=0.20$ and $\beta=12$ (in thousands of dollars). Find the probability that an excess exceeds 30 thousand dollars. (Answer: $(1+0.20(30)/12)^{-5}=1.5^{-5}=0.1317$.)
At threshold $u=50$, a GPD fit has $\xi=0.20$ and $\beta_u=15$. What scale does threshold stability predict at $v=70$? (Answer: $\beta_{70}=15+0.20(70-50)=19$.)
In 500 hypothetical losses, 40 exceed $u$. A Pareto fit estimates $\alpha=2$. Estimate the unconditional probability of exceeding $3u$, using the observed threshold-exceedance fraction. (Answer: $(40/500)(3^{-2})=0.0089$, or about 0.89%.)
A GPD method-of-moments fit uses excess mean $m=4$ and empirical variance $s^2=8$. Find $\widehat{\xi}$ and $\widehat{\beta}$. (Answer: $\widehat{\xi}=\tfrac12(1-16/8)=-0.50$; $\widehat{\beta}=4(1.50)=6$.)

What You'll Learn