Diagnosing Tail Fit In Actuarial Model Selection

Why Diagnose the Upper Tail?

For many hypothetical insurance portfolios, a small number of large losses can matter disproportionately to excess-of-loss pricing, capital estimates, and aggregate-loss projections. A severity model can describe the bulk of observed losses reasonably well while understating the probability or size of very large losses. Model-selection criteria such as AIC and BIC summarize fit across the likelihood contribution of the full data set; they do not, by themselves, show whether a model represents the upper tail adequately.

Tail diagnosis therefore asks a focused question: does the candidate model reproduce the observed behavior of large losses? The answer is not based on a single point or plot. It combines empirical exceedance rates, upper-tail probability or quantile plots, and the sensitivity of quantities such as the probability of exceeding a threshold or the expected payment above it. These diagnostics supplement, rather than replace, the overall fit measures.

Definition: For a nonnegative loss $X$, the survival function is $S_X(x)=P(X>x)=1-F_X(x)$. An upper-tail diagnostic compares observed large losses or empirical exceedance probabilities with those implied by a fitted model. A tail-sensitive quantity is a quantity whose value depends especially on losses beyond a high threshold, such as $S_X(d)$ or $E[(X-d)_+]$, where $(z)_+=\max(z,0)$.

Empirical and Model-Based Tail Comparisons

At a threshold $x$, the empirical exceedance probability from $n$ complete observations $X_1,\ldots,X_n$ is the proportion exceeding $x$:

\widehat{S}_X(x)=\frac{1}{n}\sum_{i=1}^{n} I(X_i>x).

This estimate is easy to interpret, but its precision depends on how many observations exceed the threshold. If only a few observations lie above $x$, one additional exceedance can change the empirical rate substantially. A fitted model supplies a smooth estimate $S_X(x)$, but that estimate depends on the model family and fitted parameters. Comparing both estimates at several relevant thresholds is more informative than focusing on one threshold.

A probability plot compares empirical probabilities with model probabilities. A quantile plot instead compares observed order statistics with model quantiles. For ordered observations $X_{(1)}\le\cdots\le X_{(n)}$, one common plotting position for $X_{(j)}$ is $p_j=j/(n+1)$. Its corresponding empirical survival position is $1-p_j$. These plotting positions avoid assigning an observed point probability exactly zero or one. Other plotting-position conventions exist, so a comparison should use the same stated convention throughout.

In a quantile plot, the horizontal coordinate may be the fitted model quantile $F_X^{-1}(p_j)$, and the vertical coordinate the observed $X_{(j)}$. Points well above a reference line indicate that the observations are larger than the model quantiles at those plotting positions. A tail probability plot can instead compare observed and fitted exceedance probabilities over selected thresholds. Either plot should be read for patterns across the upper tail, not as a formal proof that one model is correct.

Worked Example: Comparing Tail Exceedances With Overall Fit

A hypothetical insurer fits two severity models to the same $n=200$ complete loss observations. The fitted exponential model has mean $\beta=100$ (in thousands of dollars) and $-2\ell=2410$, with $k=1$ free parameter. The fitted Lomax model has $\theta=100$, $\alpha=2$, and $-2\ell=2416$, with $k=2$. There are 16 observed losses above $300$ thousand dollars. Compare AIC and the fitted tail probabilities at that threshold.

Method. For a model with $k$ free parameters, use $\mathrm{AIC}=-2\ell+2k$. The exponential survival function is $S_X(x)=e^{-x/\beta}$. For the Lomax model, $S_X(x)=(1+x/\theta)^{-\alpha}$.

Step 1: Calculate AIC. For the exponential model, $\mathrm{AIC}=2410+2(1)=2412$. For the Lomax model, $\mathrm{AIC}=2416+2(2)=2420$. The AIC difference is $2420-2412=8$, so AIC favors the exponential model on this whole-sample comparison.

Step 2: Calculate the empirical exceedance rate. The observed rate above $300$ is $16/200=0.0800$.

Step 3: Calculate each fitted exceedance probability. Under the exponential model, $S_X(300)=e^{-300/100}=e^{-3}=0.0498$ to four decimals. Under the Lomax model, $S_X(300)=(1+300/100)^{-2}=4^{-2}=0.0625$.

Step 4: Compare with the observations. The models imply $200(0.0498)=9.96$ and $200(0.0625)=12.50$ exceedances, respectively. The observed count is 16. The Lomax model is closer at this threshold: its absolute rate discrepancy is $0.0800-0.0625=0.0175$, compared with $0.0800-0.0498=0.0302$ for the exponential model.

AIC favors the exponential model overall, while the threshold comparison shows the Lomax model closer to the observed exceedance rate at this particular point. This does not establish that the Lomax model is the better model in every respect; it identifies a specific tail feature that the overall criterion does not display. The example avoids treating an overall information criterion as if it were a direct test of upper-tail fit.

Reading Upper-Tail Quantile Plots

A quantile plot asks whether the largest observed order statistics are in line with the fitted model’s upper quantiles. The interpretation depends on the axes: with observed losses on the vertical axis and fitted quantiles on the horizontal axis, points above the equality line indicate observed losses larger than the corresponding fitted quantiles. A cluster of such points in the upper tail suggests the fitted model may be too light-tailed for those data.

The following common severity models illustrate why tail shape matters. An exponential model with mean $\beta>0$ has $S_X(x)=e^{-x/\beta}$ for $x\geq 0$, and quantile $F_X^{-1}(p)=-\beta\ln(1-p)$. A Lomax model with scale $\theta>0$ and shape $\alpha>0$ has $S_X(x)=(1+x/\theta)^{-\alpha}$ for $x\geq0$, and quantile $F_X^{-1}(p)=\theta((1-p)^{-1/\alpha}-1)$. These expressions allow the same plotting positions to be compared under each fitted model.

Worked Example: Comparing Upper Order Statistics With Model Quantiles

In a hypothetical sample of $n=15$ losses, the three largest observations, in thousands of dollars, are $140$, $250$, and $500$. Compare them with fitted exponential and Lomax quantiles. The exponential fit has $\beta=100$; the Lomax fit has $\theta=100$ and $\alpha=2$. Use plotting positions $p_j=j/(n+1)$.

Method. The observations are $X_{(13)},X_{(14)},X_{(15)}$, so their plotting positions are $13/16=0.8125$, $14/16=0.8750$, and $15/16=0.9375$. Calculate each model quantile at those probabilities and compare observed minus fitted values.

Step 1: Exponential quantiles. At $p=0.8125$, the quantile is $-100\ln(0.1875)=167.4$. At $p=0.8750$, it is $-100\ln(0.125)=207.9$. At $p=0.9375$, it is $-100\ln(0.0625)=277.3$. Values are in thousands of dollars, rounded to one decimal.

Step 2: Lomax quantiles. At $p=0.8125$, the quantile is $100((0.1875)^{-1/2}-1)=130.9$. At $p=0.8750$, it is $100((0.125)^{-1/2}-1)=182.8$. At $p=0.9375$, it is $100((0.0625)^{-1/2}-1)=300.0$.

Plotting position	Observed loss	Exponential quantile	Observed minus exponential	Lomax quantile	Observed minus Lomax
0.8125	140	167.4	-27.4	130.9	9.1
0.8750	250	207.9	42.1	182.8	67.2
0.9375	500	277.3	222.7	300.0	200.0

Step 3: Interpret the pattern. Both models understate the largest observation at plotting position $0.9375$, and the Lomax quantile is closer there. At $0.8750$, however, the exponential quantile is closer. The three points therefore do not support a simple claim that one model fits every upper-tail point better. A plot makes this changing pattern visible; the small number of upper order statistics also cautions against treating it as conclusive evidence.

The method avoids comparing raw losses with quantiles at mismatched probabilities. Pairing each order statistic with its stated plotting position is essential to interpreting the plot.

Why Tail Fit Changes Excess-Loss Estimates

One useful tail-sensitive quantity is the expected payment per loss above a deductible $d$, before any policy limit: $E[(X-d)_+]$. This is not the conditional mean loss given that a loss exceeds $d$. It is the unconditional expected amount above the threshold, counting zero for losses at or below it. For a nonnegative severity with a finite expectation, the identity below links this expected excess to the entire survival curve above $d$.

E[(X-d)_+]=\int_d^\infty S_X(x)\,dx.

To justify the identity, write $(X-d)_+=\int_d^\infty I(X>x)\,dx$. For a fixed loss value $X$, the indicator is one from $d$ up to $X$ if $X>d$, and is zero throughout if $X\le d$; thus the integral equals $(X-d)_+$. Taking expectations and interchanging the nonnegative integral and expectation gives $\int_d^\infty P(X>x)\,dx$, which is the stated result. The integral requires a finite value when the expected excess is to be finite.

Worked Example: Expected Excess Above a High Threshold

A hypothetical excess-of-loss layer attaches at $d=200$ thousand dollars and has no upper limit. Compare the expected payment per loss under an exponential model with mean $\beta=100$ and a Lomax model with $\theta=100$, $\alpha=2$. Both models are nonnegative and have finite means.

Method. Integrate each survival function above $d$. For the exponential model, $E[(X-d)_+]=\int_d^\infty e^{-x/\beta}\,dx=\beta e^{-d/\beta}$. For the Lomax model with $\alpha>1$, $E[(X-d)_+]=\int_d^\infty(1+x/\theta)^{-\alpha}\,dx =\frac{\theta}{\alpha-1}(1+d/\theta)^{1-\alpha}$.

Step 1: Exponential result. Substituting $d=200$ and $\beta=100$ gives $100e^{-2}=13.5335$ thousand dollars, or $\$13,533.53$ per loss when rounded to cents.

Step 2: Lomax result. Substituting $\theta=100$, $\alpha=2$, and $d=200$ gives $\frac{100}{2-1}(1+200/100)^{1-2}=100(3)^{-1}=33.3333$ thousand dollars, or $\$33,333.33$ per loss when rounded to cents.

Step 3: Check the interpretation. The Lomax expected excess is larger even though its exceedance probability at $200$ is $S_X(200)=1/9=0.1111$, below the exponential probability $e^{-2}=0.1353$. The probability of crossing the attachment and the expected amount above it measure different features: fewer exceedances can still produce a larger expected excess if the conditional losses above the attachment are larger.

The method avoids inferring expected excess solely from the probability of exceeding the attachment. A tail comparison should consider both how often losses cross a threshold and how far beyond it they extend.

Common Tail-Diagnostic Traps

Confusing exceedance with non-exceedance. If 16 of 200 observations exceed a threshold, the empirical exceedance rate is $16/200=0.0800$, not $184/200=0.9200$. The latter is the observed proportion that does not exceed it.
Using a high-tail estimate as if it were precise. With few observations beyond a high threshold, the empirical rate is sensitive to individual losses. Report the exceedance count as well as the rate, and interpret the plot with that limitation in mind.
Assuming a better overall criterion guarantees a better tail fit. AIC and BIC reflect the fitted likelihood over the data used. Neither criterion directly measures closeness at one particular high threshold.
Multiplying an unconditional mean by an exceedance probability. In general, $E[(X-d)_+]\ne E[X]S_X(d)$. This incorrectly treats the loss amount as independent of whether it exceeds $d$. For the Lomax example, that shortcut gives $100(1/9)=11.1111$ thousand dollars, whereas integration of the survival function gives $33.3333$ thousand dollars. The shortcut is not a valid general identity.
Over-reading the largest point in a quantile plot. A single large loss can have substantial influence on the apparent tail pattern. Check several upper-tail points and, where relevant, compare fitted exceedance probabilities across thresholds.

Key takeaway: Use overall fit criteria to summarize whole-sample model support, then use upper-tail plots and threshold-specific calculations to inspect the model features relevant to large losses. State which tail quantity is being compared, retain the observed exceedance count, and distinguish evidence of a pattern from certainty about the true severity distribution. This treatment is aligned with the SOA Exam ASTAM (Advanced Short-Term Actuarial Mathematics) syllabus.

Check Your Understanding

Work each problem using the stated model and units. Probabilities are given to four decimals.

A fitted exponential severity model has mean $125$ thousand dollars. Calculate the model probability that a loss exceeds $400$ thousand dollars. (Answer: $0.0408$)
A fitted Lomax model has $\theta=150$ thousand dollars and $\alpha=3$. Calculate $S_X(300)$. (Answer: $0.0370$)
A sample contains $n=19$ complete losses. Using plotting positions $p_j=j/(n+1)$, what plotting position and empirical survival position correspond to the largest observation? (Answer: $p_{19}=0.9500$; empirical survival position $0.0500$)
For an exponential severity with mean $80$ thousand dollars, calculate the expected payment per loss above a $160$-thousand-dollar attachment, with no limit. Give dollars to the cent. (Answer: $\$10,826.82$)
A Lomax severity has $\theta=120$ thousand dollars and $\alpha=2$. Calculate the expected payment per loss above $d=240$ thousand dollars, with no limit. Give dollars to the cent. (Answer: $\$40,000.00$)

What You'll Learn