Pareto Severity Estimation and Tail Index

Why the Pareto Tail Index Matters

Severity models describe the sizes of individual losses. When large losses are important, the model’s tail behavior matters as much as its average. The Pareto Type I distribution is useful for studying this behavior because its tail decreases as a power of the loss amount. Its parameter $\alpha$, called the tail index, controls both the rate of that decrease and which moments of loss are finite.

This tutorial uses complete, independent observations from a continuous Pareto Type I model. The scale parameter $\theta$ is positive, and losses cannot be below $\theta$. We distinguish estimation when $\theta$ is known from estimation when both parameters are unknown. Likelihood formulas below assume observations are not censored or truncated.

Definition: A loss $X$ has a Pareto Type I distribution with scale $\theta>0$ and tail index $\alpha>0$ if its density and survival function are $$ f_X(x)=\frac{\alpha\theta^\alpha}{x^{\alpha+1}},\quad x\geq\theta, \qquad S_X(x)=P(X>x)=\left(\frac{\theta}{x}\right)^\alpha,\quad x\geq\theta. $$ The distribution function is $F_X(x)=0$ for $x<\theta$ and $F_X(x)=1-(\theta/x)^\alpha$ for $x\geq\theta$.

The scale sets the minimum possible loss, while the tail index controls how quickly the probability of exceeding a high threshold declines. For example, for any $c\geq1$, $P(X>c\theta)=c^{-\alpha}$. A smaller $\alpha$ gives a heavier tail: the probability of a loss many times the scale is larger.

Moments and Their Existence

For any real $r$, the raw moment can be evaluated by integrating the density over its support. The integral converges precisely when $r<\alpha$.

E[X^r] =\alpha\theta^\alpha\int_\theta^\infty x^{r-\alpha-1}\,dx =\frac{\alpha\theta^r}{\alpha-r},\qquad r<\alpha.

If $r\geq\alpha$, the integral over the upper tail diverges. In particular, the mean exists only for $\alpha>1$, and the second raw moment exists only for $\alpha>2$. When $\alpha>2$, subtracting the squared mean from the second raw moment gives the variance.

E[X]=\frac{\alpha\theta}{\alpha-1}\quad(\alpha>1), \qquad \operatorname{Var}(X)=\frac{\alpha\theta^2}{(\alpha-1)^2(\alpha-2)} \quad(\alpha>2).

These existence conditions are important when fitting by moments: a formula for a mean or variance cannot be used as a finite population moment when its condition fails. More generally, since $X\geq\theta>0$, every real-order raw moment with $r<\alpha$ is finite; for positive-order moments, the condition is simply $r<\alpha$.

Worked Example: A Moment Estimate With Known Scale

Suppose a hypothetical portfolio has complete independent losses $100, 200, 300,$ and $500$ dollars. Assume a Pareto Type I model with known scale $\theta=\$100$, and estimate $\alpha$ by matching the sample mean to the model mean. Then assess whether the fitted model has a finite mean and variance.

Method. With known scale, equate the sample mean $\bar{x}$ to $E[X]=\alpha\theta/(\alpha-1)$. This requires $\alpha>1$.

Step 1: Calculate the sample mean. The total is $100+200+300+500=\$1{,}100$, so $\bar{x}=1{,}100/4=\$275$.

Step 2: Solve the moment equation. Set $275=100\alpha/(\alpha-1)$. Multiplying by $\alpha-1$ gives $275\alpha-275=100\alpha$, so $175\alpha=275$, and $\widehat{\alpha}_{\mathrm{MOM}}=275/175=11/7\approx1.5714$.

Step 3: Check the moments. Since $11/7>1$, the fitted mean exists and equals $100(11/7)/(4/7)=\$275$, as required by the moment match. Since $11/7<2$, the fitted variance is infinite; the finite-variance formula does not apply.

The estimate matches the sample mean but does not imply that the fitted distribution has finite variance. The common mistake is to calculate a variance by substituting $\widehat{\alpha}_{\mathrm{MOM}}$ into the variance formula without first checking whether $\widehat{\alpha}_{\mathrm{MOM}}>2$.

Method of Moments Estimation

When the scale $\theta$ is known and the sample mean $\bar{x}$ is greater than $\theta$, matching the model mean gives a direct estimate:

\bar{x}=\frac{\alpha\theta}{\alpha-1} \quad\Longrightarrow\quad \widehat{\alpha}_{\mathrm{MOM}}=\frac{\bar{x}}{\bar{x}-\theta}.

A valid sample from this model has every observation at least $\theta$, so $\bar{x}\geq\theta$. If $\bar{x}=\theta$, all observations must equal $\theta$; the displayed finite estimate is then undefined, and the mean-matching equation has no finite solution. This is a boundary case, not a reason to substitute an arbitrary large value.

If both parameters are unknown, two moment equations can be used when the sample moments support a solution. Let $m_1$ be the sample mean and $m_2$ the sample second raw moment, calculated with divisor $n$. Set $s_m^2=m_2-m_1^2$, the corresponding moment variance. For a Pareto Type I model with $\alpha>2$, the squared coefficient of variation is $1/[\alpha(\alpha-2)]$. Thus, when $s_m^2>0$, the two-parameter moment estimates are

\widehat{\alpha}_{\mathrm{MOM}} =1+\sqrt{1+\frac{m_1^2}{s_m^2}}, \qquad \widehat{\theta}_{\mathrm{MOM}} =m_1\frac{\widehat{\alpha}_{\mathrm{MOM}}-1}{\widehat{\alpha}_{\mathrm{MOM}}}.

This solution has $\widehat{\alpha}_{\mathrm{MOM}}>2$, as required for matching a finite variance. It is not a suitable variance-based fit when the underlying model has infinite variance, and a sample moment match does not establish that the Pareto model is appropriate.

Worked Example: Moment Estimates With Both Parameters Unknown

A hypothetical complete sample, in thousands of dollars, has moment mean $m_1=2$ and moment variance $s_m^2=1$, using divisor $n$ for the empirical raw moments. Fit a two-parameter Pareto Type I model by moments and verify the fitted moments.

Method. Match the first two raw moments through the mean and variance formulas; this requires a finite variance, so the fitted $\alpha$ must exceed 2.

Step 1: Estimate the tail index. Here $m_1^2/s_m^2=2^2/1=4$. Therefore $\widehat{\alpha}_{\mathrm{MOM}}=1+\sqrt{1+4}=1+\sqrt{5}\approx3.2361$.

Step 2: Estimate the scale. The scale estimate is $2(3.2361-1)/3.2361\approx1.3819$ thousand dollars. Using the unrounded expression, it is $2\sqrt{5}/(1+\sqrt{5})$ thousand dollars.

Step 3: Check the fitted moments. The fitted mean is $\widehat{\alpha}\widehat{\theta}/(\widehat{\alpha}-1)=2$ thousand dollars by construction. The fitted variance is $\widehat{\alpha}\widehat{\theta}^{\,2}/[(\widehat{\alpha}-1)^2(\widehat{\alpha}-2)]=1$ squared thousand dollars, also by construction. The estimate exceeds 2, so both formulas used are valid.

The common mistake is to use the usual sample variance with divisor $n-1$ while treating it as the second moment-matching quantity $m_2-m_1^2$. For this method, use the stated moment variance with divisor $n$.

Maximum Likelihood Estimation

For independent complete observations $x_1,\ldots,x_n$ from a Pareto Type I model with known scale $\theta$, all observations must satisfy $x_j\geq\theta$. The likelihood as a function of $\alpha$ is the product of their densities. Taking logarithms gives a convenient expression to differentiate.

L(\alpha)=\alpha^n\theta^{n\alpha}\prod_{j=1}^n x_j^{-(\alpha+1)}, \qquad \ell(\alpha)=n\ln\alpha+n\alpha\ln\theta-(\alpha+1)\sum_{j=1}^n\ln x_j.

Differentiating the log-likelihood gives the score $n/\alpha-\sum_{j=1}^n\ln(x_j/\theta)$. If the sum of log ratios is positive, setting the score to zero yields a finite stationary point. The second derivative is $-n/\alpha^2<0$, so the log-likelihood is strictly concave and the stationary point is its maximum.

\widehat{\alpha}_{\mathrm{MLE}} =\frac{n}{\displaystyle\sum_{j=1}^n\ln(x_j/\theta)}, \qquad \text{if }\sum_{j=1}^n\ln(x_j/\theta)>0.

The condition matters. If every observation equals the known scale, the sum is zero and $L(\alpha)$ is proportional to $\alpha^n$, which grows without bound as $\alpha\to\infty$. There is then no finite MLE; the formula must not be used with a zero denominator. If $\theta$ is unknown, the likelihood also depends on the support restriction $\theta\leq\min_j x_j$. For a nonconstant sample, the joint MLE sets $\widehat{\theta}=\min_jx_j$ and uses the corresponding log-ratio formula for $\alpha$. If all observations are equal, the likelihood is unbounded as $\alpha\to\infty$, so there is no finite joint MLE either.

Check the model and data.
Confirm the Pareto Type I support, whether the scale is known, and that observations are complete and at least the stated scale.

Choose the estimator.
For moments, identify which population moments are being matched and check their existence conditions. For likelihood, use the appropriate support and likelihood.

Check boundary cases.
Before reporting an estimate, check for a zero log-ratio sum or a moment equation with no finite solution.

Interpret the fitted tail.
Use the fitted $\alpha$ to assess finite moments and the rate at which exceedance probabilities decrease.

Worked Example: Maximum Likelihood With Known Scale

Assume five hypothetical independent, complete losses follow a Pareto Type I distribution with known scale $\theta=\$1{,}000$. The observed losses are $\$1{,}000,\$2{,}000,\$4{,}000,\$8{,}000,$ and $\$16{,}000$. Find the maximum-likelihood estimate of $\alpha$, and determine whether the fitted model has a finite mean.

Method. Apply the known-scale likelihood formula to the log ratios. It applies if at least one observation exceeds the scale.

Step 1: Form the ratios. The ratios $x_j/\theta$ are $1,2,4,8,16$. Their logarithms sum to $\ln1+\ln2+\ln4+\ln8+\ln16=0+(1+2+3+4)\ln2=10\ln2$.

Step 2: Estimate $\alpha$. Since $10\ln2\approx6.9315>0$, $\widehat{\alpha}_{\mathrm{MLE}}=5/(10\ln2)\approx0.7213$, rounded to four decimal places.

Step 3: Check the mean condition. The fitted tail index is below 1, so the fitted model has no finite mean. The mean formula is not applicable.

The calculation is verified by the strictly negative second derivative of the log-likelihood, which ensures this stationary point is a maximum. The common mistake is to use the number of observations as the denominator of the estimate; the denominator is the sum of log ratios, not $n$.

Interpreting the Tail Index

The survival function makes the role of $\alpha$ explicit. Raising the threshold by a factor $c$ multiplies the exceedance probability by $c^{-\alpha}$. When comparing two models with the same scale, even a modest difference in $\alpha$ can produce a large difference in the probability of losses far above that scale. A tail-index estimate below 1 signals an infinite model mean; an estimate between 1 and 2 gives a finite mean but infinite variance; an estimate above 2 gives finite mean and variance.

Worked Example: Comparing Extreme-Loss Probabilities

Consider two hypothetical Pareto Type I loss models with the same scale $\theta=\$500$, but tail indices $\alpha=1.5$ and $\alpha=3$. Compare their means, variances, and probabilities of exceeding $\$10{,}000$, which is $20\theta$.

Step 1: Compare means. For $\alpha=1.5$, the mean is $1.5(500)/(1.5-1)=\$1{,}500$. For $\alpha=3$, it is $3(500)/(3-1)=\$750$. Both means are finite because both indices exceed 1.

Step 2: Compare variances. The model with $\alpha=1.5$ has infinite variance because $1.5\leq2$. For $\alpha=3$, the variance is $3(500)^2/[(3-1)^2(3-2)]=\$187{,}500$ squared.

Step 3: Compare exceedance probabilities. Since the threshold is $20\theta$, the probabilities are $20^{-1.5}\approx0.0112$ and $20^{-3}=1/8{,}000=0.000125$, respectively. The first is about $0.0111803/0.000125\approx89.4$ times the second.

Thus the lower-index model has twice the mean and a much larger probability of this extreme loss, while its variance is not finite. The common mistake is to compare tail indices without using the same scale and threshold: the power-law comparison here holds because both models share the same scale.

Common Mistakes and Practical Checks

Ignoring moment conditions. Check $\alpha>1$ before using the mean and $\alpha>2$ before using the variance. For a general raw moment of real order $r$, the condition is $r<\alpha$.
Using the MLE when its denominator is zero. If all complete observations equal a known scale, the likelihood is unbounded as $\alpha$ increases; there is no finite MLE.
Confusing parameterizations. Pareto distributions are written in more than one form. Confirm that the model has support $x\geq\theta$ and survival $(\theta/x)^\alpha$ before using these formulas.
Treating a fitted tail index as proof of model fit. Estimation gives parameter values under the assumed model; it does not establish that the data follow a Pareto distribution.
Applying complete-data likelihood to altered observations. Censoring, truncation, or modified payments change the likelihood and require a different data model.

Key takeaway: For a Pareto Type I model, $\alpha$ governs both moment existence and extreme-loss probabilities. Estimate it only after confirming the support, estimation assumptions, moment conditions, and any boundary cases.

Check Your Understanding

For each problem, state the relevant assumption and check that the estimator or moment formula applies.

A complete sample of hypothetical losses is $200,400,600$ dollars from a Pareto Type I model with known scale $\theta=\$200$. Estimate $\alpha$ by matching the mean, then state whether the fitted variance is finite. (Answer: $\widehat{\alpha}_{\mathrm{MOM}}=2$; the fitted mean is finite but the variance is infinite.)
Four complete losses from a known-scale Pareto Type I model with $\theta=\$500$ are $500,1{,}000,2{,}000,4{,}000$. Calculate the MLE of $\alpha$ and the fitted probability of exceeding $\$5{,}000$. (Answer: $\widehat{\alpha}_{\mathrm{MLE}}=4/(6\ln2)\approx0.9618$; $P(X>5{,}000)=10^{-\widehat{\alpha}}\approx0.1092$.)
For a two-parameter Pareto moment fit, suppose $m_1=1{,}200$ dollars and the moment variance $s_m^2=480{,}000$ dollars squared, both calculated with divisor $n$. Find the estimates and check the variance condition. (Answer: $\widehat{\alpha}_{\mathrm{MOM}}=3$, $\widehat{\theta}_{\mathrm{MOM}}=\$800$; the fitted variance is finite.)
A fitted Pareto Type I model has $\theta=\$100$ and $\alpha=1.4$. Is $E[X^{1.5}]$ finite? Is $E[X^{-2}]$ finite? (Answer: $E[X^{1.5}]$ is infinite because $1.5\geq1.4$; $E[X^{-2}]$ is finite because $-2<1.4$.)
Under a known-scale Pareto Type I model with $\theta=\$1{,}000$, every observation in a complete sample equals $\$1{,}000$. What does the likelihood imply about a finite MLE for $\alpha$? (Answer: There is no finite MLE; the likelihood is proportional to $\alpha^n$ and is unbounded as $\alpha\to\infty$.)

What You'll Learn