Estimate Severity from Deductible-Truncated Claims

Why Deductible Truncation Changes the Likelihood

A severity data set may contain only claims that exceed a deductible. A claim below the deductible is then absent from the severity records, rather than recorded as a small claim. Consequently, the observed losses are not an unselected sample from the ground-up severity distribution. Their likelihood must account for the fact that every recorded value passed the threshold.

This distinction matters when estimating the cost distribution used for pricing, reserving, or risk analysis. If the selection mechanism is omitted, the fitted model can mistake the higher average of selected claims for a feature of all claims. The methods here concern severity estimation conditional on claims being observed; they do not estimate the claim frequency or the number of losses below the deductible.

Definition: Let $X$ be a continuous ground-up loss with density $f_X$ and survival function $S_X(x)=P(X>x)$. If a claim is recorded only when $X>d$, where $d$ is the deductible threshold, the observed severity has the conditional density $$ f_{X\mid X>d}(x)=\frac{f_X(x)}{S_X(d)},\qquad x>d. $$ The denominator is the probability that a loss is included in the data. Thus the conditional density is the ground-up density renormalized over the observable range.

In this tutorial, an observed value $x_i$ is the ground-up loss, not the insurer’s payment $x_i-d$. If a data system stores only the payment under an ordinary deductible and the payment is positive, the ground-up loss can be reconstructed by adding the deductible, subject to the contract terms and data definitions. The likelihood below uses the ground-up values.

Assume independent losses from a common severity model, and assume each recorded loss is included exactly when it exceeds its known threshold. If every claim has the same threshold $d$, the likelihood for $n$ observed losses is

L(\theta\mid x_1,\ldots,x_n) =\prod_{i=1}^n\frac{f_X(x_i;\theta)}{S_X(d;\theta)}, \qquad x_i>d.

Here $\theta$ denotes the model parameter or parameter vector. When the threshold differs by claim, with known $d_i$, the corresponding likelihood is

L(\theta\mid x_1,\ldots,x_n) =\prod_{i=1}^n\frac{f_X(x_i;\theta)}{S_X(d_i;\theta)}, \qquad x_i>d_i.

These expressions condition on the thresholds and on having observed the selected claims. They are not a model for the number of claims that exceed the thresholds. If losses can have an atom at a threshold, or are discrete, the conditional probability mass function and the appropriate inclusion probability must be used instead of the continuous density formula.

Why the Survival Probability Appears

The likelihood adjustment follows directly from conditional probability. For a small interval of width $\Delta x$ around an observed value $x>d$, the probability of a loss in that interval, given that the loss exceeds $d$, is approximately the unconditional probability of the interval divided by the probability of exceeding $d$:

P(x\le X<x+\Delta x\mid X>d) =\frac{P(x\le X<x+\Delta x)}{P(X>d)} \approx \frac{f_X(x)\Delta x}{S_X(d)}.

Dividing by $\Delta x$ and taking the limit gives $f_{X\mid X>d}(x)=f_X(x)/S_X(d)$. The denominator is parameter-dependent in many models, so it cannot be dropped when maximizing the likelihood. It corrects for the fact that values in the unobserved part of the distribution were never eligible to appear in the sample.

Identify what was observed.
State whether each data value is a ground-up loss, a payment, or another quantity, and identify the threshold that determines whether it appears.

Write the conditional contribution.
For a continuous loss observed above $d_i$, use $f_X(x_i;\theta)/S_X(d_i;\theta)$.

Maximize the full likelihood.
Retain the survival-probability terms, take logarithms if useful, and solve the score equation while checking parameter restrictions.

Exponential Severity: A Closed-Form Estimate

Suppose $X$ has an exponential distribution with rate $\lambda>0$, so $f_X(x)=\lambda e^{-\lambda x}$ and $S_X(x)=e^{-\lambda x}$ for $x\ge 0$. For a common threshold $d$, the conditional density is

f_{X\mid X>d}(x) =\frac{\lambda e^{-\lambda x}}{e^{-\lambda d}} =\lambda e^{-\lambda(x-d)},\qquad x>d.

The likelihood and log-likelihood therefore reduce to

L(\lambda)=\lambda^n\exp\!\left[-\lambda\sum_{i=1}^n(x_i-d)\right], \qquad \ell(\lambda)=n\log\lambda-\lambda\sum_{i=1}^n(x_i-d).

Differentiating gives $d\ell/d\lambda=n/\lambda-\sum_i(x_i-d)$. Setting this equal to zero yields the maximum likelihood estimate. The second derivative is $-n/\lambda^2<0$, so the stationary point is a maximum.

\widehat{\lambda} =\frac{n}{\sum_{i=1}^n(x_i-d)}.

The same result applies with known claim-specific thresholds $d_i$, replacing $x_i-d$ by $x_i-d_i$. The conditional exponential excesses $X_i-d_i$ have exponential rate $\lambda$. This follows from the memoryless property, or directly from the conditional density above.

Worked Example: Estimating an Exponential Rate Above a Common Deductible

A hypothetical portfolio records four ground-up losses only when they exceed a deductible of $100. The recorded losses are $120, $150, $130, and $190. Assume independent losses from an exponential severity model with unknown rate $\lambda$. Estimate $\lambda$ by conditional maximum likelihood, and compare it with the estimate obtained by ignoring the observation threshold.

Method and setup. Because the data contain only losses above $100, use the conditional likelihood. The excesses above the threshold are $120-100=20$, $150-100=50$, $130-100=30$, and $190-100=90$ dollars. Their sum is $20+50+30+90=190$ dollars.

Conditional estimate. With $n=4$, $\widehat{\lambda}=4/190=0.0210526$ per dollar. Rounded to four significant figures, this is $0.02105$ per dollar. The fitted mean excess above the deductible is $1/\widehat{\lambda}=47.5$ dollars.

What happens if selection is ignored? Treating the four observed losses as an ordinary, untruncated exponential sample gives a sum of $120+150+130+190=590$ dollars and the estimate $4/590=0.0067797$ per dollar. This is not the likelihood for the stated observation process.

Check. The conditional estimate implies an average excess of $1/0.0210526=47.5$ dollars, equal to the observed excess average $190/4=47.5$. The incorrect calculation instead uses the average observed ground-up loss, $590/4=147.5$, as though the sample included losses below the deductible. Common mistake avoided: omitting the survival-probability adjustment and treating selected losses as an unselected sample.

Pareto Severity Above a Known Threshold

A second useful closed-form case is the Pareto Type I model with known minimum parameter $x_m$ and unknown shape parameter $\alpha>0$. Its density and survival function are

f_X(x)=\alpha x_m^\alpha x^{-\alpha-1},\qquad S_X(x)=\left(\frac{x_m}{x}\right)^\alpha,\qquad x\ge x_m.

Suppose all observed losses exceed a common threshold $d\ge x_m$. Dividing the density by $S_X(d)$ gives the conditional density $\alpha d^\alpha x^{-\alpha-1}$ for $x>d$. For $n$ observed losses, its log-likelihood is

\ell(\alpha) =n\log\alpha+n\alpha\log d-(\alpha+1)\sum_{i=1}^n\log x_i.

Differentiating and setting the score to zero gives $\widehat{\alpha}=n/\sum_i\log(x_i/d)$. The second derivative is $-n/\alpha^2<0$, confirming a maximum. This result assumes $x_m$ is known and $d\ge x_m$; it is not the same estimation problem as estimating both Pareto parameters.

Worked Example: Pareto Shape from Claims Above a High Threshold

A hypothetical dataset contains three ground-up losses, $200, $400, and $200, and includes only losses above $100. Assume a Pareto Type I severity model with known $x_m=\$50$ and unknown $\alpha$. Estimate $\alpha$ using the conditional likelihood, then compare with the estimate from incorrectly treating the observations as an untruncated sample.

Method and setup. Since $d=100\ge x_m=50$, the conditional Pareto estimate applies. The likelihood uses logarithms of ratios to the observation threshold: the ratios are $200/100=2$, $400/100=4$, and $200/100=2$.

Conditional estimate. The denominator is $\log 2+\log 4+\log 2=\log 2+2\log 2+\log 2=4\log 2$. Since $\log 2\approx0.693147$, this is approximately $2.772589$. Thus $\widehat{\alpha}=3/(4\log 2)\approx1.0820$.

Untruncated calculation. If the known minimum $x_m=50$ is used as though the sample were untruncated, the ratios become $4,8,4$. Their log sum is $\log 4+\log 8+\log 4=2\log 2+3\log 2+2\log 2=7\log 2$, not $8\log 2$. Therefore the untruncated estimate is $3/(7\log 2)\approx0.6184$.

Check. Each observed loss exceeds both the deductible threshold and $x_m$, so the data satisfy the model’s support requirements. The conditional denominator uses $x_i/d$, not $x_i/x_m$, because the sample was selected above $d$. Common mistake avoided: using the underlying model’s minimum as the likelihood reference while ignoring the higher observation threshold.

Different Thresholds and the Limits of the Method

When each claim has its own known deductible $d_i$, the correct log-likelihood is

\ell(\theta) =\sum_{i=1}^n\left[\log f_X(x_i;\theta)-\log S_X(d_i;\theta)\right].

For an exponential model, this becomes $n\log\lambda-\lambda\sum_i(x_i-d_i)$, so $\widehat{\lambda}=n/\sum_i(x_i-d_i)$. This requires that the thresholds are known and that the losses share the same exponential rate. If thresholds are associated with different risk classes having different severity parameters, the corresponding parameter must be used for each class rather than pooling all excesses under one common model.

Worked Example: Exponential Estimate with Claim-Specific Deductibles

A hypothetical insurer’s severity records include three ground-up losses, $130, $180, and $250. The known thresholds for those claims are respectively $100, $150, and $200. Assume independent losses from one exponential severity model with common unknown rate $\lambda$. Find the conditional maximum likelihood estimate and show why using the total losses would be wrong.

Method and setup. Each loss is conditioned on exceeding its own threshold. The corresponding excesses are $130-100=30$, $180-150=30$, and $250-200=50$ dollars. Their sum is $30+30+50=110$ dollars.

Conditional estimate. There are $n=3$ records, so $\widehat{\lambda}=3/110=0.0272727$ per dollar, or $0.02727$ per dollar to four significant figures. The fitted mean excess is $1/\widehat{\lambda}=110/3=\$36.67$, rounded to the nearest cent.

Incorrect untruncated estimate. The ground-up losses sum to $130+180+250=560$ dollars. Applying the ordinary exponential estimate gives $3/560=0.0053571$ per dollar, which does not account for the three selection thresholds.

Check. The observed mean excess is $110/3=\$36.67$, the reciprocal of the conditional estimate. The likelihood’s exponential term is $\exp[-110\lambda]$, since the threshold-specific contributions depend on the three excesses, not on the sum of the ground-up losses. Common mistake avoided: subtracting one pooled deductible or ignoring the different thresholds altogether.

Common Exam Traps and Practical Interpretation

Using the wrong observed quantity. Determine whether the records contain ground-up losses or payments. A deductible payment is not automatically the ground-up loss.
Dropping the selection probability. In a conditional likelihood, $S_X(d_i;\theta)$ generally depends on the parameter. Omitting it changes the estimate.
Assuming the threshold is common. If thresholds differ by claim, each likelihood contribution must use its own $d_i$.
Using a continuous density for a discrete model or a model with atoms. In that setting, use the relevant conditional probability mass function and inclusion probability.
Confusing severity estimation with frequency estimation. A likelihood conditional on recorded claims estimates the severity distribution among selected claims. It does not, by itself, estimate how many claims were hidden below the deductible.

The conditional likelihood is particularly useful when the records contain only claims that pass a reporting or coverage threshold. Its central discipline is simple: model not just the observed values, but also the rule that made those values observable. This tutorial is aligned with the SOA Exam ASTAM (Advanced Short-Term Actuarial Mathematics) syllabus.

Check Your Understanding

For each problem, use the stated observation rule and distinguish the conditional estimate from any deliberately incorrect untruncated calculation.

A hypothetical severity file contains losses of $52, $60, and $68, and records only losses above a common deductible of $40. Assume an exponential severity distribution with unknown rate. Find the conditional maximum likelihood estimate of the rate per dollar. (Answer: $3/60=0.05000$ per dollar.)
Three ground-up losses of $90, $105, and $135 are recorded only above a common threshold of $75. Under an exponential model, calculate the conditional rate estimate and its implied mean excess above the threshold. (Answer: $\widehat{\lambda}=3/105=0.02857$ per dollar; implied mean excess $=\$35.00$.)
A Pareto Type I model has known $x_m=\$25$. The recorded losses are $200, $400, and $200, and only losses above $d=\$100$ are observed. Find the conditional maximum likelihood estimate of $\alpha$. (Answer: $3/(4\log 2)\approx1.0820$.)
Three losses are recorded with respective thresholds of $20, $50, and $80. Their ground-up values are $35, $90, and $95. Under a common exponential severity model, find the conditional rate estimate. (Answer: $3/(15+40+15)=3/70\approx0.04286$ per dollar.)
A file records the losses $125, $150, and $175 only when they exceed a common deductible of $100. Assume an exponential model. Calculate both the correct conditional rate estimate and the incorrect rate estimate obtained by ignoring selection. (Answer: conditional $3/150=0.02000$ per dollar; ignoring selection $3/450=0.006667$ per dollar.)

What You'll Learn