Correcting Truncated and Censored Operational Loss Data

Why Incomplete Loss Records Need a Different Likelihood

Operational-loss databases rarely capture every event in the same way. A reporting threshold may exclude small losses entirely, while a policy limit or data-system cap may record a large loss only as “at least” a stated amount. These mechanisms change what the observed data mean. Fitting a model as though every record were a complete, uncensored observation can distort both the estimated event rate and the severity distribution.

The key distinction is whether an event is missing because it falls below a threshold or whether the event is present but its exact value is unknown beyond a censoring point. Likelihood methods represent each observation according to what the data actually reveal.

Definition: Left truncation at reporting threshold $r$ means only losses satisfying $X>r$ enter the observed loss data; losses at or below $r$ are absent. Right censoring at amount $c$ means an event is observed, but when its loss reaches or exceeds $c$, only the information $X\ge c$ is retained. Truncation removes observations from the data set; censoring retains an observation with incomplete severity information.

Let $X$ denote a loss, with density $f_X(x)$, distribution function $F_X(x)$, and survival function $S_X(x)=P(X>x)=1-F_X(x)$. Assume losses are independent draws from a severity distribution, and assume the recording mechanism and its threshold or cap are known. For frequency, let $N$ be the number of loss events over exposure $E$, with $N\sim\operatorname{Poisson}(E\lambda)$, where $\lambda$ is the event rate per unit exposure. These assumptions are modeling choices, not consequences of the observation mechanism; they should be checked for the application.

Likelihood Contributions Under Truncation

If a severity is observed only because it exceeds $r$, its density conditional on being recorded is

f_{X\mid X>r}(x)=\frac{f_X(x)}{S_X(r)},\qquad x>r.

The denominator is essential: it normalizes the density over the range from which observations can enter the database. For $n$ independent reported losses $x_1,\ldots,x_n$, the severity likelihood conditional on the number observed is

L_{\text{sev}}(\theta)=\prod_{j=1}^{n}\frac{f_X(x_j;\theta)}{S_X(r;\theta)}.

Here $\theta$ denotes the severity parameter or parameters. Omitting the survival denominator treats the reported losses as a random sample from the entire loss distribution, even though values at or below $r$ could not have appeared.

The missing events also affect frequency. Under Poisson frequency and independent severities, each event is recorded with probability $S_X(r)$. Poisson thinning therefore gives

N_r\sim\operatorname{Poisson}\big(E\lambda S_X(r)\big), \qquad L_{\text{freq}}(\lambda,\theta) =\frac{\exp\{-E\lambda S_X(r;\theta)\} \{E\lambda S_X(r;\theta)\}^{n}}{n!}.

For a given severity model, the maximum-likelihood estimate of the underlying event rate is $\widehat{\lambda}=n/[E S_X(r;\widehat{\theta})]$, not $n/E$. The observed rate $n/E$ is the rate of losses above the threshold, rather than the rate of all losses.

Why the truncation density has this form: For a small interval of width $\Delta x$ around $x>r$, the probability of a loss both being near $x$ and being reportable is approximately $f_X(x)\Delta x$. Conditional on the event being reportable, divide by $P(X>r)=S_X(r)$. Thus the conditional probability is approximately $[f_X(x)/S_X(r)]\Delta x$, giving the stated density as $\Delta x$ shrinks.

Worked Example: Correcting Both Frequency and Severity for a Reporting Threshold

A hypothetical operational database covers $E=2$ exposure-years. It includes six losses above a reporting threshold of $r=\$500$: $\$700,\$800,\$900,\$1{,}000,\$1{,}100,\$1{,}200$. Assume all losses follow an exponential distribution with unknown scale $\theta$, so $f_X(x)=\theta^{-1}e^{-x/\theta}$ and $S_X(x)=e^{-x/\theta}$ for $x\ge0$. Event frequency is Poisson with rate $\lambda$ per exposure-year. Estimate severity and the underlying event rate.

Method—fit the conditional severity distribution, then correct the observed count. For an exponential loss, the conditional density above $r$ is

\frac{f_X(x)}{S_X(r)} =\frac{1}{\theta}e^{-(x-r)/\theta},\qquad x>r.

The observed excesses over the threshold are $200,300,400,500,600,700$ dollars. Their total is $2{,}700$, so their mean is $2{,}700/6=\$450$. The conditional log-likelihood is $-6\log\theta-2{,}700/\theta$. Setting its derivative, $-6/\theta+2{,}700/\theta^2$, equal to zero gives $\widehat{\theta}=\$450$.

The estimated probability that a loss exceeds the threshold is

\widehat{S}_X(500)=e^{-500/450}=e^{-1.1111}=0.3292.

The estimated recorded-loss rate is $6/2=3$ per exposure-year. Correcting for the estimated probability of reporting gives

\widehat{\lambda} =\frac{6}{2(0.3292)} =9.113\text{ events per exposure-year}.

Check: $2(9.113)(0.3292)\approx6.000$, matching the observed expected count above the threshold. Treating all six records as the full event count would instead give $3$ events per exposure-year and materially understate frequency. Treating the severities as untruncated would estimate the exponential scale by their raw mean, $\$5{,}700/6=\$950$, rather than by the mean excess of $\$450$. Both errors arise from ignoring the reporting rule.

The frequency correction applies when events below $r$ are unobserved and their count is unavailable. If a separate system provides the total number of all events, including below-threshold events, the frequency information is different: do not discard that count and replace it with a tail-only count model. The likelihood must match exactly which counts and severities are known.

Likelihood Contributions Under Censoring

For an exact, uncensored observation $x<c$, the likelihood contribution is its density $f_X(x)$. For a loss recorded as reaching a known cap $c$, the exact severity is unknown, but the observation tells us $X\ge c$. Its likelihood contribution is therefore the probability of that event, $S_X(c)$. For $m$ exact observations $x_j$ and $q$ right-censored observations at respective caps $c_k$, the likelihood is

L(\theta)= \prod_{j=1}^{m} f_X(x_j;\theta) \prod_{k=1}^{q} S_X(c_k;\theta).

This expression assumes censoring amounts are known and do not depend on the unobserved severity in an additional, unmodeled way. A deterministic policy limit or system cap is a common example. If the censoring mechanism is informative or unknown, the displayed likelihood alone may not describe the data-generation process.

Worked Example: Estimating Severity With Losses Capped at a Known Amount

A hypothetical portfolio records four operational events. Two exact losses are $\$200$ and $\$300$; two other records are each shown as “$\$500$ or more” because the system caps recorded amounts at $\$500$. Assume independent exponential severities with unknown scale $\theta$, no reporting threshold, and a known deterministic cap. Estimate $\theta$.

Method—use density contributions for exact values and survival contributions for capped values. The likelihood is

L(\theta) =f_X(200)f_X(300)S_X(500)^2 =\theta^{-2}\exp\left(-\frac{200+300+500+500}{\theta}\right).

The two exact observations each contribute a factor $\theta^{-1}$. The two censored observations contribute survival probabilities and therefore add $500+500$ to the exponential's total amount, but they do not add further factors of $\theta^{-1}$. The log-likelihood, apart from constants, is $-2\log\theta-1{,}500/\theta$. Its derivative is $-2/\theta+1{,}500/\theta^2$. Setting this equal to zero gives

\widehat{\theta}=\frac{1{,}500}{2}=\$750.

Check: the fitted mean of $\$750$ is greater than the cap, which is consistent with two known losses of at least $\$500$. If the capped values were incorrectly treated as exact, the estimated scale would be $(200+300+500+500)/4=\$375$, substantially lower. If capped records were simply discarded, the estimate would be $(200+300)/2=\$250$, also too low. Both calculations throw away information in the censoring statement.

When Truncation and Censoring Occur Together

A database may include only losses above a reporting threshold and cap the recorded amount at a higher value. For a recorded exact loss $x$ with $r<x<c$, the contribution conditional on inclusion is $f_X(x)/S_X(r)$. For a record censored at $c>r$, the contribution conditional on having passed the reporting threshold is

P(X\ge c\mid X>r)=\frac{S_X(c)}{S_X(r)}.

The denominator accounts for truncation; the numerator accounts for the censored tail. These are separate parts of the observation mechanism and should not be substituted for one another.

Worked Example: A Reporting Threshold and a Censoring Cap in One Data Set

Suppose a hypothetical operational system records only losses above $r=\$400$, and records any loss of at least $c=\$1{,}200$ as “$\$1{,}200$ or more.” The reported records are exact losses of $\$700$ and $\$900$, plus one loss censored at $\$1{,}200$. Assume independent exponential severities with unknown scale $\theta$. Estimate $\theta$ from the severity records conditional on inclusion.

Method—use conditional density for each exact loss and conditional survival for the censored loss. The likelihood is

L(\theta) =\frac{f_X(700)}{S_X(400)} \frac{f_X(900)}{S_X(400)} \frac{S_X(1{,}200)}{S_X(400)} =\theta^{-2} \exp\left(-\frac{(700-400)+(900-400)+(1{,}200-400)}{\theta}\right).

The three excess amounts relevant to the exponent are $300$, $500$, and $800$, totaling $1{,}600$. Only the two exact observations contribute factors of $\theta^{-1}$; the censored observation contributes a survival probability. Thus the log-likelihood is $-2\log\theta-1{,}600/\theta$, apart from constants. Its derivative is $-2/\theta+1{,}600/\theta^2$, so

\widehat{\theta}=\frac{1{,}600}{2}=\$800.

Check: substituting $\theta=800$ makes the score $-2/800+1{,}600/800^2=0$. Using the ordinary density $f_X$ without conditioning would omit the three threshold denominators; treating the censored record as an exact $\$1{,}200$ loss would add an incorrect density factor and bias the estimate.

Bias Patterns and Exam Traps

The direction of bias depends on the observation mechanism and on the incorrect treatment. A reporting threshold generally makes the observed sample severity larger than the full loss distribution because smaller losses cannot appear. Ignoring truncation and fitting an untruncated severity model to these records therefore tends to overstate typical severity. Ignoring unreported events also understates frequency; the observed count is only the tail count.

Censoring has a different trap. A capped amount does not say that the loss equals the cap. Treating it as exact assigns too much probability to that single value and may substantially pull a severity estimate downward. Dropping censored records loses their lower-bound information; the resulting direction of bias depends on the model and the censoring pattern, but the calculation is not a valid likelihood for the recorded data.

Observation mechanism	What is known	Likelihood contribution
Exact loss $x$	The loss value is $x$	$f_X(x)$
Truncated exact loss $x>r$	The loss is known exactly and passed the threshold	$f_X(x)/S_X(r)$, conditional on inclusion
Right-censored loss at $c$	The loss is at least $c$	$S_X(c)$
Truncated and right-censored at $c>r$	The loss passed $r$ and is at least $c$	$S_X(c)/S_X(r)$, conditional on inclusion

Before fitting, establish whether the data set contains all events, only above-threshold events, or events with capped values. Confirm whether a value at the cap is explicitly flagged as censored or is an exact amount. A recorded amount equal to a limit is not enough by itself to distinguish those cases. These checks support estimates that better reflect the underlying risk, rather than the database's recording rules. Correcting the observation mechanism is aligned with the CAS Exam 9 (Risk Management for Actuaries) syllabus.

Check Your Understanding

For each problem, identify the observation mechanism and apply the corresponding likelihood or frequency adjustment.

Losses are exponential with known scale $\theta=\$600$. Only losses above $r=\$300$ are recorded. Across $E=4$ exposure-years, 5 losses are recorded. Estimate the underlying Poisson event rate per exposure-year. (Answer: $5/[4e^{-300/600}]=2.061$ events per exposure-year.)
Three exact exponential losses are $\$100,\$250,\$350$, and one additional loss is right-censored at $\$400$. With no threshold and unknown scale, find the maximum-likelihood estimate of $\theta$. (Answer: $(100+250+350+400)/3=\$366.67$.)
An exponential severity sample is left-truncated at $r=\$200$. The exact reported losses are $\$500,\$700,\$900$. Find the conditional maximum-likelihood estimate of the scale. (Answer: $[(500-200)+(700-200)+(900-200)]/3=\$500$.)
Only losses above $r=\$100$ are recorded; amounts at or above $c=\$700$ are censored at $c$. For an exponential model, the data are one exact loss of $\$400$ and one censored loss at $\$700$. Find the conditional maximum-likelihood estimate of $\theta$. (Answer: $[(400-100)+(700-100)]/1=\$900$.)

Observation mechanism	What is known	Likelihood contribution
Exact loss \(x\)	The loss value is \(x\)	\(f_X(x)\)
Truncated exact loss \(x>r\)	The loss is known exactly and passed the threshold	\(f_X(x)/S_X(r)\), conditional on inclusion
Right-censored loss at \(c\)	The loss is at least \(c\)	\(S_X(c)\)
Truncated and right-censored at \(c>r\)	The loss passed \(r\) and is at least \(c\)	\(S_X(c)/S_X(r)\), conditional on inclusion

What You'll Learn