What Empirical Estimates Can Use Incomplete Records

Introduction

Incomplete records do not all provide the same kind of information. An exact loss value can be placed at a particular point in a sample distribution. A right-censored record only says that the loss exceeds a recorded threshold. A missing record may provide no value constraint at all. The practical question is therefore not simply whether a record is incomplete, but which calculations the observed information supports.

This tutorial separates direct summaries of observed records from summaries of the underlying losses that cannot be identified from those records alone. That distinction is aligned with the SOA Exam ASTAM (Advanced Short-Term Actuarial Mathematics) syllabus.

What a Right-Censored Record Tells Us

For individual $i$, let $X_i$ be the loss or event time of interest. Under right censoring, the observed value is $Y_i=\min(X_i,C_i)$, where $C_i$ is the censoring time. Let $I_i=1$ indicate that $X_i$ was observed exactly, and $I_i=0$ indicate right censoring. Thus, $I_i=1$ reveals $X_i=Y_i$, while $I_i=0$ reveals only $X_i>Y_i$. The examples below treat a loss value and an event time alike for this purpose; the unit and interpretation must be stated for each application.

Definition: A directly calculable summary is a quantity determined by the recorded values and their observation status, without filling in unobserved losses. Examples include the number of exact records, the number of censored records, the sum of observed follow-up times, and the mean among exact records. A direct calculation is not automatically an estimate of a population quantity or of the full-sample loss distribution.

A complete-data empirical distribution would place every one of $n$ losses at its exact value. Its empirical cumulative distribution function (CDF) is $F_n(t)=n^{-1}\sum_{i=1}^n \mathbf{1}(X_i\le t)$. With censoring, some $X_i$ are unknown, so this expression cannot generally be evaluated exactly. The observed CDF of exact records can be calculated, but it describes only the exact-record subset.

What Can Be Said About the Full-Sample CDF?

For right-censored data, consider a fixed threshold $t$. An exact loss at or below $t$ is definitely counted in the full-sample CDF. A censored record with $Y_i<t$ might have a loss at or below $t$, or it might exceed $t$. A censored record with $Y_i\ge t$ is known to have $X_i>t$, and so cannot contribute to $F_n(t)$. Exact losses above $t$ also cannot contribute.

\frac{\#\{i:I_i=1,\ Y_i\le t\}}{n} \ \leq\ F_n(t)\ \leq\ \frac{\#\{i:I_i=1,\ Y_i\le t\}+\#\{i:I_i=0,\ Y_i<t\}}{n}.

These are bounds, not a point estimate. The strict inequality $Y_i<t$ in the upper bound matters: if censoring occurs at $t$, the record says $X_i>t$, not $X_i\le t$. The corresponding bounds for the empirical survival proportion $S_n(t)=n^{-1}\sum_i\mathbf{1}(X_i>t)$ are obtained by subtracting the CDF bounds from 1.

Count definite contributions.
Count exact observations at or below $t$. They must be included in the full-sample CDF.

Identify unresolved records.
A right-censored observation strictly below $t$ could have an underlying value on either side of $t$.

Exclude records known to exceed $t$.
Exact observations above $t$ and censored observations at or above $t$ cannot be in the CDF numerator.

Divide by the full sample size.
The bounds concern all $n$ underlying observations, not just the exact records.

Worked Examples

Worked Example: Bounding a CDF at a Loss Threshold

A hypothetical insurer has six loss records, with amounts measured in thousands of dollars. The observed pairs $(Y_i,I_i)$ are $(40,1),(55,0),(70,1),(80,0),(90,1),(100,0)$. Find the possible range of the full-sample empirical CDF at $t=60$, and compare it with the exact-record CDF.

Method. Apply the right-censoring bounds with $n=6$. The exact losses are 40, 70, and 90. The records censored at 55, 80, and 100 establish only that their underlying losses exceed those respective thresholds.

Lower bound. Of the exact losses, only 40 is at or below 60. The lower bound is therefore $1/6=0.1667$, to four decimal places.

Upper bound. The censored record at 55 is strictly below 60 and could have an underlying loss at or below 60. The records censored at 80 and 100 cannot contribute. The upper bound is $(1+1)/6=2/6=0.3333$, to four decimal places.

Exact-record comparison. Among the three exact records, one of three is at or below 60, so the exact-record CDF is $1/3=0.3333$. This is a summary of the exact-record subset, not proof that the full-sample CDF equals 0.3333. The full-sample value is only bounded between 0.1667 and 0.3333.

Check. The unresolved record can add at most one to the count, making the only possible full-sample counts 1 or 2 out of 6. Treating the censored value 55 as exact would select one of those possibilities without evidence.

Worked Example: Mean Bounds When Losses Have a Maximum

A hypothetical group has five losses, measured in thousands of dollars. Three are exact: 12, 18, and 25. The other two are right-censored at 20 and 30. Assume every underlying loss is at most 50. Calculate the exact-record mean and bounds for the mean of all five losses.

Exact-record mean. The exact values sum to $12+18+25=55$. There are three exact records, so their mean is $55/3=18.3333$ thousand dollars, or $18,333.33 to the nearest cent. It is not the mean of all five losses.

Lower bound for the full-sample mean. The censored losses exceed 20 and 30, respectively. Replacing them by their censoring thresholds gives the infimum of the possible total: $12+18+25+20+30=105$. Thus the full-sample mean is strictly greater than $105/5=21$ thousand dollars. The inequality is strict because both censored losses exceed their thresholds.

Upper bound for the full-sample mean. Under the stated maximum of 50, the largest possible total is $12+18+25+50+50=155$. The mean is therefore at most $155/5=31$ thousand dollars. The full-sample mean lies in $(21,31]$ thousand dollars.

Check and trap avoided. The exact-record mean is about $18,333.33, while the full-sample mean must exceed $21,000. Those statements are consistent because they describe different groups. Treating the censoring thresholds as exact would give $105/5=21$ thousand dollars as if it were the actual mean; the strict lower bound shows why that calculation is not justified.

Worked Example: Counts and Observed Exposure Are Directly Calculable

In a hypothetical study of time to a first claim, six policies have observed times in years and event indicators $(Y_i,I_i)$: $(1,1),(2,1),(2,0),(3,1),(4,0),(4,0)$. Calculate the exact event count, the total observed time, and the event count per observed year. Then bound the fraction of the six underlying event times at or below 4 years.

Observed counts. There are three exact events, at 1, 2, and 3 years, and three right-censored records, at 2, 4, and 4 years.

Total observed time. Sum all six recorded times, including both exact event times and censoring times: $1+2+2+3+4+4=16$ policy-years.

Events per observed year. The directly calculated ratio is $3/16=0.1875$ events per policy-year. It is a rate formed from the observed event count and observed exposure. Interpreting it as a rate for a wider population requires appropriate assumptions; it is not by itself a complete empirical distribution of event times.

Bound at 4 years. All three exact events are at or before 4, so the lower bound is $3/6=0.5000$. The censored record at 2 years is strictly below 4 and could have an event by 4; the two records censored at 4 establish that their event times exceed 4. The upper bound is $(3+1)/6=4/6=0.6667$, to four decimal places.

Check. The data determine three events and 16 observed policy-years exactly. They do not determine whether the policy censored at year 2 has its event by year 4, which explains the range from 0.5000 to 0.6667.

Other Incomplete Records and Assumptions

The same discipline applies to other incomplete-record types. A left-censored loss reported as at most $L$ is known to satisfy $X_i\le L$, but its exact location below $L$ is unknown. An interval-censored record reported between $L$ and $U$ constrains the value to that interval, subject to the convention used for its endpoints. A record marked missing, with no recorded threshold or interval, supplies no such value constraint. Counts of these record types are directly available; an exact loss distribution generally is not.

A complete-case summary is calculated using only records with exact values. For example, the complete-case mean is the sum of exact losses divided by the number of exact losses. It can be reported as a property of that subset. Using it to represent all losses requires justification that the exact records are sufficiently representative, or a separate estimation method and assumptions. Censoring that depends on the underlying loss can make the exact subset especially unrepresentative.

An assumption such as noninformative censoring can support methods that estimate a distribution from censored records. It does not turn a censored value into an exact value, nor does it make the complete-data empirical CDF directly calculable. The Kaplan-Meier estimator and related methods are distribution-estimation tools; their construction and interpretation are separate from the direct summaries and finite-sample bounds developed here.

Justification for the CDF bounds: For each individual, the indicator $\mathbf{1}(X_i\le t)$ is known to equal 1 when an exact value is at or below $t$, and it is known to equal 0 for an exact value above $t$ or a censored value at or above $t$. For a censored value below $t$, the indicator may be either 0 or 1. Summing the known ones gives the smallest possible numerator; adding every unresolved indicator as 1 gives the largest possible numerator. Dividing by $n$ gives the stated bounds.

Common Mistakes

Calling the exact-record CDF the full-sample CDF. The former uses only exact records; the latter uses all underlying observations, including unknown values.
Using a censoring threshold as an exact loss. A record censored at $c$ says $X_i>c$, not $X_i=c$.
Including censoring at the threshold in the CDF upper bound. At $Y_i=t$, right censoring establishes $X_i>t$, so that record cannot be counted in $F_n(t)$.
Forgetting the denominator. Bounds for the full-sample CDF are divided by the total number of records, not the number of exact records.
Giving an observed event rate a distributional interpretation. Events divided by observed exposure is a calculable ratio; further interpretation depends on assumptions and the intended target.

Key takeaway: Report direct counts, exposure totals, and exact-record summaries for what they are. For the full loss distribution, use the information in censored records to establish bounds, and do not claim a point estimate unless a stated estimation method and its assumptions support one.

Check Your Understanding

For each problem, distinguish a direct calculation from a statement about the full set of underlying values.

Five right-censored records have observed pairs $(4,1),(6,0),(9,1),(12,0),(15,1)$. Find the bounds for the full-sample empirical CDF at $t=10$. (Answer: $2/5\le F_n(10)\le3/5$, or 0.4000 to 0.6000.)
Four losses, in thousands of dollars, are exact at 5, 11, and 14, or censored at 8. Find the full-sample CDF bounds at 10. (Answer: $1/4\le F_n(10)\le2/4$, or 0.2500 to 0.5000.)
Four event-time records are $(7,1),(13,1),(10,0),(20,0)$. Find the bounds for the fraction of underlying event times at or below 12. (Answer: $1/4\le F_n(12)\le2/4$, or 0.2500 to 0.5000.)
Three losses are exact at 10 and 20, or censored at 15; all losses are at most 40. Find bounds for the full-sample mean. (Answer: strictly greater than $45/3=15$ and at most $70/3=23.3333$ thousand dollars.)
Two policies have exact events at 0.5 and 1.5 years; two others are censored at 2 and 3 years. Calculate events per observed policy-year. (Answer: $2/(0.5+1.5+2+3)=2/7=0.2857$ events per policy-year.)

What You'll Learn