Why Compare a Model With the Observed Loss Distribution?
A fitted loss model summarizes a population with a smooth or otherwise specified distribution. The observed losses, by contrast, form a finite sample. A direct way to inspect how those two descriptions differ is to compare the empirical distribution function with the candidate cumulative distribution function at observed loss values. This comparison shows, at each chosen loss threshold, what proportion of the sample is at or below that threshold and what proportion the model assigns there.
This tutorial concerns constructing that empirical function and making the point-by-point comparison. It does not turn the differences into a formal goodness-of-fit statistic. In particular, a list of pointwise differences is not itself a decision rule for accepting or rejecting a model.
If the ordered observations are \(X_{(1)}\le \cdots \le X_{(n)}\) and are all distinct, then \(\widehat F_n(X_{(j)})=j/n\). Between consecutive observations the function is constant; it jumps by \(1/n\) at each observation. When losses are tied, all observations at the same value contribute to the jump at that value. The value of the function at a jump includes the entire jump, because the definition counts observations satisfying \(X_i\le x\).
The empirical function describes the sample, not the unknown population distribution. A candidate model, written \(F_X(x)\), gives its own cumulative probability at a threshold \(x\). The difference \(\widehat F_n(x)-F_X(x)\) is a signed comparison: a positive value means the sample proportion at or below \(x\) exceeds the model probability there; a negative value means it is smaller.
Constructing the Step Function
A reliable construction starts by sorting the data, then counting cumulative observations rather than assigning a separate rank to every distinct loss value. At a tied loss, use the cumulative count after including every observation in that tie. This is especially important when a question asks for the empirical function at an observed loss.
Write the complete observations from smallest to largest, retaining repeated values.
At a value \(x\), count every observation less than or equal to \(x\), including all ties at \(x\).
The cumulative count divided by \(n\) is \(\widehat F_n(x)\).
Compute \(F_X(x)\) at each requested observed value, then report the two values and, if useful, their signed difference.
Worked Comparisons
Worked Example: A Repeated Loss Under an Exponential Model
A hypothetical insurer records five complete claim severities, in thousands of dollars: \(4, 7, 7, 12, 20\). Compare the empirical distribution function with a candidate exponential loss model having mean 10 thousand dollars. For this model, \(F_X(x)=1-e^{-x/10}\) for \(x\ge 0\). Find both functions at each distinct observed loss and give the empirical-minus-model difference.
Method. Sort the sample and use cumulative counts, including both claims of 7 at the threshold \(x=7\). The sample size is \(n=5\). The empirical values are \(1/5\) at 4, \(3/5\) at 7, \(4/5\) at 12, and \(5/5\) at 20.
For the model values, substitute each threshold into \(1-e^{-x/10}\):
At \(x=4\), \(F_X(4)=1-e^{-0.4}=1-0.6703=0.3297\). At \(x=7\), \(F_X(7)=1-e^{-0.7}=1-0.4966=0.5034\). At \(x=12\), \(F_X(12)=1-e^{-1.2}=1-0.3012=0.6988\). At \(x=20\), \(F_X(20)=1-e^{-2}=1-0.1353=0.8647\). Values are rounded to four decimal places.
| Observed loss \(x\) | Cumulative count | \(\widehat F_n(x)\) | \(F_X(x)\) | \(\widehat F_n(x)-F_X(x)\) |
|---|---|---|---|---|
| 4 | 1 | 0.2000 | 0.3297 | -0.1297 |
| 7 | 3 | 0.6000 | 0.5034 | 0.0966 |
| 12 | 4 | 0.8000 | 0.6988 | 0.1012 |
| 20 | 5 | 1.0000 | 0.8647 | 0.1353 |
At 7, the empirical value is \(3/5\), not \(2/5\): there is one observation at 4 and two observations equal to 7, all of which are at or below 7. The negative difference at 4 means the model assigns a larger cumulative probability there than the observed fraction. At 7, 12, and 20, the observed fractions are larger. This comparison describes the direction and size of discrepancies at these points; it does not by itself determine whether the model passes a formal test.
Worked Example: Checking a Uniform Candidate Model
A hypothetical sample of four complete repair losses, measured in thousands of dollars, is \(2,6,10,14\). A proposed model is uniform on the interval from 0 to 16 thousand dollars. Construct the empirical and candidate cumulative values at every observed loss.
Method. The observations are already ordered and distinct. Their cumulative counts are 1, 2, 3, and 4, so the empirical values are \(1/4=0.25\), \(2/4=0.50\), \(3/4=0.75\), and \(4/4=1.00\).
For a uniform distribution on \([0,16]\), \(F_X(x)=x/16\) when \(0\le x\le16\). Therefore, \(F_X(2)=2/16=0.125\), \(F_X(6)=6/16=0.375\), \(F_X(10)=10/16=0.625\), and \(F_X(14)=14/16=0.875\).
The empirical-minus-model differences are, in order, \(0.25-0.125=0.125\), \(0.50-0.375=0.125\), \(0.75-0.625=0.125\), and \(1.00-0.875=0.125\). Thus, at each observed loss, the sample cumulative proportion exceeds the candidate cumulative probability by 0.125. The empirical function remains constant between the observed losses, whereas the uniform candidate CDF increases continuously over its support.
Check. The final empirical value is 1 because all four observations are no greater than 14; the candidate value is \(14/16\), not 1, because the proposed model still assigns probability to losses between 14 and 16.
Worked Example: Reading Jumps and Comparing a Second Exponential Model
Six complete losses, in thousands of dollars, are \(1,2,2,5,8,11\). Compare their empirical values at the distinct observed losses with a candidate exponential model having mean 5 thousand dollars. Use \(F_X(x)=1-e^{-x/5}\) for \(x\ge0\), and round probabilities to four decimal places.
Method. The cumulative counts at \(1,2,5,8,11\) are respectively \(1,3,4,5,6\), giving empirical values \(1/6=0.1667\), \(3/6=0.5000\), \(4/6=0.6667\), \(5/6=0.8333\), and \(6/6=1.0000\).
Evaluate the candidate model at each threshold. At 1, \(F_X(1)=1-e^{-0.2}=0.1813\). At 2, \(F_X(2)=1-e^{-0.4}=0.3297\). At 5, \(F_X(5)=1-e^{-1}=0.6321\). At 8, \(F_X(8)=1-e^{-1.6}=0.7981\). At 11, \(F_X(11)=1-e^{-2.2}=0.8892\).
| \(x\) | Cumulative count | \(\widehat F_n(x)\) | \(F_X(x)\) | Difference |
|---|---|---|---|---|
| 1 | 1 | 0.1667 | 0.1813 | -0.0146 |
| 2 | 3 | 0.5000 | 0.3297 | 0.1703 |
| 5 | 4 | 0.6667 | 0.6321 | 0.0346 |
| 8 | 5 | 0.8333 | 0.7981 | 0.0352 |
| 11 | 6 | 1.0000 | 0.8892 | 0.1108 |
The jump at 2 is \(2/6=0.3333\), because two losses equal 2. The empirical value at 2 is the accumulated proportion \(3/6\), not just the jump size. The values show that the empirical cumulative proportion at 2 is notably above the candidate probability, while at 1 it is slightly below. Intermediate differences are rounded after subtraction of the displayed four-decimal values; using unrounded exponential values gives the same displayed results.
Why the Empirical Function Uses Cumulative Counts
The indicator definition gives a direct justification. For any fixed threshold \(x\), each term \(I(X_i\le x)\) is 1 exactly when observation \(i\) falls at or below \(x\). Summing these indicators counts the observations in that set, and dividing by \(n\) converts the count to a sample fraction. Therefore, the empirical distribution function is exactly the observed cumulative proportion, not an estimate formed by interpolating between observations.
There is also a useful probability interpretation when \(X_1,\ldots,X_n\) are independent observations from a population with CDF \(F_X\). Each indicator \(I(X_i\le x)\) has expectation \(F_X(x)\), so
Thus, at a fixed threshold, the empirical proportion is an unbiased estimate of the population cumulative probability under independent sampling. This does not mean it equals the population CDF in a particular sample; sampling variation is precisely why the two curves need not coincide.
Conventions, Scope, and Common Traps
The simple empirical function above assumes that the data are complete, individually observed losses. The observations may be tied, but each sampled loss must be represented. If the data instead contain censoring or truncation, directly treating only the recorded values as a complete sample can misrepresent the loss distribution; an estimator appropriate to that observation mechanism is needed.
- Using the wrong count at a tie. At an observed value \(x\), count every loss \(\le x\), including the full group equal to \(x\).
- Reporting the jump instead of the cumulative value. The jump at a tied value is the number tied divided by \(n\); \(\widehat F_n(x)\) also includes every smaller observation.
- Using a different denominator. For the ordinary empirical distribution function, divide by the total number \(n\) of complete observations, not by \(n+1\) or by the number of distinct loss amounts.
- Confusing a model probability with a sample proportion. \(F_X(x)\) is supplied by the candidate model, while \(\widehat F_n(x)\) is computed from the data. Their difference can be negative or positive.
- Comparing at inconsistent thresholds. Evaluate both functions at the same loss value and use the same loss units and scale.
For example, in the sample \(1,2,2,5,8,11\), a common incorrect calculation at \(x=2\) uses \(2/6\), counting only the two observations equal to 2. The correction is to count all observations at or below 2: \(1,2,2\), so \(\widehat F_n(2)=3/6=0.5000\). The value \(2/6\) is the jump size at 2, not the empirical CDF at 2.
An empirical distribution function is a compact descriptive tool for loss data and a useful first comparison with a proposed model. It preserves the actual sample ordering and tie structure, while the candidate CDF supplies the model-implied cumulative probabilities. These pointwise comparisons help identify where the sample and model differ; formal goodness-of-fit procedures require additional definitions and calibration.
Check Your Understanding
For each problem, construct or interpret empirical cumulative values as requested. Probabilities in the answers are rounded to four decimal places where needed.
- A hypothetical sample of complete loss amounts is \(3,3,9,15\), in thousands of dollars. Find \(\widehat F_n(3)\), \(\widehat F_n(9)\), and \(\widehat F_n(15)\). (Answer: \(0.5000,\ 0.7500,\ 1.0000\), respectively.)
- Five complete losses are \(1,4,4,7,9\). A candidate model is uniform on \([0,10]\). At \(x=4\), find the empirical value, candidate CDF value, and empirical-minus-model difference. (Answer: \(0.6000,\ 0.4000,\ 0.2000\).)
- Three complete losses are \(2,5,8\), and the candidate model is exponential with mean 4. At \(x=5\), calculate \(\widehat F_n(5)\), \(F_X(5)\), and their difference. (Answer: \(0.6667,\ 0.7135,\ -0.0468\).)
- Four complete losses are \(2,2,6,10\). What is the jump size at 2, and what is \(\widehat F_n(2)\)? (Answer: Jump size \(2/4=0.5000\); \(\widehat F_n(2)=2/4=0.5000\).)
- A loss study records only whether each claim exceeded a reporting threshold; exact losses below the threshold are unavailable. Should the ordinary empirical distribution function of the recorded amounts be treated as the complete-sample empirical CDF of loss severity? Explain. (Answer: No. The observations are not complete individual losses; the reporting mechanism must be accounted for with an appropriate method.)