Why Compare Two Distribution Functions?
A fitted severity model is useful only if it describes the pattern of observed losses reasonably well. The empirical distribution function summarizes the sample directly, while a candidate distribution function represents the model. The one-sample Kolmogorov-Smirnov (KS) statistic measures the largest vertical separation between these two functions.
The calculation is compact, but its two directions must be handled carefully. At each observation, the empirical distribution function jumps upward. The largest gap in one direction can occur just after that jump; the largest gap in the other direction can occur just before it. Using the wrong side of a jump can change the result.
Finding the Gaps at the Jumps
Write the observations in increasing order as \(x_{(1)}\le \cdots \le x_{(n)}\). If the observations are distinct, then at \(x_{(i)}\), the empirical distribution function has just reached \(i/n\). Immediately before that point, it is \((i-1)/n\). Thus the two one-sided statistics can be calculated from the ordered observations:
The formula for \(D_n^-\) uses the value of \(F\) at the observation and the empirical level just before its jump. Because \(F\) is continuous, its value immediately before \(x_{(i)}\) approaches \(F(x_{(i)})\). The statistic is a supremum, so this limiting value is included even if the gap is not attained strictly to the left of the jump.
List the observed losses from smallest to largest and assign ranks \(i=1,\ldots,n\).
Calculate \(F(x_{(i)})\) at every ordered observation, using the stated model and its parameters.
For each rank, calculate \(i/n-F(x_{(i)})\) and \(F(x_{(i)})-(i-1)/n\).
The largest first-direction gap is \(D_n^+\), the largest second-direction gap is \(D_n^-\), and the larger of the two is \(D_n\).
Why These Observation Points Are Enough
Between consecutive observations, \(F_n(x)\) is constant. Since \(F\) is a nondecreasing continuous CDF, it increases or stays constant over that interval. Therefore \(F_n(x)-F(x)\) cannot increase between jumps; its largest value on such an interval is at the left side of the interval, immediately after an empirical jump. Conversely, \(F(x)-F_n(x)\) cannot decrease between jumps; its largest value is approached immediately before the next jump. These are precisely the two kinds of gaps in the formulas.
The same reasoning covers the ends of the sample. Before the first observation, \(F_n(x)=0\), and the largest possible positive gap \(F(x)-F_n(x)\) is approached just before the first jump. After the last observation, \(F_n(x)=1\), so \(F_n(x)-F(x)\) cannot exceed its value just after that last jump. The formula therefore checks every location where either direction can reach its supremum.
Worked Calculations
Worked Example: A Uniform Model With Distinct Largest Gaps
A hypothetical portfolio has five positive loss observations, in dollars: \(1, 3, 4, 8,\) and \(9\). Compare them with a fully specified Uniform\((0,10)\) candidate severity model. Calculate \(D_n^+\), \(D_n^-\), and \(D_n\). Use the model CDF \(F(x)=x/10\) on \(0\le x\le10\).
Method. The observations are already ordered, and \(n=5\). At rank \(i\), compare \(F(x_{(i)})\) first with \(i/5\), then with \((i-1)/5\).
| \(i\) | \(x_{(i)}\) | \(F(x_{(i)})\) | \(i/5-F(x_{(i)})\) | \(F(x_{(i)})-(i-1)/5\) |
|---|---|---|---|---|
| 1 | 1 | 0.10 | 0.20 - 0.10 = 0.10 | 0.10 - 0.00 = 0.10 |
| 2 | 3 | 0.30 | 0.40 - 0.30 = 0.10 | 0.30 - 0.20 = 0.10 |
| 3 | 4 | 0.40 | 0.60 - 0.40 = 0.20 | 0.40 - 0.40 = 0.00 |
| 4 | 8 | 0.80 | 0.80 - 0.80 = 0.00 | 0.80 - 0.60 = 0.20 |
| 5 | 9 | 0.90 | 1.00 - 0.90 = 0.10 | 0.90 - 0.80 = 0.10 |
The largest first-direction gap is \(D_n^+=0.20\), at rank 3. The largest second-direction gap is \(D_n^-=0.20\), at rank 4. Thus $$ D_n=\max(0.20,0.20)=0.20. $$ The two one-sided maxima occur at different observations, which is entirely possible.
Check. Both maxima are among the listed nonnegative gaps, and neither direction has a value greater than \(0.20\). Therefore the absolute maximum is \(0.20\). This method avoids the common mistake of looking for a single observation that must produce both one-sided maxima.
Worked Example: A Shifted Uniform Candidate
A hypothetical set of five claim severities is \(3,5,8,11,\) and \(13\). The candidate model is Uniform\((2,14)\), with CDF \(F(x)=(x-2)/12\) for \(2\le x\le14\). Find both one-sided statistics and the overall KS statistic.
Method. The sample size is \(n=5\). Evaluate the shifted and scaled CDF, not \(x/14\): the model starts at 2 and has width \(14-2=12\).
The model CDF values are: $$ F(3)=\frac{1}{12}=0.0833,\quad F(5)=\frac{3}{12}=0.2500,\quad F(8)=\frac{6}{12}=0.5000, $$ $$ F(11)=\frac{9}{12}=0.7500,\qquad F(13)=\frac{11}{12}=0.9167. $$ For the first direction, the rank fractions are \(0.20,0.40,0.60,0.80,1.00\). The gaps are $$ 0.20-0.0833=0.1167,\quad 0.40-0.2500=0.1500,\quad 0.60-0.5000=0.1000, $$ $$ 0.80-0.7500=0.0500,\quad 1.00-0.9167=0.0833. $$ Therefore \(D_n^+=0.1500\).
For the other direction, the empirical levels just before the jumps are \(0,0.20,0.40,0.60,0.80\). The gaps are $$ 0.0833-0=0.0833,\quad 0.2500-0.20=0.0500,\quad 0.5000-0.40=0.1000, $$ $$ 0.7500-0.60=0.1500,\quad 0.9167-0.80=0.1167. $$ Thus \(D_n^-=0.1500\), and \(D_n=\max(0.1500,0.1500)=0.1500\).
Check. The two directions tie at different ranks: rank 2 for \(D_n^+\) and rank 4 for \(D_n^-\). The calculation avoids the common mistake of treating the model as Uniform\((0,14)\), which would use the wrong lower endpoint and width.
Worked Example: An Exponential Candidate and a Tail Observation
A hypothetical sample of five losses, in thousands of dollars, is \(0.4,1.2,2.4,4.0,\) and \(6.0\). Compare the data with a fully specified exponential candidate model with mean \(2\) thousand dollars, so \(F(x)=1-e^{-x/2}\) for \(x\ge0\). Calculate the KS statistic. Report the final values to four decimal places.
Method. The exponential rate is \(1/2\) per thousand dollars. Evaluate the CDF at the five ordered losses. The approximate values are $$ F(0.4)=0.1813,\quad F(1.2)=0.4512,\quad F(2.4)=0.6988,\quad F(4.0)=0.8647,\quad F(6.0)=0.9502. $$
For \(D_n^+\), subtract these values from the post-jump levels \(0.20,0.40,0.60,0.80,1.00\): $$ 0.20-0.1813=0.0187,\quad 0.40-0.4512=-0.0512,\quad 0.60-0.6988=-0.0988, $$ $$ 0.80-0.8647=-0.0647,\quad 1.00-0.9502=0.0498. $$ The largest value is \(D_n^+=0.0498\). Negative entries are retained in the list of differences but cannot be the maximum here.
For \(D_n^-\), subtract the pre-jump levels \(0,0.20,0.40,0.60,0.80\) from the same model CDF values: $$ 0.1813-0=0.1813,\quad 0.4512-0.20=0.2512,\quad 0.6988-0.40=0.2988, $$ $$ 0.8647-0.60=0.2647,\quad 0.9502-0.80=0.1502. $$ Therefore \(D_n^-=0.2988\), and $$ D_n=\max(0.0498,0.2988)=0.2988. $$
Check. The maximum occurs in the second direction at the third observation. The result is less than 1, as required for the difference between two CDF values. This calculation avoids the mistake of using the exponential rate \(2\) when the stated mean is \(2\); the rate is the reciprocal of the mean.
Common Mistake: Using the Wrong Side of a Jump
For the Uniform\((0,10)\) sample \(1,3,4,8,9\) above, an incorrect calculation of \(D_n^-\) might subtract the post-jump level \(i/n\), rather than the pre-jump level \((i-1)/n\). That calculation gives $$ F(x_{(i)})-\frac{i}{n} = -0.10,\ -0.10,\ -0.20,\ 0.00,\ -0.10. $$ Its maximum is \(0.00\), which incorrectly suggests there is no positive gap in the \(F-F_n\) direction. The correct comparison uses the pre-jump levels and gives $$ F(x_{(i)})-\frac{i-1}{n} = 0.10,\ 0.10,\ 0.00,\ 0.20,\ 0.10, $$ so \(D_n^-=0.20\). The distinction is not cosmetic: the empirical CDF has value \(i/n\) at the observation, but its left-hand limit is \((i-1)/n\).
Interpretation and Scope
The KS statistic is a distance: a value of \(0.20\), for example, means that at some point the empirical and candidate CDFs differ by 0.20 in probability. It does not by itself say whether the candidate model should be accepted or rejected. A goodness-of-fit decision also requires an appropriate reference distribution or critical value, and the calibration depends on the testing setup.
The formulas above are for comparison with a continuous candidate CDF. They give the statistic when the model parameters are treated as specified. If parameters were estimated from the same observations, the statistic can still be calculated in the same way using the fitted CDF, but the usual critical values for a fully specified model do not automatically apply. Discrete models and tied observations also require care because their jump structure differs from the continuous setting.
A practical check is to verify three items before reporting a result: the observations are in increasing order, the model CDF uses the correct parameters and support, and each direction uses its own empirical level. Reporting \(D_n^+\), \(D_n^-\), and \(D_n\) separately makes the calculation auditable and shows which direction drives the overall discrepancy.
Check Your Understanding
For each problem, calculate both one-sided gaps and the overall statistic. Round probabilities to four decimal places where needed.
- A sample of four losses is \(2,7,11,19\). Compare it with a Uniform\((0,20)\) candidate model. Find \(D_n^+\), \(D_n^-\), and \(D_n\). (Answer: \(D_n^+=0.2000,\ D_n^-=0.2000,\ D_n=0.2000\).)
- Four measurements are \(6,8,12,14\), and the candidate is Uniform\((5,15)\). Calculate the three KS quantities. (Answer: \(D_n^+=0.2000,\ D_n^-=0.2000,\ D_n=0.2000\).)
- A sample is \(0.2,0.7,1.5,3.0\) from a hypothetical severity model. Compare it with an exponential candidate having mean 1, so \(F(x)=1-e^{-x}\). (Answer: \(D_n^+=0.0687,\ D_n^-=0.2769,\ D_n=0.2769\).)
- Five losses are \(1,2,6,7,9\), and the candidate model is Uniform\((0,10)\). Calculate the two directional maxima and the overall statistic. (Answer: \(D_n^+=0.2000,\ D_n^-=0.2000,\ D_n=0.2000\).)
- Four observations are \(1,3,7,9\). The candidate is Uniform\((0,8)\), whose CDF is 1 for values at or above 8. Calculate the statistic, taking care with the observation beyond the model's upper endpoint. (Answer: \(D_n^+=0.1250,\ D_n^-=0.3750,\ D_n=0.3750\).)