Purpose and Assumptions
A goodness-of-fit test asks whether observed data are consistent with a specified probability model. For a continuous loss variable, the one-sample Kolmogorov–Smirnov (KS) test measures the largest vertical distance between the empirical distribution function and the model’s cumulative distribution function. A complete test requires more than calculating one distance: the sample must be ordered, deviations on both sides must be checked, and the resulting statistic must be compared with a critical value calibrated for the test’s assumptions.
The procedure in this tutorial is for a fully specified continuous null model. In particular, the model’s distribution and parameters are given independently of the observations being tested. The observations are assumed to be independent and identically distributed under the null hypothesis. When parameters are estimated from the same data, the usual critical values for a fully specified model do not generally apply.
For ordered observations \(x_{(1)}<\cdots<x_{(n)}\), the empirical CDF rises by \(1/n\) at each observation. The two one-sided components are
The positive component detects places where the empirical CDF is below the model CDF just before or at an observation; the negative component checks the other direction, using the empirical CDF’s value immediately before its jump. A negative candidate in either maximum is not the maximum if a larger candidate is available. The overall statistic is nonnegative because it is the maximum absolute distance.
Why the Two Deviation Formulas Work
Between consecutive observations, \(F_n(x)\) is constant while \(F_0(x)\) is continuous and nondecreasing. At observation \(x_{(i)}\), the empirical CDF jumps from \((i-1)/n\) to \(i/n\). The largest discrepancy on the two sides of that jump is therefore represented by comparing the model CDF at the observation with both adjacent empirical levels.
Taking the maximum over all jumps gives \(D_n^+\) and \(D_n^-\). This also explains why using only \(i/n-F_0(x_{(i)})\) is incomplete: it checks only one direction. The supremum distance can be driven by either side of an empirical jump.
The test hypotheses are \(H_0\): the observations follow \(F_0\), versus \(H_1\): they do not follow \(F_0\). At significance level \(\alpha\), reject \(H_0\) when \(D_n\) exceeds the appropriate critical value \(c_{\alpha,n}\). The critical value must correspond to the sample size, significance level, and calibration being used. For the examples below, the stated critical values are for a fully specified continuous model and a 5% test. Failure to reject means the sample does not provide sufficient evidence against the model at that level; it does not establish that the model is true.
Carrying Out the Test
A reliable calculation keeps the ordered observations, model CDF values, and both deviation columns together. This reduces the risk of losing an index or overlooking the maximum in the less obvious direction.
Record the null CDF, its parameters, the significance level, and the applicable critical value. Confirm that the model is fully specified and continuous.
Write the observations as \(x_{(1)}<\cdots<x_{(n)}\), then evaluate \(F_0(x_{(i)})\) at every ordered observation.
For each rank \(i\), calculate \(i/n-F_0(x_{(i)})\) and \(F_0(x_{(i)})-(i-1)/n\). Find the maximum in each column.
Set \(D_n=\max(D_n^+,D_n^-)\), compare it with \(c_{\alpha,n}\), and state the decision in the context of the hypothetical loss model.
Worked Example: Testing a Uniform Loss Model
A hypothetical insurer records five independent, uncensored losses, in thousands of dollars: \(3,9,17,24,28\). Test the fully specified model \(X\sim\operatorname{Uniform}(0,30)\) at the 5% level. Use critical value \(c_{0.05,5}=0.565\).
Method and setup. Under the null, \(F_0(x)=x/30\) for \(0\le x\le30\). The data are already ordered, and the model and its parameters are specified rather than estimated from this sample. For \(n=5\), the empirical levels before and after ranks 1 through 5 are \(0,0.2\), \(0.2,0.4\), \(0.4,0.6\), \(0.6,0.8\), and \(0.8,1\).
| \(i\) | \(x_{(i)}\) | \(F_0(x_{(i)})\) | \(i/5-F_0(x_{(i)})\) | \(F_0(x_{(i)})-(i-1)/5\) |
|---|---|---|---|---|
| 1 | 3 | 0.1000 | 0.2000 − 0.1000 = 0.1000 | 0.1000 − 0.0000 = 0.1000 |
| 2 | 9 | 0.3000 | 0.4000 − 0.3000 = 0.1000 | 0.3000 − 0.2000 = 0.1000 |
| 3 | 17 | 0.5667 | 0.6000 − 0.5667 = 0.0333 | 0.5667 − 0.4000 = 0.1667 |
| 4 | 24 | 0.8000 | 0.8000 − 0.8000 = 0.0000 | 0.8000 − 0.6000 = 0.2000 |
| 5 | 28 | 0.9333 | 1.0000 − 0.9333 = 0.0667 | 0.9333 − 0.8000 = 0.1333 |
Statistic. The largest entry in the positive-deviation column is \(D_n^+=0.1000\). The largest entry in the negative-deviation column is \(D_n^-=0.2000\), occurring at the fourth observation. Therefore \(D_n=\max(0.1000,0.2000)=0.2000\).
Decision. Since \(0.2000<0.565\), do not reject \(H_0\) at the 5% level. These hypothetical losses do not provide sufficient evidence against the specified uniform model. The calculation checks both directions: in particular, the fourth observation gives \(0.8000-3/5=0.2000\), which is the overall maximum.
Common mistake avoided. Checking only the third observation for \(D_n^-\) would give \(0.1667\) and miss the larger fourth-observation value. The maximum must be checked across every rank, not just where one candidate first appears large.
Worked Example: A Specified Exponential Loss Model
A hypothetical portfolio has five independent loss observations, in thousands of dollars: \(2,5,10,20,40\). Under the null model, \(X\) has an exponential distribution with mean 20, so \(F_0(x)=1-e^{-x/20}\) for \(x\ge0\). At the 5% level, use \(c_{0.05,5}=0.565\). Determine the test statistic and conclusion.
Method and model values. The parameter is the specified mean 20, not an estimate calculated from these five losses. Evaluating the CDF gives \(F_0(2)=1-e^{-0.1}=0.0952\), \(F_0(5)=1-e^{-0.25}=0.2212\), \(F_0(10)=1-e^{-0.5}=0.3935\), \(F_0(20)=1-e^{-1}=0.6321\), and \(F_0(40)=1-e^{-2}=0.8647\), rounded to four decimals.
| \(i\) | \(x_{(i)}\) | \(F_0(x_{(i)})\) | \(i/5-F_0(x_{(i)})\) | \(F_0(x_{(i)})-(i-1)/5\) |
|---|---|---|---|---|
| 1 | 2 | 0.0952 | 0.2000 − 0.0952 = 0.1048 | 0.0952 − 0.0000 = 0.0952 |
| 2 | 5 | 0.2212 | 0.4000 − 0.2212 = 0.1788 | 0.2212 − 0.2000 = 0.0212 |
| 3 | 10 | 0.3935 | 0.6000 − 0.3935 = 0.2065 | 0.3935 − 0.4000 = −0.0065 |
| 4 | 20 | 0.6321 | 0.8000 − 0.6321 = 0.1679 | 0.6321 − 0.6000 = 0.0321 |
| 5 | 40 | 0.8647 | 1.0000 − 0.8647 = 0.1353 | 0.8647 − 0.8000 = 0.0647 |
Statistic and decision. The column maxima are \(D_n^+=0.2065\) and \(D_n^-=0.0952\), so \(D_n=0.2065\). Because \(0.2065<0.565\), do not reject the specified exponential model at the 5% level.
Check. The third candidate in the negative column is \(-0.0065\), but it cannot be the maximum when other entries in that column are positive. It is retained in the calculation to show that each rank was checked; the maximum is \(0.0952\).
Worked Example: Detecting a Large Deviation in One Direction
A hypothetical insurer observes eight independent losses, in thousands of dollars: \(55,62,68,74,82,88,93,98\). Test \(X\sim\operatorname{Uniform}(0,100)\) at the 5% level using \(c_{0.05,8}=0.454\).
Method. For this model \(F_0(x)=x/100\). The ordered observations are given, so the CDF values are \(0.55,0.62,0.68,0.74,0.82,0.88,0.93,0.98\). The empirical levels after each rank are \(0.125,0.250,0.375,0.500,0.625,0.750,0.875,1.000\), and the levels before each rank are \(0,0.125,0.250,0.375,0.500,0.625,0.750,0.875\).
| \(i\) | \(F_0(x_{(i)})\) | Positive deviation | Negative deviation |
|---|---|---|---|
| 1 | 0.55 | 0.125 − 0.55 = −0.425 | 0.55 − 0.000 = 0.550 |
| 2 | 0.62 | 0.250 − 0.62 = −0.370 | 0.62 − 0.125 = 0.495 |
| 3 | 0.68 | 0.375 − 0.68 = −0.305 | 0.68 − 0.250 = 0.430 |
| 4 | 0.74 | 0.500 − 0.74 = −0.240 | 0.74 − 0.375 = 0.365 |
| 5 | 0.82 | 0.625 − 0.82 = −0.195 | 0.82 − 0.500 = 0.320 |
| 6 | 0.88 | 0.750 − 0.88 = −0.130 | 0.88 − 0.625 = 0.255 |
| 7 | 0.93 | 0.875 − 0.93 = −0.055 | 0.93 − 0.750 = 0.180 |
| 8 | 0.98 | 1.000 − 0.98 = 0.020 | 0.98 − 0.875 = 0.105 |
Statistic and conclusion. The largest positive deviation is \(D_n^+=0.020\), while the largest negative deviation is \(D_n^-=0.550\). Thus \(D_n=0.550\), which exceeds \(0.454\). Reject \(H_0\) at the 5% level. The sample provides evidence against the specified uniform model. Here the decisive discrepancy is in the negative component, so calculating only \(D_n^+\) would lead to the wrong decision.
Common mistake avoided. The negative-deviation formula uses \((i-1)/n\), the empirical level just before the \(i\)-th jump. At the first observation this is zero, giving \(0.55-0=0.55\); replacing it with \(i/n=0.125\) would incorrectly reduce that candidate to \(0.425\).
Interpreting and Reporting the Result
A useful report gives the null model and its parameters, sample size, significance level, both one-sided maxima, the overall statistic, the critical value, and a contextual decision. Reporting only “the test passed” is imprecise: a non-rejection is not proof that the model is correct, and the observed statistic should remain visible so that the strength of the discrepancy is clear.
The calibration matters. The stated critical values above are for the fully specified continuous null model. If a location, scale, or other parameter is fitted using the same losses, the distribution of the test statistic changes, and a critical value for a fully specified model is generally not valid. Similarly, the usual continuous-model calculation presumes a continuous distribution and independent observations. A test should not be presented as having the standard calibration if those conditions are not met.
Check Your Understanding
For each problem, calculate both one-sided deviations before deciding. Values are hypothetical, and the supplied critical values apply to the stated fully specified continuous null models.
- Four independent losses are \(5,18,31,46\), in thousands of dollars. Test \(X\sim\operatorname{Uniform}(0,50)\) at 5%, using critical value \(0.624\). (Answer: \(D_n^+=0.1500,\ D_n^-=0.1700,\ D_n=0.1700\); do not reject.)
- Five independent losses are \(8,20,44,60,72\), in thousands of dollars. Test \(X\sim\operatorname{Uniform}(0,80)\) at 5%, using critical value \(0.565\). (Answer: \(D_n^+=0.1500,\ D_n^-=0.1500,\ D_n=0.1500\); do not reject.)
- Four independent losses are \(1,4,12,25\), in thousands of dollars. Test an exponential model with specified mean 10 at 5%, using critical value \(0.624\). Round probabilities to four decimals. (Answer: \(D_n^+=0.1703,\ D_n^-=0.1988,\ D_n=0.1988\); do not reject.)
- Six independent losses are \(40,60,75,82,90,96\), in thousands of dollars. Test \(X\sim\operatorname{Uniform}(0,100)\) at 5%, using critical value \(0.521\). (Answer: \(D_n^+=0.0400,\ D_n^-=0.4333,\ D_n=0.4333\); do not reject.)
- Eight independent losses are \(50,61,70,78,85,90,95,99\), in thousands of dollars. Test \(X\sim\operatorname{Uniform}(0,100)\) at 5%, using critical value \(0.454\). (Answer: \(D_n^+=0.0100,\ D_n^-=0.5000,\ D_n=0.5000\); reject.)