Kolmogorov-Smirnov Critical Values For Specified Models

Why the Critical Value Depends on the Calibration

A Kolmogorov-Smirnov (KS) test compares the empirical distribution of a sample with a proposed cumulative distribution function. The observed statistic measures the largest vertical difference between them. To decide whether that difference is unusual, it must be compared with a reference distribution calibrated to the test’s assumptions. For the standard one-sample KS test, the relevant assumptions include independent observations from a fully specified, continuous null distribution.

The critical value is not a universal cutoff. Its choice depends on the sample size $n$, the significance level $\alpha$, and whether a finite-sample or asymptotic reference value is being used. A cutoff chosen for the wrong $n$, $\alpha$, or model-fitting situation can reverse a test conclusion.

Definition: For independent observations $X_1,\ldots,X_n$ from a fully specified continuous null cumulative distribution function $F_0$, let $F_n(x)$ be the empirical distribution function. The one-sample KS statistic is $$ D_n=\sup_x|F_n(x)-F_0(x)|. $$ At significance level $\alpha$, a critical value $d_{n,\alpha}$ is selected from the null distribution of $D_n$. The usual rejection rule is to reject the null hypothesis when $D_n>d_{n,\alpha}$. Tabulated values may be rounded, so follow the table’s stated convention if a statistic equals a displayed cutoff.

Why a Fully Specified Continuous Model Has a Common Reference Distribution

Although $F_0$ may describe claim severities, repair costs, or another continuous quantity, the null distribution of $D_n$ does not depend on the particular shape or units of $F_0$. The probability integral transform explains why. If $X$ has a continuous distribution function $F_0$, then $U=F_0(X)$ has a Uniform$(0,1)$ distribution. Applying this transformation to each independent observation converts the null sample into independent uniform observations.

The empirical distribution and the proposed distribution transform together: the largest vertical difference is therefore distributed as the largest difference between the empirical distribution of a uniform sample and the uniform distribution function. This distribution depends on the sample size, not on the specific continuous $F_0$.

Assume the null model is correct.
Each $X_j$ is independent and has the fully specified continuous distribution function $F_0$.

Transform the observations.
Set $U_j=F_0(X_j)$. Under the null, each $U_j$ is Uniform$(0,1)$, and the transformed observations remain independent.

Use the corresponding reference distribution.
The finite-sample distribution of the KS statistic is now the one for a uniform sample of size $n$. Choose its cutoff using $n$ and $\alpha$.

This justification also identifies an important condition: $F_0$ must be specified before examining the sample, rather than estimated from those same observations. When parameters are fitted, the transformed observations do not have the same standard reference distribution for the test statistic. Standard fully specified-model critical values then do not automatically give the intended significance level.

Finite-Sample Critical Values

For modest sample sizes, use finite-sample critical values when they are available. The table below gives commonly tabulated values rounded to three decimals. They are suitable for illustrating the selection process; in an application, use the specified table or software calibration and its stated rounding convention.

Sample size $n$	$\alpha=0.10$	$\alpha=0.05$	$\alpha=0.01$
10	0.368	0.409	0.489
20	0.264	0.294	0.352
40	0.190	0.210	0.252

Two directions are useful checks on table selection. With $n$ fixed, a smaller $\alpha$ requires stronger evidence to reject, so its critical value is larger. With $\alpha$ fixed, increasing $n$ generally makes the critical value smaller: with more observations, a smaller maximum discrepancy can be statistically unusual.

The value of $D_n$ itself does not determine the decision without the chosen significance level. A statistic can exceed the 10% cutoff but not the 5% cutoff, indicating evidence against the null at the former level but not at the latter. “Do not reject” is not a conclusion that the null model has been proved correct.

Worked Example: Choosing Between Two Significance Levels

A hypothetical insurer models positive repair-cost amounts as exponential with mean $4{,}000$ dollars, so the fully specified null distribution is $F_0(x)=1-\exp(-x/4000)$ for $x\geq 0$. For a sample of $n=10$ independent costs, suppose the calculated KS statistic is $D_{10}=0.380$. Determine the decision at $\alpha=0.10$ and $\alpha=0.05$, using the finite-sample table.

Step 1: Check the calibration conditions. The null model and its mean are specified, rather than estimated from these ten observations. The exponential distribution is continuous, and the observations are assumed independent. Thus the specified-model table is appropriate.

Step 2: Select the cutoffs. In the $n=10$ row, the 10% critical value is $0.368$, and the 5% critical value is $0.409$.

Step 3: Compare at each level. At 10%, $0.380>0.368$, so reject the null. At 5%, $0.380<0.409$, so do not reject the null.

Check. The 5% threshold is larger than the 10% threshold, as it should be. The result is consistent: the statistic is significant at the less stringent 10% level but not at the more stringent 5% level. The common mistake of using $0.368$ at 5% would give the wrong decision there.

Worked Example: A Stricter Test for a Specified Uniform Model

A hypothetical portfolio has independent loss amounts modeled as Uniform$(0,10{,}000)$, with both endpoints specified in advance. A sample of $n=20$ produces $D_{20}=0.310$. Decide whether to reject at $\alpha=0.05$ and at $\alpha=0.01$, using finite-sample critical values.

Step 1: Verify the model conditions. The null distribution is continuous, its parameters are specified, and the observations are assumed independent. Use the $n=20$ row.

Step 2: Read the cutoffs. The 5% cutoff is $0.294$, and the 1% cutoff is $0.352$.

Step 3: Make the comparisons. Since $0.310>0.294$, reject at 5%. Since $0.310<0.352$, do not reject at 1%.

Check. The observed statistic falls between the two cutoffs: $0.294<0.310<0.352$. Therefore the decisions differ in the expected direction. Reporting “reject at 1%” would incorrectly treat the stricter test as having a lower cutoff.

Asymptotic Critical Values

For sufficiently large $n$, an asymptotic approximation can be useful. Under a fully specified continuous null model, the distribution of $\sqrt{n}D_n$ approaches the Kolmogorov limiting distribution. Its cumulative distribution function, for $x>0$, is

P(\sqrt{n}D_n\leq x)\ \longrightarrow\ 1-2\sum_{j=1}^{\infty}(-1)^{j-1}\exp(-2j^2x^2).

The resulting approximate critical value for $D_n$ is $c_\alpha/\sqrt{n}$. Common constants are $c_{0.10}\approx1.224$, $c_{0.05}\approx1.358$, and $c_{0.01}\approx1.628$. These provide an approximation, not an exact finite-sample value. In particular, finite-sample cutoffs are often preferable for small samples, and a problem’s specified reference table takes precedence over an asymptotic approximation.

Worked Example: Applying the Large-Sample Approximation

A hypothetical claim-severity model is lognormal with $\ln X\sim N(8,0.36)$, where the first parameter is the mean and the second is the variance on the log scale. The parameters are fully specified. For $n=100$ independent positive claims, suppose the observed statistic is $D_{100}=0.142$. Use the asymptotic 5% critical value to make a decision.

Step 1: Select the approximation. The model is continuous and fully specified. Use $c_{0.05}=1.358$ and $n=100$.

Step 2: Calculate the cutoff. Since $\sqrt{100}=10$, $$ d_{100,0.05}\approx\frac{1.358}{\sqrt{100}}=\frac{1.358}{10}=0.1358. $$

Step 3: Compare. The statistic is $0.142$, and $0.142>0.1358$. Thus, using this asymptotic approximation, reject the null at the 5% level.

Check. Equivalently, $\sqrt{100}D_{100}=10(0.142)=1.420$, which exceeds $1.358$. Both comparisons give the same decision. The cutoff is reported to four decimal places; the statistic was supplied as $0.142$.

Common Exam Traps and Practical Checks

Critical-value questions often test the calibration before they test the arithmetic. A reliable check is to write down $n$, $\alpha$, whether parameters were estimated, and which reference values are being used before comparing the statistic with a cutoff.

Reversing the significance-level direction. Lowering $\alpha$ raises the critical value. For example, at $n=10$, the 5% cutoff $0.409$ exceeds the 10% cutoff $0.368$.
Ignoring sample size. A cutoff for $n=10$ cannot be substituted for one at $n=20$. The reference distribution changes with $n$.
Mixing exact and asymptotic values. Identify whether a value comes from a finite-sample table or a large-sample approximation, and do not present an approximation as exact.
Using specified-model cutoffs after fitting parameters. The standard distribution-free calibration described here assumes $F_0$ is fully specified. Estimating parameters from the tested sample changes the calibration.
Calling non-rejection proof of fit. Failing to reject means the statistic did not cross the selected threshold; it does not establish that the null model is true.

Key takeaway: For a one-sample KS test against a fully specified continuous model, choose the reference cutoff using the sample size and significance level, then compare $D_n$ with it. Smaller $\alpha$ means a larger cutoff. Use finite-sample values when appropriate, and do not apply this standard calibration automatically when the model parameters were estimated from the sample.

Check Your Understanding

For each problem, assume observations are independent and the stated continuous null model is fully specified unless the question says otherwise. Use the finite-sample table above where requested.

A hypothetical severity sample has $n=10$, $D_{10}=0.400$, and a fully specified Weibull null model. Using the table, what is the decision at $\alpha=0.05$ and at $\alpha=0.10$? (Answer: Reject at 0.10 because $0.400>0.368$; do not reject at 0.05 because $0.400<0.409$.)
A fully specified continuous model gives $D_{20}=0.300$. Using the finite-sample values, determine the decision at $\alpha=0.05$ and $\alpha=0.01$. (Answer: Reject at 0.05 because $0.300>0.294$; do not reject at 0.01 because $0.300<0.352$.)
For $n=64$, use the asymptotic 5% constant $1.358$. Calculate the approximate cutoff and decide whether $D_{64}=0.165$ is significant at 5%. (Answer: $1.358/\sqrt{64}=1.358/8=0.16975$, approximately $0.170$; do not reject because $0.165<0.16975$.)
For $n=100$, use the asymptotic 1% constant $1.628$. Decide whether $D_{100}=0.170$ is significant at 1%. (Answer: The cutoff is $1.628/\sqrt{100}=0.1628$; reject because $0.170>0.1628$.)
A candidate estimates the mean of an exponential severity model from the same $n=40$ observations used in a KS test, then proposes using the specified-model 5% cutoff $0.210$. Is that calibration automatically justified? (Answer: No. The model parameter was estimated from the tested sample, so the standard fully specified-model calibration is not automatically valid.)

What You'll Learn