Why the Critical Value Must Match the Calibration
An Anderson-Darling (AD) statistic is not compared with a universal cutoff. The correct critical value depends on the null distribution and, in particular, on whether its parameters were specified before the data were observed or estimated from the sample being tested. A cutoff calibrated under the wrong setup can change the test decision even when the statistic itself has been calculated correctly.
This tutorial focuses on selecting and interpreting that cutoff. It assumes a random sample of independent observations from a continuous distribution and a stated significance level. The AD statistic itself gives extra weight to discrepancies near the tails; the earlier material on the Anderson-Darling statistic establishes its construction.
For ordered observations \(x_{(1)}\le \cdots \le x_{(n)}\) and a continuous null cumulative distribution function \(F_0\), write \(u_{(i)}=F_0(x_{(i)})\). The usual AD statistic is
If the model is composite, \(F_0\) in this expression is replaced by the fitted distribution function \(F_{\hat\theta}\), where \(\hat\theta\) is the parameter estimate. That substitution is straightforward; choosing the null distribution of the resulting statistic is not.
Why the Fully Specified Continuous Case Is Special
Under a simple null hypothesis with continuous \(F_0\), the probability integral transform gives \(U_i=F_0(X_i)\sim \operatorname{Uniform}(0,1)\). Independence of the observations also gives independence of the \(U_i\). Therefore, after ordering, the statistic’s null distribution depends on the sample size \(n\), but not on the particular continuous \(F_0\).
This is the reason a critical-value table for a fully specified continuous model can be used across distribution families, provided the sample size convention and significance level match. It does not mean that one fixed numerical critical value applies to every sample size. A finite-sample table, an asymptotic cutoff, and a table intended for a fitted model are not interchangeable.
The argument fails when parameters are estimated from the same observations. The values \(F_{\hat\theta}(X_i)\) are linked through the common estimate \(\hat\theta\); they are not a sample of independent uniform variables under the simple-null argument. Fitting often makes the observed sample look closer to the fitted model than a fresh sample would look to a model fixed in advance. Consequently, the fitted-model AD statistic has a different null distribution.
Match the Reference to the Test Being Performed
Before looking up a cutoff, identify four features: the distribution family, whether parameters are fixed or estimated, the estimation procedure, and the significance level. For estimated parameters, the reference must also match the parameter-fitting convention—for example, maximum likelihood rather than an unrelated estimator—unless the calibration method explicitly reproduces the estimator used.
| Testing situation | Appropriate calibration | Important qualification |
|---|---|---|
| Fully specified continuous \(F_0\) | Simple-null AD table or simulation for the stated \(n\) | The null law is distribution-free, but the cutoff depends on \(n\) and the calibration convention. |
| Normal family with estimated mean and variance | Normality-specific fitted-parameter table or its matching adjusted statistic | Do not compare raw \(A_n^2\) with a critical value tabulated for the adjusted statistic. |
| Another family with estimated parameters | Family- and estimator-appropriate table or parametric bootstrap | A normality cutoff is not a generic fitted-model cutoff. |
| Discrete, tied, truncated, or censored observations | A calibration designed for that observation and sampling scheme | The ordinary continuous, complete-data calibration does not automatically apply. |
A common normality convention uses the adjusted statistic
Under this convention, one widely used normality reference gives critical values for \(A_n^{2*}\) of 0.576, 0.656, 0.787, 0.918, and 1.092 at significance levels 15%, 10%, 5%, 2.5%, and 1%, respectively. These are not critical values for the unadjusted statistic, and they are not a general table for every estimated distribution family. Always follow the convention stated with the table or software output.
Worked Applications
Worked Example: A Fully Specified Continuous Model
A hypothetical insurer tests whether 30 independent, complete claim severities follow a fully specified continuous model \(F_0\). No parameters were estimated from these 30 claims. The calculated statistic is \(A_{30}^2=2.41\). A supplied finite-sample reference table for the fully specified continuous null at \(n=30\) gives upper-tail critical values 2.35 at 5% and 3.10 at 1%. Decide at both levels.
Method. Since every parameter of \(F_0\) was fixed before observing the data, use the simple-null table for \(n=30\). The statistic is in the rejection region when it exceeds the upper-tail critical value.
At 5%, \(2.41>2.35\), so reject the null model. At 1%, \(2.41<3.10\), so do not reject it. The outcomes differ because the 1% test requires stronger evidence against the null than the 5% test.
Check. The statistic is above the less stringent 5% cutoff but below the more stringent 1% cutoff, so the two decisions are consistent. The table values here are supplied specifically for this example; they are not a universal pair of AD cutoffs.
Common mistake avoided. Do not select a fitted-normal critical value simply because the observations are severities or because the test is an AD test. Calibration follows the null model and fitting procedure actually used.
Worked Example: Estimated Normal Parameters and the Matching Adjustment
A hypothetical portfolio has \(n=20\) independent complete observations. A normal family is fitted by estimating its mean and variance from these observations. The calculated raw statistic is \(A_{20}^2=0.77\). Use the normality convention above and its 5% critical value, 0.787, to make the test decision.
Method. Parameters were estimated, so use the fitted-normal calibration. First calculate the adjustment factor:
Then calculate the statistic on the scale used by the supplied critical value:
Because \(0.80320625>0.787\), reject normality at the 5% level. No intermediate value was rounded; to four decimal places the adjusted statistic is 0.8032.
Check. The decision uses the adjusted statistic and the adjusted-statistic cutoff. The raw value \(0.77\) is not on that scale.
Common mistake avoided. Comparing \(0.77\) directly with \(0.787\) would give “do not reject.” That calculation mismatches the statistic with the cutoff. The correction is to transform the statistic first, then compare like with like.
Worked Example: Calibrating a Fitted Weibull Model by Bootstrap
A hypothetical analyst fits a two-parameter Weibull distribution to 25 independent complete losses by maximum likelihood. The observed fitted-model AD statistic is 0.61. A parametric bootstrap is run with \(B=10{,}000\) samples: each sample has size 25 from the fitted Weibull model, and the two Weibull parameters are re-estimated by maximum likelihood in every sample. Of the bootstrap statistics, 327 are at least 0.61. Estimate the upper-tail p-value and decide at 5%.
Method. Each bootstrap replicate must repeat the estimation step, because the observed statistic was calculated after fitting parameters. With the common plus-one estimate,
The estimated p-value is below 0.05, so reject the fitted Weibull model at the 5% level. Rounded to four decimal places, the p-value is 0.0328.
Check. A p-value of about 0.033 is consistent with an observed statistic above the bootstrap 95th percentile: only about 3.3% of calibrated statistics are at least as large.
Common mistake avoided. Simulating from the fitted Weibull model but keeping the original parameter estimates fixed in every replicate would omit the fitting effect. Re-estimating in each replicate makes the simulated statistics reflect the same fitting procedure as the observed statistic.
Calibration Choices and Their Limits
A table is useful when it explicitly identifies the model, sample-size convention, estimator, statistic scale, and tail probability. If any of these do not match the test, use another reference. In practice, a parametric bootstrap is a flexible option for a composite null: fit the null model, simulate samples under that fitted model, refit the parameters in each sample, calculate the AD statistic each time, and compare the observed statistic with the simulated distribution.
The bootstrap is an approximation because it simulates from estimated rather than known parameters, and its precision depends on the number of replicates. It should reproduce the actual fitting procedure, including constraints or transformations used in the estimation. It also relies on the null family being a reasonable model for generating the bootstrap samples.
The ordinary probability-integral-transform argument requires a continuous fully specified CDF and complete independent observations. Rounded data may contain ties; discrete losses do not transform to continuous uniform observations; and censoring or truncation changes what is observed. In those settings, ordinary continuous simple-null cutoffs are not automatically valid. Use a method calibrated for the actual observation scheme rather than applying a continuous table without justification.
Check Your Understanding
For each problem, identify the appropriate calibration before making the requested comparison.
- A fully specified continuous null model is tested using \(n=18\) complete observations. The statistic is 2.20. A supplied simple-null table for this sample size gives the 5% critical value 2.10. What is the decision at 5%? (Answer: Reject, because \(2.20>2.10\).)
- A fitted-normal test has \(n=10\), raw statistic \(A_{10}^2=0.60\), and uses the adjusted-normal convention. Calculate \(A_{10}^{2*}\) to four decimals. (Answer: \(0.60(1+0.075+0.0225)=0.6585\).)
- For the preceding fitted-normal test, use a 5% adjusted-statistic critical value of 0.787. What is the decision? (Answer: Do not reject, because \(0.6585<0.787\).)
- A fitted gamma model is assessed by bootstrap. In \(B=4{,}999\) replicates, 84 statistics equal or exceed the observed statistic. Use the plus-one estimate and decide at 5%. (Answer: \(\hat p=85/5000=0.0170\); reject at 5%.)
- A continuous family is fitted by maximum likelihood, but an analyst uses a fully specified-null table because the family is continuous. What is the calibration error? (Answer: Estimation changes the statistic’s null distribution; use a calibration for that family and estimation procedure, or a bootstrap that refits the parameters in every replicate.)