Diagnosing Tail Departures With Anderson-Darling

Reading an Anderson–Darling Result

An Anderson–Darling statistic can indicate a poor fit without revealing, by itself, whether the difficulty lies among unusually small or unusually large losses. That distinction matters in actuarial work: a model may describe the center of a severity distribution reasonably well while understating the chance of very large claims, or it may misrepresent losses near the lower end. Diagnosing the direction of departure requires examining the ordered observations and their fitted probabilities, not just the final statistic.

The formulas below apply to independent, identically distributed, complete observations from a continuous distribution whose cumulative distribution function $F$ is specified. For a fitted model, the statistic can still be decomposed this way, but its rejection decision must use a calibration appropriate to the parameter-estimation procedure. Censoring, truncation, or substantial ties require methods designed for those data; the ordinary continuous-sample formula should not be applied without adjustment.

Definition: Let $X_{(1)}\le \cdots \le X_{(n)}$ be the ordered observations and let $F$ be the continuous model CDF. The Anderson–Darling statistic is $$ A^2=-n-\frac{1}{n}\sum_{i=1}^{n}(2i-1)\left[\ln F(X_{(i)})+\ln\{1-F(X_{(n+1-i)})\}\right]. $$ Define the lower-tail and upper-tail contributions, respectively, by $$ L_i=-\frac{2i-1}{n}\ln F(X_{(i)}),\qquad R_i=-\frac{2i-1}{n}\ln\{1-F(X_{(n+1-i)})\}. $$ Then $A^2=\sum_{i=1}^{n}L_i+\sum_{i=1}^{n}R_i-n$.

The $L_i$ terms inspect observations starting at the smallest; the $R_i$ terms inspect observations starting at the largest. Large $L_i$ values point to unusually small observations under the model, while large $R_i$ values point to unusually large observations. The subtracted $n$ is part of the statistic, so the sum of tail contributions is not itself $A^2$.

The coefficient $2i-1$ increases as $i$ moves from the extreme rank toward the center in each of these indexed tail sums. It does not assign the largest coefficient to the most extreme rank. Tail sensitivity comes from the logarithmic terms and the statistic’s overall weighting, not from claiming that these coefficients alone increase toward the extremes.

Why Tail Contributions Can Become Large

For a continuous model, the probability integral transform says that $U=F(X)$ is uniform on $(0,1)$ when $X$ follows $F$. Thus, under the model, the transformed ordered observations should be consistent with ordered uniform values. An observation far into the lower fitted tail has $F(X_{(1)})$ close to zero; one far into the upper fitted tail has $1-F(X_{(n)})$ close to zero.

The logarithm makes these small probabilities consequential. As $p$ approaches zero, $-\ln(p)$ grows without bound. For example, changing a tail probability from $0.10$ to $0.01$ changes its negative logarithm from about $2.3026$ to $4.6052$. This explains why a small fitted probability for an extreme observation can produce a conspicuous tail contribution, even though the coefficient at the most extreme rank is $1$.

\lim_{p\to 0^+}-\ln(p)=\infty

The decomposition follows directly by separating the two logarithms in the defining sum. The first logarithm for each $i$ gives $L_i$; the second gives $R_i$. Summing those terms and retaining the original $-n$ yields $A^2=\sum L_i+\sum R_i-n$. This identity makes tail diagnosis possible without changing the test statistic.

Order the losses and evaluate fitted probabilities.
Calculate $F(X_{(i)})$ for lower-tail checks and $1-F(X_{(n+1-i)})$ for upper-tail checks.

Calculate contributions by tail.
Use $L_i$ and $R_i$ separately. Inspect which terms are large and which observations produce them.

Compare empirical and fitted tail behavior.
Ask whether the sample contains more extreme losses than the fitted model would suggest, or whether unusually small losses are responsible.

Make the test decision using the correct calibration.
A tail diagnosis explains the source of a large statistic; the calibrated critical value or p-value determines whether the evidence is sufficient to reject.

Worked Tail Decompositions

Worked Example: A Balanced Set of Fitted Probabilities

A hypothetical portfolio has four complete independent losses. Under a specified exponential severity model with mean $\theta=1$ loss unit, the CDF is $F(x)=1-e^{-x}$, for $x\ge 0$. Suppose the ordered losses are $0.1054,\ 0.5108,\ 0.9163,\ 2.3026$ loss units. Their fitted CDF values, to four decimals, are $0.1000,\ 0.4000,\ 0.6000,\ 0.9000$. Decompose $A^2$ by tail.

Lower tail. Here $n=4$, so the coefficients divided by $n$ are $1/4,\ 3/4,\ 5/4,\ 7/4$. The four lower contributions are

\begin{aligned} L_1&=-\tfrac14\ln(0.10)=0.5756,\\ L_2&=-\tfrac34\ln(0.40)=0.6872,\\ L_3&=-\tfrac54\ln(0.60)=0.6385,\\ L_4&=-\tfrac74\ln(0.90)=0.1844. \end{aligned}

Their sum is $0.5756+0.6872+0.6385+0.1844=2.0857$, using the displayed rounded terms.

Upper tail. Starting from the largest loss, the fitted survival probabilities are $0.10,\ 0.40,\ 0.60,\ 0.90$. Therefore the $R_i$ contributions are the same four values, and $\sum R_i=2.0857$.

Statistic and interpretation. Using the rounded tail sums, $A^2=2.0857+2.0857-4=0.1714$. The contributions are balanced: neither tail has an unusually small fitted probability relative to the other in this decomposition. The decision to reject still requires the appropriate calibrated reference distribution; this calculation alone does not provide a critical value.

Method and trap avoided. The method is to calculate each tail separately and then combine them. A small statistic is not proof that every part of the distribution fits perfectly, and the unadjusted sum of contributions is not $A^2$; the $-n$ term must be included.

Worked Example: An Unusually Large Loss

A hypothetical insurer observes four complete independent losses. A specified exponential model has mean $\theta=1$ loss unit and CDF $F(x)=1-e^{-x}$, for $x\ge0$. Suppose the ordered losses are $0.2231,\ 0.5108,\ 0.9163,\ 4.6052$ loss units, giving fitted CDF values $0.20,\ 0.40,\ 0.60,\ 0.99$. Calculate the tail totals and identify the direction of the departure.

Lower-tail total. Applying the coefficients $1/4,\ 3/4,\ 5/4,\ 7/4$ to the ordered CDF values gives

\begin{aligned} L_1&=-\tfrac14\ln(0.20)=0.4024,\\ L_2&=-\tfrac34\ln(0.40)=0.6872,\\ L_3&=-\tfrac54\ln(0.60)=0.6385,\\ L_4&=-\tfrac74\ln(0.99)=0.0176. \end{aligned}

Thus $\sum L_i=0.4024+0.6872+0.6385+0.0176=1.7457$.

Upper-tail total. In descending-loss order, the fitted survival probabilities are $0.01,\ 0.40,\ 0.60,\ 0.80$. Hence

\begin{aligned} R_1&=-\tfrac14\ln(0.01)=1.1513,\\ R_2&=-\tfrac34\ln(0.40)=0.6872,\\ R_3&=-\tfrac54\ln(0.60)=0.6385,\\ R_4&=-\tfrac74\ln(0.80)=0.3905. \end{aligned}

The upper-tail total is $1.1513+0.6872+0.6385+0.3905=2.8675$. Therefore $A^2=1.7457+2.8675-4=0.6132$, with rounding performed only at the final displayed results. The upper-tail total exceeds the lower-tail total, and the largest loss has fitted survival probability $0.01$. The evidence in this decomposition is therefore concentrated more in the upper tail: the observed largest loss is unusually large under the specified model.

Method and trap avoided. Inspect both the tail totals and the individual terms. The large $R_1$ is driven by $-\ln(0.01)$, not by a coefficient of $7$: at the most extreme upper rank, the coefficient is $1$, so the factor in $R_1$ is $1/4$.

Worked Example: An Unusually Small Loss

For a separate hypothetical group of four complete independent losses, suppose the specified exponential model again has mean $\theta=1$ loss unit, with $F(x)=1-e^{-x}$, for $x\ge0$. The ordered losses are $0.0101,\ 0.2231,\ 0.5108,\ 1.6094$ loss units, giving fitted CDF values $0.01,\ 0.20,\ 0.40,\ 0.80$. Calculate the tail totals and interpret them.

Lower-tail total. The contributions are

\begin{aligned} L_1&=-\tfrac14\ln(0.01)=1.1513,\\ L_2&=-\tfrac34\ln(0.20)=1.2071,\\ L_3&=-\tfrac54\ln(0.40)=1.1454,\\ L_4&=-\tfrac74\ln(0.80)=0.3905. \end{aligned}

Thus $\sum L_i=1.1513+1.2071+1.1454+0.3905=3.8943$.

Upper-tail total. Starting from the largest loss, the fitted survival probabilities are $0.20,\ 0.60,\ 0.80,\ 0.99$. Therefore

\begin{aligned} R_1&=-\tfrac14\ln(0.20)=0.4024,\\ R_2&=-\tfrac34\ln(0.60)=0.3831,\\ R_3&=-\tfrac54\ln(0.80)=0.2789,\\ R_4&=-\tfrac74\ln(0.99)=0.0176. \end{aligned}

The upper-tail total is $0.4024+0.3831+0.2789+0.0176=1.0820$. Thus $A^2=3.8943+1.0820-4=0.9763$, using the rounded totals. The lower-tail contribution is much larger, led by the smallest loss with fitted CDF $0.01$. This points to a lower-tail departure, rather than unusually large losses, as the main source of the statistic in this example.

Method and trap avoided. The $R_i$ terms use survival probabilities for observations read from largest to smallest. Substituting CDF values for those survival probabilities would reverse the upper-tail diagnosis.

Interpreting Direction and Avoiding Overclaims

A large upper-tail contribution is evidence that one or more large observed losses have small fitted survival probabilities. In practical terms, the model may be too light-tailed for the observed sample, or its fitted location or scale may be inadequate. A large lower-tail contribution indicates that one or more small observed losses have small fitted CDF values. The direction is diagnostic, not a unique explanation: the same tail pattern can arise from a wrong distributional shape, an incorrect parameter value, data errors, or an unmodeled feature of the population.

A useful complementary check is to compare observed and fitted tail probabilities at a threshold. If $k$ of $n$ complete observations exceed $c$, the empirical exceedance proportion is $k/n$, while the model’s exceedance probability is $1-F(c)$. This comparison is not a replacement for the Anderson–Darling statistic, but it can make the direction of an apparent upper-tail departure easier to explain. For lower-tail checks, compare the proportion at or below $c$ with $F(c)$.

Worked Example: Checking an Upper-Tail Pattern at a Threshold

A hypothetical sample contains $n=20$ complete independent losses. A specified exponential model has mean $\theta=100$ dollars, so $F(x)=1-e^{-x/100}$ and $S_X(x)=e^{-x/100}$ for $x\ge0$. Four observed losses exceed $c=\$300$. Compare the empirical and fitted exceedance probabilities.

Empirical exceedance probability. The observed proportion is $4/20=0.2000$.

Fitted exceedance probability. The model gives $S_X(300)=e^{-300/100}=e^{-3}=0.0498$, to four decimal places. In a sample of 20, the corresponding expected number of exceedances is $20(0.0498)=0.9960$.

Comparison. The empirical proportion, $0.2000$, is about $0.2000/0.0498=4.0161$ times the fitted probability. This is a notable descriptive upper-tail discrepancy: four losses exceed the threshold, compared with about one expected under the model. It supports checking whether the upper tail contributes strongly to the Anderson–Darling statistic, but it does not by itself determine the test decision or establish why the discrepancy occurred.

Method and trap avoided. Compare proportions with probabilities, or counts with expected counts, using matching scales. Comparing the observed count $4$ directly with the probability $0.0498$ mixes units.

Common Mistakes

Reading the coefficient backward. In the defined $L_i$ and $R_i$ sums, the most extreme rank has coefficient $1$; the coefficients increase as $i$ moves inward. For $n=4$, the coefficients are $1,3,5,7$, not a coefficient of $7$ on the extreme-rank term.
Blaming the coefficient alone for tail sensitivity. The logarithmic terms can become large when a fitted tail probability is close to zero. The coefficients do not increase toward the extremes in these indexed sums.
Calling a large statistic an upper-tail failure without inspecting contributions. The statistic combines both tails. Compare $\sum L_i$ with $\sum R_i$ and inspect the largest individual terms.
Confusing an explanation with a decision. Tail contributions identify where a discrepancy appears. Rejection still depends on a critical value or p-value calibrated for the model and estimation procedure.
Using the ordinary formula mechanically on nonstandard data. Censoring, truncation, or ties can change the appropriate test construction and calibration.

Key takeaway: Separate the Anderson–Darling statistic into its lower- and upper-tail contributions, then trace large terms to the corresponding fitted CDF or survival probabilities. The decomposition diagnoses the direction of departure; calibrated inference determines whether the model is rejected.

Check Your Understanding

For each problem, identify the indicated tail behavior or compute the requested contribution. Use probabilities as given and round numerical answers to four decimals.

For $n=4$, an ordered sample has fitted CDF values $0.08,\ 0.30,\ 0.70,\ 0.92$. What is the most extreme upper-tail contribution $R_1$? (Answer: $0.6314$)
For $n=5$, the smallest observation has fitted CDF $0.02$. Calculate its lower-tail contribution $L_1$. (Answer: $0.7824$)
A sample has lower-tail total $2.10$ and upper-tail total $3.70$, with $n=5$. Calculate $A^2$ and identify the more prominent tail contribution. (Answer: $A^2=0.8000$; upper tail)
In a sample of 30 losses, six exceed a threshold. A fitted model assigns exceedance probability $0.08$ at that threshold. Calculate the empirical exceedance proportion and the model’s expected exceedance count; state the direction suggested by the comparison. (Answer: empirical proportion $0.2000$, expected count $2.4$; the observed sample suggests more upper-tail exceedances than fitted)
For $n=4$, an analyst assigns coefficient $7$ to the most extreme lower-tail term with fitted CDF $0.01$. State the correct coefficient and calculate the correct $L_1$. (Answer: coefficient $1$; $L_1=0.25[-\ln(0.01)]=1.1513$)

What You'll Learn