Introduction: Selection Is a Decision, Not a Single Statistic
A severity model can look acceptable under one criterion and less convincing under another. Information criteria summarize a trade-off between fit and complexity, goodness-of-fit tests examine discrepancies in specified parts of the distribution, and tail diagnostics focus attention on the losses that can drive a high excess-of-loss payment. A pricing decision requires putting these pieces together rather than treating any one of them as a verdict.
This workshop uses a hypothetical set of 120 complete, positive loss observations, each measured in dollars. Three continuous severity models are fitted by maximum likelihood to the same observations: an exponential model, a lognormal model, and a two-parameter Weibull model. The observations are assumed independent and identically distributed for the likelihood calculations. There is no truncation or censoring in this case.
The exponential candidate has one estimated parameter, the rate \(\lambda\), and density \(f_X(x)=\lambda e^{-\lambda x}\) for \(x>0\). The lognormal candidate has \(X\sim\operatorname{Lognormal}(\mu,\sigma^2)\), meaning \(\ln X\sim N(\mu,\sigma^2)\), with two estimated parameters. The Weibull candidate has density \(f_X(x)=(\beta/\eta)(x/\eta)^{\beta-1}e^{-(x/\eta)^\beta}\), \(x>0\), with shape \(\beta>0\) and scale \(\eta>0\). All three models are fitted to the same loss observations; the Weibull and lognormal each use two estimated parameters.
Worked Example: Comparing Three Fitted Severity Models
Worked Example: Comparing Three Fitted Severity Models
Suppose the fitted-model report for the 120 hypothetical losses gives the maximized log likelihoods shown below. For the exponential fit, the estimated rate is \(\hat\lambda=0.0004\) per dollar. For the lognormal fit, \(\hat\mu=7.4\) and \(\hat\sigma=0.9\). The two-parameter Weibull fit has reported maximized log likelihood \(-1047.000\). Calculate AIC and BIC and identify the model preferred by each criterion.
Method. Apply both formulas to each model, using \(n=120\), the same data, and the number of estimated parameters in that model.
| Model | \(k\) | \(\ell\) | AIC calculation | BIC calculation |
|---|---|---|---|---|
| Exponential | 1 | \(-1058.886\) | \(2117.772+2=2119.772\) | \(2117.772+\ln(120)=2122.559\) |
| Lognormal | 2 | \(-1045.629\) | \(2091.258+4=2095.258\) | \(2091.258+2\ln(120)=2100.833\) |
| Weibull | 2 | \(-1047.000\) | \(2094.000+4=2098.000\) | \(2094.000+2\ln(120)=2103.575\) |
For example, the lognormal AIC is \(-2(-1045.629)+2(2)=2091.258+4=2095.258\), or 2095.26 to two decimals. Also, \(\ln(120)=4.787492\), so its BIC is \(2091.258+2(4.787492)=2100.833\), or 2100.83. The lognormal has the smallest AIC and BIC among these candidates. Its AIC advantage over the Weibull is 2.74, while its BIC advantage is 2.74 as well: these two models have the same parameter count, so both criteria rank them by their maximized likelihood.
Check. The exponential has the largest AIC and BIC, consistent with its lower maximized log likelihood. BIC’s penalty for one parameter is \(\ln(120)=4.787492\), greater than AIC’s penalty of 2; nevertheless, this does not change the ranking here.
Common mistake avoided: Counting the lognormal as having three parameters by including a separate “lognormal” indicator would give an incorrect AIC of \(2091.258+6=2097.258\). The distribution family is not an extra parameter within the fitted lognormal model: only \(\mu\) and \(\sigma\) are estimated, so \(k=2\).
Why the Criteria Penalize Complexity
The \(-2\ell\) term rewards a model for assigning greater likelihood to the observations; the penalty term discourages adding parameters simply to improve fit. AIC uses a fixed penalty of 2 per estimated parameter, while BIC uses \(\ln(n)\) per parameter. As \(n\) increases, BIC’s penalty per parameter increases, which often makes BIC more conservative about added complexity.
A short justification for the AIC form is useful. Under regularity conditions, the expected log-likelihood of a fitted model on new data is not the same as its maximized in-sample log-likelihood: parameter estimation makes the in-sample value optimistic. For a model with \(k\) freely estimated parameters, the leading large-sample optimism on the \(-2\)-log-likelihood scale is approximately \(2k\). Subtracting this optimism-adjustment from the in-sample fit, in the equivalent minimization convention, gives \(-2\ell+2k\). This is an approximation, not an exact finite-sample identity; it relies on regularity and does not make AIC a goodness-of-fit test. BIC uses a different asymptotic argument and is not simply a more accurate AIC.
The comparison is meaningful only if the likelihoods use the same observations and compatible likelihood definitions. Comparing a likelihood from truncated observations with one from untruncated observations, or comparing a likelihood for a transformed response without the required transformation adjustment, can produce a misleading ranking.
Worked Example: A Grouped Fit Check and Tail Diagnostics
Worked Example: A Grouped Fit Check and Tail Diagnostics
Consider the exponential fit from the case above, with estimated rate \(0.0004\) per dollar. Four prespecified loss intervals have observed counts \(38,34,29,19\). Under the fitted model, their expected counts are \(39.56,26.52,29.69,24.23\), which sum to 120. Calculate the Pearson chi-square statistic, its degrees of freedom, and the approximate upper-tail probability. Then compare observed and fitted exceedance counts above \$8,000 and \$12,000.
Grouped test. The Pearson statistic is the sum of squared observed-minus-expected differences divided by expected counts:
There are four cells and one parameter estimated from the data, so the usual reference degrees of freedom are \(4-1-1=2\), provided the cell probabilities were not separately estimated and the grouping was specified appropriately. For a chi-square random variable with 2 degrees of freedom, the upper-tail probability is \(e^{-X^2/2}\). Thus \(e^{-3.3162/2}\approx0.1905\). This test does not reject the exponential fit at a 5% level. It also does not prove the model is correct. The expected counts are all above 5 here; if cells were sparse, the chi-square approximation would need reconsideration, for example by combining defensible adjacent cells or using an appropriate alternative calibration.
Tail check. In the 120 losses, suppose 6 exceed \$8,000 and 3 exceed \$12,000. The empirical proportions are \(6/120=0.0500\) and \(3/120=0.0250\). The exponential survival function is \(S_X(x)=e^{-\lambda x}\). At \$8,000 it gives \(e^{-0.0004(8000)}=e^{-3.2}\approx0.0408\), or \(120e^{-3.2}\approx4.89\) expected exceedances. At \$12,000 it gives \(e^{-4.8}\approx0.0082\), or \(120e^{-4.8}\approx0.99\) expected exceedances.
For the fitted lognormal model, the exceedance probability at threshold \(K\) is \(1-\Phi((\ln K-\mu)/\sigma)\), where \(\Phi\) is the standard normal distribution function. At \$8,000 this is approximately \(0.0389\), or \(120(0.0389)\approx4.67\) expected exceedances. At \$12,000 it is approximately \(0.0134\), or \(120(0.0134)\approx1.61\) expected exceedances. The observed counts are higher than both models’ expected counts at both thresholds, especially at \$12,000 relative to the exponential expectation. But counts of 6 and 3 are small: they provide a warning to investigate tail fit, not decisive evidence that one model is superior.
A graphical check should examine the empirical survival curve against fitted survival curves, particularly in the upper tail. An exponential model has \(\ln S_X(x)=-\lambda x\), a straight line when log survival is plotted against loss \(x\). A lognormal model instead has \(\ln X\) normally distributed; a normal-probability plot of logged losses can help assess that shape. Plotting choices matter, and the few largest observations are highly variable. Graphs should support, not replace, numerical checks and an understanding of the intended use.
Worked Example: From Severity Choice to Layer Pricing
Worked Example: From Severity Choice to Layer Pricing
A hypothetical insurer is considering an excess-of-loss layer that pays \(\min((X-8000)^+,4000)\) per claim, where \(X\) is the ground-up loss in dollars and \(a^+=\max(a,0)\). Use the fitted lognormal model \(X\sim\operatorname{Lognormal}(7.4,0.9^2)\) to calculate expected payment per claim. Compare it with the exponential model having rate \(0.0004\) per dollar. If the expected claim frequency is 0.08 claims per policy-year, calculate the layer’s indicated pure premium per policy-year under each model. Ignore expenses, risk loadings, and other contract features.
Method and justification. For a nonnegative loss, the expected payment above attachment \(K\) is the stop-loss expectation \(\operatorname{E}[(X-K)^+]=\int_K^\infty S_X(x)\,dx\). This follows by writing \((X-K)^+=\int_K^\infty \mathbf{1}_{\{X>x\}}\,dx\) and taking expectations. A layer attaching at \(K\) and ending at \(M\) therefore has expected payment \(\operatorname{E}[(X-K)^+]-\operatorname{E}[(X-M)^+]\).
For a lognormal loss, set \(Z=(\ln X-\mu)/\sigma\). The tail probability is \(P(X>K)=\Phi((\mu-\ln K)/\sigma)\). To obtain the truncated first moment, integrate \(x f_X(x)\) above \(K\); completing the square in the normal density gives \(\operatorname{E}[X\mathbf{1}_{\{X>K\}}]=e^{\mu+\sigma^2/2}\Phi((\mu+\sigma^2-\ln K)/\sigma)\). Subtracting \(K P(X>K)\) yields the formula below.
Here \(e^{7.4+0.9^2/2}=e^{7.805}\approx \$2{,}452.84\). Evaluating the formula with the unrounded normal probabilities gives an excess expectation of approximately \$164.42 at \(K=\$8{,}000\), and approximately \$70.63 at \(K=\$12{,}000\). The layer payment is the difference of these unrounded expectations; to the nearest cent it is approximately \$93.78 per claim. The displayed excess expectations are rounded individually, so subtracting just those displayed amounts can differ by one cent.
For an exponential severity with rate \(\lambda\), \(S_X(x)=e^{-\lambda x}\), and integrating the survival function gives \(\operatorname{E}[(X-K)^+]=e^{-\lambda K}/\lambda\). Thus the expected layer payment is
Under the stated frequency assumption, pure premium is expected frequency times expected payment per claim. The lognormal indication is \(0.08(\$93.78)\approx\$7.50\) per policy-year; the exponential indication is \(0.08(\$81.33)\approx\$6.51\) per policy-year. These are hypothetical pure premiums, not final rates: they exclude expenses, uncertainty loadings, and any contractual or portfolio adjustments.
Check. Both model indications are below the expected payment per claim, as they should be when multiplied by a frequency below one. The lognormal indication is higher because its fitted upper tail produces a larger expected payment for this layer.
Putting the Evidence Together
In this case, both AIC and BIC prefer the lognormal among the three fitted candidates. The grouped chi-square check does not reject the exponential model at a 5% level, but the test is about the chosen grouping and can have limited power against tail differences. The tail counts and the layer calculations show why a non-rejection of the grouped test should not be interpreted as proof that the exponential is appropriate for pricing this layer.
The practical conclusion is conditional on the intended use. For a central description of losses, the information criteria and overall fit checks are relevant. For a high attachment layer, the tail shape and the stability of its estimated parameters deserve particular scrutiny. The lognormal produces the larger layer indication in this example, but a pricing actuary should also assess parameter uncertainty, sensitivity to the largest observations, and whether the fitted model is suitable for the portfolio and exposure period.
- Use AIC and BIC only after confirming that models were fitted to the same data under comparable likelihoods.
- Count estimated parameters correctly, and use the appropriate sample size in BIC.
- Interpret a goodness-of-fit p-value as evidence about a specified test, not as a probability that the model is true.
- Inspect the portion of the distribution that drives the decision; a good central fit does not guarantee a good tail fit.
- Translate severity differences into the actual payment function before drawing a pricing conclusion.
Check Your Understanding
Work each problem using the stated assumptions. Probabilities are rounded to four decimals where shown, and dollar amounts are rounded to the nearest cent.
- Two severity models are fitted to the same \(n=80\) losses. Model A has \(k=1\) and \(\ell=-210\); Model B has \(k=2\) and \(\ell=-205\). Calculate both AIC and BIC values and identify the preferred model under each criterion. (Answer: AIC values are 422 and 414; BIC values are 424.38 and 418.76; Model B is preferred by both.)
- A four-cell goodness-of-fit test has observed counts \(24,26,18,12\) and expected counts \(20,20,20,20\). One parameter was estimated from the data. Calculate the Pearson statistic and its degrees of freedom; give the approximate upper-tail probability. (Answer: \(X^2=6.00\), degrees of freedom \(=2\), upper-tail probability \(e^{-3}\approx0.0498\).)
- Among 100 hypothetical losses, 4 exceed \$6,000. A fitted exponential model gives \(S_X(6000)=0.0300\), while a fitted lognormal gives \(S_X(6000)=0.0500\). Calculate each model’s expected number of exceedances and state which has the closer expected count to the observed count, without claiming the comparison proves either model is correct. (Answer: Exponential 3 and lognormal 5 expected exceedances; the exponential is closer by count, with absolute discrepancies 1 and 1, so they are equally close.)
- An exponential severity model has rate \(\lambda=0.0002\) per dollar. Find expected payment per loss for a layer attaching at \$5,000 with a \$5,000 limit. (Answer: \((e^{-1}-e^{-2})/0.0002\approx\$1{,}162.72\) per loss.)
- A hypothetical layer has expected payment per claim of \$125 under one fitted model and \$110 under another. Expected frequency is 0.06 claims per policy-year. Calculate each indicated pure premium and the difference, then name one item excluded from this pure-premium calculation. (Answer: \$7.50 and \$6.60 per policy-year, a difference of \$0.90; expenses, risk loadings, or other contract adjustments are excluded.)