Interpreting AIC Differences And Relative Support

Introduction

An AIC table ranks candidate models, but the rank alone does not show how separated the candidates are. A model with the second-lowest AIC might be close to the best-scoring model, or it might be far behind. The difference from the minimum AIC gives a useful scale for describing that separation.

That scale needs careful interpretation. An AIC difference can be converted to a measure of relative support on the AIC scale, but it is not a hypothesis-test statistic, a p-value, or the probability that a candidate model is true. This tutorial develops the calculation, its interpretation, and the limits of the conclusions it supports.

Definition: For a set of candidate models fitted to the same data using a consistent likelihood basis, let $\mathrm{AIC}_i$ denote the AIC of model $i$. The AIC difference for model $i$ is $$ \Delta_i=\mathrm{AIC}_i-\min_j(\mathrm{AIC}_j). $$ The minimum-AIC model has $\Delta_i=0$, and all other differences are nonnegative. The relative support of model $i$, measured against the minimum-AIC model, is $$ r_i=\exp\left(-\frac{\Delta_i}{2}\right). $$

From AIC Differences to Relative Support

AIC combines fit and a penalty for the number of estimated parameters. For a model with maximized log likelihood $\hat{\ell}$ and $k$ estimated parameters, $\mathrm{AIC}=-2\hat{\ell}+2k$. Smaller AIC is preferred under the information-criterion objective. Once the AIC values have been calculated, interpreting their differences requires subtracting the minimum and applying the exponential transformation.

The factor of one-half in the exponent matters. For example, a difference of 4 gives relative support $\exp(-2)$, not $\exp(-4)$. Also, $r_i$ compares one model with the minimum-AIC model; it is not a probability distribution across the candidates.

Derivation: The AIC-based relative support of model $i$ is defined by exponentiating minus one-half of its AIC difference from the best-scoring model. Since $\Delta_i=\mathrm{AIC}_i-\mathrm{AIC}_{\min}$, this gives $$ r_i=\exp\left[-\frac{\mathrm{AIC}_i-\mathrm{AIC}_{\min}}{2}\right]. $$ For two models $i$ and $b$, the ratio of their relative supports is $$ \frac{r_i}{r_b} =\frac{\exp(-\Delta_i/2)}{\exp(-\Delta_b/2)} =\exp\left[-\frac{\Delta_i-\Delta_b}{2}\right]. $$ Thus, when $b$ is the minimum-AIC model, $\Delta_b=0$ and $r_b=1$; the ratio of the best model’s support to model $i$’s support is $\exp(\Delta_i/2)$. The half-difference appears because AIC is on the $-2$ log-likelihood scale, with a parameter penalty included.

These values are often called relative likelihoods or relative support values in an AIC comparison. The terminology does not mean that they are probabilities that the models are correct. They summarize how the candidates compare on the AIC criterion, given the candidate set and the data used in the calculation.

Worked Example: A Small Difference Between Two Severity Models

A hypothetical insurer compares two distributions for 120 independent, positive claim severities, measured in thousands of dollars. The candidate models are a Weibull distribution with fitted shape $1.5$ and scale $7.0$, and a gamma distribution with fitted shape $2.4$ and scale $3.1$. No interest-rate or mortality model is involved. The fitted log likelihoods and parameter counts are: Weibull, $\hat{\ell}=-204.3$ with $k=2$; gamma, $\hat{\ell}=-205.0$ with $k=2$. Calculate and interpret the relative support.

Method. Calculate each AIC, identify the minimum, subtract it from both values, and then apply $r_i=\exp(-\Delta_i/2)$.

For the Weibull model,

\mathrm{AIC}_{W}=-2(-204.3)+2(2)=408.6+4=412.6.

For the gamma model,

\mathrm{AIC}_{G}=-2(-205.0)+2(2)=410.0+4=414.0.

The minimum is $412.6$, so $\Delta_W=412.6-412.6=0$ and $\Delta_G=414.0-412.6=1.4$. The relative support values are

r_W=\exp(0)=1,\qquad r_G=\exp(-1.4/2)=\exp(-0.7)\approx 0.4966.

Thus, the gamma model has about $0.4966$ times the AIC-scale relative support of the Weibull model. Equivalently, the Weibull-to-gamma support ratio is $\exp(0.7)\approx 2.0138$, calculated from the unrounded exponential expression. The difference is small enough that the gamma model should not be described as having no support; this comparison alone does not settle which severity model is most useful for every actuarial purpose.

Check. The gamma AIC is larger, so its difference is positive and its relative support is below 1, consistent with the result.

Guidelines for the Scale

AIC differences are most useful as a graded measure, not as a pass-or-fail cutoff. The following conventions are common rules of thumb. They are guides to describing separation among the candidates, not universal decision thresholds.

AIC difference $\Delta_i$	Typical description
About 0 to 2	Little separation; the candidate may have substantial support relative to the minimum-AIC model.
About 2 to 4	Some separation; the candidate has less support, but may still be a plausible contender.
About 4 to 7	Considerably less support than the minimum-AIC model.
Greater than about 10	Very little support relative to the minimum-AIC model in this candidate set.

The corresponding relative support falls exponentially as $\Delta_i$ increases. For example, $\Delta_i=2$ gives $\exp(-1)\approx0.3679$, whereas $\Delta_i=10$ gives $\exp(-5)\approx0.0067$. The same difference has the same relative-support calculation regardless of the absolute size of the AIC values. A difference of 3 between AIC values near 100 has the same transformed value as a difference of 3 between AIC values near 1,000.

The amount of separation should be reported with the actual difference and, where useful, the relative support. Avoid turning a rule of thumb into a guarantee: a modest AIC difference does not prove that two models are practically interchangeable, and a large difference does not establish that the lowest-AIC model is adequate.

Worked Example: Interpreting a Set of Three Candidates

A hypothetical portfolio has 85 independent positive claim severities, in thousands of dollars. Three fitted candidates are a gamma distribution with shape $1.8$ and scale $4.0$, a lognormal distribution with $\mu=1.20$ and $\sigma=0.75$ on the log scale, and a Weibull distribution with shape $1.3$ and scale $6.2$. No interest-rate or mortality model is involved. Their maximized log likelihoods are respectively $-101.15$, $-102.75$, and $-105.15$, and each has $k=2$ estimated parameters. Find each AIC difference and relative support, then describe the results.

Method. First use $\mathrm{AIC}=-2\hat{\ell}+2k$ for each candidate. Then compare each AIC with the smallest one.

For the gamma model, $\mathrm{AIC}_{G}=-2(-101.15)+2(2)=202.30+4=206.30$. For the lognormal model, $\mathrm{AIC}_{LN}=-2(-102.75)+2(2)=205.50+4=209.50$. For the Weibull model, $\mathrm{AIC}_{W}=-2(-105.15)+2(2)=210.30+4=214.30$. The minimum AIC is $206.30$.

\Delta_G=206.30-206.30=0,\quad \Delta_{LN}=209.50-206.30=3.20,\quad \Delta_W=214.30-206.30=8.00.

The relative support values are

r_G=1,\qquad r_{LN}=\exp(-3.20/2)=\exp(-1.6)\approx0.2019,\qquad r_W=\exp(-8.00/2)=\exp(-4)\approx0.0183.

On this AIC scale, the lognormal model has about $0.2019$ times the gamma model’s relative support, while the Weibull model has about $0.0183$ times the gamma model’s relative support. The differences of $3.2$ and $8.0$ indicate meaningful separation, especially for the Weibull candidate, but they do not say that the gamma model has an 80% probability of being true. The values are comparisons to the minimum-AIC model, not probabilities.

Check. The gamma model has the smallest AIC and therefore relative support 1. Both other models have larger AIC values and relative support below 1.

Compare Models, Not Hypothesis-Test Results

An AIC difference is not a likelihood-ratio test statistic. It does not have a reference chi-square distribution merely because it can be calculated from log likelihoods. In particular, one should not use the difference as a test statistic, attach a p-value to it without a justified testing procedure, or conclude that a model has been statistically rejected because its AIC is larger.

Nor is $r_i$ a posterior probability. It is not normalized across the candidate models, and it does not incorporate prior probabilities. Even if relative support values are converted into normalized Akaike weights, those weights remain criterion-based summaries rather than probabilities that the models are true. AIC comparison also does not replace checking whether the fitted models are adequate for the intended actuarial use.

Worked Example: A Material Difference and the Half-Exponent Trap

A hypothetical actuary fits two models to the same 60 independent positive equipment-loss severities, measured in dollars. A Pareto type II model with fitted scale $2{,}600$ and shape $1.7$ has $\hat{\ell}=-387.85$ and $k=2$. A lognormal model with fitted log-scale parameters $\mu=1.10$ and $\sigma=0.80$ has $\hat{\ell}=-390.60$ and $k=2$. No interest-rate or mortality model is involved. Calculate the AIC difference and relative support of the lognormal model. State what the result does not establish.

Method. Calculate both AIC values, subtract the minimum, and use the half-difference in the exponent.

For the Pareto type II model,

\mathrm{AIC}_{P}=-2(-387.85)+2(2)=775.70+4=779.70.

For the lognormal model,

\mathrm{AIC}_{LN}=-2(-390.60)+2(2)=781.20+4=785.20.

The Pareto type II model has the minimum AIC. The lognormal difference is $\Delta_{LN}=785.20-779.70=5.50$, so its relative support is

r_{LN}=\exp(-5.50/2)=\exp(-2.75)\approx0.0639.

The Pareto-to-lognormal support ratio is $\exp(2.75)\approx15.6426$. Both values come from the unrounded exponential expressions; taking the reciprocal of a rounded value such as $0.0639$ can produce a slightly different ratio.

Common mistake. Using $\exp(-5.50)$ gives approximately $0.0041$. That is incorrect because the AIC difference is on a $-2$ log scale. The correct calculation uses $\exp(-5.50/2)\approx0.0639$. The result indicates substantially less AIC-scale support for the lognormal model in this comparison; it does not give the probability that the lognormal model is false, nor is it a hypothesis-test result.

Check. Since the lognormal model has a larger AIC, its support must be less than 1. A difference of $5.5$ produces a value below 1, as required.

Common Mistakes and Reporting

Forgetting to subtract the minimum. Use $\Delta_i=\mathrm{AIC}_i-\mathrm{AIC}_{\min}$, not the raw AIC value in the exponential.
Dropping the one-half. Use $\exp(-\Delta_i/2)$. The half is required by the AIC scale.
Reversing the comparison. Relative support against the minimum is at most 1. If a calculated value exceeds 1, check the subtraction order or state explicitly that you are calculating the ratio in the opposite direction.
Calling relative support a probability. A value such as $0.20$ means one candidate has one-fifth the relative support of the reference model on this scale. It does not mean there is a 20% chance that the candidate is true.
Making a test conclusion from an AIC difference. AIC ranks models by an information criterion. A hypothesis-test conclusion requires a testing procedure with appropriate assumptions and calibration.

Key takeaway: Find the minimum AIC, compute each nonnegative difference $\Delta_i$, and calculate $\exp(-\Delta_i/2)$. Interpret the result as relative support on the AIC scale for the candidate set and data—not as a p-value or a probability that a model is true.

Check Your Understanding

For each problem, calculate or interpret the AIC difference using the stated comparison. Round relative support values to four decimals.

Two severity models fitted to the same hypothetical claims data have AIC values 250.6 and 252.4. Find the second model’s difference from the minimum and its relative support. (Answer: $\Delta=1.8$; relative support $=\exp(-0.9)\approx0.4066$.)
A candidate model has AIC $345.0$, while the minimum AIC is $340.2$. Calculate its relative support and the minimum-AIC model’s support ratio over it. (Answer: $\Delta=4.8$; relative support $\approx0.0907$; minimum-to-candidate ratio $\exp(2.4)\approx11.0232$.)
Three models have AIC values $92.4$, $92.4$, and $98.4$. Give their differences and relative support values in the same order. (Answer: Differences $0,0,6.0$; relative supports $1.0000,1.0000,0.0498$.)
A model’s AIC is 9.0 larger than the minimum. An analyst reports its relative support as $\exp(-9)$. Identify the error and give the correct value. (Answer: The exponent must use half the difference; relative support $=\exp(-4.5)\approx0.0111$.)
A candidate’s relative support is $0.1353$. What is its AIC difference from the minimum, to two decimals, and does the value mean the candidate has a 13.53% probability of being true? (Answer: $\Delta=-2\ln(0.1353)\approx4.00$; no, it is relative AIC-scale support, not a probability that the model is true.)

What You'll Learn