Akaike Weights And Evidence Ratios

Introduction

An information criterion provides a score for comparing fitted candidate models, with smaller values preferred under that criterion. Akaike weights convert criterion differences into a normalized summary of relative support. They are useful when several candidates are being compared, but their meaning is narrower than their name may suggest: a weight is not the probability that a model is true.

This tutorial focuses on calculating weights and ratios correctly, reading a ratio in the direction stated, and describing the result within the candidate set being considered. The candidates must be fitted to comparable data and evaluated using the same criterion; otherwise the comparison is not interpretable.

Definition: For $R$ candidate models with AIC values $\mathrm{AIC}_1,\ldots,\mathrm{AIC}_R$, define $\Delta_r=\mathrm{AIC}_r-\min_s(\mathrm{AIC}_s)$. The relative likelihood for candidate $r$ is $L_r=\exp(-\Delta_r/2)$. Its Akaike weight is $$ w_r=\frac{\exp(-\Delta_r/2)}{\sum_{s=1}^{R}\exp(-\Delta_s/2)}. $$ The weights are nonnegative and sum to 1 before rounding. A pairwise Akaike evidence ratio for candidates $r$ and $s$ is $w_r/w_s$.

From Criterion Differences to Relative Support

The best-scoring candidate has $\Delta=0$, so its relative likelihood is 1. Each other candidate’s relative likelihood is between 0 and 1. The weights normalize these quantities by their total, so each weight describes one candidate’s share of the normalized support across the specified set.

The evidence ratio is a comparison between two candidates. A ratio of 3 for candidate $r$ relative to candidate $s$ means that candidate $r$ has three times the Akaike relative support of candidate $s$ in this comparison. It does not mean that $r$ has a three-in-four probability of being correct, or that the data are three times as likely under $r$ in the ordinary likelihood-ratio sense.

\frac{w_r}{w_s} = \frac{\exp(-\Delta_r/2)}{\exp(-\Delta_s/2)} = \exp\left(-\frac{\mathrm{AIC}_r-\mathrm{AIC}_s}{2}\right).

The normalizing denominator cancels in a pairwise ratio. Since $\Delta_r-\Delta_s=\mathrm{AIC}_r-\mathrm{AIC}_s$, the last expression can be calculated directly from the two AIC values. A lower AIC gives the larger weight and a ratio above 1 when it is placed in the numerator.

Check the comparison.
Confirm that all candidates use the same observations and the same criterion, and that the model labels are clear.

Find differences.
Subtract the minimum criterion value from each candidate’s value to obtain $\Delta_r$.

Calculate relative likelihoods.
Evaluate $\exp(-\Delta_r/2)$, retaining adequate precision in intermediate calculations.

Normalize and interpret.
Divide each relative likelihood by their sum for weights. For a pairwise ratio, retain the stated numerator and denominator model labels.

Worked Calculations

Worked Example: Three Claim-Frequency Candidates

A hypothetical insurer fits three claim-frequency models to the same portfolio observations. Their AIC values are 208.6 for Model A, 211.0 for Model B, and 213.2 for Model C. Calculate the three Akaike weights and the evidence ratio for Model A relative to Model B. State what that ratio means. Method: normalize relative likelihoods calculated from AIC differences. Common mistake avoided: rounding exponential values too early or treating a normalized weight ratio as a different quantity from the direct pairwise ratio.

Step 1: Find the minimum and differences. The minimum AIC is 208.6, so $ \Delta_A=208.6-208.6=0$, $ \Delta_B=211.0-208.6=2.4$, and $ \Delta_C=213.2-208.6=4.6$.

Step 2: Calculate relative likelihoods. $$ L_A=e^0=1,\qquad L_B=e^{-2.4/2}=e^{-1.2}\approx0.301194,\qquad L_C=e^{-4.6/2}=e^{-2.3}\approx0.100259. $$ Their total, calculated using unrounded values, is $1+0.3011942119+0.1002588437\approx1.4014530556$.

Step 3: Normalize. $$ w_A=\frac{1}{1.4014530556}\approx0.7135,\quad w_B=\frac{0.3011942119}{1.4014530556}\approx0.2149,\quad w_C=\frac{0.1002588437}{1.4014530556}\approx0.0715. $$ The displayed weights are rounded to four decimal places. Their displayed sum is 0.9999 because of rounding; before rounding, the weights sum to 1.

Step 4: Find the requested ratio. $$ \frac{w_A}{w_B} = \exp\left(-\frac{208.6-211.0}{2}\right) =e^{1.2} \approx3.3201. $$ Thus Model A has about 3.32 times the Akaike relative support of Model B within these candidates. Model C affects the normalized weights’ common denominator, but that denominator cancels from $w_A/w_B$.

Check. The ratio of the unrounded weights is $0.713543\ldots/0.214916\ldots\approx3.3201$, agreeing with the direct calculation.

Worked Example: Adding a Candidate Without Relabeling the Pair

Two hypothetical severity models fitted to the same claim data have AIC values 90 for Model A and 92 for Model B. A third candidate, Model C, has AIC 91. Calculate the weights for all three models and the evidence ratio for the original pair, Model A relative to Model B. Method: normalize all three relative likelihoods, then calculate the requested ratio using the original labels. Common mistake avoided: assuming the second-lowest-AIC model is the original Model B.

Step 1: Keep the labels attached to the AIC values. The values are A = 90, B = 92, and C = 91. The minimum remains 90, giving differences $\Delta_A=0$, $\Delta_B=2$, and $\Delta_C=1$.

Step 2: Calculate relative likelihoods and their total. $$ L_A=1,\qquad L_B=e^{-1}\approx0.367879,\qquad L_C=e^{-0.5}\approx0.606531. $$ $$ L_A+L_B+L_C =1+0.367879+0.606531 \approx1.974410. $$

Step 3: Normalize each candidate’s value. $$ w_A\approx\frac{1}{1.974410}=0.5065,\qquad w_B\approx\frac{0.367879}{1.974410}=0.1863,\qquad w_C\approx\frac{0.606531}{1.974410}=0.3072. $$ The values are rounded to four decimal places. The intermediate values, not these rounded weights, are used for the ratio.

Step 4: Compare the original pair. $$ \frac{w_A}{w_B} = \exp\left(-\frac{90-92}{2}\right) =e^1\approx2.7183. $$ Model A has about 2.72 times the Akaike relative support of Model B. Model C falls between them in AIC, so its weight is larger than Model B’s. If candidates are sorted by increasing AIC, the first-to-second ratio is instead $w_A/w_C\approx1.6487$; that is not the requested A-to-B ratio.

Check. The original pair still has ratio $e^1$, even after adding Model C. The set changed the individual normalized weights, not the ratio between the original two candidates.

Worked Example: A Two-Candidate Severity Comparison

A hypothetical analyst compares two severity distributions fitted to the same observed claims. Model P has AIC 147.8 and Model Q has AIC 151.0. Calculate each weight when these are the only candidates, then report the evidence ratio for P relative to Q. Method: convert the AIC gap to a relative likelihood and normalize across the two candidates. Common mistake avoided: interpreting the larger weight as a probability that the model is correct.

Step 1: Calculate the difference. Model P has the lower AIC, and $\Delta_P=0$, while $\Delta_Q=151.0-147.8=3.2$.

Step 2: Calculate and normalize the relative likelihoods. $$ L_P=1,\qquad L_Q=e^{-3.2/2}=e^{-1.6}\approx0.201897. $$ The sum is $1+0.201897=1.201897$, using the unrounded exponential for the division. Therefore $$ w_P\approx\frac{1}{1.201897}=0.8320,\qquad w_Q\approx\frac{0.201897}{1.201897}=0.1680. $$

Step 3: Calculate and interpret the ratio. $$ \frac{w_P}{w_Q} =\exp\left(-\frac{147.8-151.0}{2}\right) =e^{1.6}\approx4.9530. $$ Within this two-model candidate set, P has about 4.95 times Q’s Akaike relative support. Its weight of 0.8320 is not a 0.8320 probability that P is the true severity distribution.

Check. The two weights sum to 1.0000 at the displayed precision, and their unrounded ratio agrees with $e^{1.6}$.

Why the Ratio Formula Holds

The pairwise identity follows directly from the definition of the weights. Let $D=\sum_{s=1}^{R}\exp(-\Delta_s/2)$, the same positive normalization constant for every candidate. Then $w_r=\exp(-\Delta_r/2)/D$ and $w_s=\exp(-\Delta_s/2)/D$. Dividing cancels $D$:

\frac{w_r}{w_s} = \frac{\exp(-\Delta_r/2)/D}{\exp(-\Delta_s/2)/D} = \exp\left(-\frac{\Delta_r-\Delta_s}{2}\right) = \exp\left(-\frac{\mathrm{AIC}_r-\mathrm{AIC}_s}{2}\right).

The final equality holds because subtracting the same minimum AIC from both values does not change their difference. The ratio therefore depends only on the two AIC values, while each normalized weight depends on the whole candidate set through $D$.

Interpreting Weights Carefully

Akaike weights summarize relative support under an information criterion, conditional on the specified candidates and data. They can help communicate how concentrated the comparison is: a single large weight indicates that one candidate dominates this particular set, while more distributed weights indicate that support is shared. Neither pattern establishes that the candidate set contains the true data-generating process.

Several qualifications matter in actuarial applications:

The candidate set matters. Adding or removing a model changes the normalization and can change every displayed weight. An omitted plausible model is not represented in the weights.
Weights are not probabilities of truth. They are not posterior probabilities, and an evidence ratio is not automatically a likelihood ratio or a Bayesian Bayes factor.
The comparison must be valid. AIC comparisons require candidates evaluated on comparable observations with the same response and compatible likelihood basis. Comparing scores calculated from different data sets can produce misleading rankings and weights.
Use one criterion consistently. Calculate all relative likelihoods from the same selected information criterion; do not mix AIC values for some candidates with another criterion for others.
Keep labels and precision. Compute with unrounded values, and write the numerator and denominator model names before interpreting a ratio.

For example, an evidence ratio of 0.4 for Model R relative to Model S means R has 0.4 times S’s support, or equivalently S has $1/0.4=2.5$ times R’s support. It does not mean that R is “40% likely to be correct.” Reporting the comparison in words, with the candidate set and criterion named, prevents this common overstatement.

Common Exam Trap

A frequent error is to compute $w_r/w_s$ from rounded weights, or to compare entries after sorting the models and accidentally change the labels. The correct ratio can be obtained directly from the AIC difference, which avoids both problems:

Common mistake and correction: If Model A has AIC 90, Model B has AIC 92, and an added Model C has AIC 91, the weights in increasing-AIC order are approximately 0.5065, 0.3072, and 0.1863. Dividing the first by the second gives $1.6487$, the A-to-C ratio, not the A-to-B ratio. For A relative to B, keep the labels and calculate $\exp(-(90-92)/2)=2.7183$. The added candidate changes normalized weights but cannot change this pairwise ratio.

A second trap is rounding intermediate exponentials. In the three-frequency example, $e^{-1.2}\approx0.301194$, which rounds to 0.3012 at four decimal places, not 0.3010. Carrying unrounded values through the normalization gives the correctly rounded weights. State the requested precision and round only the final results.

Key takeaway: Akaike weights are normalized relative likelihoods for a specified, comparable candidate set. A pairwise evidence ratio is the ratio of two weights, equivalently $\exp(-(\mathrm{AIC}_r-\mathrm{AIC}_s)/2)$. Always preserve the candidate labels and interpret the result as relative support, not as the probability that a model is true.

Check Your Understanding

For each problem, preserve the model labels and use unrounded values in intermediate calculations.

Two hypothetical claim-cost models fitted to the same data have AIC values 120 and 123. Find the evidence ratio for the AIC-120 model relative to the AIC-123 model, to four decimal places. (Answer: $e^{1.5}\approx4.4817$; the AIC-120 model has about 4.4817 times the relative support.)
Three hypothetical frequency candidates have AIC values 300 for A, 302 for B, and 306 for C. Calculate all three Akaike weights to four decimal places. (Answer: $w_A=0.7054,\ w_B=0.2595,\ w_C=0.0351$.)
Models A, B, and C have AIC values 81, 82.5, and 80.5, respectively. Find their weights to four decimal places and the ratio for A relative to B. (Answer: $w_A=0.3628,\ w_B=0.1714,\ w_C=0.4658$; $w_A/w_B=e^{0.75}\approx2.1170$.)
A candidate R has AIC 220.4 and candidate S has AIC 219.0, based on the same hypothetical severity observations. Find the ratio for R relative to S to four decimal places and interpret its direction. (Answer: $e^{-0.7}\approx0.4966$; R has about 0.4966 times S’s relative support, so S has about 2.0138 times R’s.)
An analyst adds a candidate with a lower AIC to an existing set. Which statement is necessarily correct: the original candidates’ normalized weights remain unchanged, or their pairwise ratio remains unchanged? (Answer: The pairwise ratio remains unchanged; the normalized weights can change because the candidate-set denominator changes.)

What You'll Learn