Tutorials › Regression Analysis (Undergraduate) › Criterion Differences and Akaike Weights

Home  /  Tutorials  /  Information Criteria and Model Comparison

Information Criteria and Model Comparison · Tutorial 638 of 991

Criterion Differences and Akaike Weights

Calculate AIC differences and Akaike weights, then interpret what their relative support does—and does not—say about competing regression models.

Advanced 10 min read

What You'll Learn

  • Calculate each model’s AIC difference from the minimum AIC.
  • Convert AIC differences into relative likelihoods using the exponential transformation.
  • Normalize relative likelihoods to obtain Akaike weights.
  • Interpret weights as relative support within a specified candidate set.
  • Explain why an Akaike weight is not the probability that a model is true.
  • Check that models are being compared on a consistent data and likelihood basis.

From AIC Rankings to Relative Support

An AIC table ranks candidate models, but a ranking alone does not show how far apart their criterion values are. A model with the smallest AIC is ranked first whether the next model is only slightly higher or substantially higher. AIC differences and Akaike weights add a useful scale: they summarize how the candidate models compare relative to the best AIC in the set.

These quantities are conditional on the models being compared. They do not establish that any candidate model is correct, and they are not probabilities that the models are true. Their interpretation also depends on the AIC values being meaningfully comparable: the models should be fitted to the same observations, with the same response and a consistent likelihood basis. The earlier material on comparing models with different response scales explains why comparisons across incompatible likelihood scales require special care.

Definition: For a candidate set of \(R\) models, let \(\mathrm{AIC}_i\) be the AIC of model \(i\), and let \(\mathrm{AIC}_{\min}=\min_i(\mathrm{AIC}_i)\). The AIC difference for model \(i\) is $$ \Delta_i=\mathrm{AIC}_i-\mathrm{AIC}_{\min}. $$ Its relative likelihood is \(L_i=\exp(-\Delta_i/2)\). The Akaike weight for model \(i\) is $$ w_i=\frac{\exp(-\Delta_i/2)}{\sum_{j=1}^{R}\exp(-\Delta_j/2)}. $$

The minimum-AIC model has \(\Delta_i=0\), so its relative likelihood is \(\exp(0)=1\). Every other model has \(\Delta_i\geq 0\), and therefore a relative likelihood no greater than 1. The denominator in the weight formula rescales all these relative likelihoods so their weights sum to 1. A weight is thus a model’s share of the total relative support assigned across this particular candidate set.

Why the Exponential Transformation Appears

For a fitted model, AIC is calculated from the maximized log-likelihood and the parameter count. Write the maximized log-likelihood for model \(i\) as \(\ell_i\), and its parameter count as \(k_i\). Then \(\mathrm{AIC}_i=-2\ell_i+2k_i\). The parameter count must be counted consistently, including the estimated parameters required by the model’s likelihood.

The transformation has a direct comparison interpretation. For two models \(i\) and \(j\), their relative likelihoods have ratio

$$ \frac{\exp(-\Delta_i/2)}{\exp(-\Delta_j/2)} =\exp\left(-\frac{\Delta_i-\Delta_j}{2}\right) =\exp\left(-\frac{\mathrm{AIC}_i-\mathrm{AIC}_j}{2}\right). $$

The last equality holds because subtracting each model’s difference from the same minimum cancels \(\mathrm{AIC}_{\min}\). Equivalently, substituting the AIC formula gives

$$ -\frac{\mathrm{AIC}_i-\mathrm{AIC}_j}{2} =(\ell_i-\ell_j)-(k_i-k_j). $$

Thus an AIC difference determines a ratio of relative likelihoods on the exponential scale. Dividing each relative likelihood by their sum makes the weights add to one:

$$ \sum_{i=1}^{R} w_i =\frac{\sum_{i=1}^{R}\exp(-\Delta_i/2)} {\sum_{j=1}^{R}\exp(-\Delta_j/2)} =1. $$

This normalization is an arithmetic property of the definition. It does not turn the weights into probabilities that a model is true. AIC weights are often described as relative support in the candidate set, based on the AIC comparison and its information-theoretic rationale. That interpretation is useful, but it remains relative and set-dependent.

Worked Example: Three Models With AIC Differences of 0, 1, and 3

Suppose three hypothetical regression models for the same outcome, fitted to the same observations, have AIC values 128.4, 129.4, and 131.4. Assume their AIC values were calculated on a common likelihood basis. Find the differences and weights, then interpret them.

The minimum is 128.4. Subtracting it from each AIC gives

$$ \Delta_1=128.4-128.4=0,\qquad \Delta_2=129.4-128.4=1,\qquad \Delta_3=131.4-128.4=3. $$

The relative likelihoods are

$$ \exp(-0/2)=1,\qquad \exp(-1/2)\approx0.6065307,\qquad \exp(-3/2)\approx0.2231302. $$

Their sum is approximately \(1+0.6065307+0.2231302=1.8296608\). Dividing each relative likelihood by this total gives

$$ w_1\approx\frac{1}{1.8296608}=0.5465,\qquad w_2\approx\frac{0.6065307}{1.8296608}=0.3315,\qquad w_3\approx\frac{0.2231302}{1.8296608}=0.1220. $$

The weights sum to approximately 1.0000, subject to rounding. Within this three-model set, the first model has about 0.5465 of the total relative support, the second about 0.3315, and the third about 0.1220. This does not mean there is a 54.65% probability that the first model is true. It summarizes the models’ relative AIC-based support among these candidates.

Calculating and Reading Akaike Weights

A reliable calculation keeps three stages separate: first find the minimum AIC, then calculate each difference, and only then transform and normalize. The exponential uses \(-\Delta_i/2\), not \(-\Delta_i\); omitting the division by 2 changes the comparison.

1
Find the minimum AIC.
Identify the lowest AIC in the candidate set. This model’s difference will be zero.
2
Calculate each difference.
For every candidate, subtract the minimum AIC from its AIC. Check that no difference is negative.
3
Calculate relative likelihoods.
For each difference, calculate \(\exp(-\Delta_i/2)\).
4
Normalize and interpret.
Add the relative likelihoods, divide each by that sum, and report the weights as relative support within the stated candidate set.

The factor of one-half makes a difference of 2 correspond to a relative likelihood of \(\exp(-1)\), not \(\exp(-2)\). More generally, the relative likelihood for a model \(i\), compared with model \(j\), is \(\exp[-(\mathrm{AIC}_i-\mathrm{AIC}_j)/2]\). This ratio is not itself a normalized weight unless there are only two models and the denominator for normalization is also included.

Worked Example: A Close AIC Comparison

Consider a second hypothetical candidate set, fitted to the same observations and response. Its AIC differences are 0, 0.4, and 4. Calculate the Akaike weights. Retaining the decimal difference is important; rounding 0.4 to 0 would alter the result.

The relative likelihoods are

$$ \exp(0)=1,\qquad \exp(-0.4/2)=\exp(-0.2)\approx0.8187308,\qquad \exp(-4/2)=\exp(-2)\approx0.1353353. $$

The sum is \(1+0.8187308+0.1353353\approx1.9540660\). Therefore,

$$ w_1\approx\frac{1}{1.9540660}=0.5118,\qquad w_2\approx\frac{0.8187308}{1.9540660}=0.4190,\qquad w_3\approx\frac{0.1353353}{1.9540660}=0.06926. $$

The first two candidates account for most of the relative support in this set; the third has less. These weights describe the AIC comparison, not the chance that any candidate is the data-generating model. They also depend on which candidates were included in the denominator.

Weights Depend on the Candidate Set

Akaike weights are normalized across the models supplied. Consequently, changing the candidate set can change the weights even when the AICs of models retained in both sets do not change. Adding or removing a candidate changes the denominator, and therefore changes the share assigned to each model. This is why a weight should always be reported with a description of the set over which it was calculated.

A weight is also not an absolute measure of adequacy. If every candidate model represents the outcome poorly, the weights still sum to one and one candidate still has the largest weight. The calculation answers a relative question: how is the AIC-based support distributed among these specified candidates? It does not answer whether the candidate set includes a good account of the data.

Worked Example: Four Candidates and a Weight Check

For a third hypothetical analysis, four regression models are fitted to the same response values and observations. Suppose their AIC differences are 0, 1.5, 3, and 5. Calculate weights to four significant figures and check that the final distribution is normalized.

The relative likelihoods are

$$ 1,\qquad \exp(-1.5/2)=\exp(-0.75)\approx0.4723666,\qquad \exp(-3/2)\approx0.2231302,\qquad \exp(-5/2)\approx0.0820850. $$

The denominator is \(1+0.4723666+0.2231302+0.0820850\approx1.7775818\). The weights are therefore

$$ w_1\approx0.5626,\qquad w_2\approx0.2657,\qquad w_3\approx0.1255,\qquad w_4\approx0.04618. $$

Their rounded sum is \(0.5626+0.2657+0.1255+0.04618=0.99998\), which is 1 up to rounding. The first candidate receives the largest relative share, while the fourth receives the smallest. The fourth still has a positive weight because its AIC difference is finite; a small weight is not the same as a probability of exactly zero that the model is true.

Common Mistake: Treating a Weight as a Probability of Truth

Wrong reasoning: “The first model’s Akaike weight is 0.55, so there is a 55% probability that this model is the true regression model.” This treats the normalized weights as posterior probabilities over a set of mutually exclusive truths. That is not what the AIC-weight calculation establishes.

Correction: “Among the specified candidate models, this model receives about 55% of the total relative support calculated from their AIC values.” The number is conditional on the candidate set and the comparability of the AIC calculations. It does not account for models that were not considered, nor does it prove that any candidate is correct. AIC weights are model-comparison summaries, not probabilities that models are true.

Other calculation checks are straightforward but important: the smallest AIC must produce a zero difference and relative likelihood 1; all differences must be nonnegative; all relative likelihoods must be between 0 and 1; and the normalized weights must sum to approximately 1. If any check fails, revisit the subtraction, the \(-\Delta_i/2\) exponent, or the normalization.

Key takeaway: AIC differences measure distance from the minimum AIC in the candidate set. Exponentiating \(-\Delta_i/2\) gives relative likelihoods, and normalizing them gives Akaike weights. Interpret those weights as relative AIC-based support within that set—not as probabilities that models are true.

Check Your Understanding

Answer each question using the definitions and calculations in this tutorial.

  1. Three models have AIC values 72.6, 74.6, and 76.6. What are their AIC differences?
  2. For a model with \(\Delta_i=2\), write its relative-likelihood calculation and give its approximate value.
  3. Explain why the Akaike weight for a candidate can change when another candidate is added, even if the first candidate’s AIC does not change.
  4. A candidate has an Akaike weight of 0.70. State a correct interpretation and explain why this is not a 70% probability that the model is true.
  5. List two checks that can help identify an error in a table of AIC differences and Akaike weights.

Previous

Next

Back to Tutorials