Model Averaging With Akaike Weights

Why Average Across Candidate Models?

AIC-based selection identifies a model with relatively strong support among the candidates, but selecting only that model can make a prediction depend too heavily on a small difference in fit. Model averaging offers another approach: it combines each candidate model’s estimate of the same actuarial quantity, giving more weight to models with lower AIC.

The output is an estimate conditional on a specified candidate set and information criterion. It is not automatically a probability statement that any one model is true. The averaging is meaningful only when all included model-specific estimates refer to the same target, such as expected severity for the same coverage and population, or pure premium per exposure.

Definition: For candidate models $M_1,\ldots,M_K$, let $\mathrm{AIC}_i$ be the AIC for model $M_i$, calculated on the same data using the same likelihood convention. Define $\Delta_i=\mathrm{AIC}_i-\min_j\mathrm{AIC}_j$. The relative likelihood is $r_i=\exp(-\Delta_i/2)$, and the Akaike weight is $w_i=r_i/\sum_{j=1}^K r_j$. Thus $w_i\geq 0$ and $\sum_i w_i=1$.

Akaike weights express relative support within the candidate set as measured by AIC. Adding or removing candidate models can change the normalized weights, even if the AIC values of the models retained are unchanged. A weight is not, without additional assumptions and interpretation, the probability that the model is true.

Calculating a Model-Averaged Estimate

Suppose model $M_i$ gives an estimate $\hat{\theta}_i$ of a common actuarial quantity $\theta$. The model-averaged estimate is the weighted sum of those estimates. The weights are normalized first, so the result lies on the same scale as the individual estimates.

\hat{\theta}_{\mathrm{avg}}=\sum_{i=1}^K w_i\hat{\theta}_i.

This is an estimate of the common target, not an average of model labels or parameter names. For instance, a severity mean from one model can be averaged with a severity mean from another if both estimate the same population quantity under the same coverage definition. But coefficients that have different meanings across models should not be averaged merely because they are both called “parameters.”

Worked Example: Averaging Expected Severities

A hypothetical insurer fits three candidate severity models to the same covered-loss data. Their AIC values are $210$, $210+2\ln 2$, and $210+2\ln 4$. The models estimate expected covered severities of $\$1{,}000$, $\$1{,}200$, and $\$1{,}600$, respectively. Calculate the Akaike weights and model-averaged expected severity.

Method. Calculate the AIC differences from the minimum, convert each difference to a relative likelihood, normalize, and take the weighted estimate.

The minimum AIC is $210$, so the differences are $0$, $2\ln 2$, and $2\ln 4$. Their relative likelihoods are $1$, $\exp(-\ln 2)=1/2$, and $\exp(-\ln 4)=1/4$. The sum is $1+1/2+1/4=7/4$. Therefore the weights are $4/7$, $2/7$, and $1/7$.

The weighted estimate is \[ \hat{\theta}_{\mathrm{avg}}=\frac{4}{7}(1000)+\frac{2}{7}(1200)+\frac{1}{7}(1600) =\frac{4000+2400+1600}{7} =\frac{8000}{7} =\$1{,}142.86. \]

Rounded to the nearest cent, the model-averaged expected severity is $\$1{,}142.86$. As a check, the weights sum to $1$, and the estimate lies between the smallest and largest model-specific estimates. This method avoids the mistake of reporting the estimate from the minimum-AIC model alone when the task asks for a model-averaged estimate.

Within-Model and Between-Model Variation

A weighted average of predictions describes a central estimate, but it does not by itself describe uncertainty. If the target is a random quantity $X$, let $\mu_i=E[X\mid M_i]$ and $\sigma_i^2=\operatorname{Var}(X\mid M_i)$. Treat the Akaike weights as the mixing probabilities for a model-averaged predictive distribution. The total variance then has two components: expected variation within models and variation between their means.

Theorem: With model weights $w_i$, model-specific means $\mu_i$, and model-specific variances $\sigma_i^2$, the model-averaged mean and variance are \[ \mu_{\mathrm{avg}}=\sum_i w_i\mu_i,\qquad \operatorname{Var}_{\mathrm{avg}}(X)=\sum_i w_i\sigma_i^2+\sum_i w_i(\mu_i-\mu_{\mathrm{avg}})^2. \] The first variance term is within-model variation; the second is between-model variation.

To justify the formula, let $M$ be the selected model indicator, with $P(M=M_i)=w_i$. Conditional on $M_i$, $E[X^2\mid M_i]=\sigma_i^2+\mu_i^2$. Averaging over models gives $E[X^2]=\sum_i w_i(\sigma_i^2+\mu_i^2)$, while $E[X]=\mu_{\mathrm{avg}}$. Subtracting the square of the overall mean yields \[ \operatorname{Var}(X)=\sum_iw_i\sigma_i^2+\sum_iw_i\mu_i^2-\mu_{\mathrm{avg}}^2. \] Since $\sum_iw_i(\mu_i-\mu_{\mathrm{avg}})^2=\sum_iw_i\mu_i^2-\mu_{\mathrm{avg}}^2$, the stated decomposition follows. The between-model term is zero only when all positively weighted models have the same mean.

The interpretation of $\sigma_i^2$ depends on the target. For a predictive loss, it can represent variation in future losses conditional on the model; if parameter uncertainty is also included in each model’s predictive distribution, it can include that uncertainty as well. The formula does not automatically include uncertainty from estimating the Akaike weights themselves. It also does not justify treating an estimate’s standard error as a predictive variance unless that is the uncertainty being analyzed.

Worked Example: Decomposing Predictive Variance

For a hypothetical annual loss quantity measured in thousands of dollars, three candidate models have Akaike weights $4/7$, $2/7$, and $1/7$. Their conditional means are $100$, $110$, and $120$, and their conditional variances are $25$, $36$, and $49$, in squared thousands of dollars. Find the model-averaged mean and variance.

Method. First calculate the weighted mean. Then calculate the weighted within-model variance and the weighted squared deviations of the model means from the overall mean.

The average mean is \[ \mu_{\mathrm{avg}}=\frac{4(100)+2(110)+1(120)}{7} =\frac{740}{7} =105.7143. \]

The within-model component is \[ \sum_iw_i\sigma_i^2 =\frac{4(25)+2(36)+1(49)}{7} =\frac{221}{7} =31.5714. \]

The deviations from the average mean are $100-740/7=-40/7$, $110-740/7=30/7$, and $120-740/7=100/7$. Thus the between-model component is \[ \sum_iw_i(\mu_i-\mu_{\mathrm{avg}})^2 =\frac{4(1600)+2(900)+1(10000)}{343} =\frac{18200}{343} =53.0612. \]

The total model-averaged variance is $31.5714+53.0612=84.6327$ squared thousands of dollars. The check is that the total exceeds the within-model component because the model means differ, contributing positive between-model variation. All displayed results are rounded to four decimal places.

Average the Target, Not Its Ingredients

When each model supplies a prediction built from several quantities, calculate the target separately under each model before averaging. In general, the weighted average of model-specific products is not the product of the separately weighted averages. This matters for pure premiums, where frequency and severity predictions can vary together across candidate models.

Worked Example: Model-Averaging a Pure Premium

Three hypothetical frequency-severity models have Akaike weights $4/7$, $2/7$, and $1/7$. Their expected annual claim counts per exposure are $0.08$, $0.10$, and $0.12$; their expected severities are $\$5{,}000$, $\$4{,}600$, and $\$4{,}300$. Calculate the model-averaged pure premium, defined here as expected claim count per exposure multiplied by expected severity within each model.

Method. Calculate each model’s pure premium first, then average those premiums with the Akaike weights.

Model 1 gives $0.08(\$5{,}000)=\$400$ per exposure. Model 2 gives $0.10(\$4{,}600)=\$460$. Model 3 gives $0.12(\$4{,}300)=\$516$. Therefore, \[ \mathrm{PP}_{\mathrm{avg}}=\frac{4(400)+2(460)+1(516)}{7} =\frac{3036}{7} =\$433.71\text{ per exposure}. \]

As a check on the tempting alternative, the separately averaged frequency is $[4(0.08)+2(0.10)+0.12]/7=0.64/7$, and the separately averaged severity is $[4(5000)+2(4600)+4300]/7=33500/7$. Their product is $21440/49=\$437.55$ per exposure, not $\$433.71$. The difference arises because the model-specific frequency and severity predictions are paired within each model. Averaging each ingredient first loses that pairing. The correction is to form the target within each model before weighting it.

Common Interpretation Traps

Using raw relative likelihoods as weights. Relative likelihoods generally do not sum to one. Divide each by their sum before averaging.
Calling a weight the probability a model is true. It is a normalized measure of relative AIC support within the specified candidate set, not a universal truth probability.
Ignoring between-model variation. Averaging model-specific variances alone omits uncertainty from differing model means in a model-averaged predictive distribution.
Averaging unlike quantities. Ensure that every model-specific output represents the same target, population, coverage, time period, and units.
Assuming the variance formula covers everything. The usual decomposition treats the weights as fixed and includes only the conditional uncertainty represented by each model’s variance plus variation between model means.

Key takeaway: Normalize relative likelihoods to obtain Akaike weights, average estimates only when they describe a common target, and separate within-model variance from between-model variation when describing a model-averaged predictive distribution.

Check Your Understanding

For each problem, calculate weights before forming an average when needed. Answers are rounded as indicated.

Three models have AIC differences $0$, $2\ln 3$, and $2\ln 9$, and estimate the same expected payment as $\$70$, $\$100$, and $\$160$. Find the Akaike weights and the model-averaged estimate. (Answer: weights $9/13, 3/13, 1/13$; estimate $\$90.77$)
Two models have AIC values $250$ and $250+2\ln 4$. Their predictions of a common quantity are $40$ and $100$. Find the weights and averaged prediction. (Answer: weights $4/5, 1/5$; prediction $52$)
Three models have weights $1/2,1/3,1/6$, conditional means $10,13,19$, and conditional variances $4,9,16$. Find the model-averaged mean and total variance. (Answer: mean $12.5$; within-model variance $7.6667$, between-model variance $10.2500$, total variance $17.9167$)
Two frequency-severity models have Akaike weights $3/4$ and $1/4$. Their expected claim counts per exposure are $0.05$ and $0.10$, and their expected severities are $\$4{,}000$ and $\$3{,}000$. Find the model-averaged pure premium by averaging the model-specific premiums. (Answer: $(3/4)(\$200)+(1/4)(\$300)=\$225$ per exposure)
Three models have AIC values $300$, $302$, and $304$. Calculate the Akaike weight of the second model to four decimal places. (Answer: $\exp(-1)/[1+\exp(-1)+\exp(-2)]=0.2447$)

What You'll Learn