Why BIC Penalizes Complexity
A more complex GLM can usually fit the data at least as well as a simpler one, because additional parameters give it more flexibility. But a better fit on the data used to estimate the model does not, by itself, justify keeping every added rating variable or coefficient. A model-selection criterion makes the tradeoff explicit: it rewards fit and penalizes complexity.
The Bayesian information criterion, or BIC, penalizes each estimated parameter according to the sample size. As the sample grows, its penalty per parameter increases. This makes BIC a useful complement to AIC when comparing candidate GLMs for ratemaking. Neither criterion guarantees that the preferred model is correctly specified, and BIC is not a substitute for checking predictive performance, calibration, or practical constraints.
AIC uses the same fit term but a different complexity penalty:
AIC charges 2 units for each additional parameter, regardless of \(n\); BIC charges \(\ln(n)\) units per parameter. Therefore, when \(n>e^2\), BIC’s per-parameter penalty is larger than AIC’s. This does not mean BIC must always select the simpler model: a sufficiently large improvement in log-likelihood can outweigh its penalty.
Comparing Two Candidate GLMs
For two candidates fitted to the same observations and response, write model \(A\) for the simpler candidate and model \(B\) for the more complex candidate. The difference in their BIC values is
If this difference is negative, model \(B\) has the lower BIC; if it is positive, model \(A\) does. When \(B\) adds \(q=k_B-k_A\) parameters, it is favored precisely when its increase in maximized log-likelihood exceeds the threshold
The equivalent AIC threshold is \(\ell_B-\ell_A>q\). The BIC threshold grows with sample size; the AIC threshold does not. This comparison assumes the candidates use the same response data and a comparable likelihood. Do not compare raw BIC values from fits to different observations, different response definitions, or incompatible likelihoods and interpret the difference as a model-selection result.
Why the BIC Penalty Depends on Sample Size
BIC’s penalty can be justified as a large-sample approximation to a Bayesian model’s marginal likelihood. The marginal likelihood averages the likelihood over possible parameter values using a prior, rather than evaluating it only at the maximum-likelihood estimate. For a regular, identifiable model with fixed parameter dimension, a smooth likelihood, an interior maximum, and a prior density positive near that maximum, a local approximation around the maximum gives the intuition.
The likelihood is largest near the maximum-likelihood estimate. In a regular model, its curvature near that point grows roughly in proportion to \(n\).
With \(k\) parameters, the region contributing substantially to the integrated likelihood has approximate volume proportional to \(n^{-k/2}\). The likelihood there is approximately \(L\) at its maximum.
The integrated likelihood is therefore approximately \(L n^{-k/2}\), apart from terms that do not grow with \(n\). Its negative twice-log value is \(-2\ell+k\ln(n)\), which is the BIC form.
This derivation explains why BIC has a sample-size-dependent penalty and why, under its regularity conditions, it is often described as an approximation to a Bayesian model comparison. It is an asymptotic approximation, not an exact posterior probability or a guarantee that the selected GLM is true. It can also be unreliable for settings that violate the assumptions, such as non-identifiable models or cases where the effective parameter dimension changes with sample size.
Worked Comparisons for Poisson Ratemaking GLMs
The examples use hypothetical claim-frequency data. In each case, policy-year claim counts \(Y_i\) are modeled as conditionally independent Poisson random variables with means \(\mu_i\), where \(\log(\mu_i)=\log(e_i)+x_i^\mathsf{T}\beta\). Here \(e_i\) is known exposure, \(\log(e_i)\) is an offset, and \(\beta\) contains the estimated coefficients. The Poisson dispersion is fixed at 1. The log-likelihoods below are stipulated outputs from fitting the specified candidates; use those fitted values and count the estimated coefficients, but not the known offset, in \(k\). Criteria are rounded to three decimals at the final step.
Worked Example: AIC and BIC Disagree on a Large Portfolio
A hypothetical insurer fits two Poisson frequency GLMs to the same \(n=1{,}200\) policy-years. Both use exposure offsets. Model \(A\) has an intercept and four rating coefficients, so \(k_A=5\), with maximized log-likelihood \(\ell_A=-842.0\). Model \(B\) adds three coefficients for a rating factor, so \(k_B=8\), with \(\ell_B=-835.8\). Calculate both criteria and identify their preferred models.
Method and fit change. The log-likelihood improvement is \(\ell_B-\ell_A=-835.8-(-842.0)=6.2\). The added parameters are \(q=8-5=3\). Thus the fit contribution to the BIC difference is \(-2(6.2)=-12.4\), while the complexity contribution is \(3\ln(1{,}200)\).
BIC. Since \(\ln(1{,}200)=7.090076836\), the criteria are
The difference is \(1{,}728.32061469-1{,}719.45038418=8.87023051\), or 8.870 to three decimals. BIC favors the simpler model \(A\).
AIC. The criteria are \(\operatorname{AIC}_A=1{,}684.0+2(5)=1{,}694.0\) and \(\operatorname{AIC}_B=1{,}671.6+2(8)=1{,}687.6\). AIC favors \(B\), with a difference of \(1{,}687.6-1{,}694.0=-6.4\). The two criteria disagree because BIC charges \(7.090\) per added parameter here, while AIC charges 2. The common mistake is to treat improved fit as sufficient reason to select \(B\); the calculation shows that BIC’s larger penalty outweighs that improvement. Both calculations check against the difference formula: \(-12.4+3(7.090076836)=8.87023051\).
Worked Example: A One-Parameter Addition at 3,000 Observations
A second hypothetical Poisson frequency analysis uses the same \(n=3{,}000\) policy-years for both models, with known exposure offsets. Model \(A\) has four estimated coefficients and \(\ell_A=-620.0\). Model \(B\) adds one coefficient and has \(\ell_B=-616.2\). Calculate the BIC difference and compare the AIC decision.
Method. The fitted log-likelihood improves by \(-616.2-(-620.0)=3.8\), and \(q=1\). Therefore the fit improvement contributes \(-2(3.8)=-7.6\) to the BIC difference.
Using \(\ln(3{,}000)=8.006367568\), the BIC difference is
The positive difference favors model \(A\). For verification, \(\operatorname{BIC}_A=1{,}240.0+4(8.006367568)=1{,}272.02547027\), and \(\operatorname{BIC}_B=1{,}232.4+5(8.006367568)=1{,}272.43183784\); their difference is \(0.40636757\). AIC instead gives \(\operatorname{AIC}_A=1{,}240.0+8=1{,}248.0\) and \(\operatorname{AIC}_B=1{,}232.4+10=1{,}242.4\), favoring \(B\). The common mistake is miscalculating \(-7.6+\ln(3{,}000)\): it is approximately \(0.4064\), not \(0.404\). The positive sign matters because it means BIC favors the smaller model.
Worked Example: When a Small Sample Makes the Added Term Worthwhile
For a hypothetical small portfolio of \(n=40\) policy-years, two Poisson frequency GLMs use the same observations and exposure offsets. Model \(A\) has three estimated coefficients and log-likelihood \(-102.0\); model \(B\) adds one coefficient and has log-likelihood \(-99.5\). Determine which candidate each criterion favors.
Method. The log-likelihood gain is \(-99.5-(-102.0)=2.5\), so the fit term in the BIC difference is \(-5.0\). The added-parameter penalty is \(\ln(40)=3.688879454\). Hence
The negative difference favors \(B\). Direct calculation checks the result: \(\operatorname{BIC}_A=204.0+3(3.688879454)=215.06663836\), and \(\operatorname{BIC}_B=199.0+4(3.688879454)=213.75551782\). AIC gives \(\operatorname{AIC}_A=204.0+6=210.0\) and \(\operatorname{AIC}_B=199.0+8=207.0\), also favoring \(B\). Here the BIC threshold for one additional parameter is \(\ln(40)/2=1.844439727\) in log-likelihood gain; the actual gain of 2.5 exceeds it. The common mistake is to assume BIC always favors the simpler model. It favors the more complex model whenever the fit improvement is large enough to offset its penalty.
Using BIC Responsibly
BIC gives a concise fit-versus-complexity comparison, but the calculation is only meaningful when its inputs are appropriate. In a GLM analysis, check the definition of the observational unit and ensure \(n\) refers to the independent units represented by the likelihood. Do not substitute total exposure or total claim count for \(n\) simply because those quantities appear elsewhere in the ratemaking analysis. For aggregated data, use the sample-size convention appropriate to the likelihood formulation and apply it consistently across candidates.
Also check the parameter count. Include every estimated regression coefficient and any other estimated likelihood parameter included in the model. Do not count a known offset as a parameter. If candidates use different response data or likelihood conventions, their information-criterion values may not be comparable in the intended way. Finally, a lower BIC is a relative result among the candidates examined; it does not establish that the model is calibrated, stable across rating segments, or operationally suitable.
The practical interpretation is that BIC becomes more demanding about added complexity as the sample size increases. A candidate with extra parameters must earn that complexity through an adequate improvement in maximized likelihood. AIC may still prefer it because AIC applies a fixed penalty. Reporting the criterion, its inputs, and the comparison set makes this tradeoff transparent. The approach is aligned with the CAS Exam 8 (Advanced Ratemaking) syllabus.
Check Your Understanding
For each problem, compare the candidates using the stated criterion. Treat the log-likelihoods and parameter counts as fitted-model outputs, and use the same data for both candidates.
- Two hypothetical Poisson frequency GLMs are fitted to \(n=500\) policy-years. Model \(A\) has \(k_A=3\) and \(\ell_A=-300.0\); model \(B\) has \(k_B=4\) and \(\ell_B=-296.5\). Calculate \(\operatorname{BIC}_B-\operatorname{BIC}_A\) and state the BIC preference. (Answer: \(-0.7854\); model \(B\).)
- At \(n=10{,}000\), a candidate adds one coefficient and improves the maximized log-likelihood by 5.5. What is the BIC difference, complex minus simple, and which model does BIC favor? (Answer: \(-1.7897\); the more complex model.)
- At \(n=2{,}500\), a candidate adds one coefficient and improves the maximized log-likelihood by 3.2. Calculate the BIC difference, complex minus simple, and identify the preferred model. (Answer: \(1.4240\); the simpler model.)
- At \(n=2{,}000\), a candidate adds two coefficients and improves the maximized log-likelihood by 5.0. State the BIC and AIC differences, complex minus simple, and each preference. (Answer: BIC difference \(5.2018\), favoring the simpler model; AIC difference \(-6.0000\), favoring the more complex model.)
- At \(n=12\), a candidate adds one coefficient and improves the maximized log-likelihood by 1.0. Calculate the BIC difference, complex minus simple, and compare the AIC decision. (Answer: BIC difference \(0.4849\), favoring the simpler model; AIC difference \(0.0000\), a tie.)