Why BIC Uses Both Fit and Complexity
A more flexible model can often achieve a higher maximized likelihood because it has more ways to adapt to the observations. Comparing likelihoods alone can therefore favor added parameters even when the improvement in fit is modest. The Bayesian Information Criterion, or BIC, combines a model’s maximized likelihood with a penalty for its number of estimated parameters. It is a useful way to compare candidate models when they are fitted to the same observations and the same response.
BIC is a score, not a hypothesis test or a direct probability that a model is correct. When candidate models are compared on a consistent basis, a lower BIC indicates the preferred score under this criterion. The value by itself is not an absolute measure of model quality.
The first term, \(-2\ell(\hat{\theta})\), reflects fit: a larger maximized log likelihood makes this term smaller. The second term, \(k\ln(n)\), is the complexity penalty. It grows with the number of estimated parameters and, for \(n>1\), with sample size. The two terms are on the same numerical scale and must be combined; the fit term is not itself the BIC.
Calculating and Comparing BIC
A reliable calculation follows the same sequence each time: identify the likelihood and its maximized log likelihood, determine the number of free parameters, use the correct sample size, and then calculate both terms. For independent observations, \(n\) counts the observations represented in the likelihood. Do not substitute the number of parameters, the number of model types, or a total such as the sum of observed claim amounts.
For each candidate, record its maximized log likelihood, its free-parameter count \(k\), and the common sample size \(n\).
Multiply the maximized log likelihood by \(-2\). In particular, if the log likelihood is negative, this term is positive.
Multiply \(k\) by the natural logarithm of \(n\), not by \(n\) itself or by the base-10 logarithm.
Compute \(-2\ell(\hat{\theta})+k\ln(n)\) for each candidate. The lower score is preferred by BIC.
Worked Example: Comparing Exponential and Gamma Severity Models
A hypothetical insurer fits two models to the same 100 independent, uncensored positive claim severities, measured in thousands of dollars. The exponential model has rate parameter \(\hat{\lambda}=0.10\) per thousand dollars and maximized log likelihood \(-120.4\). The gamma model uses shape \(\hat{\alpha}=4.0\) and scale \(\hat{\theta}=2.4\) thousand dollars, with maximized log likelihood \(-115.0\). Treat the reported fit values as the results of maximizing each model’s likelihood. Calculate and compare BIC. The method is direct BIC calculation; it avoids choosing a model from log likelihood alone.
Exponential model. The rate \(\lambda\) is its single estimated parameter, so \(k=1\). The sample size is \(n=100\), and \(\ln(100)=4.605170\). Its fit term is \( -2(-120.4)=240.8 \), and its penalty is \(1(4.605170)=4.605170\). Therefore,
Gamma model. Both the shape and scale are estimated, so \(k=2\). Its fit term is \( -2(-115.0)=230.0 \), and its penalty is \(2(4.605170)=9.210340\). Therefore,
The gamma model has the lower BIC: \(239.210340<245.405170\). Its maximized log likelihood is higher by \(5.4\), improving the fit term by \(2(5.4)=10.8\). Its extra parameter increases the penalty by \(4.605170\). The fit improvement outweighs that added penalty, so gamma is preferred by BIC. Rounded to four decimal places, the BIC values are 239.2103 and 245.4052, respectively.
Why an Added Parameter Must Improve Fit
A useful way to understand the penalty is to compare two models directly. Suppose model 2 has one more free parameter than model 1, and both use the same \(n\) observations. Write their maximized log likelihoods as \(\ell_2\) and \(\ell_1\). Subtracting their BIC scores gives
Model 2 has the lower BIC exactly when this difference is negative. Rearranging the inequality gives
This is a comparison rule, not an assumption that the larger model will always improve the likelihood. It shows why a higher log likelihood alone is insufficient: the improvement must exceed the penalty for the added parameter. If the models differ by \(r\) parameters rather than one, the corresponding condition is \(\ell_2-\ell_1>(r/2)\ln(n)\).
Worked Example: A Three-Model Comparison With a Small Sample
A hypothetical analyst fits three severity models to the same 25 independent, uncensored positive losses. The exponential model has rate \(\hat{\lambda}=0.08\) per thousand dollars and maximized log likelihood \(-42.0\). The Weibull model has shape \(\hat{c}=1.4\) and scale \(\hat{\lambda}=15.0\) thousand dollars, with maximized log likelihood \(-39.5\). The gamma model has shape \(\hat{\alpha}=2.2\) and scale \(\hat{\theta}=6.0\) thousand dollars, with maximized log likelihood \(-39.0\). Treat the reported log likelihoods as the fitted results. Compare the three models using BIC. This calculation highlights that the best log likelihood need not give the best BIC.
The exponential model estimates one rate parameter, so \(k=1\). The Weibull model estimates its shape and scale, so \(k=2\). The gamma model also estimates shape and scale, so \(k=2\). All use \(n=25\), for which \(\ln(25)=3.218876\).
For the exponential model, the fit term is \(-2(-42.0)=84.0\), and the penalty is \(1(3.218876)=3.218876\). Thus,
For the Weibull model, the fit term is \(-2(-39.5)=79.0\), and the penalty is \(2(3.218876)=6.437752\). Thus,
For the gamma model, the fit term is \(-2(-39.0)=78.0\), and the penalty is also \(2(3.218876)=6.437752\). Thus,
Gamma has the lowest BIC, followed by Weibull and then exponential. Since Weibull and gamma have the same parameter count and use the same observations, their penalties are equal; the gamma model’s better maximized log likelihood therefore gives it the lower BIC. Rounded to four decimal places, the scores are 87.2189, 85.4378, and 84.4378, respectively.
Sample Size and the Meaning of the Score
The penalty depends on \(n\), so the sample size is part of the calculation rather than a descriptive detail. For example, with \(n=64\), adding one parameter increases the penalty by \(\ln(64)=4.158883\). The added parameter must improve maximized log likelihood by more than half that amount, \(2.079442\), for the model to have a lower BIC. This follows directly from the comparison inequality above.
BIC is useful because it gives a consistent numerical trade-off between fit and complexity. It does not establish that the selected model describes every feature of the data well, and it does not replace checking whether the model assumptions are defensible. A model with the lowest BIC among a set of poor candidates is still only the best-scoring candidate in that set.
For a meaningful comparison, use the same observations and response and calculate the maximized likelihood consistently across candidates. In particular, do not compare a score based on one data set with a score based on another, or silently change how likelihood contributions are included. Such differences can make the numerical comparison misleading.
Common BIC Traps
- Using the wrong sign. If \(\ell(\hat{\theta})=-30\), the fit term is \(-2(-30)=60\), not \(-60\).
- Using the wrong logarithm or penalty. The formula uses \(k\ln(n)\). It is not \(kn\), and it does not use \(\log_{10}(n)\).
- Miscounting parameters. Count the free parameters estimated for the candidate model, not the observations or the number of labels used to describe it.
- Selecting by likelihood alone. A larger maximized log likelihood improves fit, but a more complex model also receives a larger penalty.
- Comparing inconsistent inputs. Candidate scores are not fairly compared if their likelihoods are based on different observations or inconsistent likelihood calculations.
Check Your Understanding
For each problem, calculate the requested BIC value or comparison. Round numerical BIC values to four decimal places.
- A model fitted to 40 independent observations has maximized log likelihood \(-50.5\) and estimates \(k=2\) free parameters. Calculate its BIC. (Answer: \(108.3778\))
- Two models use the same 20 independent positive losses. Model A is exponential with estimated rate \(\hat{\lambda}=0.12\) and maximized log likelihood \(-30.0\). Model B is Weibull with estimated shape \(\hat{c}=1.3\), scale \(\hat{\lambda}=9.0\), and maximized log likelihood \(-27.8\). Which model has lower BIC? (Answer: Model B; \(\operatorname{BIC}_A=62.9957\) and \(\operatorname{BIC}_B=61.5915\))
- Two models fitted to the same 100 observations differ by one parameter. What must be the minimum log-likelihood improvement for the model with the additional parameter to have lower BIC? State the strict inequality. (Answer: The improvement must be greater than \(\frac{1}{2}\ln(100)=2.3026\))
- Model A has \(k=2\) and maximized log likelihood \(-20.0\); model B has \(k=3\) and maximized log likelihood \(-18.9\). They use the same 16 observations. Calculate both BIC values and identify the lower-scoring model. (Answer: \(\operatorname{BIC}_A=45.5452\), \(\operatorname{BIC}_B=46.1177\); model A has the lower BIC.)