BIC and Parsimony in Time-Series Selection

Why BIC Is Used in Time-Series Selection

When two time-series models fit the observed data, the more complex model will often achieve a higher maximized likelihood simply because it has more flexibility. A selection criterion must balance that improvement against the cost of estimating additional parameters. The Bayesian information criterion, or BIC, makes this trade-off by adding a penalty that depends on both the number of estimated parameters and the sample size.

A lower BIC indicates the preferred candidate among the models being compared. It does not establish that a model is true, guarantee good forecasts, or replace checks of residual adequacy. The candidate set should already have a defensible time-series basis, and the models’ likelihoods must be comparable.

Definition: For a model fitted by maximum likelihood, let $\hat L$ be its maximized likelihood, let $\ell(\hat\theta)=\log\hat L$ be its maximized log-likelihood, let $n$ be the number of observations contributing to that likelihood, and let $k$ be the number of estimated model parameters. Then $$ \operatorname{BIC}=-2\ell(\hat\theta)+k\log n. $$ The logarithm is the natural logarithm. Among comparable candidates, the model with the smaller BIC is preferred by this criterion.

The first term measures lack of fit: a larger maximized likelihood means a smaller value of $-2\ell(\hat\theta)$. The second term penalizes complexity. In an ARMA or ARIMA setting, $k$ generally includes estimated autoregressive and moving-average coefficients, any estimated mean, intercept, or drift, and the innovation variance. Include other estimated coefficients, such as regression effects, when they are part of the fitted model. A parameter fixed in advance is not estimated and is not counted. Use the same counting convention for every candidate.

The sample size $n$ is the number of observations used in the likelihood being compared, not automatically the number of calendar periods in a broader data set. A BIC comparison requires a consistent likelihood basis and sample definition across candidates. If candidates were fitted to different observations or their likelihoods are not comparable, their BIC values should not be ranked mechanically; the earlier material on comparable likelihoods addresses this issue.

Computing and Comparing BIC

It is often easiest to compare models by calculating a difference. Suppose a larger model adds $q$ estimated parameters to a smaller model. Define $G$ as the improvement in fit measured by the decrease in $-2\ell(\hat\theta)$:

G=2\bigl(\ell_{\text{large}}-\ell_{\text{small}}\bigr).

A positive $G$ means the larger model has a higher maximized likelihood. If both models use the same $n$, the BIC difference is

\operatorname{BIC}_{\text{large}}-\operatorname{BIC}_{\text{small}} =-G+q\log n.

Thus, the additional parameters are favored by BIC only when the improvement in fit exceeds their total penalty $q\log n$. A negative difference favors the larger model; a positive difference favors the smaller one.

Worked Example: Comparing Two ARMA Candidates

A hypothetical monthly series has $n=80$ observations used in both likelihoods. Candidate A has maximized log-likelihood $-120.4$ and $k=3$ estimated parameters. Candidate B has maximized log-likelihood $-116.9$ and $k=5$. Calculate both BIC values and identify the preferred candidate. For an additional comparison, calculate AIC using $\operatorname{AIC}=-2\ell(\hat\theta)+2k$.

Method and parameter check. Use the stated parameter counts consistently. Candidate B estimates two more parameters, and its log-likelihood is higher, so its fit improves. That alone does not determine which BIC is smaller.

Candidate A. Its fit term is $-2(-120.4)=240.8$. Since $\log(80)=4.3820266$, its penalty is $3(4.3820266)=13.1461$. Therefore,

\operatorname{BIC}_A=240.8+13.1461=253.9461.

Candidate B. Its fit term is $-2(-116.9)=233.8$. Its penalty is $5(4.3820266)=21.9101$, so

\operatorname{BIC}_B=233.8+21.9101=255.7101.

Candidate A has the smaller BIC, so BIC prefers the simpler candidate. For AIC, the calculations are $240.8+2(3)=246.8$ for A and $233.8+2(5)=243.8$ for B. AIC prefers B. This example shows that improved fit does not guarantee a lower BIC. Values are rounded to four decimal places; the ranking is unchanged by rounding.

Common mistake avoided: Do not select B solely because its log-likelihood is higher. The selection criterion includes a penalty, and the penalty can reverse the ranking.

Why the BIC Penalty Often Favors Simpler Models

AIC adds $2$ for each estimated parameter, whereas BIC adds $\log n$. For $n>e^2$, $\log n>2$, so BIC has the larger per-parameter penalty. For a comparison adding $q$ parameters, its penalty exceeds AIC’s by $q(\log n-2)$. This explains why BIC often selects simpler models than AIC for moderate or large samples. It does not mean BIC always chooses the simpler candidate: a sufficiently large improvement in likelihood can outweigh the penalty.

One motivation for BIC comes from approximating a model’s integrated likelihood, also called its marginal likelihood. Under regularity conditions, that approximation yields a fit term and a penalty that grows with sample size. The following outline gives the reason for the $k\log n$ term; it is a large-sample approximation, not an exact identity for every time-series model.

Average likelihood over parameter values.
For parameter vector $\theta$ and prior density $\pi(\theta)$, the integrated likelihood is $m(y)=\int L(\theta)\pi(\theta)\,d\theta$. It rewards a model for fitting well across a region of parameter values, rather than only at its best-fitting point.

Approximate near the maximum.
For a regular model with $k$ identifiable parameters, a smooth positive prior near an interior maximum, and likelihood curvature that grows proportionally to $n$, the likelihood is concentrated in a region with width of order $n^{-1/2}$ in each parameter direction. The resulting volume factor is of order $n^{-k/2}$.

Take minus twice the logarithm.
The leading terms are $\log m(y)\approx \ell(\hat\theta)-(k/2)\log n+O(1)$. Multiplying by $-2$ gives $-2\log m(y)\approx -2\ell(\hat\theta)+k\log n+O(1)$. BIC retains the first two terms and omits terms that do not grow with $n$.

This justification explains the increasing complexity penalty, but it also identifies assumptions behind the approximation. Irregular or poorly identified models, parameter values on boundaries, and unsuitable likelihood comparisons can weaken the rationale. BIC is a useful selection criterion, not a substitute for verifying that the fitted time-series model is adequate.

Worked Example: Testing Whether One Extra Parameter Is Worthwhile

For a hypothetical series with $n=50$, a smaller model has $-2\ell(\hat\theta)=180.0$ and $k=3$. An expanded model adds one estimated parameter and has $-2\ell(\hat\theta)=175.8$, so $k=4$. Determine the BIC decision and compare it with AIC.

Method. The fit improvement is the decrease in $-2\ell$: $G=180.0-175.8=4.2$. The additional BIC penalty is $\log(50)=3.912023$. Therefore, the BIC change from the smaller to the expanded model is

\Delta\operatorname{BIC}=-4.2+\log(50)=-4.2+3.912023=-0.287977.

The negative change means the expanded model has the smaller BIC. Equivalently, its BIC is lower by about $0.2880$. The individual scores verify the result: the smaller model has BIC $180.0+3(3.912023)=191.7361$, while the expanded model has BIC $175.8+4(3.912023)=191.4481$.

For AIC, the change is $-4.2+2=-2.2$, so AIC also prefers the expanded model. In this case the likelihood improvement exceeds both penalties.

Common mistake and correction: It is incorrect to say that BIC rejects the added parameter because $4.2<\log(50)=3.9120$; that inequality is false. Since $4.2>\log(50)=3.9120$, BIC prefers the expanded model, and its BIC is smaller by approximately $4.2-\log(50)=0.2880$. AIC also prefers the expanded model because its penalty $2$ is less than $4.2$. Values are rounded to four decimal places.

Reading the Result in Context

A BIC difference is a comparison of penalized fit, not a direct measure of forecast accuracy. If two candidates have close BIC values, the criterion may not provide a decisive practical distinction. Consider the time-series structure, parameter plausibility, and residual diagnostics as well. In particular, a low BIC does not excuse residual autocorrelation or another systematic pattern that indicates model inadequacy.

Also distinguish the number of observations from the number of parameters. Increasing $n$ changes the penalty for each parameter, even if the models’ parameter counts remain fixed. The logarithm grows slowly, but for a given additional parameter the BIC penalty eventually becomes larger than AIC’s fixed penalty of $2$. Conversely, for small $n$, $\log n$ can be less than $2$; it is not correct to claim that BIC always imposes a larger penalty.

Worked Example: AIC and BIC Rank Candidates Differently

A hypothetical weekly series provides $n=120$ observations for both candidates. Model S has maximized log-likelihood $-182.6$ with $k=4$. Model C has maximized log-likelihood $-178.7$ with $k=6$. Calculate the BIC values and use the difference formula to verify the ranking. Then check the AIC ranking.

Model S. Its fit term is $-2(-182.6)=365.2$. Since $\log(120)=4.7874917$, the penalty is $4(4.7874917)=19.1500$. Thus $\operatorname{BIC}_S=365.2+19.1500=384.3500$.

Model C. Its fit term is $-2(-178.7)=357.4$. Its penalty is $6(4.7874917)=28.7250$. Thus $\operatorname{BIC}_C=357.4+28.7250=386.1250$. BIC prefers S.

As a check, C adds $q=2$ parameters and improves the fit by $G=2(-178.7+182.6)=7.8$. Hence

\operatorname{BIC}_C-\operatorname{BIC}_S=-7.8+2\log(120)=-7.8+9.5750=1.7750.

This agrees with the direct scores up to rounding: C’s BIC is higher. For AIC, the scores are $365.2+2(4)=373.2$ for S and $357.4+2(6)=369.4$ for C, so AIC prefers C. BIC’s added penalty for the two parameters is $2\log(120)$, which exceeds AIC’s added penalty of $4$. Values are rounded to four decimal places.

Common mistake avoided: When using the difference formula, count the added parameters as $q=2$, not the total parameter count of the larger model. The difference formula compares only the additional penalty.

Common Exam Traps

Reversing the direction of the score. Smaller BIC is preferred. Do not select the model with the largest log-likelihood without applying the penalty.
Using the wrong sign for fit improvement. If the larger model decreases $-2\ell$ by $G$, its BIC change is $-G+q\log n$, not $G+q\log n$.
Omitting a parameter. Count every estimated quantity under the stated convention, including estimated innovation variance and mean or drift terms.
Using different sample sizes without checking comparability. BIC includes $\log n$, and the likelihood term itself depends on which data were used. A ranking is not automatically meaningful when likelihoods are based on different observations.
Claiming BIC always favors simpler models than AIC. For $n>e^2$, its per-parameter penalty is larger, but likelihood improvement can still make a more complex model’s BIC lower.

Key takeaway: Calculate $-2\ell(\hat\theta)+k\log n$ using a consistent likelihood, sample size, and parameter count. For nested candidates with the same $n$, BIC favors added parameters only when their improvement in $-2\ell$ exceeds $q\log n$. Always interpret that ranking alongside model adequacy.

Check Your Understanding

For each original comparison, use lower BIC as the selection rule. Unless a question says otherwise, assume candidates use the same observations and comparable maximum-likelihood conventions.

A candidate adds one parameter to a model fitted to $n=25$ observations and decreases $-2\ell$ by $3.0$. Calculate the BIC change for the expanded model and state the preference. (Answer: $-3+\log(25)=0.2189$; BIC prefers the smaller model.)
For $n=64$, a one-parameter expansion improves the maximized log-likelihood from $-90.0$ to $-87.7$. Find the BIC change and preferred model. (Answer: The decrease in $-2\ell$ is $4.6$, so $\Delta\operatorname{BIC}=-4.6+\log(64)=-0.4411$; BIC prefers the expanded model.)
At $n=100$, Model A has $\ell=-150.0$ and $k=2$; Model B has $\ell=-146.5$ and $k=4$. Calculate both BIC values and select one. (Answer: $\operatorname{BIC}_A=300+2\log(100)=309.2103$; $\operatorname{BIC}_B=293+4\log(100)=311.4207$; BIC prefers A.)
A time-series model estimates an AR(1) coefficient, an MA(1) coefficient, an intercept, and an innovation variance. A second model estimates an AR(2) structure, an MA(1) coefficient, an intercept, and the same variance parameter. With $n=40$, their maximized log-likelihoods are $-52.3$ and $-49.9$, respectively. Count parameters and determine the BIC preference. (Answer: The counts are $4$ and $5$; the fit improvement is $4.8$, and $\Delta\operatorname{BIC}=-4.8+\log(40)=-1.1111$, so BIC prefers the second model.)
At $n=16$, a one-parameter expansion decreases $-2\ell$ by $3.5$. Does BIC favor the expanded model, and does AIC favor it? (Answer: BIC change $=-3.5+\log(16)=-0.7274$, so BIC favors the expanded model; AIC change $=-3.5+2=-1.5$, so AIC also favors it.)

What You'll Learn