Why Differencing Complicates Model Comparison
When two ARIMA candidates use different differencing orders, they may not be fitted to the same effective observations. For example, first differencing a series of length \(T\) produces \(T-1\) differences, while second differencing produces \(T-2\). A reported information criterion may therefore combine a log likelihood based on one number of observations with a criterion for another model based on a different number.
That matters because information criteria balance fit against complexity using a particular likelihood and sample size. A smaller reported value is not automatically evidence that one model is better if the two values come from likelihoods defined on different data, transformed responses, or initial-condition conventions. The key question is not merely whether both models were fitted to the same original series. It is whether their likelihoods are comparable.
What Differencing Changes
For a time series \(Y_1,\ldots,Y_T\), first differencing forms \(\Delta Y_t=Y_t-Y_{t-1}\). Second differencing forms \(\Delta^2Y_t=\Delta Y_t-\Delta Y_{t-1}\). Differencing can help make a series suitable for an ARMA model, but it changes the response being modelled and can remove initial observations from a conditional likelihood calculation.
If a model is fitted conditionally to the differenced series, a common simple count is \(T-d\) usable observations for differencing order \(d\). This is a useful warning about changing effective sample size, not a universal formula for every software likelihood. Exact likelihood methods, conditional methods, and treatments of initial values can count or handle observations differently. The analyst must use the sample size associated with the likelihood actually used.
A further issue is that a likelihood for \(\Delta Y_t\) is not automatically the same likelihood as one for \(Y_t\). Differencing is not a one-to-one transformation of the full data: the differences alone do not determine the initial level. Thus, a likelihood calculated on differenced values cannot simply be treated as a likelihood for the original series without accounting for the model and the initial information.
Information Criteria and Their Inputs
Let \(\ell\) be the maximized log likelihood for a specified data set and likelihood convention, \(k\) the number of estimated model parameters, and \(n\) the number of observations associated with that likelihood. The usual criteria are:
The AICc expression requires \(n-k-1>0\). In these formulas, \(k\) counts estimated parameters under the adopted likelihood, including such quantities as AR or MA coefficients, a mean or drift when estimated, and the innovation variance when it is estimated. Differencing order \(d\) is a chosen model order, not ordinarily a continuously estimated likelihood parameter counted in \(k\). The same parameter-counting convention must be used across candidates.
The value of \(n\) is not simply the number of rows in the original data file. It is the sample size relevant to the likelihood and, for AICc and BIC, must be defined consistently with that likelihood. If one candidate's output uses \(n=T-1\) and another's uses \(n=T\), substituting each reported value into a criterion does not make the resulting scores automatically comparable.
Worked Example: Why Two Reported AIC Values May Mislead
A hypothetical analyst fits two models to a series of 60 monthly observations. Model A is ARIMA(1,1,0), fitted by a conditional likelihood to 59 first differences. Its reported log likelihood is \(\ell_A=-82.0\), with \(k_A=2\). Model B is ARIMA(1,0,1), fitted to 60 original observations; its reported log likelihood is \(\ell_B=-84.5\), with \(k_B=3\). Calculate the reported AIC values and decide whether they establish a ranking.
Method. Calculate each displayed AIC using its reported log likelihood and parameter count, then check whether the likelihoods are on a common basis. The common mistake to avoid is interpreting the smaller arithmetic result as a valid model comparison before checking the inputs.
For Model A:
For Model B:
The reported values are 168 and 175, but Model A's conditional likelihood is based on differenced observations and 59 contributions, while Model B's is based on original observations and 60 contributions. These reported values do not establish that Model A is preferable.
Suppose both candidates are then refitted using a suitable common likelihood for the same observed series and consistent initial-condition treatment. The resulting log likelihoods are \(\ell_A=-81.8\) and \(\ell_B=-82.4\), with the same parameter counts. Now:
On this common basis, Model A has the smaller AIC by \(170.8-167.6=3.2\). This supports preferring A under AIC among these candidates; it does not make the earlier, non-comparable AIC values valid.
Why a Common Likelihood Basis Matters
Information criteria are built from a maximized log likelihood and a penalty. The likelihood is a sum of log-density contributions for the observations included in the model. If one candidate has a different set of contributions, or models a different response without an appropriate likelihood connection, its log likelihood is not measuring fit on the same basis.
This gives a short justification for the common-sample requirement. Suppose two models describe the same observed data \(y\), and use likelihoods \(L_A(y)\) and \(L_B(y)\) with comparable constants and initial-condition treatment. Their log likelihoods evaluate the competing models against the same data. The fit difference in AIC is then:
Both parts of this difference have a meaningful interpretation: the first compares fit to the same data, and the second compares the parameter penalties. If instead \(\ell_A\) is calculated from one transformed sample and \(\ell_B\) from another response sample, the first term mixes unlike quantities. The formula still produces a number, but its ranking may not answer the intended model-selection question.
Even equal \(n\) does not fix every problem. If one likelihood is for \(\Delta Y_t\) and another is for \(Y_t\), using the same count of observations does not by itself make them likelihoods for the same response. A common-sample refit must use an appropriate common likelihood formulation, not just truncate or pad the data until the sample counts match.
Determine whether each criterion comes from the original series or a differenced series, and whether the likelihood is exact, conditional, or otherwise adjusted.
Identify the observations and initial conditions contributing to each likelihood. Record the corresponding \(n\), rather than assuming it is the raw series length.
When the likelihood bases differ, refit candidates using the same observed response data and consistent likelihood and initialization conventions, if a valid common formulation is available.
Use consistent \(n\) and parameter-counting rules for AIC, AICc, or BIC. Then interpret the criterion alongside model diagnostics and the modelling purpose.
Worked Comparisons After Common-Sample Refitting
Worked Example: Comparing AIC, AICc, and BIC on a Common Basis
Two hypothetical ARIMA candidates are fitted to the same 48-observation likelihood basis. Candidate A has \(\ell_A=-63.2\) and \(k_A=2\); candidate B has \(\ell_B=-62.6\) and \(k_B=4\). Calculate all three criteria and identify the ranking. Use \(n=48\) for both models.
Method. Apply the same \(n\) and the stated parameter counts to each criterion. The common mistake to avoid is comparing only log likelihoods: B fits better by likelihood, but pays a larger complexity penalty.
For A:
For B:
For AICc, candidate A's correction is \(2(2)(3)/(48-2-1)=12/45=0.2667\), so \(\operatorname{AICc}_A=130.6667\). Candidate B's correction is \(2(4)(5)/(48-4-1)=40/43=0.9302\), so \(\operatorname{AICc}_B=134.1302\).
For BIC, \(\log(48)\approx3.8712\). Thus:
All three criteria prefer A in this example. B has a higher maximized log likelihood because \(-62.6>-63.2\), but that improvement is not enough to offset its larger penalty. The comparison is interpretable because the likelihood basis and \(n\) are common.
Worked Example: Aligning the Conditional Sample After Differencing
A hypothetical monthly series contains 72 observations. An analyst wants to compare an ARIMA model with \(d=0\) and one with \(d=2\). For a conditional comparison, the analyst decides to condition on the same first two observations and evaluate each model's conditional likelihood for \(Y_3,\ldots,Y_{72}\). A suitable fitting procedure reports \(\ell_0=-96.1,\ k_0=3\) and \(\ell_2=-94.8,\ k_2=4\). Calculate AIC, AICc, and BIC using the common likelihood sample size, and explain what the results do and do not show.
Method. The common evaluated response sequence contains \(72-2=70\) observations, so use \(n=70\) for both criteria. The common mistake to avoid is giving the \(d=2\) model a smaller \(n\) merely because differencing ordinarily loses initial values, even though both stated conditional likelihoods use the same response sequence and conditioning information.
For \(d=0\):
For \(d=2\):
The AIC value is lower for \(d=2\) by \(198.2-197.6=0.6\). For AICc, the corrections are \(2(3)(4)/(70-3-1)=24/66=0.3636\) and \(2(4)(5)/(70-4-1)=40/65=0.6154\). Therefore, \(\operatorname{AICc}_0=198.5636\) and \(\operatorname{AICc}_2=198.2154\).
Since \(\log(70)\approx4.2485\), the BIC values are:
AIC and AICc slightly prefer \(d=2\), while BIC prefers \(d=0\). This is not an arithmetic contradiction: the criteria penalize complexity differently. The result is meaningful only because the example stipulates comparable conditional likelihoods for the same response sequence and conditioning information. In an actual analysis, matching the number 70 alone would not establish those conditions.
Common Traps and Practical Interpretation
A common exam trap is to notice that a differenced model has fewer usable observations, insert its reported log likelihood into AIC, and rank it directly against an undifferenced model fitted to more observations. The calculation may be arithmetically correct and the conclusion still unjustified. First determine whether the likelihoods measure fit to the same response data.
Another trap is to force the same \(n\) by discarding early observations from one candidate while leaving the likelihoods defined for different transformed responses. That makes the counts match, but does not necessarily make the likelihoods comparable. If a common, valid likelihood formulation is unavailable, report that the information-criterion comparison is not supported by those outputs rather than presenting a misleading ranking.
Finally, a criterion ranking is not a substitute for checking whether the model is adequate. AIC, AICc, and BIC compare candidates under their likelihood assumptions; residual diagnostics and the intended use of the model still matter. When the comparison is valid, state the criterion, the likelihood basis, the effective \(n\), and the parameter-counting convention so the ranking can be interpreted.
Check Your Understanding
For each problem, check both the arithmetic and whether the stated information supports a fair comparison.
- A series has 40 observations. A \(d=1\) model reports a conditional likelihood based on 39 differences, while a \(d=0\) model reports one based on all 40 original observations. Is comparing their reported AIC values automatically valid? (Answer: No; the likelihoods use different response data and effective samples, so a common likelihood basis is needed.)
- Two candidates use a common likelihood basis with \(\ell_A=-50.0,\ k_A=2\) and \(\ell_B=-48.5,\ k_B=4\). Calculate both AIC values and state the preferred model by AIC. (Answer: \(\operatorname{AIC}_A=104\), \(\operatorname{AIC}_B=105\); A is preferred.)
- With \(n=30\), candidate A has \(\ell_A=-40.0,\ k_A=2\), and candidate B has \(\ell_B=-39.5,\ k_B=3\). On a common likelihood basis, calculate AICc to three decimals and choose the smaller value. (Answer: \(\operatorname{AICc}_A=84.444\), \(\operatorname{AICc}_B=85.923\); A is preferred.)
- A series has 55 observations, and candidates with \(d=0\) and \(d=2\) are to be evaluated conditionally on the same first two observations. How many response observations are in the common sequence \(Y_3,\ldots,Y_{55}\), and is that count alone enough to prove the likelihoods are comparable? (Answer: 53 observations; no, the likelihood and initialization conventions must also be comparable.)