Repairing an ARIMA Model After Residual Autocorrelation

Why Residual Autocorrelation Calls for a Model Review

An ARIMA model is intended to capture the serial dependence relevant to forecasting. After fitting it, the one-step-ahead residuals should behave approximately like white noise: they should have mean near zero, roughly stable variance, and no material autocorrelation. A visible residual pattern means the model may have left predictable structure unexplained. It does not, by itself, identify the correct repair.

The repair is a sequence of decisions. First locate the suspicious lag or pattern. Then decide whether it suggests a missing ordinary or seasonal AR or MA term, or whether the differencing choice deserves review. Finally, refit and inspect the new residuals. Adding terms until a diagnostic looks acceptable can produce an unnecessarily complex model, while differencing again can remove useful signal or create artificial dependence.

Definition: For a fitted ARIMA model, an in-sample one-step residual at time \(t\) is \(e_t=y_t-\hat{y}_{t|t-1}\), where \(\hat{y}_{t|t-1}\) is the forecast of \(y_t\) made using information available through time \(t-1\). Residual autocorrelation is association between residuals at different time lags. A residual sequence is approximately white noise when its mean is near zero, its variance is reasonably stable, and its autocorrelations are small enough to be consistent with uncorrelated innovations.

Locate the Pattern Before Changing the Model

Inspect the residual time plot, the residual ACF, and—when useful—the residual PACF. A single notable spike at lag 1 suggests remaining short-term dependence. A spike at seasonal lag \(s\), or repeated spikes at \(s,2s,\ldots\), suggests an unmodeled seasonal pattern. A gradual decay is different from a single isolated spike: it can point to broader dependence or an inadequate differencing or trend specification. These are clues, not automatic order-selection rules.

For \(n\) residuals, a commonly used approximate reference bound for an individual residual autocorrelation is \( \pm 1.96/\sqrt{n} \). This is a rough, large-sample reference under a white-noise null; it is not a guarantee that every spike inside the bound is harmless or that every spike outside it requires a new parameter. Looking at many lags increases the chance of seeing at least one apparently unusual spike by chance.

Diagnostic: The Ljung–Box statistic for testing whether autocorrelations through lag \(h\) are jointly zero is

Q(h)=n(n+2)\sum_{k=1}^{h}\frac{\hat{\rho}_k^2}{n-k},

where \(\hat{\rho}_k\) is the residual sample autocorrelation at lag \(k\). Under the approximate null distribution, \(Q(h)\) is compared with a chi-square distribution. A common degrees-of-freedom adjustment for an ARIMA model is \(h-r\), where \(r\) is the number of estimated ordinary and seasonal AR and MA coefficients. The approximation requires \(h>r\), and its calibration is not exact in every setting, especially for short series or when the fitted model is complicated.

The Ljung–Box test assesses several lags together; it does not say which model term to add. The individual residual ACF and PACF help locate the pattern, while knowledge of the fitted specification and the differencing rationale helps choose a defensible candidate revision.

How the Joint Diagnostic Works

The Ljung–Box statistic is a scaled sum of squared sample autocorrelations. Under a white-noise null, residual autocorrelations are approximately centered at zero, with sampling variability of order \(1/n\). Squaring and summing measures the total departure across the chosen lags. The factor \(n(n+2)/(n-k)\) adjusts the contribution at lag \(k\) for finite sample size. Under the null and suitable large-sample conditions, the adjusted sum is approximately chi-square, with a degrees-of-freedom correction for fitted AR and MA coefficients.

Thus a small p-value is evidence that at least some autocorrelation remains among the tested lags. A large p-value means the test did not detect sufficient evidence against the joint null; it does not prove that the residuals are independent, normally distributed, or otherwise ideal for every purpose.

Locate.
Check the residual plot and ACF/PACF for ordinary lags, seasonal lags, and a repeated or slowly decaying pattern. Compare individual spikes with their approximate reference bounds.

Form a targeted revision.
Consider the smallest plausible ordinary or seasonal AR/MA order change. If the pattern instead raises concern about differencing, review whether the original series required that difference before adding another one.

Refit and recheck.
Inspect the revised residuals and use a joint check over relevant lags. Confirm that the chosen test has positive adjusted degrees of freedom and interpret it alongside the plots.

Keep the repair proportionate.
Retain a revision only when it addresses the diagnosed pattern without creating a less plausible or unnecessarily complex specification.

Worked Repairs

Worked Example: A Remaining Ordinary-Lag Pattern

A hypothetical insurer models monthly claim counts with ARIMA\((0,1,1)\). There are \(n=40\) usable residuals. The first two residual autocorrelations are \(\hat{\rho}_1=0.42\) and \(\hat{\rho}_2=0.08\). Assess the signal and choose a reasonable next candidate.

Method. Compare the individual correlations with the approximate bound, then calculate a joint Ljung–Box statistic through lag 2 for the current model.

The reference bound is

\frac{1.96}{\sqrt{40}}=\frac{1.96}{6.3246}\approx 0.310

The lag-1 value, \(0.42\), exceeds \(0.310\), while the lag-2 value, \(0.08\), does not. The lag-1 result is the main clue. The current model has one estimated MA coefficient, so the adjusted degrees of freedom through lag 2 are \(2-1=1\).

Calculate the joint statistic:

\begin{aligned} Q(2) &=40(42)\left(\frac{0.42^2}{40-1}+\frac{0.08^2}{40-2}\right)\\ &=1680\left(\frac{0.1764}{39}+\frac{0.0064}{38}\right)\\ &=1680(0.0045231+0.0001684)\\ &\approx 7.88. \end{aligned}

Against a chi-square distribution with 1 degree of freedom, the 5% critical value is about \(3.84\). Since \(7.88>3.84\), the joint test rejects the white-noise autocorrelation null at that level. A reasonable next candidate is ARIMA\((1,1,1)\), because the prominent ordinary lag-1 dependence motivates testing an added ordinary AR term. The ACF/PACF pattern and the refitted residuals must still be checked; the spike alone does not establish that this is the unique correct model.

Check. The lag-1 autocorrelation exceeds its approximate bound, and the computed joint statistic exceeds its stated critical value. The decision to test an added term is therefore consistent with both diagnostics. Common mistake avoided: treating \(0.42\) as proof that one specific term must be added, without refitting and checking the revised residuals.

Worked Example: A Seasonal Residual Spike

A hypothetical quarterly benefit-payment series is fitted with seasonal period \(s=4\) using ARIMA\((0,1,1)(0,1,0)_4\). For \(n=48\) residuals, the autocorrelations at lags 1 through 4 are \(0.02,-0.03,0.04,0.50\), respectively. Diagnose the pattern and identify a targeted candidate change.

The approximate bound is \(1.96/\sqrt{48}=1.96/6.9282\approx0.283\). Only the lag-4 autocorrelation, \(0.50\), exceeds this bound. Its position at the seasonal period, rather than at an ordinary short lag, is the key clue.

For the current model, \(r=1\), because it has one estimated nonseasonal MA coefficient. Through lag 4, the adjusted degrees of freedom are \(4-1=3\). The Ljung–Box calculation is

\begin{aligned} Q(4)=48(50)\left( &\frac{0.02^2}{47}+ \frac{(-0.03)^2}{46}+ \frac{0.04^2}{45}+ \frac{0.50^2}{44} \right)\\ =2400(&0.00000851+0.00001957+0.00003556+0.00568182)\\ &\approx13.79. \end{aligned}

The 5% chi-square critical value with 3 degrees of freedom is about \(7.815\); \(13.79>7.815\), so the joint test detects remaining autocorrelation through lag 4. A targeted candidate is to add one seasonal AR or seasonal MA term and compare the residual patterns after refitting. The given single seasonal spike does not identify which term is correct by itself; the seasonal ACF/PACF shape and model fit must guide that choice.

Check. The dominant spike is at \(s=4\), and the joint statistic exceeds the stated critical value. A seasonal, rather than indiscriminate ordinary, model revision is warranted. Common mistake avoided: treating the fourth-lag spike as if it were a lag-1 effect simply because it is the first conspicuous spike.

Worked Example: Investigating a Negative Lag-1 Spike

A hypothetical monthly series is fitted with ARIMA\((0,1,1)\). The analyst notices residual \(\hat{\rho}_1=-0.58\) among \(n=36\) residuals and considers adding another difference. Explain why the differencing choice should be reviewed before making that change. Suppose a candidate ARMA\((1,0)\) model fitted to the undifferenced series has residual autocorrelations \(0.10,0.04,-0.03,0.02\) through lag 4.

For the first model, the individual bound is \(1.96/\sqrt{36}=1.96/6=0.327\), so \(-0.58\) is outside the bound. Its one-lag Ljung–Box statistic is

Q(1)=36(38)\frac{(-0.58)^2}{36-1} =1368\frac{0.3364}{35} \approx13.15.

This flags residual dependence, but a strong negative lag-1 pattern after differencing can also prompt a review for possible over-differencing. It is not a reason to difference again automatically. The candidate undifferenced model has \(r=1\); for \(h=4\), its adjusted degrees of freedom are \(4-1=3\). Its statistic is

\begin{aligned} Q(4)=36(38)\left( &\frac{0.10^2}{35}+\frac{0.04^2}{34} +\frac{(-0.03)^2}{33}+\frac{0.02^2}{32} \right)\\ =1368(&0.00028571+0.00004706+0.00002727+0.00001250)\\ &\approx0.51. \end{aligned}

Since \(0.51<7.815\), the candidate does not reject the joint autocorrelation null at the 5% level through lag 4. This is evidence that the undifferenced candidate has removed the detected autocorrelation pattern, not proof that it is the best model. The analyst should verify that the undifferenced specification is appropriate for the series and check residual plots and relevant seasonal lags before retaining it.

Check. The first model has a large negative lag-1 residual correlation; the candidate model has a small joint statistic. The comparison supports reviewing the original differencing choice rather than reflexively increasing \(d\). Common mistake avoided: assuming every residual autocorrelation is repaired by taking another difference.

Common Traps and Practical Checks

A model repair should address the location and form of the observed dependence. An ordinary AR or MA addition is not interchangeable with a seasonal addition. Likewise, an extra difference is not interchangeable with adding a short-memory term. After each change, inspect whether the targeted pattern diminished and whether a new pattern appeared elsewhere.

Adding terms based on one isolated spike. A single exceedance among many inspected lags may occur by chance. Consider its location, the surrounding ACF/PACF pattern, and a joint test.
Ignoring the fitted-parameter adjustment. The Ljung–Box degrees of freedom are not automatically \(h\). If \(h-r\leq0\), that comparison is not usable with this adjustment; choose a suitable larger \(h\), while recognizing that short samples limit the approximation.
Calling a large p-value proof of white noise. A test can lack power, particularly with few observations. Check plots and meaningful seasonal lags as well.
Repairing dependence by repeated differencing. Differencing should be supported by the behavior of the series. Strong negative residual autocorrelation after differencing is a reason to investigate, not an automatic instruction to difference again.
Stopping after refitting. A changed model must be checked using its own residuals. A successful reduction at one lag does not rule out dependence at another.

Key takeaway: Residual autocorrelation is a diagnostic signal, not a model-order prescription. Locate the pattern, make the smallest plausible targeted revision, then recheck the revised residuals using both graphical evidence and an appropriate joint test. This diagnostic workflow is aligned with the SOA Exam SRM (Statistics for Risk Modeling) syllabus.

Check Your Understanding

For each case, identify the main diagnostic implication or calculate the requested result. Treat the stated approximations and test conditions as given.

A model has \(n=50\) residuals and \(\hat{\rho}_1=0.31\). Calculate the approximate 95% reference bound and state whether this spike exceeds it. (Answer: The bound is approximately \(\pm0.2772\); yes, \(0.31\) exceeds it.)
A monthly model with seasonal period 12 has a clear residual ACF spike at lag 12, while lags 1 through 5 are small. What type of model change should be investigated first? (Answer: Investigate a targeted seasonal AR or MA term, guided by the seasonal ACF/PACF; do not treat the spike as an ordinary lag-1 pattern.)
For \(n=60\), a model with two estimated AR/MA coefficients has residual autocorrelations \(0.12,0.08,-0.05\) through lag 3. Calculate \(Q(3)\) and give the adjusted degrees of freedom. (Answer: \(Q(3)\approx1.48\); adjusted degrees of freedom \(=3-2=1\), so it does not exceed the 5% critical value \(3.84\).)
A series was differenced once, after which its fitted model has a large negative residual autocorrelation at lag 1. What should be reviewed before adding another difference? (Answer: Review whether differencing was needed and whether it may have been excessive; compare a defensible less-differenced candidate and check its residuals.)
After adding a seasonal term, a Ljung–Box test has a large p-value, but the residual ACF still has a clear spike at a later seasonal multiple. Is the revised model automatically adequate? (Answer: No. Check the later seasonal dependence, the test horizon and degrees of freedom, and the residual plots; a large p-value does not prove white noise.)

What You'll Learn