Residual Autocorrelation and Portmanteau Diagnostics

Why Check Residual Autocorrelation?

An estimated ARMA model is intended to summarize serial dependence in a time series. If the model has captured that dependence adequately, its fitted innovations should have no systematic linear association across time. Residual autocorrelation diagnostics examine this requirement: they ask whether the residuals still contain dependence the fitted model has not explained.

These diagnostics do not prove that a model is correct. A residual ACF plot can reveal patterns worth investigating, while a portmanteau test provides a formal test of a joint null hypothesis about a selected range of lags. Both require judgment: results depend on the sample size, the lag range, and the fact that the residuals come from an estimated model.

Residuals and Their Sample Autocorrelations

Suppose $n$ observations are used to fit a stationary, invertible ARMA($p,q$) model. Write the fitted innovation residuals as $\hat{\varepsilon}_1,\ldots,\hat{\varepsilon}_n$. The residuals estimate the unobserved innovations; they are not the true innovations themselves. If the model includes an estimated mean, residuals are often centered before calculating their autocorrelations.

Definition: Let $\bar{\hat{\varepsilon}}=n^{-1}\sum_{t=1}^{n}\hat{\varepsilon}_t$. The sample residual autocorrelation at lag $k$ is $$ r_k= \frac{\displaystyle\sum_{t=k+1}^{n} (\hat{\varepsilon}_t-\bar{\hat{\varepsilon}}) (\hat{\varepsilon}_{t-k}-\bar{\hat{\varepsilon}})} {\displaystyle\sum_{t=1}^{n} (\hat{\varepsilon}_t-\bar{\hat{\varepsilon}})^2}, \qquad k=1,\ldots,m. $$ Here $m$ is the largest lag included in the diagnostic. The value $r_k$ measures linear association between residuals $k$ time steps apart.

A residual ACF plot displays $r_k$ against $k$. If the fitted model is adequate, the bars should generally fluctuate around zero without a persistent pattern. For a rough visual reference, plots often show limits near $\pm 1.96/\sqrt{n}$, motivated by the approximate sampling variation of autocorrelations under white noise. These are not exact confidence limits for residual autocorrelations from an estimated ARMA model, and they are not simultaneous limits covering all plotted lags.

Worked Example: Interpreting a Residual ACF Plot

Suppose a hypothetical ARMA model is fitted to $n=100$ monthly observations. Its residual ACF bars at lags 1 through 5 are $0.10,\ 0.08,\ 0.06,\ 0.04,\ 0.02$. The rough reference limits are $$ \pm\frac{1.96}{\sqrt{100}}=\pm 0.196. $$

Every listed bar lies within these limits. The gradual decline is not, by itself, strong evidence of residual autocorrelation: individual-bar limits are only a visual guide, and the five autocorrelations are considered together by a portmanteau test. The plot gives no obvious large residual correlation, but it does not establish that the model is correct or that other assumptions hold.

The Ljung–Box Portmanteau Test

A portmanteau test combines residual autocorrelations over a specified range instead of testing each lag separately. The Ljung–Box version scales the sum to improve its finite-sample behavior relative to the earlier Box–Pierce statistic.

Definition: For residual autocorrelations $r_1,\ldots,r_m$ from $n$ observations, the Ljung–Box statistic is $$ Q_{\mathrm{LB}}=n(n+2)\sum_{k=1}^{m}\frac{r_k^2}{n-k}. $$ The null hypothesis is that residual autocorrelations through lag $m$ are jointly zero. For a fitted ARMA($p,q$) model, the conventional large-sample reference distribution is $\chi^2_{m-p-q}$, provided $m>p+q$ and the model and regularity conditions are suitable.

In this setting, $H_0$ states that the residual autocorrelations at the tested lags are jointly zero; $H_a$ states that at least one is nonzero. The degrees-of-freedom adjustment $p+q$ accounts for estimating the autoregressive and moving-average coefficients. The conventional ARMA adjustment does not subtract an additional degree of freedom for an estimated mean. Other model structures or software procedures may use a model-specific adjustment, so the reference distribution used should be checked.

The test is approximate, not an exact finite-sample result. Its usual calibration relies on a correctly specified stationary, invertible ARMA model, regular parameter estimation, and a sufficiently large sample. The underlying innovation assumption is zero-mean, serially uncorrelated innovations with constant variance. Gaussian innovations are not required for the basic diagnostic interpretation under suitable large-sample conditions, but small samples or violations of regularity can make the chi-square approximation unreliable.

Worked Example: Calculating the Ljung–Box Statistic

Consider a hypothetical ARMA(1,1) fit to $n=100$ observations. Suppose its residual autocorrelations at lags 1 through 6 are $0.10,\ 0.08,\ 0.06,\ 0.04,\ 0.05,\ 0.03$. Calculate the Ljung–Box statistic and its adjusted degrees of freedom.

Here $n(n+2)=100(102)=10200$, and the six contributions are $$ \begin{aligned} Q_{\mathrm{LB}} &=10200\left( \frac{0.10^2}{99}+ \frac{0.08^2}{98}+ \frac{0.06^2}{97}+ \frac{0.04^2}{96}+ \frac{0.05^2}{95}+ \frac{0.03^2}{94} \right)\\ &=1.03030+0.66612+0.37856+0.17000+0.26842+0.09766\\ &\approx 2.611. \end{aligned} $$ The fitted model has $p+q=1+1=2$ estimated ARMA coefficients. With $m=6$, the reference degrees of freedom are $6-2=4$. The upper-tail p-value is $$ P(\chi^2_4\ge 2.611)\approx 0.625. $$

At significance level $\alpha=0.05$, this p-value does not provide evidence against the joint null of zero residual autocorrelation through lag 6. It does not show that the null is true; it says that this diagnostic did not detect such dependence at the chosen lag range and sample size.

Why the Degrees-of-Freedom Adjustment Is Needed

If the innovations were observed directly, sample autocorrelations under a white-noise null would have approximately independent, zero-centered variation at a fixed set of lags in large samples. Fitted residuals differ: estimating AR and MA coefficients uses information in the same data and tends to constrain some low-lag residual correlations. Treating all $m$ terms as unconstrained would therefore use a reference distribution that does not reflect parameter fitting.

A short asymptotic justification makes the adjustment intuitive. Under the null and regular estimation, the vector of the first $m$ residual autocorrelations has approximately zero-centered normal variation after accounting for the fitted parameters. Estimation imposes approximately $p+q$ constraints on that vector. In the conventional ARMA result, the remaining effective dimension is $m-(p+q)$. The Ljung–Box weighted sum of squared autocorrelations is then calibrated approximately by a chi-square distribution with that many degrees of freedom: $$ Q_{\mathrm{LB}}\ \dot{\sim}\ \chi^2_{m-p-q}. $$ This is an asymptotic reference result, not a claim that the fitted residual correlations are exactly independent or that the chi-square approximation is exact for every sample.

Worked Example: A Small Statistic with Adjusted Degrees of Freedom

A hypothetical AR(2) model is fitted to $n=80$ observations. At $m=8$ lags, suppose the residual autocorrelations are $0.12,\ -0.08,\ 0.05,\ 0,\ 0,\ 0,\ 0,\ 0$. The model has $p+q=2$ fitted ARMA coefficients, so the adjusted reference distribution has $8-2=6$ degrees of freedom.

Since $n(n+2)=80(82)=6560$, the nonzero contributions are $$ \begin{aligned} Q_{\mathrm{LB}} &=6560\left(\frac{0.12^2}{79} +\frac{(-0.08)^2}{78} +\frac{0.05^2}{77}\right)\\ &=\frac{94.464}{79}+\frac{41.984}{78}+\frac{16.4}{77}\\ &\approx 1.1957+0.5383+0.2130\\ &\approx 1.947. \end{aligned} $$

For $\chi^2_6$, the upper-tail p-value is approximately $0.925$. At $\alpha=0.05$, there is little evidence from this test against zero residual autocorrelation through lag 8. A high p-value is not proof of adequate specification: the test may have limited power, especially in a modest sample or against dependence not well represented by the selected lags.

Choosing Lags and Reading the Evidence

The lag limit $m$ should be selected before interpreting the test, with the sampling frequency, plausible dependence horizon, and sample size in mind. A very small $m$ can miss dependence at later lags. A very large $m$ adds many noisy terms and can weaken the usefulness of the approximation, particularly when $n$ is not large. The requirement $m>p+q$ is necessary for positive degrees of freedom, but it does not by itself guarantee a good approximation.

A significant portmanteau statistic indicates evidence that at least one residual autocorrelation among the tested lags is nonzero. It does not identify which lag is responsible, nor does it specify a unique replacement model. Inspect the residual ACF to locate possible patterns, then consider whether the model needs a different ARMA structure or another relevant specification change. Conversely, a nonsignificant result is a failure to detect the tested dependence, not a demonstration that all innovations are independent.

Worked Example: A Significant Joint Diagnostic

For a hypothetical ARMA(1,1) fit to $n=120$ observations, suppose residual autocorrelations at lags 1 through 6 are $0.25,\ 0.18,\ 0.12,\ 0.08,\ 0.03,\ 0$. Here $n(n+2)=120(122)=14640$, and the adjusted degrees of freedom are $6-1-1=4$.

The contributions are $$ \begin{aligned} Q_{\mathrm{LB}} &=14640\left( \frac{0.25^2}{119}+\frac{0.18^2}{118} +\frac{0.12^2}{117}+\frac{0.08^2}{116} +\frac{0.03^2}{115}+\frac{0^2}{114}\right)\\ &=\frac{915}{119}+\frac{474.336}{118} +\frac{210.816}{117}+\frac{93.696}{116} +\frac{13.176}{115}+0\\ &\approx 7.6891+4.0198+1.8018+0.8077+0.1146\\ &\approx 14.433. \end{aligned} $$ For $\chi^2_4$, the upper-tail p-value is approximately $0.0060$. Because $0.0060<0.05$, reject $H_0$ at the 5% significance level. There is evidence of residual autocorrelation at one or more of the first six lags, suggesting that the fitted model has left some serial dependence unexplained. The statistic and p-value do not establish which model change would be appropriate.

Common Mistakes and Practical Limits

Using $m$ degrees of freedom automatically. For the conventional ARMA($p,q$) residual test, use $m-p-q$, not $m$, when the test's assumptions and adjustment apply.
Reading each ACF bar as a separate final test. Approximate reference limits help identify conspicuous bars, but they do not account for inspecting many lags at once. The portmanteau statistic assesses a joint range.
Calling a nonsignificant result proof of white noise. A large p-value means the test did not find sufficient evidence against its null; it does not establish that every residual dependence is absent.
Interpreting rejection as identification. Rejection indicates remaining serial correlation somewhere in the tested range, not its cause or the correct revised model.
Ignoring the tested range. A test through lag $m$ gives no direct conclusion about lags beyond $m$.

Residual autocorrelation is one part of model assessment, not a complete innovation assessment. A model can have weak residual ACF evidence and still have other problems. The central diagnostic question here is narrower: after fitting the ARMA dependence, is there evidence of linear serial dependence remaining in the residuals over the lags examined?

Check Your Understanding

Answer these questions using the definitions and procedures above.

What does a residual autocorrelation at lag 3 measure?
For an ARMA(2,1) model tested through lag $m=8$, what conventional chi-square degrees of freedom are used, assuming the adjustment is applicable?
Why are the usual $\pm1.96/\sqrt{n}$ ACF reference limits only a rough visual guide for fitted residuals?
A Ljung–Box test has p-value $0.03$ at $\alpha=0.05$. State the conclusion in context without claiming that the test identifies a particular model change.
Why does a nonsignificant portmanteau test not prove that the fitted innovations are independent?