Worked ARMA Identification and Estimation from Sample ACF

From Sample Correlations to a Fitted ARMA Model

A sample autocorrelation function (ACF) is often the first clue to the dependence structure of an economic time series. A gradually decaying ACF and a PACF without a clear cutoff can suggest an autoregressive moving-average model. Those patterns are clues, not proofs: in a finite sample, sampling variation can blur the theoretical patterns, and several orders may be plausible. The useful workflow is to propose a small set of models, estimate their parameters, and check whether the resulting residuals behave like uncorrelated innovations.

The examples below use a hypothetical simulated series. Its values are not real economic observations. Assume it was generated as a stationary, zero-mean Gaussian ARMA(1,1) process, and that the mean is treated as known to be zero for this illustration. In an application, the mean specification would also need to be considered.

For an ARMA(1,1), the theoretical ACF does not cut off after a fixed lag. Instead, for \(k\geq 1\), it decays geometrically:

This contrasts with the idealized patterns often used as clues for pure AR or pure MA models: a pure AR process has a PACF that cuts off in the population, while a pure MA process has an ACF that cuts off. For mixed models, neither function generally has a sharp cutoff. In a sample, the approximate white-noise reference limits \(\pm1.96/\sqrt{n}\) can help flag correlations that stand out, but they are a screening guide rather than a formal order-selection rule.

Worked Example: Proposing an Order from the ACF and PACF

Moment Estimation for ARMA(1,1)

The theoretical ACF gives a simple moment-based route to an ARMA(1,1) fit. If the lag-1 correlation is nonzero, the geometric decay implies \(\rho_2/\rho_1=\phi\). Replacing population correlations by sample autocorrelations gives the moment estimate \(\hat{\phi}_{\mathrm{mom}}=\hat{\rho}_2/\hat{\rho}_1\). Then the equation for \(\rho_1\) can be solved for \(\theta\). Moment estimation is useful for illustrating the connection between an ACF pattern and model parameters, and it can provide starting values for likelihood optimization. It is not a substitute for checking the fit.

The equation for \(\theta\) is quadratic, so it can have two roots. The invertibility restriction \(|\theta|<1\) guides which root is admissible under the model convention used here. Once the parameters are obtained, the lag-zero sample autocovariance \(c_0\), which is the sample variance under the chosen centering, can be used to estimate the innovation variance:

Why the Moment Relationship Holds

The moment calculation follows directly from the theoretical ACF, rather than from a rule of thumb. For \(k\geq 1\), the ARMA(1,1) ACF satisfies \(\rho_k=\phi^{k-1}\rho_1\). Setting \(k=2\) gives \(\rho_2=\phi\rho_1\). If \(\rho_1\neq0\), division by \(\rho_1\) yields \(\phi=\rho_2/\rho_1\). Replacing these population correlations by their sample estimates gives the moment equation used above. This substitution explains both the appeal and the limitation of the method: the estimate inherits sampling variability from the sample ACF, and can be unstable when \(\hat{\rho}_1\) is close to zero.

The expression for the innovation variance follows from the theoretical lag-zero variance, $$ \gamma_0=\sigma^2\frac{1+\theta^2+2\phi\theta}{1-\phi^2}. $$ Solving for \(\sigma^2\) and replacing the parameters and \(\gamma_0\) by their moment estimates gives the stated formula. These relationships assume a stationary ARMA(1,1) model with the specified innovation structure; the formulas should not be applied mechanically to a series with an unaddressed trend or other misspecification.

Likelihood Estimation and Residual Checking

Moment estimates match selected theoretical correlations to sample correlations. Gaussian maximum likelihood instead chooses parameters to make the observed series most plausible under the model. With stationary initialization, the likelihood can be evaluated through the one-step prediction errors and their variances from the model’s innovation recursion. A constrained numerical optimizer searches over the parameter space, enforcing \(|\phi|<1\), \(|\theta|<1\), and \(\sigma^2>0\). The likelihood estimate need not equal the moment estimate: it uses the full likelihood, not just two sample autocorrelations.

A fitted model is not adequate merely because its parameter estimates satisfy the constraints. The fitted innovations or residuals should have no systematic remaining autocorrelation. A portmanteau statistic summarizes residual autocorrelations over a selected set of lags. Its reference degrees of freedom are commonly adjusted for the fitted AR and MA parameters; the choice of lag range and the approximation should be appropriate to the sample and model.

Common Pitfalls and Practical Interpretation

• Treating an ACF pattern as a unique identification. A declining ACF and a non-cutting-off PACF suggest candidate orders, but finite-sample patterns are noisy. Treat them as a guide to models worth fitting, not as a definitive selection test.
• Keeping every algebraic moment root. The moment equation may yield multiple values of \(\theta\). Under the convention here, an invertible fit requires \(|\theta|<1\); a root outside that range is not the admissible invertible solution.
• Assuming a successful optimizer guarantees a good model. An optimizer can return valid parameter values even when residuals retain serial correlation. Check residual dependence and other relevant innovation properties.
• Calling a non-significant diagnostic proof of adequacy. A large portmanteau \(p\)-value is a failure to find evidence against the tested null, not the probability that the model is true. The result depends on the lag range, sample size, and test’s approximate reference distribution.

This workflow connects the descriptive information in the sample ACF and PACF to parameter estimation and model checking. The ACF proposes structure; moment equations provide a transparent calculation; likelihood estimation uses the full model; and residual checks ask whether important dependence remains unexplained.