Stable Autoregressive Filters and Resonance

From an autoregressive equation to a filter

A moving-average filter forms its output from a weighted sum of present and past inputs. An autoregressive equation instead includes past outputs. When that equation is stable, it can also be represented as a causal linear filter: the present output is a weighted sum of current and past noise. This representation makes the frequency response and spectrum calculable. Its denominator also reveals how poles can create resonant peaks.

Work on a probability space \((\Omega,\mathcal{F},\mathbb{P})\). Let \((\varepsilon_n)_{n\in\mathbb{Z}}\) be a real, zero-mean white-noise sequence: \(\mathbb{E}[\varepsilon_n]=0\) and \(\operatorname{Cov}(\varepsilon_n,\varepsilon_m)=\sigma^2\mathbf{1}_{\{n=m\}}\), where \(0<\sigma^2<\infty\). The time index is \(n\in\mathbb{Z}\), and both input and output take values in \(S=\mathbb{R}\). White noise here means uncorrelated observations with a common finite variance; independence is not required.

The variable \(z\) records a delay: \(z^j\) corresponds to \(j\) time steps into the past. The root condition places every pole of \(H(z)=1/A(z)\) outside the unit circle. In the Fourier convention used here, the spectral density \(f\) and covariance sequence \(\gamma(k)=\operatorname{Cov}(X_{n+k},X_n)\) satisfy \(f(\omega)=(1/(2\pi))\sum_{k\in\mathbb{Z}}\gamma(k)e^{-ik\omega}\) and \(\gamma(k)=\int_{-\pi}^{\pi}e^{ik\omega}f(\omega)\,d\omega\).

Proof. If \(A\) has degree zero, then \(A=1\), \(h_0=1\), and \(h_j=0\) for \(j>0\); the claims reduce to \(X_n=\varepsilon_n\). Otherwise, factor \(A(z)=\prod_{\ell=1}^{p}(1-z/r_\ell)\), listing roots with their multiplicities. Partial fractions express \(1/A(z)\) as a finite sum of terms \(C(1-z/r)^{-m}\), with constants \(C\), roots \(r\), and positive integers \(m\). The power-series coefficient of \(z^j\) in such a term is \(C\binom{j+m-1}{m-1}r^{-j}\). Because \(|r|>1\), its absolute values are bounded by a polynomial in \(j\) times a geometrically decreasing sequence, and hence are summable. Thus \(\sum_j|h_j|<\infty\), in particular \(\sum_jh_j^2<\infty\). The variance of a finite tail \(\sum_{j=u}^{v}h_j\varepsilon_{n-j}\) is \(\sigma^2\sum_{j=u}^{v}h_j^2\), which tends to zero as \(u\) tends to infinity. This proves mean-square convergence.

Multiplying the power series by \(A(z)\) gives \(A(z)\sum_{j\ge0}h_jz^j=1\). Equating coefficients shows that applying the autoregressive left-hand side to \(X_n\) leaves \(\varepsilon_n\): the coefficient of the current input is \(1\), and every coefficient of an older input is \(0\). The mean is zero. For any integer \(k\), mean-square convergence and white-noise covariance give \(\gamma(k)=\sigma^2\sum_{j\ge0}h_{j+k}h_j\) when \(k\ge0\), with \(\gamma(-k)=\gamma(k)\). These covariances depend only on the lag, so the output is weakly stationary.

Finally, absolute summability permits multiplication and rearrangement of the two absolutely convergent series in the Fourier calculation. The Fourier series of \(\gamma\) is \(\sigma^2(\sum_{j\ge0}h_je^{-ij\omega})(\sum_{j\ge0}h_je^{ij\omega})\): matching terms with index difference \(k\) gives the covariance at lag \(k\). The factors are complex conjugates because the coefficients are real. Dividing by \(2\pi\) gives \(f(\omega)=\sigma^2|H(e^{-i\omega})|^2/(2\pi)\). \(\square\)

First examples: stability and pole location

For a real second-order filter, a conjugate pair of poles produces a frequency-selective response. Pole angle indicates where the response tends to be large; pole radius controls how close the poles are to the unit circle. The exact peak need not occur at the pole angle, so it is useful to minimize the denominator rather than infer the peak from a sketch.

Proof. Factoring the polynomial gives \(A(z)=(1-ae^{i\theta}z)(1-ae^{-i\theta}z)\), which yields the two stated roots. At \(z=e^{-i\omega}\), the squared modulus is \([1+a^2-2a\cos(\theta-\omega)][1+a^2-2a\cos(\theta+\omega)]\). Using \(\cos(\theta-\omega)+\cos(\theta+\omega)=2\cos(\theta)\cos(\omega)\) and \(\cos(\theta-\omega)\cos(\theta+\omega)=\cos^2(\theta)+\cos^2(\omega)-1\) gives the displayed \(D(x)\). Its coefficient of \(x^2\) is \(4a^2>0\). Differentiating gives \(D'(x)=8a^2x-4a(1+a^2)\cos(\theta)\), which vanishes at \(x_*\). A convex quadratic reaches its minimum on \([-1,1]\) at its vertex when the vertex is in the interval; if it lies outside, the minimum is at the nearer endpoint. Since \(x=\cos\omega\), an interior minimizer gives the stated frequency, and the evenness of the denominator gives the negative-frequency peak as well. \(\square\)

How pole radius shapes resonance

For a conjugate pair, moving the poles closer to the unit circle generally makes the response sharper and taller near its resonant frequencies. This is a qualitative statement about a stable family: each member must still have \(a<1\). At the boundary \(a=1\), the stability condition fails. The pole angle identifies the central frequency in the limit as \(a\) approaches \(1\), while the exact finite-\(a\) peak is determined by the denominator calculation above.

A stability pitfall

A common mistake is to use the autoregressive polynomial as the transfer function rather than its reciprocal. For the AR(1) equation in the first example, that mistake gives \(H(e^{-i\omega})=1-\tfrac12e^{-i\omega}\), and at \(\omega=0\) it gives \(H(1)=\tfrac12\). The correct equation is \(A(L)X_n=\varepsilon_n\); solving for the output applies the inverse filter, so \(H(1)=1/A(1)=1/(1-\tfrac12)=2\). The spectral density uses the squared modulus of this reciprocal, not the squared modulus of \(A\).

Stability is also essential, not a technical decoration. If the AR(1) coefficient is \(1\), the formal causal coefficients would all equal \(1\). Their squared sum diverges, so \(\sum_{j\ge0}\varepsilon_{n-j}\) does not converge in mean square when \(\sigma^2>0\). The stable-filter spectral formula cannot be applied to that causal series. More generally, a root on or inside the unit circle violates the root condition used to obtain an absolutely summable causal response.

For a stable AR filter, the calculation has a reliable order: check the roots of \(A\), invert \(A\) to identify the transfer function, and only then evaluate \(\sigma^2|H(e^{-i\omega})|^2/(2\pi)\). For resonance, minimize \(|A(e^{-i\omega})|^2\) over frequency. This separates stability, which determines whether the causal filter exists, from resonance, which describes how its spectrum is shaped.