From Covariance Values to a Spectral Density
A spectral density can often be computed without first finding a spectral representation of the process: sum the covariance sequence against the appropriate Fourier factors. For a geometric covariance sequence, the sum is an infinite geometric series and simplifies to a rational function of frequency. For a finite moving-average process, only finitely many covariance values are nonzero, so the sum is a trigonometric polynomial. In both cases, checking the answer means more than simplifying the algebra: the density must be nonnegative, and its integral must recover the variance.
Work on a probability space \((\Omega,\mathcal{F},\mathbb{P})\). Let \((X_n)_{n\ge0}\) be a real-valued, weakly stationary discrete-time process with state space \(S=\mathbb{R}\), finite mean, and covariance sequence \(\gamma(h)=\operatorname{Cov}(X_{n+h},X_n)\), defined for integer lags \(h\). The covariance sequence is even: \(\gamma(-h)=\gamma(h)\). We use the discrete-time Fourier convention in which a spectral density \(f\), when it exists, satisfies the inversion formula below.
The first formula defines the density by Fourier summation whenever the series converges appropriately; absolute summability of \(\gamma\) is a sufficient condition for uniform convergence. The factor \(1/(2\pi)\) is part of this convention. In particular, integrating the density over \([-\pi,\pi]\) should give \(\gamma(0)=\operatorname{Var}(X_n)\), not \(2\pi\gamma(0)\). The same function is periodic with period \(2\pi\), so endpoint values at \(-\pi\) and \(\pi\) agree.
Geometric Covariance Sequences
Proof. If \(v=0\), every covariance value and the stated density are zero, so the claims hold. Suppose \(v>0\). Since \(\sum_{h\in\mathbb{Z}}|v r^{|h|}|=v(1+2\sum_{k\ge1}|r|^k)<\infty\), the series converges absolutely. Pairing the terms at lags \(k\) and \(-k\) gives $$ \sum_{h\in\mathbb{Z}}r^{|h|}e^{-ih\theta} =1+\sum_{k\ge1}r^k(e^{-ik\theta}+e^{ik\theta}) =1+\frac{r e^{-i\theta}}{1-r e^{-i\theta}}+\frac{r e^{i\theta}}{1-r e^{i\theta}}. $$ The denominators are nonzero because \(|r|<1\). Combining the terms over the common denominator \((1-r e^{-i\theta})(1-r e^{i\theta})=1-2r\cos\theta+r^2\) gives numerator \(1-r^2\). Multiplying by \(v/(2\pi)\) proves the formula. The denominator is \(|1-r e^{i\theta}|^2>0\), and \(1-r^2>0\), so \(f(\theta)\ge0\). Finally, absolute convergence permits term-by-term integration of the Fourier series. For integer \(h\), \(\int_{-\pi}^{\pi}e^{-ih\theta}\,d\theta\) is \(2\pi\) when \(h=0\) and zero otherwise; hence \(\int_{-\pi}^{\pi}f(\theta)\,d\theta=v=\gamma(0)\). \(\square\)
Worked Example: A Geometric Covariance with \(r=\tfrac12\)
Suppose the covariance sequence is \(\gamma(h)=4(1/2)^{|h|}\). Here \(v=4\) and \(r=1/2\), with \(|r|<1\), so the theorem applies. Substitution gives \(1-r^2=1-1/4=3/4\) and \(1-2r\cos\theta+r^2=1-\cos\theta+1/4=5/4-\cos\theta\). Therefore
Nonnegativity check: since \(-1\le\cos\theta\le1\), the denominator \(5-4\cos\theta\) is at least \(5-4=1\), so \(f(\theta)>0\) for every \(\theta\). Normalization check: the constant Fourier coefficient in the density series is \(\gamma(0)=4\); equivalently, integrating the absolutely convergent series leaves only that coefficient, giving \(\int_{-\pi}^{\pi}f(\theta)\,d\theta=4\). This equals the variance specified by the covariance sequence.
Finite Moving-Average Covariance Sequences
A finite moving average of order \(q\) is a process of the form \(X_n=\sum_{j=0}^{q}a_j\varepsilon_{n-j}\), where the coefficients \(a_j\) are real and \((\varepsilon_n)\) is zero-mean white noise with \(\operatorname{Cov}(\varepsilon_m,\varepsilon_n)=\sigma^2\) when \(m=n\), and zero otherwise. Here the process is indexed by integer time; restricting to \(n\ge0\) does not change its covariance calculations. In the sum for \(\gamma(h)\), a noise term contributes only when the same \(\varepsilon\) occurs in both \(X_{n+h}\) and \(X_n\). Consequently, \(\gamma(h)=0\) for \(|h|>q\).
Proof. Expand the covariance using bilinearity: $$ \gamma(h)=\sum_{j=0}^{q}\sum_{k=0}^{q}a_j a_k \operatorname{Cov}(\varepsilon_{n+h-j},\varepsilon_{n-k}). $$ A summand is nonzero exactly when \(n+h-j=n-k\), or \(k=j-h\). Re-indexing by \(j'=k\), equivalently pairing coefficients whose indices differ by \(h\), yields \(\gamma(h)=\sigma^2\sum_{0\le j\le q,\ 0\le j+h\le q}a_j a_{j+h}\); if \(|h|>q\), the index conditions have no solutions and the sum is zero. Now expand the squared modulus: $$ \left|\sum_{j=0}^{q}a_j e^{-ij\theta}\right|^2 =\sum_{j=0}^{q}\sum_{k=0}^{q}a_j a_k e^{-i(j-k)\theta}. $$ For each difference \(h=j-k\), the coefficient of \(e^{-ih\theta}\) is exactly \(\sum_{0\le k\le q,\ 0\le k+h\le q}a_k a_{k+h}\). Thus the finite Fourier sum \((2\pi)^{-1}\sum_h\gamma(h)e^{-ih\theta}\) equals the displayed squared-modulus formula. A squared modulus is nonnegative. For normalization, integrate the finite sum: all terms with \(h\ne0\) integrate to zero, while the \(h=0\) term integrates to \(\gamma(0)\). The covariance formula at \(h=0\) gives \(\gamma(0)=\sigma^2\sum_{j=0}^{q}a_j^2\). \(\square\)
Worked Example: A Moving Average of Order One
Let \(X_n=\varepsilon_n+\tfrac12\varepsilon_{n-1}\), where the white noise has variance \(\sigma^2=1\). The coefficients are \(a_0=1\) and \(a_1=1/2\). The theorem gives \(\gamma(0)=1^2+(1/2)^2=5/4\) and \(\gamma(1)=a_0a_1=1/2\). By evenness \(\gamma(-1)=1/2\), and \(\gamma(h)=0\) for \(|h|>1\). Hence
Nonnegativity check: \(\frac54+\cos\theta\ge\frac54-1=\frac14>0\). Normalization check: integrating the constant term gives \(\frac{1}{2\pi}\frac54(2\pi)=\frac54\), and the cosine term integrates to zero over \([-\pi,\pi]\). Thus the integral is \(5/4=\gamma(0)\). The polynomial form gives the same nonnegativity conclusion: \(\left|1+\tfrac12e^{-i\theta}\right|^2=\frac54+\cos\theta\).
Worked Example: A Moving Average of Order Two
Let \(X_n=\varepsilon_n+\tfrac12\varepsilon_{n-1}-\tfrac14\varepsilon_{n-2}\), with white-noise variance \(\sigma^2=4\). The coefficient vector is \((1,\tfrac12,-\tfrac14)\). Compute the nonzero covariances directly: $$ \gamma(0)=4\left(1+\frac14+\frac1{16}\right)=\frac{21}{4},\qquad \gamma(1)=4\left(\frac12-\frac18\right)=\frac32,\qquad \gamma(2)=4\left(-\frac14\right)=-1. $$ Evenness gives \(\gamma(-1)=3/2\) and \(\gamma(-2)=-1\); all other lags are zero. Fourier summation therefore gives
Nonnegativity check: the triangle inequality gives \(\left|1+\tfrac12e^{-i\theta}-\tfrac14e^{-2i\theta}\right|\ge1-\tfrac12-\tfrac14=\tfrac14\); thus \(f(\theta)\ge\frac{4}{2\pi}(\frac14)^2=\frac{1}{8\pi}>0\). Normalization check: the cosine terms integrate to zero, so the integral is \(\frac{1}{2\pi}\frac{21}{4}(2\pi)=21/4\), equal to \(\gamma(0)\). As a second check of the coefficients, expanding the squared modulus yields constant term \(1+1/4+1/16=21/16\), coefficient \(1/2-1/8=3/8\) on each complex exponential at lag one, and coefficient \(-1/4\) on each at lag two. Multiplying by \(4/(2\pi)\) gives exactly the Fourier sum above.
Checks That Catch Common Errors
For a real covariance sequence, the terms at \(h\) and \(-h\) must both appear in the Fourier sum. For example, if \(\gamma(0)=5/4\) and \(\gamma(1)=\gamma(-1)=1/2\), retaining only the nonnegative lag would give the incorrect expression \((2\pi)^{-1}(5/4+\tfrac12e^{-i\theta})\). It is generally complex-valued, so it cannot be the real spectral density of this real-valued process. Including both lags gives \((2\pi)^{-1}(5/4+\tfrac12e^{-i\theta}+\tfrac12e^{i\theta})=(2\pi)^{-1}(5/4+\cos\theta)\), which is real, nonnegative, and normalized.
Another useful check is to distinguish the variance from the integral of the density under the chosen convention: the integral is \(\gamma(0)\). The factor \(1/(2\pi)\) belongs in the density formula, while the interval of integration has length \(2\pi\). For finite moving averages, the squared-modulus expression is a direct nonnegativity check; for geometric covariances, check both \(1-r^2>0\) and \(1-2r\cos\theta+r^2=|1-r e^{i\theta}|^2>0\). These checks also make the assumptions visible: the geometric-series formula requires \(|r|<1\), while a finite moving average has only finitely many nonzero covariance lags.
Check Your Understanding
Use the Fourier convention stated above. In each calculation, include both positive and negative lags when they are nonzero.
- For \(\gamma(h)=9(1/3)^{|h|}\), write the spectral density and verify its integral using the geometric-series result.
- For \(X_n=\varepsilon_n-\tfrac13\varepsilon_{n-1}\) with white-noise variance \(1\), compute \(\gamma(0)\), \(\gamma(1)\), and \(f(\theta)\). Check nonnegativity and normalization.
- Why does a finite moving average have \(\gamma(h)=0\) whenever \(|h|\) exceeds its order?
- For the order-two example, identify the coefficient of \(e^{-2i\theta}\) in the Fourier sum and explain why it is negative.
- What goes wrong if a Fourier sum for a real covariance sequence includes positive lags but omits the corresponding negative lags?