The Spectral Representation of Stationary Sequences

From Covariances to a Random Spectral Integral

A weakly stationary sequence has a covariance that depends only on the lag between two observations. Herglotz’s theorem expresses that covariance sequence as the Fourier coefficients of a finite positive measure on the frequency circle. The spectral representation goes further: it realizes the sequence itself, in mean square, as an integral of the functions \(e^{in\lambda}\) against orthogonal random increments. The measure describes how much variance is associated with each frequency; the random increments supply the corresponding random variation.

Let \((X_n)_{n\in\mathbb{Z}}\) be a real- or complex-valued process on a probability space \((\Omega,\mathcal{F},\mathbb{P})\). It is weakly stationary if \(\mathbb{E}|X_n|^2<\infty\), its mean is constant, and its covariance depends only on the lag. Write \(\mu=\mathbb{E}X_n\), \(Y_n=X_n-\mu\), and use the convention \(\gamma(h)=\mathbb{E}[Y_{n+h}\overline{Y_n}]\). In particular, \(\gamma(0)=\mathbb{E}|Y_n|^2\) is the variance when the process is real-valued.

By Herglotz’s theorem, every weakly stationary covariance sequence has such a measure, and the measure is unique under this convention. This existence result is established in the earlier material on Herglotz’s theorem. The result below uses that measure to construct the random increments and the integral representation.

Proof. Work in the complex space of square-integrable random variables, with inner product \(\langle U,V\rangle=\mathbb{E}[U\overline V]\), and let \(H\) be the mean-square closed linear span of \(\{Y_n:n\in\mathbb{Z}\}\). Define a map first on trigonometric polynomials by \(U(\sum_n a_n e^{in\lambda})=\sum_n a_nY_n\), where only finitely many coefficients \(a_n\) are nonzero.

For such a polynomial \(p(\lambda)=\sum_n a_n e^{in\lambda}\), stationarity and the definition of \(F\) give

Thus the map preserves squared norms. In particular, if two polynomials agree \(F\)-almost everywhere, their images have squared difference norm zero, so \(U\) is well-defined on their equivalence classes in \(L^2(F)\). Trigonometric polynomials are dense in \(L^2(F)\): continuous functions are dense for a finite Borel measure on the compact frequency circle, and the Stone–Weierstrass theorem gives uniform approximation of continuous functions by trigonometric polynomials. These standard density results are used here without proof. The isometry therefore extends by mean-square limits to an isometry \(U:L^2(F)\to H\). Its range is closed, since an isometry takes a Cauchy sequence to a Cauchy sequence and \(H\) is complete. Its range contains every \(Y_n=U(e^{in\lambda})\), so it contains their closed linear span \(H\); hence its range is exactly \(H\).

For a Borel set \(A\), set \(Z(A)=U(\mathbf{1}_A)\), where \(\mathbf{1}_A\) is the indicator function. Since \(U\) takes values in the closed span of centered random variables, \(\mathbb{E}Z(A)=0\). The isometry gives \(\mathbb{E}[Z(A)\overline{Z(B)}]=\int\mathbf{1}_A\mathbf{1}_B\,dF=F(A\cap B)\), proving the increment covariance and, for disjoint sets, orthogonality.

If \(A_k\) are pairwise disjoint, the indicators of their finite unions converge in \(L^2(F)\) to the indicator of their union: the squared error after \(r\) sets is \(F(\bigcup_{k>r}A_k)\), which tends to zero because \(F\) is finite. Applying the isometry proves the claimed countable additivity in mean square. For a simple function \(f=\sum_j c_j\mathbf{1}_{A_j}\) on disjoint sets, define \(\int f\,Z(d\lambda)=\sum_j c_jZ(A_j)=U(f)\). The isometry extends this definition to every \(f\in L^2(F)\), with \(\mathbb{E}|\int f\,Z(d\lambda)|^2=\int|f|^2\,dF\). Taking \(f(\lambda)=e^{in\lambda}\) gives \(Y_n=U(f)=\int e^{in\lambda}\,Z(d\lambda)\). Finally, the isometry applied to \(e^{i(n+h)\lambda}\) and \(e^{in\lambda}\) gives the stated covariance integral. \(\square\)

The representation is a mean-square identity: the squared expected error between \(X_n-\mu\) and the integral is zero. It does not assert a pathwise integral or pointwise convergence of ordinary Fourier series. Nor does orthogonality alone imply independence; independence of disjoint spectral increments requires additional assumptions, such as suitable Gaussian structure.

Worked Representations

Reading the Representation Correctly

The isometry in the proof is the central mechanism. It matches the squared mean-square size of any finite linear combination of observations with the squared \(L^2(F)\) size of the corresponding trigonometric polynomial. Completing that match extends the correspondence from finite combinations to all square-integrable spectral functions. In particular, the covariance formula is not an additional assumption about the constructed increments: it follows from their isometry.

For disjoint sets \(A\) and \(B\), the identity \(F(A\cap B)=0\) gives \(\mathbb{E}[Z(A)\overline{Z(B)}]=0\). This is orthogonality, not a general claim of independence. Also, the measure records centered variance: its total mass is \(\gamma(0)\), not the second moment \(\mathbb{E}|X_n|^2\) when the mean is nonzero. The mean \(\mu\) is added separately in the representation.

Common mistake: losing the Fourier normalization. In the white-noise example, writing \(F(d\lambda)=2\,d\lambda\) instead of \(2\,d\lambda/(2\pi)\) would give \(F([-\pi,\pi))=2(2\pi)=4\pi\). The spectral formula at \(h=0\) would then imply variance \(4\pi\), contradicting the specified variance \(2\). With the stated convention, the correct measure is \(2\,d\lambda/(2\pi)\), whose total mass is \(2(2\pi)/(2\pi)=2\). Always check the total mass against the centered variance before using a proposed spectral measure.