Herglotz’s Theorem for Covariance Sequences

Positive Definiteness and Fourier Coefficients

A covariance sequence cannot be chosen freely: its finite covariance matrices must be nonnegative definite. Herglotz’s theorem gives an exact characterization of this constraint. It says that the sequences satisfying it are precisely the Fourier coefficient sequences of finite nonnegative measures on the circle. This result applies both to covariance sequences and to more general sequences on the integers.

We use the circle \(\mathbb{T}=\mathbb{R}/(2\pi\mathbb{Z})\), represented by angles modulo \(2\pi\), and the Fourier convention \(c_n=\int_{\mathbb{T}}e^{in\theta}\,\mu(d\theta)\), for \(n\in\mathbb{Z}\). The measure is finite and nonnegative; it need not be a probability measure.

For a weakly stationary real-valued process \((X_n)_{n\ge0}\) on a probability space \((\Omega,\mathcal{F},\mathbb{P})\), its covariance sequence is \(\gamma(h)=\operatorname{Cov}(X_{n+h},X_n)\), with the usual extension to integer lags. Its finite covariance matrices are nonnegative definite, so this sequence is positive definite. The theorem below then says that a finite nonnegative measure represents the covariances. Its mass is the variance at lag zero.

Why a Representing Measure Implies Positive Definiteness

This direction follows by writing the quadratic form as the integral of a squared absolute value. It also fixes the signs in our Fourier convention.

Proof. Suppose a finite nonnegative measure \(\mu\) represents the sequence. For any finite \(I\subset\mathbb{Z}\) and coefficients \(a_j\), linearity of integration gives \(\sum_{j,k\in I}a_j\overline{a_k}c_{j-k}=\int_{\mathbb{T}}\sum_{j,k\in I}a_j\overline{a_k}e^{i(j-k)\theta}\,\mu(d\theta)\). The integrand is \(\left|\sum_{j\in I}a_je^{ij\theta}\right|^2\), which is nonnegative. The matrix is Hermitian because \(c_{k-j}=\overline{c_{j-k}}\), as follows by conjugating the integral defining \(c_{j-k}\). Thus the sequence is positive definite. Taking \(n=0\) in the representation gives \(c_0=\int_{\mathbb{T}}1\,\mu(d\theta)=\mu(\mathbb{T})\). \(\square\)

Constructing the Measure

For the converse, we first build a sequence of nonnegative measures from finite pieces of the sequence. Let \(m(d\theta)=d\theta/(2\pi)\) be normalized length measure on the circle. For each integer \(N\ge0\), define the trigonometric polynomial \[ f_N(\theta)=\frac{1}{N+1}\sum_{j,k=0}^{N}c_{j-k}e^{-i(j-k)\theta}. \] Positive definiteness implies \(f_N(\theta)\ge0\): in its defining quadratic form use the index set \(\{0,\ldots,N\}\) and \(a_j=e^{-ij\theta}\). The polynomial is real because the defining matrices are Hermitian.

Integrating each exponential against \(m\), whose integral of \(e^{ir\theta}\) is \(1\) for \(r=0\) and \(0\) for nonzero integer \(r\), shows that \(\int f_N\,dm=c_0\). Therefore \(\mu_N(d\theta)=f_N(\theta)m(d\theta)\) is a nonnegative measure of mass \(c_0\).

Indeed, in the sum defining \(f_N\), only terms with \(j-k=n\) survive integration against \(e^{in\theta}\). There are \(N+1-|n|\) such pairs \((j,k)\), and each contributes \(c_n/(N+1)\). For each fixed \(n\), the displayed coefficient tends to \(c_n\) as \(N\) tends to infinity.

Proof. Assume \((c_n)\) is positive definite. The singleton index set shows \(c_0\ge0\). If \(c_0=0\), consider the two indices \(0,n\). Their matrix has diagonal entries \(0\) and off-diagonal entries \(c_n,c_{-n}\). A Hermitian nonnegative matrix with both diagonal entries zero must have zero off-diagonal entries: testing the quadratic form on vectors \((1,z)\) for arbitrary complex \(z\) forces its linear cross-term to vanish. Hence \(c_n=0\) for every \(n\). The zero measure represents this sequence and has mass \(c_0\).

Now suppose \(c_0>0\). The measures \(\mu_N\) constructed above are all nonnegative and have the same finite mass \(c_0\). We use the standard weak sequential compactness theorem for finite nonnegative measures of bounded mass on a compact metric space: every such sequence has a subsequence converging weakly to a finite nonnegative measure. Here weak convergence means convergence of integrals of every continuous function. This measure-theoretic compactness theorem is standard; its proof is not needed for the finite Fourier calculations here. The circle is compact, so some subsequence \(\mu_{N_r}\) converges weakly to a finite nonnegative measure \(\mu\). For fixed \(n\), the function \(e^{in\theta}\) is continuous, and therefore \(\int e^{in\theta}\,\mu(d\theta)=\lim_{r\to\infty}\int e^{in\theta}\,\mu_{N_r}(d\theta)=c_n\). The last equality follows from the coefficient formula and \(N_r\to\infty\). Taking the constant function \(1\) in weak convergence gives \(\mu(\mathbb{T})=\lim_r\mu_{N_r}(\mathbb{T})=c_0\). This proves existence and the mass statement.

For uniqueness, suppose \(\mu\) and \(\nu\) both represent the sequence. They have equal integrals of every \(e^{in\theta}\), and hence of every trigonometric polynomial. Trigonometric polynomials are uniformly dense in the continuous functions on the circle (the standard trigonometric-polynomial approximation theorem). Since \(\mu\) and \(\nu\) are finite, uniform approximation implies that their integrals agree for every continuous function: the integral error is at most the uniform approximation error times the measure’s mass. Finite Borel measures on a compact metric space are determined by their integrals against continuous functions, so \(\mu=\nu\). This proves uniqueness and completes the theorem. \(\square\)

Worked Representations

Total Mass and a Common Normalization Error

The equality \(\mu(\mathbb{T})=c_0\) is part of the representation, not an optional normalization. If \((c_n)\) is a covariance sequence, then \(c_0\) is the variance, so the representing measure has that total mass. It is a probability measure only when \(c_0=1\). When \(c_0=0\), positive definiteness forces the entire sequence to be zero, as the proof showed.

Common mistake. For the sequence \(c_0=2\), \(c_n=0\) for \(n\ne0\), using normalized length measure \(m\) alone gives Fourier coefficients \(1\) at zero and \(0\) elsewhere. That is not the stated sequence: its zero-lag coefficient is \(1\), not \(2\). The correction is to use \(2m\), whose mass and zero-lag coefficient are both \(2\). Always check the coefficient at \(n=0\) before treating a representing measure as a probability distribution.

Herglotz’s theorem is useful because it turns all finite positive-definiteness constraints into one representation by a nonnegative measure. In applications, the sequence fixes that measure uniquely, while its value at zero immediately fixes the total mass. The theorem itself does not require the measure to have a density or to be supported at finitely many points; those are additional properties of particular examples.