Why Represent a Process by Frequency?
A covariance-stationary time series can be described through how its dependence changes with the lag between observations. A complementary view asks how its variation is distributed over frequencies: is it concentrated in persistent components, in a few recurring cycles, or across a wider range? The spectral representation makes this frequency-based description precise without changing the process being studied.
The key point is that this is not merely a way to build a new process with the same autocovariances. The representation theorem expresses the given process itself as a mean-square integral of frequency-specific random increments. The measure governing those increments is its spectral distribution.
Covariance Stationarity and the Spectral Distribution
Let \(\{X_t:t\in\mathbb{Z}\}\) be a real-valued process with finite second moments. It is covariance-stationary if its mean is a constant \(\mu\), and its autocovariance depends only on the lag. Write \(\gamma(h)=\operatorname{Cov}(X_t,X_{t-h})\), so \(\gamma(h)\) does not depend on \(t\). In particular, \(\gamma(0)=\operatorname{Var}(X_t)\).
For every finite set of times and complex coefficients, the autocovariances satisfy \(\sum_j\sum_k a_j\overline{a_k}\gamma(t_j-t_k)\geq 0\). This property is called positive definiteness. Herglotz’s theorem says that a positive-definite autocovariance sequence on the integers has a unique finite, nonnegative measure on the frequency circle whose Fourier coefficients are that sequence. In this tutorial, frequency is measured in radians per time period, and the circle is represented by \([-\pi,\pi)\), with the endpoints identified.
The total-mass statement follows by setting \(h=0\), since \(e^{i0\lambda}=1\). Thus, \(F\) records how the process variance is allocated over frequencies. A point mass at a frequency represents variance concentrated at that frequency; mass spread over a set of frequencies represents variance distributed across that set. For a real-valued process, the spectral measure is symmetric around zero: positive and negative frequencies have matching mass.
The Orthogonal-Increment Representation
An orthogonal-increment random measure \(Z\) assigns a mean-zero complex random variable \(Z(A)\) to each measurable frequency set \(A\). It is countably additive in mean square, and its increments have covariance \(\mathbb{E}[Z(A)\overline{Z(B)}]=F(A\cap B)\). In particular, increments for disjoint sets are orthogonal (uncorrelated). The notation \(\int g(\lambda)\,dZ(\lambda)\) denotes a mean-square integral: it is first defined for simple functions \(g\), then extended by mean-square limits.
The existence claim can be seen by connecting the spectral measure back to the original random variables. Herglotz’s theorem supplies \(F\). In the space of square-integrable functions under \(F\), the exponential functions \(e^{it\lambda}\) have inner products \[ \int e^{i(t-s)\lambda}\,dF(\lambda)=\gamma(t-s). \] The centered process variables \(X_t-\mu\) have the same inner products, because \(\mathbb{E}[(X_t-\mu)\overline{(X_s-\mu)}]=\gamma(t-s)\). Therefore the map taking \(e^{it\lambda}\) to \(X_t-\mu\) preserves inner products on finite linear combinations. It extends by mean-square limits to the closure of those combinations. Trigonometric polynomials are dense in the relevant \(L^2(F)\) space, so the map extends to that space. Applying it to indicator functions defines \(Z(A)\); applying it to \(e^{it\lambda}\) gives the displayed representation of \(X_t-\mu\). This construction is why the representation is of the given process.
The integral also explains how the covariance formula follows. Orthogonal increments give the isometry \[ \mathbb{E}\left[\left(\int g\,dZ\right)\overline{\left(\int q\,dZ\right)}\right] =\int g(\lambda)\overline{q(\lambda)}\,dF(\lambda). \] Taking \(g(\lambda)=e^{it\lambda}\) and \(q(\lambda)=e^{is\lambda}\) yields \[ \operatorname{Cov}(X_t,X_s) =\int e^{i(t-s)\lambda}\,dF(\lambda) =\gamma(t-s). \] This calculation checks the covariance implied by the representation; the construction above establishes that the representation applies to the original \(X_t\).
Worked Representations
Worked Example: A Random Level Concentrates Variance at Zero
Let \(X_t=C\) for every integer \(t\), where \(C\) is a random variable with mean \(\mu\) and finite variance \(v\). This is a covariance-stationary process: its mean is \(\mu\), and any two observations are the same random variable, so \(\gamma(h)=v\) at every lag.
A measure placing mass \(v\) at frequency zero has the required Fourier coefficients: \[ \int e^{ih\lambda}\,dF(\lambda) =v e^{ih\cdot 0} =v \] for every \(h\). Thus \(F\) is a point mass of size \(v\) at zero. The increment \(Z(\{0\})\) is \(C-\mu\), and \[ \int e^{it\lambda}\,dZ(\lambda) =e^{it\cdot 0}Z(\{0\}) =C-\mu. \] The representation therefore recovers \(X_t-\mu\) exactly for every \(t\). All the process variance is concentrated at zero frequency because the random level does not change over time.
Worked Example: A Random Harmonic Component
Let \(X_t=A\cos(t\omega)+B\sin(t\omega)\), where \(0<\omega<\pi\). Suppose \(A\) and \(B\) have mean zero, variance \(\tau^2\), and covariance zero. These moment conditions are enough for covariance stationarity; independence is not required. The mean is zero, and for any integer \(h\), \[ \begin{aligned} \gamma(h) &=\mathbb{E}[X_tX_{t-h}]\\ &=\tau^2\{\cos(t\omega)\cos((t-h)\omega)+\sin(t\omega)\sin((t-h)\omega)\}\\ &=\tau^2\cos(h\omega). \end{aligned} \] The last expression depends only on \(h\), as required.
Place mass \(\tau^2/2\) at each of \(\omega\) and \(-\omega\). Its Fourier coefficient at lag \(h\) is \[ \frac{\tau^2}{2}e^{ih\omega}+\frac{\tau^2}{2}e^{-ih\omega} =\tau^2\cos(h\omega), \] which equals the calculated autocovariance. The increments can be written \(Z(\{\omega\})=(A-iB)/2\) and \(Z(\{-\omega\})=(A+iB)/2\). Each has variance \(\tau^2/2\), and they are orthogonal because \[ \mathbb{E}\!\left[Z(\{\omega\})\overline{Z(\{-\omega\})}\right] =\mathbb{E}\!\left[\left(\frac{A-iB}{2}\right)^2\right] =\frac{\mathbb{E}[A^2]-\mathbb{E}[B^2]-2i\mathbb{E}[AB]}{4} =0. \] Finally, \[ e^{it\omega}\frac{A-iB}{2} +e^{-it\omega}\frac{A+iB}{2} =A\cos(t\omega)+B\sin(t\omega)=X_t. \] The two spectral masses encode the positive- and negative-frequency parts of the same real harmonic component.
Worked Example: Two Harmonics and Their Variance Allocation
Consider \(X_t=A_1\cos(t\pi/3)+B_1\sin(t\pi/3)+A_2\cos(t\pi/2)+B_2\sin(t\pi/2)\). Assume all four coefficients have mean zero, are pairwise uncorrelated, and have variances \(\operatorname{Var}(A_1)=\operatorname{Var}(B_1)=4\) and \(\operatorname{Var}(A_2)=\operatorname{Var}(B_2)=1\). Each harmonic component has covariance equal to its coefficient variance times the cosine of its frequency times the lag. The uncorrelated components add, giving \[ \gamma(h)=4\cos(h\pi/3)+\cos(h\pi/2). \] This depends only on \(h\), and its value at zero is \(4+1=5\).
The spectral distribution has mass \(2\) at each of \(\pi/3\) and \(-\pi/3\), and mass \(1/2\) at each of \(\pi/2\) and \(-\pi/2\). At any lag \(h\), its Fourier coefficient is \[ 2e^{ih\pi/3}+2e^{-ih\pi/3} +\frac12e^{ih\pi/2}+\frac12e^{-ih\pi/2} =4\cos(h\pi/3)+\cos(h\pi/2). \] The total mass is \(2+2+1/2+1/2=5\), matching the process variance. The two pairs of masses identify which frequencies contribute variance, while their sizes show how much each contributes.
What the Representation Does—and Does Not—Say
The representation is an equality in mean square for each time point. In particular, the mean-square difference between \(X_t-\mu\) and its integral is zero, so they are equal almost surely for each fixed \(t\). The full sequence of integrals therefore represents the original process, not simply a newly generated series with the same autocovariance function.
Orthogonality of increments is also not independence. Disjoint frequency increments have zero covariance, but need not be statistically independent. Nor does a matching mean and autocovariance function, by itself, imply that two separately constructed processes have the same distribution. For jointly Gaussian processes, matching means and covariances do imply matching finite-dimensional distributions; without such a condition, covariance information alone is not enough.
Check that the mean is constant and \(\gamma(h)\) depends only on the lag.
Use a nonnegative measure whose Fourier coefficients equal \(\gamma(h)\); its total mass must be \(\gamma(0)\).
Orthogonal increments allocate variance across frequency sets, and integrating \(e^{it\lambda}\) against them represents \(X_t-\mu\).
Common Mistake
A common mistake is to construct a random measure with the right variance measure, calculate that the resulting process has autocovariance \(\gamma(h)\), and then claim that this represents the original \(X_t\). That calculation establishes a covariance match, not an equality with the given process. The correction is to use the orthogonal-increment representation theorem, or to construct the integral map from the closed linear span of the original centered variables in \(L^2\), as above. The distinction matters because covariance matching alone does not generally determine a process’s distribution.
Check Your Understanding
Use the definitions and examples above to answer the following questions.
- Why must the total mass of a process’s spectral distribution equal its variance?
- A process is constant over time at a random level with variance \(v\). At which frequency is its spectral mass located, and how large is that mass?
- For a harmonic process with coefficient variances \(\tau^2\), why are the spectral masses split between positive and negative frequencies?
- What does orthogonality of increments say, and why does it not by itself imply independence?
- Why is matching an autocovariance function not, on its own, enough to establish the spectral representation of a particular given process?