Frequency Weighting and Discrete Differences
A finite difference compares nearby observations of a sequence. If successive observations tend to be similar, the variance of their difference is small; if the sequence fluctuates rapidly, differences can be large. Spectral measures describe variation by frequency, and a difference weights each frequency by a known factor. This gives an exact link between frequency content and discrete-time smoothness.
Let \((X_n)_{n\ge0}\) be a real-valued weakly stationary sequence on a probability space \((\Omega,\mathcal{F},\mathbb{P})\), with finite second moments. Write \(\mu=\mathbb{E}X_n\), \(Y_n=X_n-\mu\), and \(\gamma(h)=\operatorname{Cov}(X_{n+h},X_n)\) for \(h\ge0\), extending the covariance to negative lags by \(\gamma(-h)=\gamma(h)\). Under the Fourier convention used here, its finite spectral measure \(F\) on \([-\pi,\pi]\) satisfies \(\gamma(h)=\int_{-\pi}^{\pi}e^{ih\lambda}F(d\lambda)\), and \(F([-\pi,\pi])=\gamma(0)\). The existence of this representation is Herglotz’s theorem, established in the earlier material on spectral representation.
The constant mean contributes nothing to a difference of positive order, since the coefficients in the difference sum to zero. Thus \(\Delta^r X_n=\Delta^r Y_n\). The moment \(M_{2r}\) emphasizes frequencies farther from zero through the weight \(\lambda^{2r}\). The variance \(D_r\) also emphasizes nonzero frequencies, but with a different weight. The next result gives that weight exactly.
Proof. Fix \(r\ge1\), and put \(c_k=(-1)^{r-k}\binom{r}{k}\). Since the mean cancels, expanding the variance gives \(D_r=\sum_{j=0}^r\sum_{k=0}^r c_jc_k\gamma(k-j)\). The covariance representation applies to every integer lag, so substituting it and interchanging the finite sums with the integral gives \[ D_r=\int_{-\pi}^{\pi}\left(\sum_{k=0}^r c_ke^{ik\lambda}\right)\left(\sum_{j=0}^r c_je^{-ij\lambda}\right)F(d\lambda). \] The second factor is the complex conjugate of the first. By the binomial formula, the first factor is \((e^{i\lambda}-1)^r\); their product is therefore \(|e^{i\lambda}-1|^{2r}\). Also, \(|e^{i\lambda}-1|^2=(e^{i\lambda}-1)(e^{-i\lambda}-1)=2(1-\cos\lambda)\), proving the integral identity.
For the bounds, use \(|e^{i\lambda}-1|=2|\sin(\lambda/2)|\). On \(0\le x\le\pi/2\), \(\sin x\le x\), since \(x-\sin x\) has derivative \(1-\cos x\ge0\) and is zero at zero. Also \(\sin x\ge 2x/\pi\): the second derivative of \(\sin x\) is \(-\sin x\le0\), so its graph lies above the chord joining \((0,0)\) to \((\pi/2,1)\). Applying these inequalities at \(x=|\lambda|/2\) gives \(2|\lambda|/\pi\le |e^{i\lambda}-1|\le|\lambda|\). Raise to the power \(2r\) and integrate against the nonnegative measure \(F\) to obtain the bounds. For \(r=0\), both integrals equal \(F([-\pi,\pi])=\gamma(0)\), as claimed. \(\square\)
The bounds say that the order-\(r\) difference variance and the order-\(2r\) spectral moment measure the same overall frequency-weighted scale, up to constants depending only on \(r\). They are not generally equal: the difference weight is periodic in frequency and bounded, while \(\lambda^{2r}\) increases across the chosen frequency interval. In particular, an atom at frequency zero contributes to neither quantity when \(r\ge1\).
Worked Calculations
Worked Example: White Noise
Let \((X_n)\) be mean-zero white noise with variance \(\sigma^2>0\): its covariance is \(\gamma(0)=\sigma^2\) and \(\gamma(h)=0\) for every nonzero integer \(h\). Its spectral measure is \(F(d\lambda)=\frac{\sigma^2}{2\pi}\,d\lambda\), since integrating \(e^{ih\lambda}\) over \([-\pi,\pi]\) gives \(2\pi\) when \(h=0\) and zero for nonzero integer \(h\).
The second spectral moment is \[ M_2=\frac{\sigma^2}{2\pi}\int_{-\pi}^{\pi}\lambda^2\,d\lambda =\frac{\sigma^2}{2\pi}\left(\frac{2\pi^3}{3}\right) =\frac{\sigma^2\pi^2}{3}. \] For the first difference, the spectral formula gives \[ D_1=\frac{\sigma^2}{2\pi}\int_{-\pi}^{\pi}2(1-\cos\lambda)\,d\lambda =\frac{\sigma^2}{2\pi}(4\pi)=2\sigma^2, \] because the integral of \(\cos\lambda\) over this interval is zero.
Check by a direct covariance calculation: \(\Delta X_n=X_{n+1}-X_n\). The two terms have variance \(\sigma^2\) each and covariance zero, so \(D_1=\sigma^2+\sigma^2=2\sigma^2\), agreeing with the integral. The moment bound reads \(\frac{4}{\pi^2}\frac{\sigma^2\pi^2}{3}\le2\sigma^2\le\frac{\sigma^2\pi^2}{3}\), that is, \(\frac{4}{3}\sigma^2\le2\sigma^2\le\frac{\pi^2}{3}\sigma^2\).
Worked Example: A Pair of Spectral Lines
Suppose the spectral measure has mass \(2\) at each of \(\pi/3\) and \(-\pi/3\), and no other mass. Then, for every integer \(h\), \[ \gamma(h)=2e^{ih\pi/3}+2e^{-ih\pi/3}=4\cos(h\pi/3). \] In particular, \(\gamma(0)=4\), consistent with the total spectral mass \(2+2=4\).
For every \(r\ge1\), the spectral moment and difference variance are \[ M_{2r}=2\left(\frac{\pi}{3}\right)^{2r} +2\left(\frac{\pi}{3}\right)^{2r} =4\left(\frac{\pi}{3}\right)^{2r}, \qquad D_r=2(1)^r+2(1)^r=4. \] Here \(|e^{i\pi/3}-1|^2=2(1-\cos(\pi/3))=2(1-\tfrac12)=1\), and the same value holds at \(-\pi/3\). Thus, for example, \(M_2=4\pi^2/9\) while \(D_1=4\); the two values are not identical.
Check using covariances for the first difference: \(\gamma(0)=4\) and \(\gamma(1)=4\cos(\pi/3)=2\). Since \(\operatorname{Var}(X_{n+1}-X_n)=\gamma(0)+\gamma(0)-2\gamma(1)\), the direct calculation gives \(4+4-2(2)=4=D_1\). This also verifies that the total mass and the covariance formula use the same normalization.
Worked Example: A Zero-Frequency Component and an Oscillating Component
Let \(A,U,V\) be independent, mean-zero random variables with variances \(3,1,1\), respectively, and define \(X_n=A+U\cos(n\pi/2)+V\sin(n\pi/2)\). This is a mean-zero weakly stationary sequence: independence gives \[ \gamma(h)=\operatorname{Cov}(X_{n+h},X_n) =3+\cos(h\pi/2). \] Indeed, the two trigonometric terms contribute \(\cos((n+h)\pi/2)\cos(n\pi/2)+\sin((n+h)\pi/2)\sin(n\pi/2)=\cos(h\pi/2)\). Its spectral measure is \(F=3\delta_0+\tfrac12\delta_{\pi/2}+\tfrac12\delta_{-\pi/2}\), which yields \(3+\cos(h\pi/2)\) under the stated convention.
The mass at zero contributes \(3\) to \(M_0=\gamma(0)=4\), but contributes nothing to \(M_{2r}\) or \(D_r\) for \(r\ge1\). The other two masses give \[ M_{2r}=\left(\frac{\pi}{2}\right)^{2r}, \qquad D_r=\tfrac12(2)^r+\tfrac12(2)^r=2^r, \] because \(|e^{i\pi/2}-1|^2=2(1-\cos(\pi/2))=2\), with the same value at \(-\pi/2\). In particular, \(M_2=\pi^2/4\), \(D_1=2\), and \(D_2=4\).
Check by covariance differences: \(\gamma(0)=4\), \(\gamma(1)=3\), and \(\gamma(2)=2\). Thus \(D_1=2\gamma(0)-2\gamma(1)=8-6=2\). For the second difference, whose coefficients are \(1,-2,1\), expansion gives \(D_2=6\gamma(0)-8\gamma(1)+2\gamma(2)=24-24+4=4\). Both direct calculations agree with the spectral formula.
What the Differences Say About Smoothness
For a fixed order \(r\), a small \(D_r\) means that the weighted spectral energy under \(|e^{i\lambda}-1|^{2r}\) is small. Frequencies near zero receive small weight, while frequencies away from zero receive more. Increasing \(r\) strengthens this contrast: the multiplier is raised to a higher power. The moment bounds make this interpretation precise by comparing the difference variance to \(M_{2r}\).
There are two qualifications. First, discrete-time frequencies are restricted to a bounded interval, so every moment \(M_{2r}\) is finite when the process has finite variance. A finite moment here is not, by itself, a delicate regularity condition. Second, a small difference variance is a statement about a particular order of differences at unit spacing; it should not be confused with a claim about a continuous-time derivative or a limit as the spacing tends to zero.
Common mistake. A tempting but incorrect calculation is to replace \(|e^{i\lambda}-1|^{2r}\) by \(\lambda^{2r}\) in the exact formula and conclude that \(D_r=M_{2r}\). The two weights are comparable but not equal. For the spectral lines at \(\pm\pi/3\), the order-one difference weight is \(1\) at both atoms, so \(D_1=4\); the moment weight is \((\pi/3)^2\), so \(M_2=4\pi^2/9\). The correction is to use the exact trigonometric multiplier for \(D_r\), and use \(\lambda^{2r}\) only for the spectral moment and the proven bounds.
Another useful check is the zero-frequency component in the third example. It contributes to the variance but disappears from every positive-order difference. Forgetting this can lead to the mistaken conclusion that \(D_r\) measures all of \(\gamma(0)\). It measures variation after applying the difference, and a constant-in-time component is annihilated by that operation.
Check Your Understanding
Use the spectral formula and, where possible, verify the answer from covariances.
- For a weakly stationary sequence with spectral measure \(F\), write the integral expression for the variance of its second forward difference.
- White noise has variance \(\sigma^2\). Compute \(M_2\) from its uniform spectral measure and compare it with the variance of its first difference.
- A spectral measure has masses \(1\) at each of \(\pi/2\) and \(-\pi/2\), and no other mass. Find \(M_2\) and \(D_1\).
- Why does a spectral atom at frequency zero contribute to \(\gamma(0)\) but not to \(D_r\) for \(r\ge1\)?
- State the two-sided inequality relating \(D_r\) and \(M_{2r}\), and identify which trigonometric bounds give it.