White Noise, Geometric, and Periodic Covariances

Three Basic Covariance Patterns

Autocovariance records how a process at one time is linearly associated with the same process at another time. Some familiar models produce distinctive patterns: white noise has no covariance across distinct times, a stationary AR(1) process has a geometric pattern, and a random-phase sinusoid has an oscillating pattern. Computing these sequences makes clear which model parameters control variance, persistence, sign changes, and periodicity.

Throughout, processes are real-valued and have finite second moments. A process \((X_n)_{n\ge0}\) on a probability space \((\Omega,\mathcal{F},\mathbb{P})\) is weakly stationary if its mean is constant in \(n\) and its covariance \(\operatorname{Cov}(X_m,X_n)\) depends only on \(m-n\). Its autocovariance sequence is written \(\gamma(h)\), where \(\gamma(h)=\operatorname{Cov}(X_{n+h},X_n)\) whenever both time indices are nonnegative. For a real-valued weakly stationary process, \(\gamma(-h)=\gamma(h)\).

White Noise: Covariance Only at Lag Zero

For white noise, \(\gamma(0)=\operatorname{Var}(\varepsilon_n)=\sigma^2\), while \(\gamma(h)=0\) for \(h\ne0\). Thus its covariance sequence is concentrated at lag zero. This says nothing by itself about whether the variables are independent or normally distributed; those are additional assumptions, not consequences of the covariance pattern.

Stationary AR(1): Geometric Decay

Consider the first-order autoregression \(X_n=\phi X_{n-1}+\varepsilon_n\), where \(\phi\) is a real coefficient and the innovations \((\varepsilon_n)\) have mean zero, variance \(\sigma^2>0\), and are uncorrelated at distinct times. For the usual causal stationary version, assume \(|\phi|<1\) and take the innovations to be indexed by all integers. Define, for \(n\ge0\),

The series converges in mean square: the squared mean-square distance between partial sums with terms through indices \(r\) and \(s\), where \(s>r\), is \(\sigma^2\sum_{j=r+1}^{s}\phi^{2j}\), which tends to zero as \(r\) tends to infinity because \(|\phi|<1\). The resulting process has constant mean zero. Its covariance is geometric, as the following result establishes.

Proof. It is enough first to compute \(\gamma(h)\) for \(h\ge0\). Using finite partial sums and then taking their mean-square limits, uncorrelatedness makes every product of distinct innovations have expectation zero. In the product \(X_{n+h}X_n\), the term \(\varepsilon_{n+h-j}\) matches a term \(\varepsilon_{n-k}\) exactly when \(j=h+k\). Therefore

The geometric series converges because \(|\phi|<1\). For a real weakly stationary process, covariance is symmetric in its two time arguments, so \(\gamma(-h)=\gamma(h)\); replacing \(h\ge0\) by \(|h|\) gives the stated formula for every integer lag. The formula depends on the lag alone, and the mean is constant, so the process is weakly stationary. Dividing by \(\gamma(0)=\sigma^2/(1-\phi^2)\) gives the autocorrelation \(\phi^{|h|}\). \(\square\)

The coefficient \(\phi\) controls the lag pattern. If \(0<\phi<1\), covariances are positive and decay geometrically. If \(-1<\phi<0\), their signs alternate while their magnitudes decay geometrically. If \(\phi=0\), the process reduces to white noise and all nonzero-lag covariances vanish. For the recursion with innovations uncorrelated with \(X_{n-1}\), the restriction \(|\phi|<1\) is necessary for a stationary solution with finite positive variance when \(\sigma^2>0\): stationarity and the recursion imply \(v=\phi^2v+\sigma^2\), where \(v=\operatorname{Var}(X_n)>0\), hence \(1-\phi^2=\sigma^2/v>0\).

Random-Phase Sinusoids: Oscillating Covariance

A sinusoid with a fixed, nonrandom phase is generally not weakly stationary: its mean can vary with time. Randomizing the phase uniformly removes that time dependence from the first two moments. Let \(U\) be uniform on \([0,2\pi)\) and define \(X_n=A\cos(\omega n+U)\), where \(A\) and \(\omega\) are fixed real numbers. The state space is \(\mathbb{R}\) and the time index is \(n\ge0\).

Proof. Uniformity of \(U\) gives \(\mathbb{E}[\cos(\omega n+U)]=0\), since the integral of cosine over a full period is zero. Thus \(\mathbb{E}[X_n]=0\). For times \(n\) and \(n+h\), use \(\cos a\cos b=\tfrac12\cos(a-b)+\tfrac12\cos(a+b)\). The term involving \(a+b\) has expectation zero because it contains \(2U\) with a uniform phase over a full cycle; the other term is constant in \(U\). Therefore \(\mathbb{E}[X_{n+h}X_n]=\tfrac{A^2}{2}\cos(\omega h)\). This depends only on \(h\), proving weak stationarity and the covariance formula. If \(A=0\), the formula is zero at every lag, so every positive integer is a period. Now suppose \(A\ne0\). If \(q\omega=2\pi k\) for an integer \(k\), then \(\cos(\omega(h+q))=\cos(\omega h+2\pi k)=\cos(\omega h)\), so \(q\) is a period. Conversely, if \(q\) is a period, equality at lag \(h=0\) gives \(\cos(q\omega)=\cos(0)=1\), which holds exactly when \(q\omega=2\pi k\) for some integer \(k\). This proves both directions. \(\square\)

When \(A\ne0\), an integer period exists exactly when \(\omega/(2\pi)\) is rational. If this ratio is \(p/r\) in lowest terms, with \(r>0\), the smallest positive integer period is \(r\): the condition \(q p/r\in\mathbb{Z}\) requires \(r\) to divide \(q\). If the ratio is irrational, no positive integer period exists. These statements concern the covariance sequence; they do not assert that each sample path repeats at an integer period.

Reading and Comparing the Lag Patterns

The three models can be distinguished by the shape of their autocovariances. White noise has only a lag-zero value. The causal stationary AR(1) covariance has geometrically changing magnitude, with its sign pattern determined by \(\phi\). The random-phase sinusoid has a cosine pattern; its amplitude sets the overall covariance scale and its frequency sets the oscillation across integer lags. A zero amplitude is an important degenerate case: it gives the zero process and zero covariance, whatever frequency is written in the model.

Common mistake: using \(\phi^h\) at negative lags. In the AR(1) example, writing \(\gamma(-1)=\gamma(0)\phi^{-1}=4(-2)=-8\) is wrong. The formula \(\gamma(h)=\gamma(0)\phi^h\) was derived for \(h\ge0\), and extending it that way to negative lags can even produce a covariance larger in magnitude than the variance. Covariance symmetry instead gives \(\gamma(-1)=\gamma(1)=-2\); the correct all-lag formula uses \(\phi^{|h|}\). A second useful check is the zero-amplitude sinusoid: when \(A=0\), \(\gamma(h)=0\) for all \(h\), so every positive integer is a covariance period, regardless of \(\omega\).