Identities That Every Autocovariance Sequence Must Satisfy
A weakly stationary process cannot have arbitrary covariances across its time points. Reversing the order of two variables leaves their covariance unchanged, and the Cauchy–Schwarz inequality limits the size of a covariance by the variables’ variances. Applying these facts to pairs of process values and to their differences gives useful restrictions on autocovariances. These restrictions can rule out proposed covariance values before any more extensive analysis.
Let \((X_n)_{n\in\mathbb{Z}}\) be a real-valued process on a probability space \((\Omega,\mathcal{F},\mathbb{P})\), with \(\mathbb{E}[X_n^2]<\infty\) for every \(n\). It is weakly stationary if its mean is constant, \(\mathbb{E}[X_n]=m\), and its covariance depends only on the time difference. Use the convention
where the right-hand side is independent of \(n\). The function \(\gamma\) on the integers is the autocovariance sequence. Its value at zero is the common variance, \(\gamma(0)=\operatorname{Var}(X_n)\), and is nonnegative. The process need not be Gaussian: these identities require only weak stationarity and finite second moments.
Proof. Covariance is symmetric: \(\operatorname{Cov}(U,V)=\operatorname{Cov}(V,U)\) for any square-integrable real random variables \(U,V\). Therefore \(\gamma(-h)=\operatorname{Cov}(X_{n-h},X_n)=\operatorname{Cov}(X_n,X_{n-h})=\gamma(h)\), using stationarity to identify the last covariance with lag \(h\). For the bound, Cauchy–Schwarz gives \(|\operatorname{Cov}(X_{n+h},X_n)|\leq\sqrt{\operatorname{Var}(X_{n+h})\operatorname{Var}(X_n)}=\gamma(0)\). Expanding the variance of a difference gives \(\operatorname{Var}(X_{n+h})+\operatorname{Var}(X_n)-2\operatorname{Cov}(X_{n+h},X_n)=2(\gamma(0)-\gamma(h))\); expanding the variance of a sum changes the minus sign to a plus sign. These variances are nonnegative, so in particular \(-\gamma(0)\leq\gamma(h)\leq\gamma(0)\). \(\square\)
The Cauchy–Schwarz inequality used here follows directly from nonnegative squared deviations. For centered, square-integrable random variables \(U,V\), if \(\mathbb{E}[U^2]=0\), then \(U=0\) almost surely and \(\mathbb{E}[UV]=0\). Otherwise \(\mathbb{E}[(V-tU)^2]\geq0\) for every real \(t\). The resulting quadratic in \(t\) cannot have a positive discriminant, which gives \((\mathbb{E}[UV])^2\leq\mathbb{E}[U^2]\mathbb{E}[V^2]\). Applying this to centered variables gives the covariance inequality, including the zero-variance case.
One important consequence is that if \(\gamma(0)=0\), then \(\gamma(h)=0\) at every lag. Indeed, the bound gives \(|\gamma(h)|\leq0\). More strongly, each \(X_n\) equals the constant \(m\) almost surely because it has variance zero. The almost-sure equality can be asserted for all integer \(n\) simultaneously: there are countably many indices, and the union of countably many probability-zero exceptional events still has probability zero.
Constraints from Differences
Define a lag-\(h\) difference by \(D_{n,h}=X_{n+h}-X_n\). The variance identity in the theorem says how much variability there is in this difference. In particular, \(\gamma(h)\leq\gamma(0)\), so the difference variance cannot be negative. When two differences are considered together, Cauchy–Schwarz produces further restrictions involving several lags.
Proof. Stationarity and covariance symmetry give \(\operatorname{Var}(U)=2(\gamma(0)-\gamma(h))\) and \(\operatorname{Var}(V)=2(\gamma(0)-\gamma(k))\). Expanding their covariance term by term gives \(\operatorname{Cov}(U,V)=\operatorname{Cov}(X_{n+h},X_{n+h+k})-\operatorname{Var}(X_{n+h})-\operatorname{Cov}(X_n,X_{n+h+k})+\operatorname{Cov}(X_n,X_{n+h})\). The four terms are respectively \(\gamma(k)\), \(\gamma(0)\), \(\gamma(h+k)\), and \(\gamma(h)\). Thus \(\operatorname{Cov}(U,V)=\gamma(k)-\gamma(0)-\gamma(h+k)+\gamma(h)\). Cauchy–Schwarz bounds its absolute value by \(\sqrt{\operatorname{Var}(U)\operatorname{Var}(V)}\), which equals the stated right-hand side. The factors under the square root are nonnegative by the variance identity, so the bound is defined, including when either factor is zero. \(\square\)
This inequality is a necessary condition, not a recipe that guarantees a proposed list of autocovariances is valid. It is useful because it turns known values at two lags into a restriction on another lag. The examples show how to apply it and how to check the arithmetic.
Worked Example: Symmetry and the Bound at an Odd Lag
Let \(A\) and \(B\) be independent, mean-zero random variables with variances \(1\) and \(3\), respectively. On their probability space, define \(X_n=A+(-1)^nB\) for \(n\in\mathbb{Z}\). Find \(\gamma(0)\), \(\gamma(3)\), and the variances of \(X_{n+3}-X_n\) and \(X_{n+3}+X_n\).
The mean is zero for every \(n\). Independence and the zero means make the cross-covariances vanish, so \(\operatorname{Cov}(X_{n+h},X_n)=\operatorname{Var}(A)+(-1)^{n+h}(-1)^n\operatorname{Var}(B)=1+3(-1)^h\). At lag zero, \(\gamma(0)=1+3=4\). Since \((-1)^3=-1\), \(\gamma(3)=1-3=-2\), and symmetry gives \(\gamma(-3)=-2\) as well.
The difference variance is \(2(4-(-2))=12\), while the sum variance is \(2(4+(-2))=4\). Directly, \(X_{n+3}-X_n=-2(-1)^nB\), whose variance is \(4\cdot3=12\); and \(X_{n+3}+X_n=2A\), whose variance is \(4\cdot1=4\). Both direct calculations agree with the identities, and both variances are nonnegative.
Worked Example: Constraining the Covariance at Lag Two
Suppose a weakly stationary process has \(\gamma(0)=4\) and \(\gamma(1)=3\). What does the successive-increment inequality require of \(\gamma(2)\)? Then check the restriction for the stationary process \(X_n=\sum_{j=0}^{\infty}(3/4)^j\varepsilon_{n-j}\), where the \(\varepsilon_n\) are independent, mean-zero random variables of variance \(7/4\).
Set \(h=k=1\) in the theorem. The left side becomes \(\left|\gamma(1)-\gamma(0)-\gamma(2)+\gamma(1)\right|=|3-4-\gamma(2)+3|=|2-\gamma(2)|\). The right side is \(2\sqrt{(4-3)(4-3)}=2\). Hence \(|2-\gamma(2)|\leq2\), or, after adding \(\gamma(2)\) and then subtracting \(2\), \(0\leq\gamma(2)\leq4\).
For the proposed process, the infinite sum converges in mean square because the sum of the variances of its independent terms is \((7/4)\sum_{j=0}^{\infty}(9/16)^j<\infty\). Its variance is \((7/4)/(1-9/16)=4\). For a positive lag \(h\), shared innovations in the two sums give \(\gamma(h)=(3/4)^h\gamma(0)\): each shared term in \(X_{n+h}\) has coefficient \((3/4)^h\) times its coefficient in \(X_n\), and nonshared independent terms have covariance zero. Thus \(\gamma(1)=(3/4)4=3\) and \(\gamma(2)=(9/16)4=9/4\). Substitution gives \(|2-9/4|=1/4\leq2\), and \(9/4\) lies in the required interval.
Worked Example: A Three-Lag Constraint from Successive Increments
For the process in the previous example, use \(h=2\) and \(k=1\) to constrain \(\gamma(3)\), then verify that its computed value satisfies the constraint.
Here \(\gamma(0)=4\), \(\gamma(1)=3\), and \(\gamma(2)=9/4\). The left side of the inequality is \(\left|\gamma(1)-\gamma(0)-\gamma(3)+\gamma(2)\right|=|3-4-\gamma(3)+9/4|=|5/4-\gamma(3)|\). The right side is \(2\sqrt{(4-9/4)(4-3)}=2\sqrt{(7/4)(1)}=\sqrt{7}\). Consequently \(|5/4-\gamma(3)|\leq\sqrt{7}\), equivalently \(5/4-\sqrt{7}\leq\gamma(3)\leq5/4+\sqrt{7}\).
The process formula gives \(\gamma(3)=(3/4)^3\cdot4=27/16\). Its distance from \(5/4\) is \(|5/4-27/16|=|20/16-27/16|=7/16\). Since both sides are nonnegative and \((7/16)^2=49/256\leq7=(\sqrt7)^2\), the required inequality holds.
Common Mistake: Reversing a Covariance
A common error is to change the sign of a covariance when reversing the order of its two variables. For example, suppose someone proposes \(\gamma(0)=1\) and \(\gamma(1)=3\), then argues that \(\operatorname{Cov}(X_n,X_{n+1})=-3\) because the lag has been reversed. That is incorrect: \(\operatorname{Cov}(X_n,X_{n+1})=\operatorname{Cov}(X_{n+1},X_n)=\gamma(1)=3\). The proposal itself is impossible, since Cauchy–Schwarz requires \(|\gamma(1)|\leq\gamma(0)\), which would say \(3\leq1\). Equivalently, the proposed values give \(\operatorname{Var}(X_{n+1}-X_n)=2(1-3)=-4\), an impossible negative variance.
Keep in mind that these identities supply constraints, not a complete test for whether a whole sequence is an autocovariance sequence. Checking symmetry, individual lag bounds, and selected increment inequalities is valuable, but passing those checks alone does not establish that a proposed sequence comes from a stationary process.
Check Your Understanding
Use the identities and inequalities above. When testing a proposed value, distinguish a necessary bound from a proof that the full sequence is valid.
- If \(\gamma(0)=5\) and \(\gamma(4)=-2\), what are \(\gamma(-4)\), \(\operatorname{Var}(X_{n+4}-X_n)\), and \(\operatorname{Var}(X_{n+4}+X_n)\)?
- If \(\gamma(0)=9\) and \(\gamma(2)=6\), what interval does the basic bound impose on \(\gamma(2)\), and what variance does it give for \(X_{n+2}-X_n\)?
- For \(\gamma(0)=4\), \(\gamma(1)=2\), and \(\gamma(2)=1\), substitute \(h=k=1\) into the successive-increment inequality and check whether it holds.
- Explain why \(\gamma(0)=0\) forces every autocovariance value to be zero.
- Does satisfying the basic symmetry and lag bounds prove that a proposed entire sequence is an autocovariance sequence? Explain briefly.