Weak Stationarity: What Must Stay the Same?
A process can change over time in its individual observations while keeping a stable pattern in its first two moments. Weak stationarity formalizes that idea: the mean does not depend on time, and the covariance between two observations depends on their separation, not on where the pair occurs. This is a moment-based condition; it does not, by itself, say that all features of the joint distributions are unchanged by shifting time.
Let \((\Omega,\mathcal{F},\mathbb{P})\) be a probability space, and let \((X_n)_{n\ge 0}\) be a real-valued discrete-time process. Assume each \(X_n\) is square-integrable, meaning \(\mathbb{E}[X_n^2]<\infty\). This assumption ensures that its mean and covariances are finite and defined. Write \(\operatorname{Cov}(X_n,X_m)=\mathbb{E}[(X_n-\mathbb{E}X_n)(X_m-\mathbb{E}X_m)]\).
For a real-valued process, covariance is symmetric: \(\operatorname{Cov}(X_n,X_m)=\operatorname{Cov}(X_m,X_n)\). Consequently, the autocovariance at a negative lag equals the autocovariance at the corresponding positive lag. We use the convention \(\gamma(h)=\operatorname{Cov}(X_{n+h},X_n)\) whenever \(n\ge0\) and \(n+h\ge0\). Under weak stationarity, the right side does not depend on the eligible choice of \(n\).
These equations summarize exactly what the mean and autocovariance tell us: the mean is the same at every time, every variance is the lag-zero autocovariance, and every covariance is indexed by the time difference. The autocovariance function alone does not determine the mean. Two processes can have the same autocovariance function and different constant means. Thus the complete first- and second-moment description consists of \(\mu\) together with \(\gamma\).
Proof. Suppose first that the process is weakly stationary. Its mean is constant by definition. The covariance \(\operatorname{Cov}(X_{n+h},X_n)\) depends only on the difference between its time indices, which is \(h\). Setting \(n=0\) shows that this covariance equals \(\operatorname{Cov}(X_h,X_0)\), as asserted. Conversely, suppose the mean is constant and the displayed covariance identity holds for all \(n,h\ge0\). For any \(a,b\ge0\) with \(a\ge b\), write \(a=b+h\), where \(h=a-b\ge0\). The assumed identity gives \(\operatorname{Cov}(X_a,X_b)=\operatorname{Cov}(X_h,X_0)\), which depends only on \(a-b\). If \(b>a\), symmetry of covariance gives \(\operatorname{Cov}(X_a,X_b)=\operatorname{Cov}(X_b,X_a)\), which depends only on \(b-a\). Thus every covariance depends only on the time difference, and both conditions in the definition hold. Finally, symmetry gives \(\gamma(-h)=\gamma(h)\). \(\square\)
Weak stationarity is distinct from strict stationarity. Strict stationarity requires the entire finite-dimensional distributions to be unchanged by valid time shifts. Weak stationarity requires only time-invariant means and lag-dependent covariances. In particular, checking variances alone is insufficient: covariances between different times must also have the required lag pattern.
Examples: Checking the Conditions
Worked Example: A Random Periodic Sequence
Let \(A\) and \(B\) be independent random variables with \(\mathbb{P}(A=2)=\mathbb{P}(A=-2)=\tfrac12\) and \(\mathbb{P}(B=2)=\mathbb{P}(B=-2)=\tfrac12\). For \(n\ge0\), define \(X_n=3+A\cos(n\pi/3)+B\sin(n\pi/3)\). Find its mean and autocovariance, and verify weak stationarity.
The two signs in each variable have equal probability, so \(\mathbb{E}[A]=\mathbb{E}[B]=0\). Also, \(\mathbb{E}[A^2]=\mathbb{E}[B^2]=4\), and independence gives \(\operatorname{Cov}(A,B)=0\). Therefore
The last equality is the cosine difference identity. The mean does not depend on \(n\), and the covariance depends on the lag \(h\) only, so the process is weakly stationary. Its lag values at \(h=0,1,2,3\) are \(\gamma(0)=4\cos(0)=4\), \(\gamma(1)=4\cos(\pi/3)=2\), \(\gamma(2)=4\cos(2\pi/3)=-2\), and \(\gamma(3)=4\cos(\pi)=-4\).
Check: At lag zero, the variance is \(4\), as given by \(\gamma(0)\). At lag one, \(\operatorname{Cov}(X_1,X_0)=4\cos(\pi/3)=2\), and \(\operatorname{Cov}(X_2,X_1)=4\cos(\pi/3)=2\) as well. The equal covariances at these different locations agree with the lag-only formula.
Worked Example: Why a Lag Pattern Alone Is Not Enough
Let \((\varepsilon_n)_{n\ge0}\) be independent random variables with \(\mathbb{P}(\varepsilon_n=1)=\mathbb{P}(\varepsilon_n=-1)=\tfrac12\). First define \(X_n=2n+\varepsilon_n\). Then define \(Y_n=n\varepsilon_n\). Determine which weak-stationarity conditions hold for each sequence.
For each \(n\), \(\mathbb{E}[\varepsilon_n]=0\) and \(\operatorname{Var}(\varepsilon_n)=\mathbb{E}[\varepsilon_n^2]=1\). Thus \(\mathbb{E}[X_n]=2n\), which is not constant: for example, \(\mathbb{E}[X_0]=0\) and \(\mathbb{E}[X_1]=2\). Independence gives \(\operatorname{Cov}(X_n,X_m)=0\) when \(n\ne m\), while \(\operatorname{Var}(X_n)=1\). So the covariance pattern depends only on lag, but \(X_n\) is not weakly stationary because its mean changes.
For \(Y_n=n\varepsilon_n\), the mean is \(\mathbb{E}[Y_n]=n\mathbb{E}[\varepsilon_n]=0\) at every time. Its variance is \(\operatorname{Var}(Y_n)=n^2\operatorname{Var}(\varepsilon_n)=n^2\). In particular, \(\operatorname{Var}(Y_0)=0\) and \(\operatorname{Var}(Y_1)=1\). Since variance is the covariance at lag zero, the covariance condition fails even though the mean is constant.
Check: The first sequence fails the constant-mean condition; the second fails the lag-zero covariance condition. Neither is weakly stationary. These computations show why both parts of the definition must be checked, including the mean condition and the covariance condition.
Using the Autocovariance Function
Once a process is known to be weakly stationary, \(\mu\) and \(\gamma\) give the mean and variance of any finite linear combination of its observations. This is useful in forecasting, filtering, and analyzing averages, because a potentially long covariance matrix can be filled from a one-dimensional list of lag values.
Proof. Linearity of expectation and the constant-mean condition give \(\mathbb{E}[L]=\sum_{i=0}^r c_i\mathbb{E}[X_i]=\mu\sum_{i=0}^r c_i\). For the variance, subtract this mean to get \(L-\mathbb{E}[L]=\sum_{i=0}^r c_i(X_i-\mu)\). Expanding the square inside the expectation gives \(\operatorname{Var}(L)=\sum_{i=0}^r\sum_{j=0}^r c_ic_j\mathbb{E}[(X_i-\mu)(X_j-\mu)]\). Each expectation is \(\operatorname{Cov}(X_i,X_j)=\gamma(i-j)\) by weak stationarity and the lag convention. Substitution proves the formula. The sums are finite, so their expansion and rearrangement require no interchange of infinite sums. \(\square\)
Worked Example: Variance of a Weighted Sum
Let \((\varepsilon_n)_{n\in\mathbb{Z}}\) be independent, mean-zero random variables with variance \(2\), and define \(X_n=\varepsilon_{n-1}+\varepsilon_n\) for \(n\ge0\). Compute the autocovariance function and then find the variance of \(L=X_0+2X_1-X_2\).
Each \(X_n\) has mean zero. Its variance is the sum of the variances of its two independent terms: \(\operatorname{Var}(X_n)=2+2=4\). If \(h=1\), \(X_{n+1}=\varepsilon_n+\varepsilon_{n+1}\) and \(X_n=\varepsilon_{n-1}+\varepsilon_n\) share exactly the term \(\varepsilon_n\). The covariance is therefore \(\operatorname{Var}(\varepsilon_n)=2\). If \(h\ge2\), the two sums use disjoint independent variables, so their covariance is zero. Hence \(\gamma(0)=4\), \(\gamma(1)=\gamma(-1)=2\), and \(\gamma(h)=0\) for \(|h|\ge2\). The mean is constant and the covariance depends only on lag, so this is weakly stationary.
For \(L\), the coefficients at indices \(0,1,2\) are \(1,2,-1\). The diagonal terms in the variance formula contribute \((1^2+2^2+(-1)^2)\gamma(0)=6\cdot4=24\). The pair \((0,1)\) contributes \(2(1)(2)\gamma(1)=4\cdot2=8\); the pair \((0,2)\) contributes \(2(1)(-1)\gamma(2)=0\); and the pair \((1,2)\) contributes \(2(2)(-1)\gamma(1)=-4\cdot2=-8\). Thus \(\operatorname{Var}(L)=24+8+0-8=24\).
Check by direct expansion: \(X_0=\varepsilon_{-1}+\varepsilon_0\), \(2X_1=2\varepsilon_0+2\varepsilon_1\), and \(-X_2=-\varepsilon_1-\varepsilon_2\). Therefore \(L=\varepsilon_{-1}+3\varepsilon_0+\varepsilon_1-\varepsilon_2\). Independence gives \(\operatorname{Var}(L)=(1^2+3^2+1^2+(-1)^2)\cdot2=12\cdot2=24\), matching the autocovariance calculation.
Common Pitfall and Practical Use
A common mistake is to check that each observation has the same variance and then conclude that the process is weakly stationary. Constant variance checks only \(\gamma(0)\); it says nothing about whether the covariance at lag one, two, or any other lag changes with time. For example, a process may have \(\operatorname{Var}(X_n)=1\) for every \(n\), but \(\operatorname{Cov}(X_n,X_{n+1})\) may differ between \(n=0\) and \(n=1\). It would then fail weak stationarity.
Another error is to call the mean part of the autocovariance function. The autocovariance is centered: it uses \(X_n-\mathbb{E}[X_n]\). Adding a fixed constant to every \(X_n\) changes the mean but leaves every covariance unchanged. Accordingly, report the mean \(\mu\) separately from \(\gamma\).
In applications, the lag representation is valuable because it separates location from dependence. The constant mean describes the process’s level, while \(\gamma(h)\) describes how observations \(h\) steps apart vary together. The identities \(\operatorname{Var}(X_n)=\gamma(0)\) and \(\operatorname{Cov}(X_n,X_m)=\gamma(m-n)\) are immediate once weak stationarity has been established; they should not be used as assumptions before checking the definition.
Check Your Understanding
For each question, state which weak-stationarity condition is being used and verify calculations from the definition.
- If \(\gamma(0)=5\), what is \(\operatorname{Var}(X_7)\) for a weakly stationary process?
- A process has constant mean and \(\operatorname{Cov}(X_n,X_{n+2})=3+n\). Is it weakly stationary? Explain.
- For a weakly stationary process with \(\gamma(0)=4\) and \(\gamma(1)=1\), compute \(\operatorname{Var}(X_0+X_1)\).
- Why can two weakly stationary processes have the same autocovariance function but different means?
- In the weighted-sum formula, why must both positive and negative lags be allowed?