Weak Stationarity as Mean and Covariance Invariance

Why Time-Invariance Matters

A time series can exhibit dependence across observations without having the same statistical behavior at every date. For example, its average level may drift, its variability may grow, or the covariance between two observations may depend on when they occurred, not just how far apart they were. These changes matter because many time-series summaries and models rely on a stable pattern over time.

Weak stationarity gives a precise, moment-based meaning to that stability. It requires a constant mean and a covariance between two observations that depends only on the time separating them. The definition does not require observations at different dates to be independent, and it does not assert that all features of their distributions are unchanged.

Definition: A process $\{X_t\}$, indexed by integer time $t$, is weakly stationary (also called covariance stationary) if each $X_t$ has a finite second moment and:

$E[X_t]=\mu$ for the same finite $\mu$ at every $t$; and
$\operatorname{Cov}(X_t,X_{t-h})=\gamma(h)$ for every time $t$ and lag $h$, where $\gamma(h)$ is finite and does not depend on $t$.

The second condition includes constant variance: setting $h=0$ gives $\operatorname{Var}(X_t)=\gamma(0)$ at every date. For nonzero lags, the covariance may be nonzero; weak stationarity permits dependence as long as the covariance pattern is stable under shifts in time.

What Covariance Invariance Says

The lag $h$ is a separation in time, not a particular date. For example, the covariance between observations at dates 12 and 9 has lag 3, just as the covariance between dates 40 and 37 does. Weak stationarity requires these covariances to be equal. Using the mean $\mu$, the autocovariance at lag $h$ is

\gamma(h)=\operatorname{Cov}(X_t,X_{t-h}) =E[(X_t-\mu)(X_{t-h}-\mu)].

Because covariance is symmetric, $\gamma(-h)=\gamma(h)$. The variance is the special case $\gamma(0)$, so $\gamma(0)$ is often called the variance of the process. The autocovariance function is the collection of these values across lags. This definition concerns population moments; a finite observed series can suggest whether the conditions are plausible but cannot by itself establish them for every date.

A useful way to read the definition is to compare any two pairs of dates with the same separation. If their covariance differs, the process is not weakly stationary. Conversely, checking only that the mean is constant is insufficient: the variance could change over time, or the covariance at a fixed lag could vary with the date.

Worked Example: Classifying White Noise

Suppose a hypothetical process $\{\varepsilon_t\}$ has $E[\varepsilon_t]=0$, $\operatorname{Var}(\varepsilon_t)=9$ units squared at every date, and $\operatorname{Cov}(\varepsilon_t,\varepsilon_s)=0$ whenever $t\ne s$. Does it satisfy weak stationarity?

The process has a constant finite mean, $\mu=0$. At lag zero, its autocovariance is its variance:

\gamma(0)=\operatorname{Cov}(\varepsilon_t,\varepsilon_t)=9.

For any nonzero lag $h$, the two dates $t$ and $t-h$ are distinct, so the assumed covariance is zero:

\gamma(h)=\operatorname{Cov}(\varepsilon_t,\varepsilon_{t-h})=0,\qquad h\ne 0.

Both the mean and every autocovariance are unchanged when $t$ shifts. This process is weakly stationary, with autocovariance function equal to 9 at lag zero and 0 at every nonzero lag. The conclusion follows from the stated moment conditions; the example does not require a particular distributional shape.

A Stationary AR(1) Process

An autoregressive process of order one, or AR(1), relates the current value to the previous value. Consider

X_t=c+\phi X_{t-1}+\varepsilon_t,

where $c$ is a constant, $\phi$ is a coefficient, and $\varepsilon_t$ is a mean-zero innovation with variance $\sigma_\varepsilon^2$. For the standard stationary construction, assume $|\phi|<1$, the innovations are uncorrelated across dates, and each innovation is uncorrelated with past innovations. The process is taken to have its stationary, infinite-past construction rather than an arbitrary starting value.

Under these conditions the centered process can be represented as a convergent sum of past innovations. Its mean and autocovariances can be calculated directly, which verifies the stationarity conditions rather than relying only on the label “AR(1).”

Result: For the stationary AR(1) process with $|\phi|<1$, mean-zero innovations of variance $\sigma_\varepsilon^2$, and the stated uncorrelatedness conditions, $$ \mu=\frac{c}{1-\phi},\qquad \gamma(h)=\frac{\sigma_\varepsilon^2}{1-\phi^2}\phi^{|h|}. $$

To justify the result, subtract the proposed mean $\mu=c/(1-\phi)$ from both sides of the AR(1) equation. Writing $Y_t=X_t-\mu$ gives $Y_t=\phi Y_{t-1}+\varepsilon_t$. Repeated substitution yields $Y_t=\varepsilon_t+\phi\varepsilon_{t-1}+\phi^2\varepsilon_{t-2}+\cdots$. Since $|\phi|<1$, the squared coefficients have finite sum, and this representation has a finite variance.

Every innovation has mean zero, so $E[Y_t]=0$ and $E[X_t]=\mu$. In the variance calculation, cross-products of distinct innovations have zero expectation under the uncorrelatedness assumptions. Therefore:

\operatorname{Var}(X_t)=\operatorname{Var}(Y_t) =\sigma_\varepsilon^2\sum_{j=0}^{\infty}\phi^{2j} =\frac{\sigma_\varepsilon^2}{1-\phi^2}.

For $h\ge 0$, the terms shared by $Y_t$ and $Y_{t-h}$ are the innovations dated $t-h,t-h-1,\ldots$. The coefficient on $\varepsilon_{t-h-j}$ in $Y_t$ is $\phi^{h+j}$, while its coefficient in $Y_{t-h}$ is $\phi^j$. All nonshared cross-products have zero covariance. Thus

\operatorname{Cov}(Y_t,Y_{t-h}) =\sigma_\varepsilon^2\sum_{j=0}^{\infty}\phi^{h+j}\phi^j =\frac{\sigma_\varepsilon^2\phi^h}{1-\phi^2}.

For negative $h$, covariance symmetry gives the same expression with $|h|$. Neither the mean nor this covariance depends on $t$, establishing weak stationarity.

Worked Example: Checking a Stationary AR(1) Model

Consider a hypothetical AR(1) process with $c=1$ outcome unit, $\phi=0.6$, and innovations of mean zero and variance $4$ outcome units squared. Assume the standard stationary construction and uncorrelatedness conditions stated above. Find its mean and autocovariances at lags 0, 1, and 2.

Since $|0.6|<1$, the stationary AR(1) result applies. The mean is

\mu=\frac{c}{1-\phi}=\frac{1}{1-0.6}=2.5

outcome units. The variance, or lag-zero autocovariance, is

\gamma(0)=\frac{4}{1-0.6^2} =\frac{4}{0.64}=6.25

outcome units squared. The next two autocovariances are

\gamma(1)=6.25(0.6)=3.75,\qquad \gamma(2)=6.25(0.6)^2=2.25.

These values depend on lag, not on the date of the pair: every pair one period apart has covariance 3.75 outcome units squared, and every pair two periods apart has covariance 2.25 outcome units squared. Together with the constant mean and variance, this verifies weak stationarity.

Why a Random Walk Is Not Weakly Stationary

A random walk accumulates shocks rather than reverting around a fixed level. Consider the hypothetical process $X_0=0$ and $X_t=X_{t-1}+\varepsilon_t$ for $t\ge1$, where the shocks are uncorrelated over time, have mean zero, and have variance $\sigma_\varepsilon^2>0$. Then $X_t=\sum_{j=1}^{t}\varepsilon_j$. Its mean remains zero, but its variance is

\operatorname{Var}(X_t)=t\sigma_\varepsilon^2.

The variance therefore grows with time. In addition, for $s\le t$, the covariance is

\operatorname{Cov}(X_s,X_t)=s\sigma_\varepsilon^2.

For a fixed separation $t-s$, this covariance still changes with the date $s$. Either the changing variance or the date-dependent covariance is enough to violate weak stationarity.

Worked Example: Finding the Failure in a Random Walk

Suppose a hypothetical random walk starts at $X_0=0$, with shocks having mean zero and variance $1$ unit squared, uncorrelated across time. Compare its variances at dates 1 and 3, and compare two covariances at lag 1.

Because $X_t$ sums $t$ shocks, $\operatorname{Var}(X_1)=1$ unit squared and $\operatorname{Var}(X_3)=3$ units squared. The variance is not constant. Also, $X_2=\varepsilon_1+\varepsilon_2$ and $X_3=\varepsilon_1+\varepsilon_2+\varepsilon_3$, so

\operatorname{Cov}(X_2,X_3)=\operatorname{Var}(\varepsilon_1+\varepsilon_2)=2

units squared. By contrast, $X_1=\varepsilon_1$, so

\operatorname{Cov}(X_1,X_2)=\operatorname{Var}(\varepsilon_1)=1

unit squared. Both pairs are separated by one period, but their covariances differ. The random walk is not weakly stationary.

Common Mistake and Practical Significance

A common mistake is to check only that $E[X_t]$ is constant and then call the process stationary. A random walk with zero-mean shocks illustrates why this reasoning fails: its mean is zero at each date, but its variance increases and its lag-one covariance depends on the date. The correction is to check both parts of the definition: the mean must be constant, and every covariance must be determined only by the lag.

Weak stationarity is useful because it summarizes a stable mean-and-covariance structure, which is a foundation for many time-series methods. It is a specific moment condition, not a guarantee that every feature of a process is stable. When assessing a model, state the assumptions that support its moment calculations and distinguish properties of the model from patterns merely observed in a finite sample.

Key takeaway: Weak stationarity requires finite second moments, a time-invariant mean, and covariances that are invariant to shifting both dates together. Constant variance is included as the zero-lag case; dependence across dates is allowed.

Check Your Understanding

Use the definition and calculations in this tutorial to answer the following questions.

State the two moment conditions that a process must satisfy to be weakly stationary.
A process has constant mean and variance, but its covariance at lag 2 differs between early and late dates. Is it weakly stationary? Explain.
For a mean-zero white-noise process with variance $\tau^2$ and zero covariance at distinct dates, give its autocovariance at lag zero and at nonzero lags.
For a stationary AR(1) with $c=2$, $\phi=0.5$, and innovation variance $3$, calculate the mean and the variance.
Why does a zero-mean random walk fail the weak-stationarity conditions?