Autocovariance As A Function Of Lag

Why Covariance Is Indexed by Lag

For two observations from a time series, covariance describes how their values vary together. But the covariance between observations at times 2 and 3 may not be the same as the covariance between observations at times 20 and 21. A useful simplification is possible when the series has stable mean and variance, and the covariance between two observations depends on how far apart they are rather than on their calendar positions. In that setting, we can describe dependence with a function of lag.

This tutorial defines that function and calculates it in several settings. The examples focus on the population properties of a stochastic process, not on estimates calculated from one observed data set.

Definition: Let $\{X_t\}$ be a time series indexed by integer times, with finite second moments. It is weakly stationary if (1) $E[X_t]=\mu$ is constant for every $t$, (2) $\operatorname{Var}(X_t)$ is finite and constant, and (3) $\operatorname{Cov}(X_t,X_s)$ depends on $t$ and $s$ only through their separation. For a nonnegative integer lag $h$, the autocovariance at lag $h$ is $$ \gamma(h)=\operatorname{Cov}(X_t,X_{t+h})=E[(X_t-\mu)(X_{t+h}-\mu)]. $$ For a weakly stationary series, this value is the same for every allowable $t$.

At lag 0, the two time points coincide, so $\gamma(0)=\operatorname{Var}(X_t)$. At lag 1, the observations are one time step apart; at lag 3, they are three steps apart. The unit is set by the time index: if $t$ counts months, lag 3 means three months. Autocovariance is measured in squared units of $X_t$, because it is a covariance between two values of that series.

The definition is for a process with a constant mean and covariance structure determined by separation. If the covariance changes with calendar time, one cannot describe all pairs using a single value $\gamma(h)$ for each lag. The covariance between times $t$ and $t+h$ would then need to retain its dependence on $t$.

A Derivation From Shared Random Components

A direct way to calculate autocovariance is to expand each observation and identify random terms that appear in both. The following result makes that procedure efficient for a finite weighted sum.

Result: Suppose $$ X_t=\mu+\sum_{j=0}^{q}a_j\varepsilon_{t-j}, $$ where $q$ is a fixed nonnegative integer, the $a_j$ are fixed coefficients, and $\{\varepsilon_t\}$ are independent random variables with mean 0 and common finite variance $\sigma^2$. Then the process has constant mean and variance, and for $0\le h\le q$, $$ \gamma(h)=\sigma^2\sum_{j=0}^{q-h}a_j a_{j+h}. $$ For $h>q$, $\gamma(h)=0$.

To justify the formula, the constant $\mu$ disappears when each observation is centered. Expand the covariance at lag $h$:

\operatorname{Cov}(X_t,X_{t+h}) = \operatorname{Cov}\left( \sum_{j=0}^{q}a_j\varepsilon_{t-j}, \sum_{k=0}^{q}a_k\varepsilon_{t+h-k} \right).

By bilinearity, this is a sum of terms $a_j a_k\operatorname{Cov}(\varepsilon_{t-j},\varepsilon_{t+h-k})$. Independence and the common mean imply that this covariance is 0 unless the two innovation indices match. Matching requires $t-j=t+h-k$, or $k=j+h$. When $0\le h\le q$, the matching pairs are $j=0,\ldots,q-h$, each contributing $\sigma^2a_j a_{j+h}$. If $h>q$, no indices match, so every term is 0. This also explains why the mean and variance do not change with $t$: the same coefficients and the same innovation distribution are used at every time.

Worked Example: A Two-Term Weighted Sum

A hypothetical monthly index is modeled as $X_t=40+\varepsilon_t+2\varepsilon_{t-1}$, where the innovations are independent and each has distribution $N(0,3)$. Find the mean, variance, and autocovariances at lags 0, 1, and 2 or greater.

Method. Identify which innovation terms are shared by observations at each separation. The innovation variance is $\sigma^2=3$, and the two coefficients are $a_0=1$ and $a_1=2$.

The mean is $E[X_t]=40+0+2(0)=40$. At lag 0, independence gives $\gamma(0)=\operatorname{Var}(X_t)=1^2(3)+2^2(3)=3+12=15$.

At lag 1, write both centered observations: $X_t-40=\varepsilon_t+2\varepsilon_{t-1}$ and $X_{t+1}-40=\varepsilon_{t+1}+2\varepsilon_t$. The only shared innovation is $\varepsilon_t$, with coefficient 1 in the first observation and coefficient 2 in the second. Thus $\gamma(1)=1(2)(3)=6$. At lag 2, $X_{t+2}-40=\varepsilon_{t+2}+2\varepsilon_{t+1}$ shares no innovation with $X_t-40$, so $\gamma(2)=0$. Larger lags also have no shared terms and have autocovariance 0.

Check. The finite-sum formula gives $\gamma(0)=3(1^2+2^2)=15$, $\gamma(1)=3(1)(2)=6$, and no terms for $h\ge2$. These agree with the direct calculations. A common mistake is to multiply the innovation variance by $2$ at lag 1; the contribution is instead the product of the two coefficients, $1\cdot2$, times the variance.

Dependence That Persists Across All Lags

Not all dependence comes from a short list of overlapping innovations. A random component shared by every observation can produce nonzero covariance at every positive lag. This possibility is important when a series represents repeated measurements on a unit with an unobserved, persistent characteristic.

Worked Example: A Shared Random Effect

Suppose a hypothetical portfolio indicator is $X_t=12+4U+\varepsilon_t$ at each time $t$. The shared variable $U$ has a Bernoulli distribution with $P(U=1)=0.5$; the $\varepsilon_t$ are independent $N(0,1)$ variables and are independent of $U$. Find the mean, variance, and covariance between observations at any two distinct times.

Method. Separate the common source $4U$ from the time-specific errors. Since $E[U]=0.5$, $E[\varepsilon_t]=0$, and $E[U^2]=0.5$, the mean is $12+4(0.5)=14$.

A Bernoulli variable with success probability 0.5 has variance $0.5(1-0.5)=0.25$. Therefore the variance at one time is $\operatorname{Var}(X_t)=4^2(0.25)+1=4+1=5$.

For distinct times $t$ and $s$, the time-specific errors have covariance 0. The common component contributes $\operatorname{Cov}(4U,4U)=16(0.25)=4$. Thus $\operatorname{Cov}(X_t,X_s)=4$ whenever $t\ne s$, including lag 1 and lag 5. At lag 0 the covariance is the variance, 5.

Check. The mean and variance are constant, and every pair of distinct times has covariance 4, so covariance depends only on separation (in this case, it is the same for every positive separation). The shared random effect is the source of that nonzero covariance. Treating the two observations as independent would incorrectly give covariance 0.

Negative Autocovariance From Opposite-Signed Terms

The sign of an autocovariance depends on the signs of the shared terms in the two centered observations. If a shared random variable enters one observation with a positive coefficient and the other with a negative coefficient, its contribution to covariance is negative. The next example makes this explicit.

Worked Example: A Difference of Successive Innovations

A hypothetical monthly change series is $X_t=100+\varepsilon_t-\varepsilon_{t-1}$, where the innovations are independent and each has distribution $N(0,9)$. Calculate the mean and autocovariance at lags 0, 1, and 2 or greater.

Method. Center the series and track the shared innovation. Here $E[X_t]=100$, and $X_t-100=\varepsilon_t-\varepsilon_{t-1}$.

At lag 0, the two independent terms each contribute variance 9: $\gamma(0)=9+9=18$. At lag 1, the centered observations are $X_t-100=\varepsilon_t-\varepsilon_{t-1}$ and $X_{t+1}-100=\varepsilon_{t+1}-\varepsilon_t$. The only shared term is $\varepsilon_t$; its coefficients are $+1$ and $-1$. Therefore $\gamma(1)=(1)(-1)(9)=-9$.

At lag 2, $X_{t+2}-100=\varepsilon_{t+2}-\varepsilon_{t+1}$, which shares no innovation with $X_t-100$, so $\gamma(2)=0$. The same reasoning gives $\gamma(h)=0$ for every $h\ge2$.

Check. The finite-sum calculation with coefficients $1,-1$ gives $\gamma(0)=9(1^2+(-1)^2)=18$ and $\gamma(1)=9(1)(-1)=-9$. The variance is positive, as required, while the lag-1 autocovariance is negative. The exam trap is to assume that observations sharing an innovation must have positive covariance; the coefficient signs determine the contribution.

How to Organize a Lag Calculation

For a model expressed as a sum of random components, an orderly calculation prevents missed terms and sign errors. Use the same sequence whether the series is described verbally or by a formula.

Check the process assumptions.
Identify the mean, the finite variances, and whether the covariance structure is stable over time. Do not use one lag-only function if covariance also depends on calendar time.

Center the observations.
Subtract the common mean from each time point. Constants then disappear from covariance calculations.

Align the two time points.
Write the centered expressions for $X_t$ and $X_{t+h}$, making the lag explicit.

Keep only nonzero covariance contributions.
Independent, mean-zero terms at distinct times have covariance 0. A shared term contributes the product of its two coefficients times its variance.

Check the result.
At lag 0, the answer must equal the variance. For a finite weighted sum, lags beyond the maximum separation of shared innovations have covariance 0.

A population autocovariance is not the same quantity as an estimate from a finite observed series. The function $\gamma(h)$ is defined using the process distribution and its expectations. An observed sequence provides data with which one might estimate it, but the calculation of sample autocovariance involves additional choices that are separate from this definition.

Autocovariance retains the measurement scale: multiplying a series by a constant $c$ multiplies every autocovariance by $c^2$. Consequently, two series with the same dependence pattern but different units can have very different numerical autocovariances. When interpreting a value, state both the lag and the units or scale of the process.

Key takeaway: For a weakly stationary series, $\gamma(h)$ is the covariance between observations $h$ time steps apart, and it does not depend on where that pair occurs in the series. In calculations built from independent innovations, only random terms shared across the two observations contribute.

Check Your Understanding

For each process, identify the relevant shared terms before calculating. Assume all stated random variables are independent where specified.

Let $X_t=7+\varepsilon_t+\varepsilon_{t-1}$, where the $\varepsilon_t$ are independent $N(0,2)$. Find $\gamma(0)$, $\gamma(1)$, and $\gamma(2)$. (Answer: $\gamma(0)=4,\ \gamma(1)=2,\ \gamma(2)=0$.)
Let $X_t=U+\varepsilon_t$, where $U$ is Bernoulli with $P(U=1)=0.25$, and the independent errors are $N(0,4)$. Find the mean, variance, and covariance at any positive lag. (Answer: mean $0.25$, variance $4.1875$, and positive-lag autocovariance $0.1875$.)
Let $X_t=3+\varepsilon_t-2\varepsilon_{t-1}$, where the independent innovations are $N(0,5)$. Find $\gamma(0)$, $\gamma(1)$, and all $\gamma(h)$ for $h\ge2$. (Answer: $\gamma(0)=25,\ \gamma(1)=-10,\ \gamma(h)=0$ for $h\ge2$.)
Let $X_t=2U+\varepsilon_t$, where $U\sim N(3,1)$, and the independent errors are $N(0,2)$. Find the mean, variance, and covariance at any positive lag. (Answer: mean $6$, variance $6$, and positive-lag autocovariance $4$.)
Let $X_t=5+\varepsilon_t+\tfrac12\varepsilon_{t-1}-\varepsilon_{t-2}$, where the independent innovations are $N(0,2)$. Calculate $\gamma(0)$, $\gamma(1)$, $\gamma(2)$, and $\gamma(h)$ for $h>2$. (Answer: $\gamma(0)=4.5,\ \gamma(1)=0,\ \gamma(2)=-2,\ \gamma(h)=0$ for $h>2$.)

What You'll Learn