Consistency of Sample Autocovariances

Why Estimate Autocovariances?

A time series contains information not only in its average and variability, but also in how observations at different dates move together. The population autocovariance at lag $h$ describes that dependence. In practice, it is unknown, so an analyst calculates a sample autocovariance from a finite stretch of observations. The central question here is when that calculation converges to the population quantity as the series gets longer.

The result is about a fixed lag: $h$ is held constant as the sample size increases. It is a consistency result, meaning convergence in probability, not an assertion that an estimate from any one finite sample must be close to its target. We use the usual time-series notation $X_t$ for the observation at date $t$.

Population and Sample Autocovariances

Definition: For a weakly stationary process with mean $\mu=E[X_t]$, the population autocovariance at integer lag $h$ is $$ \gamma(h)=E[(X_t-\mu)(X_{t+h}-\mu)]. $$ It does not depend on $t$. For a sample $X_1,\ldots,X_n$, define $\bar X=n^{-1}\sum_{t=1}^n X_t$. For an integer $h$ with $0\leq h<n$, the sample autocovariance used here is $$ \hat{\gamma}_n(h)=\frac{1}{n}\sum_{t=1}^{n-h}(X_t-\bar X)(X_{t+h}-\bar X). $$

The denominator convention matters for a finite-sample calculation. Some definitions use $n-h$ instead of $n$; at any fixed $h$, the ratio $(n-h)/n$ tends to one, so this change does not alter the consistency result below. At lag zero, $\hat{\gamma}_n(0)$ uses denominator $n$, so it is not the usual unbiased sample variance with denominator $n-1$.

For negative lags, stationarity gives $\gamma(-h)=\gamma(h)$. The sample covariance can likewise be defined symmetrically, so it is enough to develop the result for fixed nonnegative $h$. Autocovariance has squared units: if $X_t$ is measured in dollars, $\gamma(h)$ and $\hat{\gamma}_n(h)$ are in dollars squared.

Worked Example: Calculating a Lag-One Sample Autocovariance

Suppose a hypothetical series of six quarterly changes, in percentage points, is $4,1,3,0,2,5$. Calculate the sample autocovariance at lag one using the stated denominator convention.

The sample mean is $$ \bar X=\frac{4+1+3+0+2+5}{6}=\frac{15}{6}=2.5 $$ percentage points. The deviations in order are $1.5,-1.5,0.5,-2.5,-0.5,2.5$. The five lag-one products are $$ (1.5)(-1.5)=-2.25,\quad (-1.5)(0.5)=-0.75,\quad (0.5)(-2.5)=-1.25,\quad (-2.5)(-0.5)=1.25,\quad (-0.5)(2.5)=-1.25. $$

Their sum is $-4.25$. Therefore $$ \hat{\gamma}_6(1)=\frac{-4.25}{6}=-0.7083 $$ percentage-points squared. This is a finite-sample estimate; the calculation alone does not establish that it is close to the population lag-one autocovariance.

A Sufficient Consistency Result

Consistency requires conditions that control both extreme observations and dependence across time. One convenient set of sufficient conditions uses alpha-mixing. For a strictly stationary process, $\alpha(k)$ measures the degree of dependence between events determined by observations up to some date and events determined by observations at least $k$ dates later. The process is alpha-mixing if $\alpha(k)$ tends to zero as $k$ increases. The stronger summability condition below controls how quickly that dependence fades.

Theorem: Suppose $\{X_t\}$ is strictly stationary, has mean $\mu$, and satisfies $E|X_t-\mu|^{4+\delta}<\infty$ for some $\delta>0$. Suppose also that it is alpha-mixing and $$ \sum_{k=1}^{\infty}\alpha(k)^{\delta/(4+\delta)}<\infty. $$ Then, for every fixed nonnegative integer $h$, $$ \hat{\gamma}_n(h)\xrightarrow{p}\gamma(h). $$ These are sufficient, not necessary, conditions.

Strict stationarity ensures that the relevant moments and dependence structure do not change with the date. The moment assumption controls the variability of products of observations. The mixing-rate assumption ensures that distant parts of the series contribute limited dependence. Together, they give a law of large numbers for the products that appear in an autocovariance.

Why the Result Holds

Write $Z_t=X_t-\mu$. First consider the average that would be available if the population mean were known: $$ A_n(h)=\frac{1}{n}\sum_{t=1}^{n-h}Z_tZ_{t+h}. $$ Each product has expectation $\gamma(h)$, so $E[A_n(h)]=(n-h)\gamma(h)/n$, which tends to $\gamma(h)$ for fixed $h$.

To control variation around this expectation, define $Y_t=Z_tZ_{t+h}$. By stationarity and the moment assumption, $Y_t$ has a finite moment of order $2+\delta/2$. For products $Y_t,Y_s$ separated by more than the fixed overlap distance, the Davydov covariance inequality bounds their covariance by a constant times $\alpha(|t-s|-h)^{\delta/(4+\delta)}$. The finitely many overlapping or nearby terms have bounded covariance by the moment assumption. Consequently, the sum of the absolute covariances of the $Y_t$'s over all separations is finite. Expanding the variance of their average then gives a constant $C_h$, not depending on $n$, such that $$ \operatorname{Var}(A_n(h))\leq \frac{C_h}{n}. $$ Chebyshev's inequality implies $A_n(h)-E[A_n(h)]\xrightarrow{p}0$. Combined with its expectation limit, this proves $A_n(h)\xrightarrow{p}\gamma(h)$.

The estimator actually uses $\bar X$, not $\mu$. Let $d_n=\bar X-\mu$. The same mixing and moment conditions give $\bar X\xrightarrow{p}\mu$, so $d_n\xrightarrow{p}0$. Expanding each centered product yields $$ \hat{\gamma}_n(h)-A_n(h) =-\frac{d_n}{n}\sum_{t=1}^{n-h}(Z_t+Z_{t+h}) +\frac{n-h}{n}d_n^2. $$ The averages of the $Z_t$'s in the first term are bounded in probability: their expected absolute values are bounded by $E|Z_t|<\infty$. Thus the first term tends to zero in probability because $d_n\xrightarrow{p}0$. The second term also tends to zero, since $(n-h)/n\leq1$ and $d_n^2\xrightarrow{p}0$. Replacing the known mean by the sample mean therefore leaves the probability limit unchanged, completing the justification.

Worked Example: Consistency for a Positive-Lag Autocovariance

Consider a hypothetical stationary Gaussian AR(1) process $X_t=0.6X_{t-1}+\varepsilon_t$, where the innovations are independent $N(0,4)$ variables and the process is initialized in its stationary distribution. Find the population lag-two autocovariance and assess whether the sample autocovariance at lag two is consistent for it.

The stationary variance is $$ \gamma(0)=\frac{4}{1-0.6^2}=\frac{4}{0.64}=6.25. $$ For this AR(1) process, $\gamma(h)=0.6^h\gamma(0)$ for nonnegative $h$. Hence $$ \gamma(2)=0.6^2(6.25)=0.36(6.25)=2.25. $$

The stationary Gaussian AR(1) with $|0.6|<1$ is alpha-mixing at a geometric rate and has finite moments of every positive order. In particular, its $4+\delta$ moment is finite, and its geometric mixing rate satisfies the theorem’s summability condition. Thus, for lag two held fixed, $\hat{\gamma}_n(2)\xrightarrow{p}2.25$, in squared units of $X_t$. This statement concerns the estimator as $n$ grows; it does not give the estimate for a particular observed sample.

Worked Example: The Independent Case at a Positive Lag

Suppose a hypothetical series consists of independent, identically distributed observations with mean zero and variance $9$, measured in millimetres, and with a finite $4+\delta$ moment for some $\delta>0$. What does the consistency result imply at lag one?

Independence across dates gives $\gamma(1)=E[X_tX_{t+1}]=E[X_t]E[X_{t+1}]=0$. An independent sequence is alpha-mixing with $\alpha(k)=0$ for $k\geq1$, so the summability condition holds. The finite-moment condition also holds by assumption. Therefore $$ \hat{\gamma}_n(1)\xrightarrow{p}0. $$ The estimated autocovariance has units of millimetres squared. The limit is zero because of the specified independence and zero mean, not because every finite sample must have a zero lag-one estimate.

Checking the Scope and Avoiding Common Errors

The theorem does not say that any stationary-looking sequence has consistent sample autocovariances. Stationarity gives a stable population target, but alone it does not ensure that time averages converge to that target. The moment and dependence assumptions provide one useful route to that convergence. Other, weaker sufficient conditions are possible, but they require their own justification.

A second limitation is the fixed-lag qualification. For example, the result for $h=1$ and the result for $h=2$ each hold as $n$ grows. It does not follow that the same proof applies when the lag itself increases with $n$. Such a claim needs conditions that address the changing lag and is a separate question.

Mistake: “The process is stationary, so the sample autocovariance must be consistent.” Correction: Stationarity defines a time-invariant target, but additional conditions are needed to control time averages under dependence.
Mistake: “The estimated autocovariance is unbiased, so it is consistent.” Correction: Unbiasedness, when it holds under a particular definition and assumptions, does not by itself establish that estimation error vanishes as the sample grows.
Mistake: “The denominator $n$ prevents consistency because only $n-h$ products are included.” Correction: At fixed $h$, $(n-h)/n\to1$, so the missing fraction has no effect on the probability limit.

Key takeaway: Under suitable moment and weak-dependence conditions, the average of the fixed-lag products converges to the population autocovariance. Estimating the mean by $\bar X$ does not change that limit.

Check Your Understanding

Answer each question by referring to the definitions and conditions above.

Using the denominator convention in this tutorial, how many products enter $\hat{\gamma}_n(3)$, and what is the denominator?
Why does using $n-h$ rather than $n$ in the denominator leave the fixed-lag probability limit unchanged?
In the consistency result, what roles do the finite $4+\delta$ moment and the alpha-mixing summability condition play?
For an independent, zero-mean series with finite variance, what is the population autocovariance at a positive lag?
Why does consistency at each fixed lag not automatically establish consistency when the lag increases with sample size?