Short Memory and Long Memory

Why the Rate of Dependence Matters

A stationary series can have correlations that become small at long lags yet remain consequential over a long sample. The key question is not only whether dependence fades, but how quickly. If autocovariances shrink fast enough for their absolute values to have a finite sum, the process is usually called short memory. If positive autocovariances decay slowly enough that this sum diverges, the process is commonly described as having long memory.

This distinction affects averages over time. Under summable autocovariance decay, the variance of a partial sum grows at most at a linear rate, and—when a positive long-run variance exists—the sample mean has variance of order $1/n$. Under a common form of long memory, that variance declines more slowly than $1/n$. The distinction concerns the rate of dependence, not whether the process is stationary: both short- and long-memory processes can be covariance-stationary.

Autocovariances and the Two Memory Classes

Let $\{X_t\}$ be a covariance-stationary process with finite mean $\mu$ and autocovariance function $\gamma(h)=\operatorname{Cov}(X_t,X_{t-h})$, for integer lag $h$. Its autocorrelation at lag $h$ is $\rho(h)=\gamma(h)/\gamma(0)$, provided $\gamma(0)>0$. Since $\gamma(-h)=\gamma(h)$, it is enough to describe the nonnegative lags.

Definition: A covariance-stationary process has short memory under the standard absolute-summability convention if $\sum_{h=-\infty}^{\infty}|\gamma(h)|<\infty$. Equivalently, when $0<\gamma(0)<\infty$, its autocorrelations are absolutely summable. A process is commonly called long memory when its autocovariances are not absolutely summable; the term is especially useful when the autocovariances have a slowly decaying, nonnegative tail, so their sum diverges positively.

Absolute summability is a property of the whole tail, not a statement that every autocovariance is positive. With signs that alternate or otherwise cancel, nonsummability of absolute values alone does not establish the same partial-sum variance behavior as a positive, slowly decaying tail. The standard power-law account of long memory below makes its positive-tail condition explicit.

For a sample of length $n$, define the centered partial sum $S_n=\sum_{t=1}^{n}(X_t-\mu)$, and the sample mean $\bar X_n=n^{-1}\sum_{t=1}^{n}X_t$. These are related by $\bar X_n-\mu=S_n/n$, so $\operatorname{Var}(\bar X_n)=\operatorname{Var}(S_n)/n^2$.

How Autocovariances Determine Partial-Sum Variance

Stationarity lets us group the terms in the variance of $S_n$ by the distance between their time indices. There are $n$ individual variance terms, and for each lag $h$ between 1 and $n-1$, there are $n-h$ pairs in each time order. Thus:

\operatorname{Var}(S_n) = n\gamma(0)+2\sum_{h=1}^{n-1}(n-h)\gamma(h).

Dividing by $n^2$ gives the corresponding sample-mean variance:

\operatorname{Var}(\bar X_n) = \frac{1}{n}\left[\gamma(0)+2\sum_{h=1}^{n-1}\left(1-\frac{h}{n}\right)\gamma(h)\right].

These identities explain why dependence matters: each autocovariance contributes across many pairs. The weight $n-h$ counts how many pairs at lag $h$ occur in a partial sum, while the factor $1-h/n$ appears when the variance is written for the mean.

Summable Decay and the Long-Run Variance

If autocovariances are absolutely summable, the weighted bracket in the sample-mean formula converges to the long-run variance $V=\gamma(0)+2\sum_{h=1}^{\infty}\gamma(h)$. To see why, at every fixed lag $h$, the weight $1-h/n$ approaches 1 as $n$ grows. The absolute-summability condition ensures that the remaining tail can be made uniformly small by choosing a sufficiently large fixed lag. Therefore $\operatorname{Var}(\bar X_n)$ is asymptotically $V/n$, provided $V>0$. If $V=0$, the leading $1/n$ term vanishes and this conclusion about a positive variance rate does not apply.

Theorem: For a covariance-stationary process with absolutely summable autocovariances, $$ \lim_{n\to\infty}n\,\operatorname{Var}(\bar X_n) =\gamma(0)+2\sum_{h=1}^{\infty}\gamma(h)=V. $$ When $V>0$, the sample-mean variance has leading rate $V/n$, and the partial-sum variance has leading rate $Vn$.

The conclusion gives a variance rate, not by itself a central limit theorem or a guarantee of any particular distribution for the sample mean. Those further results require their own conditions.

Worked Example: Geometric Decay in a Stationary AR(1) Process

Suppose a hypothetical stationary AR(1) process has autoregressive coefficient $\phi=0.8$ and white-noise innovations with variance 1 squared measurement unit. Assume the innovations are uncorrelated over time and the process is in its stationary regime. Since $|\phi|<1$, its autocovariance function is $\gamma(h)=\gamma(0)\phi^{|h|}$, with $\gamma(0)=1/(1-\phi^2)$ in the stated units.

The lag-zero autocovariance is $1/(1-0.8^2)=1/0.36=2.7778$ squared units. For positive lags, the absolute autocovariances have a geometric sum:

\sum_{h=1}^{\infty}|\gamma(h)| =2.7778\sum_{h=1}^{\infty}0.8^h =2.7778\left(\frac{0.8}{1-0.8}\right) =11.111.

Adding the finite lag-zero term still gives a finite total, so the process has short memory. Its long-run variance is $V=\gamma(0)[1+2\sum_{h=1}^{\infty}0.8^h]$. Substitution gives $V=2.7778[1+2(0.8/0.2)]=2.7778(9)=25.00$ squared units.

Consequently, for large $n$, the variance of the sample mean is approximately $25.00/n$ squared units, while the partial-sum variance is approximately $25.00n$ squared units. The approximation is a large-sample variance rate, not an exact finite-sample formula.

Power-Law Decay and Long Memory

A common model for slowly decaying dependence is a positive power-law tail: for constants $C>0$ and $0<\alpha<1$, suppose $\gamma(h)\sim C h^{-\alpha}$ as $h$ grows. Here $\sim$ means the ratio of the two sides approaches 1. The exponent is below 1, so the positive tail is nonsummable: the corresponding power series $\sum_{h=1}^{\infty}h^{-\alpha}$ diverges.

The partial-sum variance formula gives the leading growth rate. For large $n$, replace the sum of $(n-h)h^{-\alpha}$ by its leading integral. With $x=h/n$, the integral is $n^{2-\alpha}\int_0^1(1-x)x^{-\alpha}\,dx$. The integral is finite because $0<\alpha<1$; evaluating it gives $1/[(1-\alpha)(2-\alpha)]$. The lag-zero term grows only as $n$, which is smaller than $n^{2-\alpha}$. Therefore:

\operatorname{Var}(S_n)\sim \frac{2C}{(1-\alpha)(2-\alpha)}n^{2-\alpha}, \qquad \operatorname{Var}(\bar X_n)\sim \frac{2C}{(1-\alpha)(2-\alpha)}n^{-\alpha}.

Because $0<\alpha<1$, the partial-sum variance grows faster than $n$, and the sample-mean variance declines more slowly than $1/n$. This is the variance-rate consequence of this positive power-law form of long memory.

Worked Example: A Power-Law Tail and Sample-Mean Variance

Consider a hypothetical stationary process with autocovariance tail $\gamma(h)\sim 4.5h^{-0.4}$, where 4.5 is measured in squared units of the series. Assume the positive-tail approximation is applicable at large lags. Since $0.4<1$, the tail is nonsummable and falls into the power-law long-memory case above.

Here $C=4.5$ and $\alpha=0.4$. The leading constant for the sample-mean variance is $2C/[(1-\alpha)(2-\alpha)]$, so substitution gives $9/[0.6(1.6)]=9/0.96=9.375$. Thus the leading-order expression is:

\operatorname{Var}(\bar X_n)\sim 9.375n^{-0.4}.

At $n=100$, this leading-order expression is $9.375(100^{-0.4})\approx1.486$ squared measurement units, since $100^{-0.4}\approx0.1585$. This is an asymptotic approximation based on the tail, not an exact variance from a fully specified autocovariance function. Its $n^{-0.4}$ rate is slower than the $n^{-1}$ rate associated with a positive long-run variance under short memory.

The Boundary Case and the Meaning of Nonsummability

The exponent $\alpha=1$ is a boundary between summable and nonsummable power-law tails. If $\gamma(h)\sim C/h$ with $C>0$, the autocovariances are not summable. In the partial-sum formula, the weighted sum behaves to leading order like $C n\sum_{h=1}^{n}1/h$, which is of order $C n\log n$. Consequently, the sample-mean variance is of order $(\log n)/n$: it still tends to zero, but more slowly than $1/n$.

For $\alpha>1$, a power-law tail $C h^{-\alpha}$ is summable, so it satisfies the short-memory summability condition. At $\alpha=1$, the harmonic tail diverges; for $0<\alpha<1$, the divergence is stronger and gives the power-law variance rates derived above.

Worked Example: Comparing Geometric and Power-Law Persistence

Compare two hypothetical autocorrelation tails, each with $\gamma(0)=1$ squared unit. In the first, $\rho(h)=0.6^h$ for $h\geq1$. In the second, the positive tail satisfies $\rho(h)\sim h^{-0.7}$. These are stipulated tail forms for comparison; the rate calculations use the stationary variance identities above.

For the geometric case, $\sum_{h=1}^{\infty}0.6^h=0.6/(1-0.6)=1.5$. The long-run variance is therefore $1+2(1.5)=4.00$ squared units, and $\operatorname{Var}(\bar X_n)$ has leading rate $4.00/n$.

For the power-law case, $\alpha=0.7$ and $C=1$. The autocorrelation tail is nonsummable because $0.7<1$. Its sample-mean variance has leading rate:

\frac{2}{(1-0.7)(2-0.7)}n^{-0.7} =\frac{2}{0.39}n^{-0.7} \approx 5.128n^{-0.7}.

At $n=100$, the two leading-rate expressions are $4/100=0.0400$ squared units for the geometric case and $5.128(100^{-0.7})\approx0.204$ squared units for the power-law case. These are leading-order comparisons, not exact finite-sample variances. They illustrate how a slowly declining positive tail can leave substantially more variation in a time average as the sample grows.

Common Pitfall: “Nonsummable” Does Not Name a Variance Rate by Itself

A tempting but incomplete argument is: “The autocovariances are not summable, so the partial-sum variance must grow faster than $n$.” Nonsummability of absolute values alone does not justify that conclusion: signs can cancel in the weighted sum that determines variance. The faster-than-linear rate derived here relies on a positive power-law tail with exponent between 0 and 1. Under absolute summability, the long-run variance result applies; outside that case, the actual autocovariance pattern and its effect on the partial-sum formula must be examined.

Key takeaway: Geometric decay is summable and gives a finite long-run variance. A positive power-law tail $C h^{-\alpha}$, with $0<\alpha<1$, is nonsummable and yields partial-sum variance of order $n^{2-\alpha}$ and sample-mean variance of order $n^{-\alpha}$. Always state the tail and sign conditions behind a claimed variance rate.

Check Your Understanding

Use the autocovariance-sum and partial-sum variance relationships to answer these questions.

State the absolute-summability condition that defines short memory under the convention used here.
If $\gamma(h)\sim C h^{-0.8}$ for $C>0$, is the tail summable? What is the leading order of the sample-mean variance?
For a short-memory process with long-run variance $V>0$, what are the leading orders of $\operatorname{Var}(S_n)$ and $\operatorname{Var}(\bar X_n)$?
Why does nonsummability of absolute autocovariances, without information about their signs or tail form, not establish a faster-than-linear partial-sum variance rate?
What is the sample-mean variance order for a positive tail $\gamma(h)\sim C/h$, and how does it compare with the short-memory rate?

What You'll Learn