Central Limit Theorems for Weakly Dependent Series

Why a Central Limit Theorem Needs More Than Stationarity

For independent observations, a central limit theorem explains why a suitably standardized sample average is approximately normal when the sample is large. Time-series observations are often dependent, so the independent-data theorem does not automatically apply. Nearby observations may move together, and their dependence changes the variability of an average even when the series has a constant mean and variance.

A dependent-data central limit theorem gives conditions under which the average is still asymptotically normal. The conditions must control both the tails of the observations and how dependence persists across time. The limiting variance then accounts for the effect of covariances at every lag, not only the variance of one observation.

Throughout, let $\{X_t\}$ be a time series with finite second moments. When the series is weakly stationary, its mean is constant and its autocovariance depends only on the lag. Write $\mu=E[X_t]$, $\gamma(h)=\operatorname{Cov}(X_t,X_{t-h})$, and $\bar X_n=n^{-1}\sum_{t=1}^n X_t$. Requiring finite second moments ensures that these autocovariances, including $\gamma(0)$, exist and are finite.

Definition: The long-run variance of a covariance-stationary series, when the sum exists, is $$ \sigma_{\mathrm{LR}}^2=\sum_{h=-\infty}^{\infty}\gamma(h) =\gamma(0)+2\sum_{h=1}^{\infty}\gamma(h). $$ Here $\gamma(-h)=\gamma(h)$ for a real-valued stationary series. This quantity describes the asymptotic variance of $\sqrt{n}(\bar X_n-\mu)$; it is not generally the variance $\gamma(0)$ of a single observation.

A Representative Mixing Central Limit Theorem

One way to formalize weak dependence is through alpha-mixing coefficients. Let $\mathcal F_{-\infty}^{0}$ contain events determined by observations at times at most 0, and let $\mathcal F_k^{\infty}$ contain events determined by observations at times at least $k$. The alpha-mixing coefficient at separation $k$ measures the largest departure from independence between these two sets of events.

Definition: The alpha-mixing coefficient at lag $k$ is $$ \alpha(k)=\sup_{A\in\mathcal F_{-\infty}^{0},\,B\in\mathcal F_k^{\infty}} \left|P(A\cap B)-P(A)P(B)\right|. $$ A process is alpha-mixing if $\alpha(k)\to 0$ as $k\to\infty$. A summability condition requires this decline to be sufficiently fast that a specified power of the coefficients has a finite sum.

The following is a standard sufficient-condition version of a dependent-data central limit theorem. It is not the only available version, and its assumptions are sufficient rather than necessary.

Theorem: Suppose $\{X_t\}$ is strictly stationary, $E[X_t]=\mu$, and $E|X_t-\mu|^{2+\delta}<\infty$ for some $\delta>0$. Suppose also that the series is alpha-mixing and $$ \sum_{k=1}^{\infty}\alpha(k)^{\delta/(2+\delta)}<\infty. $$ Then the autocovariances are absolutely summable, the long-run variance $\sigma_{\mathrm{LR}}^2=\gamma(0)+2\sum_{h=1}^{\infty}\gamma(h)$ is finite and nonnegative, and $$ \sqrt{n}(\bar X_n-\mu)\ \xrightarrow{d}\ N(0,\sigma_{\mathrm{LR}}^2). $$ If $\sigma_{\mathrm{LR}}^2>0$, division by $\sigma_{\mathrm{LR}}$ gives a standard normal limit. If $\sigma_{\mathrm{LR}}^2=0$, the displayed limit is degenerate at zero, so this standardization is not available.

The moment condition asks for a little more than a finite variance. It limits the influence of extreme observations sufficiently for the dependence restriction to control sums. The mixing condition asks that remote parts of the series become nearly independent quickly enough. The exponent $\delta/(2+\delta)$ links the required rate of mixing to the available moment.

Why Summable Dependence Identifies the Variance

The mixing and moment assumptions do more than establish a normal limit. They also make the variance in that limit well-defined. A covariance inequality for alpha-mixing variables (often called Davydov’s inequality) bounds the absolute covariance at lag $k$ by a constant times $\alpha(k)^{\delta/(2+\delta)}$, when both variables have finite $2+\delta$ moments. Thus the theorem’s summability condition implies $\sum_{k=1}^{\infty}|\gamma(k)|<\infty$.

The following calculation shows why the sum of autocovariances is the relevant variance. Since $\sqrt n(\bar X_n-\mu)=n^{-1/2}\sum_{t=1}^n(X_t-\mu)$, stationarity gives

\begin{aligned} \operatorname{Var}\!\left(\sqrt n(\bar X_n-\mu)\right) &=\frac{1}{n}\operatorname{Var}\!\left(\sum_{t=1}^n(X_t-\mu)\right)\\ &=\frac{1}{n}\left[n\gamma(0)+2\sum_{h=1}^{n-1}(n-h)\gamma(h)\right]\\ &=\gamma(0)+2\sum_{h=1}^{n-1}\left(1-\frac{h}{n}\right)\gamma(h). \end{aligned}

For each fixed lag $h$, the weight $1-h/n$ tends to 1 as $n$ grows. Absolute summability allows the weighted sum to converge to the full covariance sum: the tail can be made uniformly small by choosing a sufficiently large lag cutoff, and the finite set of remaining weights each converges to 1. Consequently, the variance above tends to $\sigma_{\mathrm{LR}}^2$. This variance calculation alone does not prove asymptotic normality; the mixing and moment assumptions in the theorem provide the additional control needed for that distributional conclusion.

Worked Example: Independent Observations as a Special Case

Suppose hypothetical observations $X_t$ are independent and identically distributed with mean $\mu=12$ units and variance $9$ units squared, with a finite $2+\delta$ moment for some $\delta>0$. Find the asymptotic distribution of the sample mean for $n=100$.

Independence makes $\alpha(k)=0$ for every $k\geq 1$, so the mixing sum is zero. Identical distribution supplies strict stationarity, and the stated moment condition supplies the required tail control. Also, $\gamma(0)=9$ and $\gamma(h)=0$ for $h\ne 0$. Therefore $\sigma_{\mathrm{LR}}^2=9$.

The theorem gives $\sqrt n(\bar X_n-12)$ approximately distributed as $N(0,9)$ for large $n$. Equivalently, for $n=100$, the sample mean is approximately normal with mean $12$ units and standard error $$ \sqrt{\frac{9}{100}}=0.3\text{ units}. $$

For example, the approximate probability that the sample mean lies within $0.6$ units of 12 is $$ P(11.4\leq \bar X_{100}\leq 12.6) \approx P(-2\leq Z\leq 2) \approx 0.9545, $$ where $Z\sim N(0,1)$. This is a large-sample approximation, not an exact statement about the finite-sample distribution.

Worked Examples With Dependence

Worked Example: A Finite-Dependent Moving-Average Series

Consider the hypothetical stationary series $X_t=\varepsilon_t+0.5\varepsilon_{t-1}$, where the innovations are independent and identically distributed with mean zero, variance $4$ units squared, and a finite $2+\delta$ moment for some $\delta>0$. Determine the limiting variance for the sample mean.

Each $X_t$ depends only on $\varepsilon_t$ and $\varepsilon_{t-1}$. Observations separated by more than one period depend on disjoint sets of independent innovations, so the series is 1-dependent and $\alpha(k)=0$ for $k\geq2$. It is strictly stationary, has mean zero, and meets the moment condition.

Using independence of distinct innovations, $$ \gamma(0)=\operatorname{Var}(\varepsilon_t+0.5\varepsilon_{t-1}) =(1+0.5^2)(4)=5, $$ and $$ \gamma(1)=\operatorname{Cov}(\varepsilon_t+0.5\varepsilon_{t-1}, \varepsilon_{t-1}+0.5\varepsilon_{t-2}) =0.5(4)=2. $$ All autocovariances at lags greater than one are zero. Thus $$ \sigma_{\mathrm{LR}}^2=5+2(2)=9. $$ It follows that $\sqrt n\,\bar X_n$ converges in distribution to $N(0,9)$. For $n=100$, the approximate standard error of the average is $\sqrt{9/100}=0.3$ units. Using only the single-period variance would instead give $\sqrt{5/100}\approx0.2236$ units, understating the uncertainty because it ignores positive covariance at lag one.

Worked Example: A Stationary Gaussian AR(1) Series

Consider the hypothetical process $X_t=0.6X_{t-1}+\varepsilon_t$, where the innovations are independent $N(0,1)$ variables and the process is in its stationary distribution. Find the variance governing the large-sample distribution of the sample mean.

Because $|0.6|<1$, the stationary solution exists. This Gaussian stationary AR(1) process is geometrically alpha-mixing, so its mixing coefficients decline at a geometric rate. Gaussian variables have finite moments of every positive order; in particular, one may take $\delta=2$. A geometric rate satisfies the required summability condition, so the theorem applies.

The stationary variance is $$ \gamma(0)=\frac{1}{1-0.6^2}=\frac{1}{0.64}=1.5625, $$ and its autocovariances are $\gamma(h)=1.5625(0.6)^h$ for $h\geq0$. Therefore, $$ \begin{aligned} \sigma_{\mathrm{LR}}^2 &=1.5625+2\sum_{h=1}^{\infty}1.5625(0.6)^h\\ &=1.5625\left(1+2\frac{0.6}{1-0.6}\right)\\ &=1.5625(4)=6.25. \end{aligned} $$ Thus $\sqrt n\,\bar X_n$ converges in distribution to $N(0,6.25)$. For $n=100$, the large-sample approximation to the standard error of the average is $\sqrt{6.25/100}=0.25$ units. Positive persistence makes this larger than the value $ \sqrt{1.5625/100}=0.125$ that would result from incorrectly treating the observations as independent.

Common Pitfall: Checking Correlations Alone

A tempting argument is: “The autocorrelations become small, so dependence is weak enough for the independent-data central limit theorem.” That reasoning is incomplete. Small or even zero correlations do not in general establish independence or alpha-mixing; a series can have uncorrelated observations while still having dependence in other features of its joint distribution. The earlier material on zero autocorrelation and dependence illustrates why correlation alone does not certify the conditions of this theorem.

The correction is to identify a valid dependence condition for the process at hand and check it together with the required moment condition. The theorem above uses alpha-mixing with a particular summability rate, not merely declining autocorrelations. Also check that the long-run variance is positive before using it to standardize the limit.

Key takeaway: Under suitable stationarity, moment, and summable-mixing conditions, a centered sample mean has an asymptotic normal distribution. Its limiting variance includes the full sequence of autocovariances, so dependence can increase or decrease the uncertainty relative to an independence calculation.

Check Your Understanding

Try these questions using the assumptions and results stated above.

Why does a theorem involving $\gamma(0)$ require finite second moments, rather than only a finite mean?
In the mixing central limit theorem, what role does the summability condition on $\alpha(k)$ play?
A stationary series has $\gamma(0)=4$, $\gamma(1)=1$, and $\gamma(h)=0$ for $h\geq2$. What is its long-run variance, if the stated central limit theorem’s other conditions hold?
Why is the ordinary independent-observation standard error generally inappropriate for a series with nonzero autocovariances?
Does a finding of zero autocorrelation at every nonzero lag, by itself, verify the alpha-mixing condition? Explain.