Serial Dependence and Long-Run Variance

Why Serial Dependence Changes Regression Uncertainty

In regression data ordered over time, neighboring observations may share shocks. If an error is unusually large this period, the next error may also tend to be large. The OLS coefficient can still target the intended parameter under suitable exogeneity conditions, but its sampling variance depends on whether the observations’ contributions to the estimating equations move together over time. A variance calculation that treats those contributions as unrelated can therefore understate or overstate uncertainty.

The key object is not only the variance of the error at a single date. It is the long-run variance of the regression score contributions: their contemporaneous covariance plus the covariances between contributions at different time lags. This tutorial develops that object and explains why a heteroskedasticity-only covariance estimate, which retains only the contemporaneous component, can be inadequate.

Model and notation: For observations $t=1,\ldots,n$, write the linear regression as $y_t=x_t'\beta+u_t$, where $x_t$ is a $k$-vector of regressors, $\beta$ is the coefficient vector, and $u_t$ is the regression error. The score contribution at date $t$ is $g_t=x_tu_t$. Let $Q$ denote the probability limit of $n^{-1}\sum_{t=1}^n x_tx_t'$, assumed finite and nonsingular.

For the variance results below, assume the regression is correctly specified in the sense that $E[x_tu_t]=0$, and that a law of large numbers gives the stated limit $Q$. For the long-run variance result, also assume the score process is stationary with finite second moments, its covariances are absolutely summable, and an appropriate central limit theorem applies to its partial sums. These conditions are standard sufficient conditions for the asymptotic argument; stationarity alone does not guarantee it.

Long-Run Variance of the Score

Definition: For a stationary score process $g_t=x_tu_t$, define its lag-$h$ covariance matrix by $\Gamma_h=E[g_tg_{t-h}']$. Its long-run variance matrix is $$ \Omega=\sum_{h=-\infty}^{\infty}\Gamma_h =\Gamma_0+\sum_{h=1}^{\infty}\left(\Gamma_h+\Gamma_h'\right). $$ The matrix $\Gamma_0$ is the contemporaneous score covariance. The other terms account for dependence between score contributions at different dates.

The long-run variance is a matrix because a regression can have several coefficients. For any fixed coefficient contrast $c$, the score component relevant to that contrast is $c'Q^{-1}g_t$; its long-run variance is $c'Q^{-1}\Omega Q^{-1}c$. As a variance, this quantity cannot be negative. The terms at positive and negative lags are paired because $\Gamma_{-h}=\Gamma_h'$; an individual lag covariance matrix need not be symmetric.

Under the stated conditions, OLS has the asymptotic representation

\sqrt{n}(\hat{\beta}-\beta) = Q^{-1}\frac{1}{\sqrt{n}}\sum_{t=1}^n g_t+o_p(1), \qquad \operatorname{Avar}\!\left(\sqrt{n}(\hat{\beta}-\beta)\right) = Q^{-1}\Omega Q^{-1}.

Here $o_p(1)$ denotes a term that converges in probability to zero. Thus the approximate covariance matrix of $\hat{\beta}$ is $n^{-1}Q^{-1}\Omega Q^{-1}$. Serial dependence affects inference through $\Omega$, even when the regression’s limiting design matrix $Q$ is unchanged.

Why Cross-Time Covariances Enter

The variance of the sum of the score contributions contains every pair of dates, not just pairs that coincide. Stationarity lets us group those pairs by their time separation. The following calculation gives the finite-sample identity behind the long-run variance.

Expand the variance of the score sum.
Since $E[g_t]=0$, the covariance of the sum is the sum of all pairwise covariance matrices.

Group terms by lag.
There are $n-h$ pairs separated by lag $h$ in each direction, so stationarity gives a weight of $1-h/n$ after dividing by $n$.

Take the long-sample limit.
Absolute summability of the lag covariances allows the weighted sum to converge to the sum over all lags, $\Omega$.

\begin{aligned} \operatorname{Var}\!\left(\frac{1}{\sqrt{n}}\sum_{t=1}^n g_t\right) &=\frac{1}{n}\sum_{t=1}^n\sum_{s=1}^n\operatorname{Cov}(g_t,g_s)\\ &=\Gamma_0+\sum_{h=1}^{n-1}\left(1-\frac{h}{n}\right)(\Gamma_h+\Gamma_h')\\ &\longrightarrow \Omega. \end{aligned}

The lag weights arise because a sample of length $n$ contains fewer pairs at large separations: only $n-h$ pairs are separated by $h$. The limit is the long-run variance when the weighted tails become negligible. This justifies why a contemporaneous variance alone does not generally describe the uncertainty of a sum of dependent contributions.

Worked Examples

Worked Example: Persistent Errors in an Intercept-Only Regression

Suppose a hypothetical time series follows $y_t=\beta+u_t$, and the errors are stationary AR(1): $u_t=\rho u_{t-1}+\varepsilon_t$, with $\rho=0.5$, $E[u_t]=0$, and $\operatorname{Var}(u_t)=4$. Assume the innovations and process satisfy the regularity conditions needed for the stationary AR(1) covariance formula. Compare the asymptotic variance of the sample mean using the long-run variance with the contemporaneous-only value.

In this regression $x_t=1$, so $g_t=u_t$ and $Q=1$. The lag-$h$ error covariance is $\Gamma_h=4(0.5)^{|h|}$. Therefore,

\Omega =4+2\sum_{h=1}^{\infty}4(0.5)^h =4\left(1+2\sum_{h=1}^{\infty}(0.5)^h\right) =4\left(\frac{1+0.5}{1-0.5}\right) =12.

The geometric series converges because $|0.5|<1$. The long-run asymptotic variance of $\hat{\beta}$ is $12/n$. A heteroskedasticity-only calculation uses the lag-zero variance, $4$, and gives $4/n$. At $n=100$, these approximate variances are $0.12$ and $0.04$, respectively, so the corresponding standard errors are $\sqrt{0.12}=0.3464$ and $\sqrt{0.04}=0.2000$, in the units of $y_t$. Positive persistence makes the contemporaneous-only calculation substantially too small here.

Worked Example: Exact Finite-Sample Variance of a Sample Mean

For a second hypothetical example, let $y_t=\beta+u_t$, where $u_t$ has variance $1$ and AR(1) correlation $\operatorname{Corr}(u_t,u_{t-h})=0.4^{|h|}$. With $n=5$, calculate the exact variance of the sample mean and compare it with the value that would result from ignoring cross-time covariances. The assumed stationary AR(1) process has zero mean errors and finite second moments.

The sample mean’s variance is $n^{-2}$ times the variance of the error sum. Equivalently, using the lag weights derived above, it is $n^{-1}$ times the weighted covariance sum:

\begin{aligned} \operatorname{Var}(\bar{y}) &=\frac{1}{5}\left[1+ 2\left\{ \left(1-\frac{1}{5}\right)(0.4) +\left(1-\frac{2}{5}\right)(0.4)^2 +\left(1-\frac{3}{5}\right)(0.4)^3 +\left(1-\frac{4}{5}\right)(0.4)^4 \right\}\right]\\ &=\frac{1}{5}\left[1+2(0.32000+0.09600+0.02560+0.00512)\right]\\ &=\frac{1.89344}{5}=0.378688. \end{aligned}

Ignoring the nonzero-lag covariances gives $1/5=0.200000$. The exact standard deviation of the sample mean is $\sqrt{0.378688}=0.6154$, compared with $\sqrt{0.2}=0.4472$ under the independent-error calculation. These are finite-sample variances for the specified model, rather than estimates from observed data. The difference illustrates how positive serial correlation can make a time series of five observations contain less information about its mean than five independent observations with the same marginal variance.

Worked Example: Long-Run Covariance for Two Score Components

Suppose a hypothetical regression has two score components with lag covariances $\Gamma_0=\begin{pmatrix}4&0\\0&2\end{pmatrix}$ and $\Gamma_1=\begin{pmatrix}1&0\\0&0.5\end{pmatrix}$, with $\Gamma_{-1}=\Gamma_1'$ and all covariances at lags of magnitude two or greater equal to zero. Assume $Q=I$, the two-dimensional identity matrix, and the score process satisfies the regularity conditions for the asymptotic variance formula. Find the long-run variance and compare it with the contemporaneous-only matrix.

The lag-one matrix is symmetric, so its contribution from the two directions is $2\Gamma_1$. Hence

\Omega =\Gamma_0+\Gamma_1+\Gamma_1' = \begin{pmatrix}4&0\\0&2\end{pmatrix} + \begin{pmatrix}1&0\\0&0.5\end{pmatrix} + \begin{pmatrix}1&0\\0&0.5\end{pmatrix} = \begin{pmatrix}6&0\\0&3\end{pmatrix}.

Because $Q=I$, the asymptotic covariance of $\sqrt{n}(\hat{\beta}-\beta)$ is $\Omega$, and the approximate covariance of $\hat{\beta}$ is $\Omega/n$. The contemporaneous-only matrix would instead give $\Gamma_0/n$. In particular, the first coefficient’s asymptotic variance is $6/n$, not $4/n$, and the second’s is $3/n$, not $2/n$. This example shows that long-run covariance is calculated for score vectors; it is not merely a label for the variance of an individual regression error.

When Heteroskedasticity-Only Covariance Estimates Fail

A heteroskedasticity-only covariance estimate allows the variance of each observation’s error to vary with its regressors, but its asymptotic “meat” contains only $\Gamma_0$. It therefore targets the appropriate score covariance when nonzero-lag score covariances are absent. In contrast, a long-run variance includes the sum of those lag covariances.

Key condition: A heteroskedasticity-only covariance estimate is generally valid for OLS asymptotic variance when the aggregate nonzero-lag score covariance is zero: $$ \sum_{h\ne 0}\Gamma_h=0. $$ A sufficient, stronger condition is $\Gamma_h=0$ at every nonzero lag. If the aggregate is nonzero and does not become asymptotically negligible, using only $\Gamma_0$ generally gives the wrong asymptotic variance.

This condition is about the score $x_tu_t$, not only about the error $u_t$. Serial correlation in errors often induces serial correlation in scores, but the two statements are not identical: changing regressors can alter or eliminate score covariances. Conversely, errors that are uncorrelated across time do not by themselves rule out dependence in score contributions if the regressors or other components have dependence. The relevant quantity for OLS covariance is the long-run covariance of $x_tu_t$.

Even when an error process is serially correlated, cross-time score covariances may be zero in a particular design; in that case the omission of lag terms need not change the asymptotic target. Also, nonzero lag covariances could in principle sum to zero. Such cancellation can make the long-run and contemporaneous matrices agree, but it should not be presumed just because an estimate based only on contemporaneous terms is convenient. The data structure and assumptions must support the covariance calculation being used.

Common Pitfall

Wrong reasoning: “The regression includes an intercept and the errors have a finite variance, so a heteroskedasticity-robust standard error accounts for all dependence.”
Correction: Allowing unequal variances across observations is not the same as including covariance between different dates. A heteroskedasticity-only estimate captures the lag-zero score covariance; serial score dependence requires the long-run variance, or an assumption that makes its nonzero-lag terms irrelevant.

Another error is to conclude that positive error autocorrelation always inflates every coefficient’s variance. The effect depends on the score covariances and the coefficient contrast: lag covariances can be negative, positive, or structured differently across coefficients. The correct comparison is between $Q^{-1}\Omega Q^{-1}$ and the contemporaneous-only counterpart, not a blanket rule about the direction of change.

Key takeaway: For dependent regression data, OLS uncertainty is governed by the long-run variance of the score contributions $x_tu_t$. Heteroskedasticity-only covariance estimates omit cross-time score covariances and are justified only when those omitted terms do not affect the asymptotic variance.

Check Your Understanding

Use the definitions and results above to answer the following questions.

What is the score contribution $g_t$ in a regression with regressor vector $x_t$ and error $u_t$?
Why does the long-run variance include both $\Gamma_h$ and $\Gamma_h'$ for positive lags?
In the intercept-only AR(1) example, what happens to the long-run variance as the positive persistence parameter increases toward one?
State a sufficient condition under which a heteroskedasticity-only covariance estimate has the correct asymptotic target.
Why is serial correlation in $u_t$ not, by itself, a complete description of the covariance relevant to OLS inference?