Deriving HAC Covariance Estimators

Why a Regression Covariance Must Account for Time

In an ordered data set, adjacent regression observations can carry related estimation errors. A covariance estimator that treats score contributions from different times as uncorrelated then omits their cross-products. Heteroskedasticity-and-autocorrelation-consistent (HAC) estimation addresses this by estimating the long-run covariance of the regression scores, then placing that estimate inside the familiar regression sandwich.

The aim here is to derive that construction and identify conditions under which it estimates the relevant asymptotic covariance. The earlier material on Serial Dependence and Long-Run Variance supplies the central idea: the variance of a long sum depends on autocovariances at every lag, not only on the lag-zero variance.

Each \(g_t\) is a \(p\)-vector, so each \(\Gamma_h\) and \(\Omega\) is a \(p\times p\) matrix. The transpose term is essential: \(\Gamma_h\) records covariance in one time direction, and \(\Gamma_h'\) supplies the corresponding reverse direction. Although errors may be heteroskedastic, HAC estimation does not require the score contributions to be uncorrelated over time.

From the Score Sum to the Sandwich

Let \(\hat\beta\) be ordinary least squares, let \(Q_n=X'X/n\), and suppose the regression model and regularity conditions give the expansion \(\sqrt{n}(\hat\beta-\beta)=Q_n^{-1}n^{-1/2}\sum_{t=1}^n g_t+o_p(1)\). This says that, to first order, coefficient estimation error is a transformed average of the score contributions. If \(Q_n\) converges to a positive-definite matrix \(Q\), and the normalized score sum has limiting covariance \(\Omega\), then the asymptotic covariance of \(\sqrt{n}(\hat\beta-\beta)\) is \(Q^{-1}\Omega Q^{-1}\). Consequently, the covariance of \(\hat\beta\) is approximately \(Q^{-1}\Omega Q^{-1}/n\).

For computation, replace the unknown score \(g_t\) by its residual-based estimate \(\hat g_t=x_t\hat u_t\), where \(\hat u_t=y_t-x_t'\hat\beta\). For lag \(h\geq 0\), define the sample autocovariance \(\hat\Gamma_h=n^{-1}\sum_{t=h+1}^{n}\hat g_t\hat g_{t-h}'\). The denominator is \(n\), including at positive lags; this convention makes the corresponding matrix formula transparent.

The same estimator can be written directly as a weighted sum of score cross-products, surrounded by the regression bread matrices:

This equality follows by substituting the definition of each \(\hat\Gamma_h\) and using \(X'X=nQ_n\). It also clarifies the distinction between a heteroskedasticity-robust covariance estimate and a HAC estimate: the lag-zero term is the same type of score outer-product sum, while HAC adds weighted cross-products between observations at different times.

Why These Autocovariances Appear

The estimator is motivated by the exact finite-sample covariance identity for a stationary, mean-zero score process. Expand the outer product of the normalized score sum. Terms with time separation \(h\) occur \(n-h\) times in each direction. Dividing by \(n\) therefore gives:

If the autocovariances are absolutely summable, the contribution from sufficiently distant lags becomes small, and the finite-sample expression approaches \(\Omega\) as \(n\) grows. The HAC estimator imitates this sum: it estimates autocovariances from the data, includes lags up to \(L\), and weights the included lags. Weighting can reduce the influence of noisy estimates at distant lags; consistency requires the truncation and weights to retain the relevant dependence as the sample grows.

Scalar Calculations and the Regression Scale

In a one-parameter calculation, the matrices reduce to scalars. The long-run covariance estimate is then a weighted estimate of the score variance plus twice the weighted score autocovariance at each included positive lag. The regression design still matters: the score covariance is transformed by the inverse design second moment on both sides.

Dependence and Bandwidth Conditions

An asymptotically valid HAC estimate needs conditions both on the score sequence and on the way lags are included. A useful set of sufficient conditions is as follows. The model has fixed \(p\), \(Q_n\) converges in probability to a positive-definite \(Q\), and the score process is stationary, has mean zero, and satisfies a central limit theorem for \(n^{-1/2}\sum_t g_t\). Its long-run covariance exists; a common sufficient condition is absolute summability of its autocovariance matrices, meaning \(\sum_{h=-\infty}^{\infty}\|\Gamma_h\|<\infty\). The residual-based score substitutions must also be accurate enough that they do not change the limiting covariance estimate.

For example, a stationary score process with a suitable finite moment of order greater than four and sufficiently fast-decaying strong-mixing dependence can provide the needed central limit and covariance-estimation results. These are sufficient conditions, not a claim that every application must use precisely that dependence model. In practice, the relevant assumptions concern the sequence of regression scores, which combines regressors and errors; describing the errors alone does not establish them.

For a lag window \(w_{h,L}=K(h/L)\), standard consistency conditions include \(K(0)=1\), suitable boundedness and continuity near zero, and weights that suppress sufficiently distant lags. The bandwidth must grow so that more lags can contribute, while remaining small relative to the sample size:

These limits express a balance. If \(L_n\) stays fixed, autocovariances beyond that fixed lag are omitted even when they contribute to \(\Omega\). If \(L_n\) grows as fast as the sample, estimates of many poorly measured distant-lag covariances can add substantial noise. Under appropriate dependence, moment, and kernel conditions, the stated bandwidth behavior allows the included range to expand while keeping it short compared with the available observations. Exact sufficient rate restrictions can depend on the dependence and moment assumptions.

Interpretation and a Common Pitfall

HAC estimation changes the estimated uncertainty of the regression coefficients; it does not itself change the OLS coefficient estimates. A coefficient standard error is in the units of that coefficient, and a confidence interval or test using it inherits the asymptotic justification only to the extent that the model, dependence, and bandwidth conditions are credible. HAC inference does not turn an associational regression coefficient into a causal effect.

Common mistake: “There are only a few autocorrelated lags in my sample, so I can hold \(L\) fixed as the sample grows and still estimate the long-run covariance.” This generally fails because the target sums over all lags: a fixed cutoff permanently discards any nonzero autocovariances beyond it. Under the standard bandwidth conditions, \(L_n\) grows, but \(L_n/n\) tends to zero. A finite-sample calculation can use a selected finite bandwidth; the asymptotic argument explains why a fixed bandwidth is not generally a consistent sequence of estimators.