Long-Run Covariance Estimation For GMM

Why GMM Needs Long-Run Covariance

GMM is built from moment observations, and those observations may be serially dependent. For example, moment vectors formed from time-series data can remain correlated across dates even when their population mean is zero. In that setting, the covariance of one date's moment vector does not by itself describe the variability of the sample average. The missing information is the sequence of covariances across time.

The matrix that collects this dependence is the long-run covariance. Estimating it with a heteroskedasticity-and-autocorrelation-consistent (HAC) estimator allows GMM inference and weighting to account for both changing variance and serial correlation. HAC is not a correction to the moment conditions themselves: it estimates the sampling variability generated when those moments are averaged over time.

Definition: Let $g_t(\theta)$ be a $q$-dimensional moment vector observed at date $t$, and suppose the true parameter is $\theta_0$, with $E[g_t(\theta_0)]=0$. Write $u_t=g_t(\theta_0)-E[g_t(\theta_0)]$, and define $\Gamma_j=E[u_tu_{t-j}']$ for $j\geq 0$. If the covariance series is well behaved, the long-run covariance matrix is $$ S=\sum_{j=-\infty}^{\infty}E[u_tu_{t-j}'] =\Gamma_0+\sum_{j=1}^{\infty}(\Gamma_j+\Gamma_j'). $$ It describes the asymptotic covariance of the scaled sample average: $\sqrt{n}\,\bar g$ has limiting covariance $S$ under a central limit theorem for the moment sequence.

The transpose in this expression matters. For vector moments, $\Gamma_j$ need not be symmetric: it records how the moment vector at one date covaries with the vector $j$ dates earlier. The paired matrix $\Gamma_j+\Gamma_j'$ is symmetric, as a covariance matrix must be. When $q=1$, the transposes have no effect and the expression reduces to the familiar sum of contemporaneous variance and twice each positive-lag covariance.

Sample HAC Estimation

In practice, the population moments are unknown and the moments are evaluated at an estimate, usually a preliminary or final GMM estimate $\hat\theta$. Let $\bar g=n^{-1}\sum_{t=1}^n g_t(\hat\theta)$, and center each observation as $\hat u_t=g_t(\hat\theta)-\bar g$. For lag $j$, estimate the autocovariance matrix by

\hat\Gamma_j=\frac{1}{n}\sum_{t=j+1}^{n}\hat u_t\hat u_{t-j}', \qquad j\geq 0.

The factor $1/n$, rather than $1/(n-j)$, is a standard convention for HAC estimation. It keeps all lag estimates on a common scale; other finite-sample normalizations exist, but must be used consistently. A kernel HAC estimator truncates and weights the lag terms:

\hat S=\hat\Gamma_0+\sum_{j=1}^{L} k_j(\hat\Gamma_j+\hat\Gamma_j').

Here $L$ is the bandwidth, the largest included lag, and $k_j$ is a kernel weight. A widely used choice is the Bartlett kernel, with $k_j=1-j/(L+1)$. It gives more weight to nearby lags and progressively less to more distant ones. For this choice, the resulting HAC estimate is positive semidefinite when constructed from the sample autocovariances above. That property is useful because a covariance estimate should not imply negative variance in some direction.

Worked Example: A Scalar Bartlett HAC Estimate

Suppose a hypothetical sequence of four centered scalar moment observations is $1,-1,1,-1$. Estimate its long-run covariance using the Bartlett kernel with bandwidth $L=1$. The sample mean is zero, so the displayed sequence is already centered.

The lag-zero estimate is $$ \hat\Gamma_0=\frac{1^2+(-1)^2+1^2+(-1)^2}{4}=1. $$ At lag one, the three available products are $(-1)(1)=-1$, $(1)(-1)=-1$, and $(-1)(1)=-1$. Thus $$ \hat\Gamma_1=\frac{-1-1-1}{4}=-0.75. $$

For $L=1$, the Bartlett weight is $k_1=1-1/(1+1)=0.5$. The HAC estimate is therefore $$ \hat S=\hat\Gamma_0+k_1(2\hat\Gamma_1) =1+0.5\{2(-0.75)\}=0.25. $$

The negative lag-one covariance reduces the estimated long-run variance relative to the contemporaneous variance of 1: alternating moments partly offset when averaged. This is a calculation from a very small hypothetical sequence, not a claim that the estimate is precise; reliable HAC estimation depends on the amount of data and the dependence assumptions.

Vector Moments and Lag Orientation

With multiple moment conditions, autocovariances are matrices. For a lag $j$, each entry of $\hat\Gamma_j$ pairs a component at time $t$ with a component at time $t-j$. It is important not to replace $\hat\Gamma_j$ by its transpose before adding both terms: the two orientations carry distinct cross-covariances.

Worked Example: Computing a Two-Moment HAC Matrix

Consider the hypothetical centered moment vectors $u_1=(1,0)'$, $u_2=(0,1)'$, $u_3=(-1,0)'$, and $u_4=(0,-1)'$. Their sample mean is $(0,0)'$. Compute a Bartlett estimate with $L=1$.

The lag-zero estimate is the average of the four outer products: $$ \hat\Gamma_0=\frac{1}{4}\sum_{t=1}^{4}u_tu_t' =\begin{pmatrix}0.5&0\\0&0.5\end{pmatrix}. $$ For lag one, form $u_tu_{t-1}'$ for $t=2,3,4$. Their sum is $$ \begin{pmatrix}0&0\\1&0\end{pmatrix} +\begin{pmatrix}0&-1\\0&0\end{pmatrix} +\begin{pmatrix}0&0\\1&0\end{pmatrix} =\begin{pmatrix}0&-1\\2&0\end{pmatrix}. $$ Dividing by $n=4$ gives $$ \hat\Gamma_1=\begin{pmatrix}0&-0.25\\0.5&0\end{pmatrix}. $$

The Bartlett weight at lag one is $0.5$. Adding the lag matrix and its transpose gives $$ \hat S=\hat\Gamma_0+0.5(\hat\Gamma_1+\hat\Gamma_1') =\begin{pmatrix}0.5&0.125\\0.125&0.5\end{pmatrix}. $$

For example, the upper-left entry of $\hat\Gamma_1$ is zero: the first-coordinate products across the three adjacent pairs are $0(1)=0$, $(-1)(0)=0$, and $0(-1)=0$, whose sum divided by four remains zero. The final HAC matrix is symmetric. Its eigenvalues are $0.5+0.125=0.625$ and $0.5-0.125=0.375$, both nonnegative, as expected for this Bartlett estimate.

Why the Estimator Can Be Consistent

A HAC estimator approximates an infinite covariance sum with a finite, weighted sum. Its consistency requires conditions that make this approximation reliable. A standard framework assumes a stationary moment sequence with finite second moments, weak enough dependence for a central limit theorem, and summable autocovariances. In particular, absolute summability of the entries of the covariance sequence ensures that distant-lag contributions eventually become small. In applications, moments are evaluated at an estimator; consistency also requires that this estimator converges to $\theta_0$ and that the moment functions behave regularly near $\theta_0$.

The bandwidth must grow with the sample size to include increasingly distant dependence, but not so quickly that noisy high-lag estimates dominate. A common asymptotic requirement is $L\to\infty$ and $L/n\to 0$, together with kernel regularity and the relevant dependence conditions. The Bartlett weights satisfy the basic tapering requirement. These conditions describe large-sample behavior; they do not guarantee that a particular bandwidth performs well in a short sample.

Center the estimated moments.
Evaluate $g_t$ at the parameter estimate and subtract the sample mean, so the covariance calculation concerns deviations around the estimated moment average.

Estimate relevant lag covariances.
For each lag through $L$, pair the current centered vector with its lagged counterpart in the stated order.

Weight and symmetrize the lag contributions.
Apply the kernel weights and include $\hat\Gamma_j+\hat\Gamma_j'$, yielding a symmetric estimate of long-run covariance.

Connection to GMM Inference and Weighting

Let $G$ denote the Jacobian matrix of the population moments with respect to the parameter, evaluated at $\theta_0$, and let $W$ be the GMM weight matrix used in estimation. Under the regularity conditions for GMM asymptotic normality, the sandwich covariance matrix for $\sqrt{n}(\hat\theta-\theta_0)$ is

(G'WG)^{-1}G'WSWG(G'WG)^{-1}.

When moments are dependent, the $S$ in this formula is the long-run covariance, not merely $\Gamma_0$. Replacing $S$ by a HAC estimate $\hat S$ provides a feasible estimate of the covariance used for standard errors, Wald statistics, and confidence regions. The covariance of $\hat\theta$ itself is estimated by dividing the asymptotic covariance of $\sqrt{n}(\hat\theta-\theta_0)$ by $n$.

Worked Example: HAC Covariance in a Scalar GMM Standard Error

Suppose a hypothetical scalar GMM problem has one parameter, one moment, estimated Jacobian $G=2$, and a fixed scalar weight $W=1$. Assume the fitted moments produce a HAC estimate $\hat S=1.44$, and the sample size is $n=400$. Compute the asymptotic standard error using the sandwich formula.

In this scalar case, the sandwich variance for the scaled estimator is $$ \frac{1}{G^2}W^2\hat S=\frac{1^2(1.44)(2^2)}{(2^2)^2} =\frac{1.44}{4}=0.36. $$ Thus the estimated variance of $\hat\theta$ is $0.36/400=0.0009$, and its standard error is $$ \widehat{SE}(\hat\theta)=\sqrt{0.0009}=0.03. $$

The estimate $1.44$ is the long-run variance of the moment sequence; $G=2$ converts moment variation into parameter variation, and division by the sample size accounts for averaging. The standard error is in the units of the parameter. This example holds the weight fixed to isolate the role of $\hat S$; it does not prescribe how to choose a GMM weight matrix.

The same long-run covariance also informs the choice of GMM weights. Different weighting matrices can yield different precision, and a reliable estimate of $S$ is needed when constructing weights intended to account for the covariance among moments. The derivation of an efficient weight is a separate step; here the essential point is that a contemporaneous covariance estimate generally omits the serial covariance contributions.

Common Mistakes and Practical Checks

Using only $\hat\Gamma_0$. This treats moments at different dates as uncorrelated. If serial dependence is present, include the weighted lag terms.
Forgetting the transpose. In vector moments, $\hat\Gamma_j$ may not be symmetric. Add $\hat\Gamma_j+\hat\Gamma_j'$, rather than doubling $\hat\Gamma_j$ itself.
Reversing the lag convention midway. Under $\Gamma_j=E[u_tu_{t-j}']$, compute each sample product as $\hat u_t\hat u_{t-j}'$. Changing the order transposes the matrix and changes its interpretation.
Treating the bandwidth as inconsequential. Too small a bandwidth can omit relevant dependence; too large a bandwidth can add noisy terms. Report the kernel and bandwidth, and consider whether the choice is plausible for the time scale of dependence.
Interpreting HAC as protection against every problem. HAC addresses heteroskedasticity and serial dependence under suitable large-sample conditions. It does not repair invalid moment restrictions, weak identification, or a misspecified model.

Key takeaway: GMM moment averages have long-run covariance $S$, which combines contemporaneous covariance with covariance across lags. A kernel HAC estimate approximates this sum; its centering, lag orientation, kernel, and bandwidth should be stated clearly because they feed directly into GMM uncertainty estimates and weighting.

Check Your Understanding

Try these questions before moving on.

Why does the long-run covariance include both $\Gamma_j$ and $\Gamma_j'$ for each positive lag when moments are vectors?
For the Bartlett kernel and bandwidth $L=2$, what are the weights $k_1$ and $k_2$?
Under what bandwidth conditions does a standard HAC consistency argument let the maximum included lag increase with sample size?
If moment observations are serially correlated, what feature of the sampling variability is missed by using only the lag-zero covariance?
In the GMM sandwich formula, where does the estimated long-run covariance enter, and what does it contribute to inference?