Sandwich Variance For GMM

Why GMM Needs a Sandwich Covariance

A GMM estimator chooses parameter values that make a vector of sample moments small, with the weighting matrix determining how discrepancies across moments enter the criterion. But the weighting matrix alone does not determine the estimator’s uncertainty. Uncertainty also depends on how sensitively the moments change with the parameter and on how variable the moments are across observations. The sandwich covariance brings these ingredients together.

The result applies to a general fixed or convergent positive-definite weighting matrix; it does not require that matrix to be an estimate of the inverse moment covariance. That distinction matters: the criterion uses a chosen metric for comparing moment discrepancies, while the sampling variance reflects the actual variability of the moments. This tutorial derives the covariance and shows how each component is estimated.

Definition: Let $m_i(\theta)$ be a $q\times 1$ vector of moment functions for observation $i$, with parameter $\theta\in\mathbb{R}^p$. Write $\bar m_n(\theta)=n^{-1}\sum_{i=1}^n m_i(\theta)$. The GMM estimator minimizes $Q_n(\theta)=\bar m_n(\theta)'W_n\bar m_n(\theta)$, where $W_n$ is a symmetric positive-definite $q\times q$ weighting matrix. At the true parameter $\theta_0$, the moment restriction is $E[m_i(\theta_0)]=0$. The moment Jacobian is $G=E[\partial m_i(\theta_0)/\partial\theta']$, a $q\times p$ matrix, and the moment covariance is $\Omega=\operatorname{Var}(m_i(\theta_0))$, a $q\times q$ matrix.

Conditions and the Asymptotic Sandwich

For the derivation below, suppose observations are independent and identically distributed, the moment restriction holds at an interior parameter $\theta_0$, and $E[m_i(\theta_0)]=0$. Assume the moment vectors have finite second moments and satisfy a central limit theorem:

\sqrt{n}\,\bar m_n(\theta_0)\ \xrightarrow{d}\ N(0,\Omega).

Assume also that $m_i(\theta)$ is differentiable in a neighborhood of $\theta_0$, that the sample Jacobian converges uniformly in that neighborhood to its population counterpart, and that $G$ has full column rank. Let $W_n\xrightarrow{p}W$, where $W$ is symmetric positive definite. Finally, the estimator is a consistent interior minimizer with a root-$n$ rate, $\hat\theta-\theta_0=O_p(n^{-1/2})$, and its first-order condition holds. These conditions ensure that the local Taylor expansion has a remainder $o_p(n^{-1/2})$ and that the sample Jacobian at the estimate can be replaced by $G$. Differentiability alone would not justify either step.

Theorem: Under these conditions, define $A=G'WG$. Full column rank of $G$ and positive definiteness of $W$ make $A$ nonsingular. Then $$ \sqrt{n}(\hat\theta-\theta_0)\ \xrightarrow{d}\ N(0,V_W), \qquad V_W=A^{-1}G'W\Omega WG A^{-1}. $$ Thus the large-sample covariance matrix of $\hat\theta$ is approximately $V_W/n$.

The expression is called a sandwich because the middle “filling” is $\Omega$, the covariance of the moments, while the two outer “bread” factors are $A^{-1}$. The factors $G$ and $W$ transmit moment uncertainty into parameter uncertainty. In general, $V_W$ is not simply $(G'WG)^{-1}$: that simpler expression omits the actual covariance structure of the moments.

Deriving the Result

Let $\bar G_n(\theta)=\partial\bar m_n(\theta)/\partial\theta'$. Differentiating the GMM criterion gives the first-order condition $\bar G_n(\hat\theta)'W_n\bar m_n(\hat\theta)=0$; the factor of two in the derivative cancels. Uniform convergence of the sample Jacobian and the root-$n$ rate justify expanding the sample moments around $\theta_0$:

\bar m_n(\hat\theta) = \bar m_n(\theta_0)+G(\hat\theta-\theta_0)+o_p(n^{-1/2}).

In the first-order condition, $\bar G_n(\hat\theta)$ can be replaced by $G$ up to a term negligible at the root-$n$ scale, and $W_n$ can be replaced by its probability limit $W$. Substituting the expansion and rearranging yields

G'WG\sqrt{n}(\hat\theta-\theta_0) = -G'W\sqrt{n}\,\bar m_n(\theta_0)+o_p(1).

Since $A=G'WG$ is nonsingular, multiply by $A^{-1}$ to obtain the asymptotic linear representation:

\sqrt{n}(\hat\theta-\theta_0) = -A^{-1}G'W\sqrt{n}\,\bar m_n(\theta_0)+o_p(1).

The central limit theorem gives the limiting covariance of $\sqrt{n}\bar m_n(\theta_0)$ as $\Omega$. For any matrix $B$, the covariance of $BZ$ is $B\operatorname{Var}(Z)B'$. Applying this identity with $B=-A^{-1}G'W$, and using the symmetry of $A$ and $W$, gives

\operatorname{Avar}\!\left(\sqrt{n}(\hat\theta-\theta_0)\right) = A^{-1}G'W\Omega WG A^{-1} = V_W.

The negative sign in the linear representation does not change the covariance: it appears on both sides of the covariance product and cancels. This derivation also shows why $\Omega$ must sit between the two weighted Jacobian factors. The weighting matrix shapes the estimator’s response to the sample moments; it is not itself a substitute for their sampling covariance.

Estimating the Components

In practice, replace the population quantities by sample estimates. The sample Jacobian at the fitted parameter is $\hat G=\partial\bar m_n(\hat\theta)/\partial\theta'$. For independent observations, a natural estimate of the moment covariance is the sample covariance of the fitted moment vectors:

\hat\Omega = \frac{1}{n}\sum_{i=1}^n \bigl(m_i(\hat\theta)-\bar m_n(\hat\theta)\bigr) \bigl(m_i(\hat\theta)-\bar m_n(\hat\theta)\bigr)'.

Using $1/(n-1)$ instead of $1/n$ gives the usual finite-sample sample covariance and is asymptotically equivalent. Centering is useful in an overidentified model because the fitted sample moments need not be exactly zero. A consistent estimate $\hat W$ of the weighting matrix’s probability limit completes the plug-in covariance:

\widehat{\operatorname{Var}}(\hat\theta) = \frac{1}{n} (\hat G'\hat W\hat G)^{-1} \hat G'\hat W\hat\Omega\hat W\hat G (\hat G'\hat W\hat G)^{-1}.

The standard error for parameter $j$ is the square root of the $j$th diagonal entry of this estimated covariance matrix. This estimator is based on independent observations. With serial or other within-group dependence, the covariance of the sample moments is not generally estimated by the independent-observation formula; long-run covariance estimation is a separate issue.

Evaluate the sample Jacobian.
Differentiate the sample moment vector with respect to the parameters and evaluate at $\hat\theta$, analytically or with a justified numerical derivative.

Estimate moment variability.
Compute the covariance across observations of $m_i(\hat\theta)$, centering by their sample mean.

Assemble the sandwich.
Insert $\hat G$, $\hat\Omega$, and $\hat W$ into the plug-in formula, retaining the factor $1/n$ for the covariance of $\hat\theta$.

Check dimensions and rank.
Confirm that $\hat G$ is $q\times p$, that $\hat\Omega$ and $\hat W$ are $q\times q$, and that $\hat G'\hat W\hat G$ is invertible.

Worked Examples

Worked Example: A General Weighting Matrix

Consider a one-parameter, two-moment model with hypothetical population quantities $G=(1,2)'$, $W=\begin{pmatrix}2&0\\0&1\end{pmatrix}$, and $\Omega=\begin{pmatrix}4&1\\1&3\end{pmatrix}$. Both $W$ and $\Omega$ are positive definite: their diagonal entries are positive and their determinants are $2$ and $11$, respectively. The scalar $G'WG$ is

G'WG=(1,2) \begin{pmatrix}2&0\\0&1\end{pmatrix} \begin{pmatrix}1\\2\end{pmatrix} =2+4=6.

The middle term is

G'W\Omega WG = (2,2) \begin{pmatrix}4&1\\1&3\end{pmatrix} \begin{pmatrix}2\\2\end{pmatrix} = (2,2)\begin{pmatrix}10\\8\end{pmatrix} =36.

Therefore $V_W=36/6^2=1$. The asymptotic variance of the estimator is $1/n$, so its asymptotic standard error is $1/\sqrt{n}$ in the parameter’s units. This calculation uses the given general weighting matrix; it does not assume that $W$ equals $\Omega^{-1}$.

Worked Example: Plugging In a Sample Moment Covariance

Suppose four hypothetical independent observations yield moments $m_i(\theta)=(a_i-\theta,\ b_i-2\theta)'$, where the recorded pairs $(a_i,b_i)$ are $(2,1),(0,2),(1,0),(1,1)$. Take $W=I$. The sample means are $\bar a=1$ and $\bar b=1$, so the criterion is $(1-\theta)^2+(1-2\theta)^2$. Its derivative is $-2(1-\theta)-4(1-2\theta)$; setting it to zero gives $10\hat\theta-6=0$, or $\hat\theta=0.6$.

The sample Jacobian is constant: $\hat G=(-1,-2)'$. The four fitted moment vectors are $(1.4,-0.2)',(-0.6,0.8)',(0.4,-1.2)'$, and $(0.4,-0.2)'$. Their mean is $(0.4,-0.2)'$, not zero. Centering each vector by this mean gives $(1,0)',(-1,1)',(0,-1)'$, and $(0,0)'$. Thus, using divisor $n-1=3$,

\hat\Omega = \frac{1}{3} \begin{pmatrix} 2&-1\\ -1&2 \end{pmatrix} = \begin{pmatrix} 2/3&-1/3\\ -1/3&2/3 \end{pmatrix}.

Here $\hat G'\hat G=5$, and $\hat G'\hat\Omega\hat G=(1,2)\hat\Omega(1,2)'=2$. Hence the estimated asymptotic variance for $\sqrt{n}(\hat\theta-\theta_0)$ is $2/25=0.08$. With $n=4$, the plug-in covariance estimate for $\hat\theta$ is $0.08/4=0.020$, and its standard error is $\sqrt{0.020}=0.1414$ parameter units. The calculation uses sample moments and does not claim that this small-sample standard error is precise.

Worked Example: Estimating a Nonlinear Moment Jacobian

For a third hypothetical illustration, let $m_i(\theta)=(y_i-\theta,\ z_i-e^\theta)'$, use $W=I$, and observe four independent pairs $(y_i,z_i)$: $(0.1,0.8),(0.3,0.8),(0.2,0.9),(0.2,0.7)$. The sample means are $\bar y=0.2$ and $\bar z=0.8$. The sample criterion is $(0.2-\theta)^2+(0.8-e^\theta)^2$. Its derivative divided by two is $h(\theta)=\theta-0.2+e^{2\theta}-0.8e^\theta$. Since $h'(\theta)=1+2e^{2\theta}-0.8e^\theta>0$ for every $\theta$ (the quadratic $1+2t^2-0.8t$ has positive minimum $0.92$ for $t>0$), $h$ is strictly increasing. Also $h(0)=0$, so $\hat\theta=0$ is the unique minimizer.

Differentiating each moment vector gives $\partial m_i(\theta)/\partial\theta'=(-1,-e^\theta)'$. Thus $\hat G=(-1,-1)'$, evaluated at $\hat\theta=0$. At that estimate, the moment vectors are $(0.1,-0.2)',(0.3,-0.2)',(0.2,-0.1)'$, and $(0.2,-0.3)'$. Their mean is $(0.2,-0.2)'$. The centered components have sums of squares $0.02$ and $0.02$, with cross-product sum zero. Using divisor three gives $\hat\Omega=\operatorname{diag}(1/150,1/150)$.

Now $\hat G'\hat G=2$, and $\hat G'\hat\Omega\hat G=1/75$. The estimated asymptotic variance is therefore $(1/75)/2^2=1/300\approx0.003333$. For $n=4$, the estimated covariance of $\hat\theta$ is $1/1200\approx0.0008333$, giving standard error $\sqrt{1/1200}\approx0.02887$ in the units of $\theta$. This example shows why the Jacobian must be evaluated at the fitted parameter in a nonlinear model.

Interpretation and Common Pitfalls

The sandwich is a sampling-uncertainty calculation under the stated moment restrictions and regularity conditions. Its middle matrix is the variability of the moment vector, and its outer factors translate that variability into parameter units. A positive-definite weighting matrix helps define the criterion, while the full-rank Jacobian ensures that changes in the parameter are distinguishable through the moments. If the Jacobian lacks full column rank, the inverse required by this formula does not exist.

A frequent mistake is to report $(\hat G'\hat W\hat G)^{-1}$ as the covariance for any GMM estimator. That expression is not the general sandwich: it leaves out $\hat\Omega$ and the surrounding weighting factors. Another common error is to force the fitted sample moments to zero. In an overidentified model, there are more moment restrictions than parameters, so minimizing the criterion generally leaves a nonzero sample moment vector. Center the moment observations when estimating their covariance.

Finally, do not confuse the estimated covariance of $\hat\theta$ with the asymptotic covariance of $\sqrt{n}(\hat\theta-\theta_0)$. The latter is $V_W$; the former is approximately $V_W/n$. Omitting $1/n$ produces a standard error too large by a factor of $\sqrt{n}$.

Key takeaway: For a general GMM weighting matrix, parameter uncertainty depends jointly on the moment Jacobian, the weighting matrix, and the covariance of the moments. Estimate all three consistently, preserve the sandwich structure, and divide the asymptotic covariance by $n$ to obtain the covariance scale for the estimator.

Check Your Understanding

Answer each question by identifying the relevant matrix or calculation.

In the sandwich formula, what does $\Omega$ measure, and how does its role differ from that of $W$?
For $q$ moments and $p$ parameters, what are the dimensions of $G$, $W$, and $G'WG$?
Why does the derivation require conditions beyond differentiability to use the Taylor expansion at the root-$n$ scale?
Why can the fitted sample moment mean be nonzero in an overidentified model, and how is the sample moment covariance centered?
If $V_W$ is the asymptotic covariance of $\sqrt{n}(\hat\theta-\theta_0)$, what matrix approximates the covariance of $\hat\theta$ itself?

What You'll Learn