Asymptotic Normality Of GMM

From Consistency to a Local Distribution

Consistency says that a GMM estimator approaches the parameter value as the sample grows. Asymptotic normality adds a finer description: it identifies the estimator’s typical error at the $1/\sqrt{n}$ scale and the covariance governing that error. This approximation is the basis for large-sample uncertainty calculations.

The key idea is that, near the true parameter, the sample moment vector changes approximately linearly with the parameter. The estimator then offsets the random sampling error in the moments by moving the parameter. The moment Jacobian determines how much movement is needed, while the moment covariance describes the size and dependence of the sampling error.

Definition: Let $\theta_0\in\mathbb{R}^k$ be the true parameter and let $g_i(\theta)\in\mathbb{R}^q$ be the moment vector for observation $i$, with $q\geq k$. Define the sample moment vector as $$ g_n(\theta)=\frac{1}{n}\sum_{i=1}^{n}g_i(\theta). $$ Under correct specification, $E[g_i(\theta_0)]=0$. The population moment Jacobian is $G=E[\partial g_i(\theta_0)/\partial\theta']$, a $q\times k$ matrix, and the moment covariance is $\Omega=\operatorname{Var}(g_i(\theta_0))$, a $q\times q$ matrix.

For a fixed symmetric positive-definite weighting matrix $W$ of dimension $q\times q$, the GMM estimator minimizes $g_n(\theta)'Wg_n(\theta)$. The results below concern the case where $W$ does not change with the sample size. A fixed weight need not be an efficient choice; it still yields a well-defined asymptotic distribution when the identification and regularity conditions hold.

Conditions for the Result

Use the following conditions to interpret the derivation. They make explicit where the limiting normal approximation comes from and when it is applicable.

Correct specification: $E[g_i(\theta_0)]=0$, and the observations are independent and identically distributed. The moment vector has finite second moments, and a central limit theorem applies to its sample mean.
Consistency and an interior solution: the estimator $\hat{\theta}$ converges in probability to $\theta_0$, and $\theta_0$ is an interior point of the parameter space. Consistency is the result established in Consistency of GMM.
Smoothness: the moments are differentiable near $\theta_0$; sample Jacobians converge to $G$ in that neighborhood, so a first-order Taylor expansion has a remainder negligible at the scale used below.
Local identification: $G$ has full column rank. Together with positive-definite $W$, this makes $G'WG$ invertible.

For a nondegenerate limiting distribution, assume also that $\Omega$ is positive definite. We write $A=G'WG$, a $k\times k$ matrix. The notation $o_p(1)$ denotes a term converging to zero in probability.

Theorem: Under these conditions, the fixed-weight GMM estimator has the first-order stochastic expansion $$ \sqrt{n}(\hat{\theta}-\theta_0) = -A^{-1}G'W\frac{1}{\sqrt{n}}\sum_{i=1}^{n}g_i(\theta_0)+o_p(1), \qquad A=G'WG. $$ Consequently, $$ \sqrt{n}(\hat{\theta}-\theta_0)\ \xrightarrow{d}\ N\!\left(0,\ A^{-1}G'W\Omega WG A^{-1}\right). $$

Here $N(0,V)$ denotes a multivariate normal distribution with mean zero and covariance matrix $V$. The minus sign in the expansion expresses the adjustment: if the sample moments at $\theta_0$ happen to be positive in a direction, the parameter shifts in the direction that reduces the weighted moments. The sign does not affect the covariance matrix.

Deriving the First-Order Expansion

Because $\theta_0$ is interior and the criterion is differentiable, an interior minimizer satisfies the first-order condition. Differentiating the criterion with respect to $\theta$ gives $$ \frac{\partial}{\partial\theta}\{g_n(\theta)'Wg_n(\theta)\} = 2G_n(\theta)'Wg_n(\theta), $$ where $G_n(\theta)=\partial g_n(\theta)/\partial\theta'$ is the sample Jacobian. Thus $G_n(\hat{\theta})'Wg_n(\hat{\theta})=0$.

Consistency and the smoothness conditions allow a Taylor expansion of the moments around $\theta_0$: $$ g_n(\hat{\theta}) = g_n(\theta_0)+G(\hat{\theta}-\theta_0) +o_p(\|\hat{\theta}-\theta_0\|). $$ The sample Jacobian in the first-order condition converges to $G$. Substituting the expansion into that condition and retaining its first-order terms gives $$ 0=G'Wg_n(\theta_0)+G'WG(\hat{\theta}-\theta_0) +o_p(n^{-1/2}). $$ The $n^{-1/2}$ order follows from the central limit theorem, which implies $g_n(\theta_0)=O_p(n^{-1/2})$; the expansion and consistency conditions ensure the discarded terms are negligible at that scale. Since $A=G'WG$ is invertible, rearrangement yields $$ \sqrt{n}(\hat{\theta}-\theta_0) = -A^{-1}G'W\sqrt{n}\,g_n(\theta_0)+o_p(1) = -A^{-1}G'W\frac{1}{\sqrt{n}}\sum_{i=1}^{n}g_i(\theta_0)+o_p(1). $$

Finally, the central limit theorem gives $n^{-1/2}\sum_i g_i(\theta_0)\xrightarrow{d}N(0,\Omega)$. Applying the linear transformation $-A^{-1}G'W$ yields the stated covariance. This is an application of the standard multivariate central limit theorem and continuous mapping result; the GMM-specific step is the expansion linking moment noise to parameter error.

Worked Applications

Worked Example: One Parameter and Two Moments

Suppose a hypothetical model has one parameter $\theta$ and two correctly specified moments. At the true value, let the moment Jacobian and moment covariance be $$ G=\begin{pmatrix}1\\2\end{pmatrix}, \qquad \Omega=\begin{pmatrix}4&1\\1&9\end{pmatrix}, \qquad W=I_2. $$ Assume independent observations, the regularity conditions above, and an interior consistent estimator. Find the limiting variance of $\sqrt{n}(\hat{\theta}-\theta_0)$.

First, $A=G'WG=1^2+2^2=5$, so $A^{-1}=1/5$. Next, $$ G'\Omega G = \begin{pmatrix}1&2\end{pmatrix} \begin{pmatrix}4&1\\1&9\end{pmatrix} \begin{pmatrix}1\\2\end{pmatrix} = \begin{pmatrix}1&2\end{pmatrix} \begin{pmatrix}6\\19\end{pmatrix} =44. $$ Therefore the asymptotic variance is $44/5^2=1.760$. The limiting distribution is $$ \sqrt{n}(\hat{\theta}-\theta_0)\xrightarrow{d}N(0,1.760). $$

For a sample of size $n$, this implies an approximate standard deviation of $\hat{\theta}$ equal to $\sqrt{1.760/n}$, in the units of $\theta$. It is a large-sample approximation, not an exact finite-sample distribution.

Worked Example: Applying the Expansion to Sample Moments

Consider a hypothetical scalar parameter measured in years, with two moments and fixed weight $$ G=\begin{pmatrix}1\\2\end{pmatrix}, \qquad W=\begin{pmatrix}1&0\\0&1/4\end{pmatrix}. $$ Suppose the consistent estimate is near $\theta_0=10.000$ years and the observed sample moments at $\theta_0$ are $g_n(\theta_0)=(0.040,-0.020)'$. Assume correct specification, an interior solution, and the regularity conditions for the expansion. Use the first-order approximation to find the parameter adjustment.

The weighted Jacobian term is $$ A=G'WG=(1,2) \begin{pmatrix}1&0\\0&1/4\end{pmatrix} \begin{pmatrix}1\\2\end{pmatrix} =1+1=2. $$ Also, $G'W=(1,0.5)$. The first-order adjustment is therefore $$ \hat{\theta}-\theta_0 \ \approx\ -A^{-1}G'Wg_n(\theta_0) =-\frac{1}{2}\{0.040+0.5(-0.020)\} =-0.0150\text{ years}. $$ Thus the first-order approximation is $\hat{\theta}\approx10.000-0.0150=9.985$ years. This is a local approximation based on the sample moments and Jacobian, not an exact solution to every nonlinear GMM problem.

Worked Example: Two Parameters and Three Moments

For a hypothetical two-parameter model, suppose the moment Jacobian, moment covariance, and fixed weight are $$ G=\begin{pmatrix}1&0\\0&1\\1&1\end{pmatrix}, \qquad \Omega=I_3,\qquad W=I_3. $$ Assume both parameters are measured in their stated units, correct specification holds, and the regularity conditions are satisfied. Compute the asymptotic covariance matrix.

The weighted Jacobian matrix is $$ A=G'G=\begin{pmatrix}2&1\\1&2\end{pmatrix}, \qquad A^{-1}=\frac{1}{3}\begin{pmatrix}2&-1\\-1&2\end{pmatrix}. $$ Because $W=\Omega=I_3$, the covariance formula simplifies to $A^{-1}G'GA^{-1}=A^{-1}$. Hence $$ \sqrt{n}(\hat{\theta}-\theta_0)\xrightarrow{d} N\!\left(0,\frac{1}{3}\begin{pmatrix}2&-1\\-1&2\end{pmatrix}\right). $$

The two components have asymptotic variances $2/3$ and covariance $-1/3$ at the $\sqrt n$ scale. For $n=300$, the corresponding approximate standard errors are $\sqrt{(2/3)/300}=0.04714$ in the first parameter’s units and $0.04714$ in the second parameter’s units. Their estimation errors are negatively correlated in this example.

What the Result Does—and Does Not—Say

The expansion separates two sources of behavior. The random term $n^{-1/2}\sum_i g_i(\theta_0)$ is the sampling fluctuation of the correctly specified moments. The matrix $-A^{-1}G'W$ maps that fluctuation into parameter error. A moment direction that changes strongly with the parameter can provide more local information; a poorly conditioned $G'WG$ can instead make the estimator sensitive to moment noise.

The theorem requires correct specification at $\theta_0$. If the population moments do not equal zero there, the central expansion around a true parameter satisfying those moments does not apply as stated. It also relies on full column rank of $G$: if a parameter direction has no effect on the moments locally, $G'WG$ is singular and this derivation cannot determine a regular, unique local distribution for that direction.

The covariance formula depends on all three ingredients: the moment Jacobian, the covariance of the moments, and the chosen weight. It is not generally correct to report $(G'WG)^{-1}$ as the covariance merely because that matrix appears in the expansion. That simplification needs additional conditions; with a general fixed weight, the full expression is $A^{-1}G'W\Omega WG A^{-1}$.

Common Pitfall

Common mistake: “The estimator minimizes the criterion, so its asymptotic covariance is just $(G'WG)^{-1}$.” Minimizing a weighted squared moment does not remove the sampling variation of the moments. The correct general covariance retains $\Omega$ between the two factors $G'W$: $$ (G'WG)^{-1}G'W\Omega WG(G'WG)^{-1}. $$ Using only $(G'WG)^{-1}$ can give an incorrect uncertainty measure unless conditions justify that simplification.

Key takeaway: Under correct specification, smoothness, consistency, and local identification, a fixed-weight GMM estimator is asymptotically normal. Its first-order error is the negative weighted projection of sample moment noise through the inverse weighted Jacobian.

Check Your Understanding

Use the definitions and expansion above to answer these questions.

What is the dimension of $G$ when there are $q$ moments and $k$ parameters?
Why does the first-order expansion contain a minus sign?
For $W=I$, $G=(1,1)'$, and $\Omega=I_2$, calculate $A$ and the asymptotic variance of $\sqrt{n}(\hat{\theta}-\theta_0)$.
What role does the central limit theorem play in deriving the limiting distribution?
Why is $(G'WG)^{-1}$ not, in general, the full asymptotic covariance matrix?

What You'll Learn