Overidentified Moments and Optimal GMM Weighting

Why Use More Moment Conditions Than Parameters?

A method-of-moments estimator uses population restrictions that a model implies, such as \(E[g(W_i,\theta_0)]=0\). When there are as many restrictions as unknown parameters, one may be able to solve the sample equations exactly. But applied models often supply more moment conditions than parameters. Sampling variation will generally make those equations impossible to satisfy all at once. Generalized method of moments (GMM) resolves this by choosing the parameter value that makes the sample moments collectively small, with a specified weighting matrix determining how discrepancies are measured.

The weight matters because moment conditions can have different scales and different amounts of sampling variability. A good weight gives less influence to noisy directions and accounts for correlation across moments. The central results are an asymptotic covariance formula for a general positive-definite weight and a demonstration that weighting by the inverse covariance of the moments is asymptotically optimal.

The vector \(\bar g_n(\theta)\) has \(q\) entries, but only \(p\) parameters can be adjusted. In general, there is no \(\theta\) making all \(q\) sample moments exactly zero. GMM instead minimizes a weighted squared distance from zero. The matrix \(W_n\) is a weight matrix; below, \(W\) denotes its probability limit. It is distinct from the observation \(W_i\).

Asymptotic Covariance for a General Weight

Write \(m(\theta)=E[g(W_i,\theta)]\), and define the \(q\times p\) derivative matrix and the covariance matrix of the moments at the true parameter by

For the result below, assume \(\theta_0\) is an interior point of the parameter space; \(E[g(W_i,\theta_0)]=0\); \(g\) is differentiable near \(\theta_0\); and the required laws of large numbers and uniform derivative convergence hold there. Assume \(E[\|g(W_i,\theta_0)\|^2]<\infty\), the central limit theorem holds for the sample moments, \(\Omega\) is positive definite, and \(G\) has full column rank \(p\). Also assume \(W_n\xrightarrow{p}W\), where \(W\) is symmetric positive definite, and that the chosen interior minimizer \(\hat\theta\) is consistent and root-\(n\) consistent. These conditions ensure local curvature in all parameter directions and allow the first-order condition to be linearized.

Derivation. The multivariate central limit theorem gives \(\sqrt n\,\bar g_n(\theta_0)\xrightarrow{d}N(0,\Omega)\). At an interior minimizer, differentiating \(Q_n\) with respect to \(\theta\) gives the first-order condition

A Taylor expansion of the sample moments around \(\theta_0\), using consistency and derivative convergence, gives

Substitute this into the first-order condition and use \(\hat G_n(\hat\theta)\xrightarrow{p}G\) and \(W_n\xrightarrow{p}W\). Since \(G'WG\) is nonsingular, the resulting equation yields

The covariance of the leading term is \(A_W\Omega A_W'\), where \(A_W=(G'WG)^{-1}G'W\). Multiplication gives the displayed expression for \(V(W)\), and the central limit theorem and Slutsky’s theorem give the stated limiting distribution. \(\square\)

Why Inverse-Covariance Weighting Is Optimal

The optimal weight is \(W=\Omega^{-1}\), or any positive scalar multiple of it. To see why, first note that the linear approximation to a GMM estimator has the form \(-A_W\sqrt n\,\bar g_n(\theta_0)\), where \(A_W=(G'WG)^{-1}G'W\) satisfies \(A_WG=I_p\). The constraint \(A_WG=I_p\) expresses that the linearized estimator responds correctly to local changes in the parameter.

Proof. Define \(A_*=(G'\Omega^{-1}G)^{-1}G'\Omega^{-1}\). Then \(A_*G=I_p\), and substituting \(W=\Omega^{-1}\) into the general covariance formula gives \(A_*\Omega A_*'=(G'\Omega^{-1}G)^{-1}\). For a general \(W\), set \(D=A_W-A_*\). Since both matrices multiplied by \(G\) equal \(I_p\), \(DG=0\). Expand:

The cross term \(A_*\Omega D'\) is zero because

Its transpose is also zero. Thus \(V(W)-V_{\mathrm{opt}}=D\Omega D'\), which is positive semidefinite because \(\Omega\) is positive definite. This proves the claim. \(\square\)

This result compares asymptotic covariance matrices in the positive-semidefinite order: no linear contrast of the parameter has greater asymptotic variance under the optimal weight than under another fixed positive-definite weight. It does not mean the optimal weight makes every sample moment vanish. Rather, it accounts for the sampling variation and correlation among those moments.

Implementation and Common Pitfalls

In practice, \(\Omega\) is usually unknown, so an optimal weight is not directly available. A common strategy is to obtain a consistent preliminary estimator, estimate the covariance of the moment vector at that estimator, and use its inverse as a subsequent weight. The asymptotic optimality result applies when the resulting weight converges in probability to \(\Omega^{-1}\), subject to the regularity conditions for the estimator. The earlier material on plug-in nuisance estimates treats the general role of consistent preliminary estimates; the key point here is that the weight must estimate the covariance of the moments, not the parameter covariance.

Common mistake: “There are more moments than parameters, so I should invert \(G\) and solve the sample equations.” This is impossible when \(G\) is \(q\times p\) with \(q>p\); it is not a square matrix, and the sample moments will generally not have a common zero. The correction is to minimize \(\bar g_n(\theta)'W_n\bar g_n(\theta)\). The matrix to invert in the local covariance formula is \(G'WG\), which is \(p\times p\) and nonsingular under full column rank of \(G\). A second mistake is to use \(\Omega\) as the optimal weight: the optimal choice is its inverse, because high-variance moment directions should count less, not more.