Linear GMM In Closed Form

From Linear Moments to a Closed-Form Estimator

Generalized method of moments (GMM) chooses parameter values that make sample moments close to zero, with closeness measured using a weighting matrix. For linear moment conditions, the sample moments are affine functions of the parameter. The resulting criterion is quadratic, so its minimizer can be derived directly rather than found by numerical optimization.

The formula is compact, but its dimensions and rank conditions matter. They determine whether the estimator is defined uniquely, and they clarify the different roles of the sample regressor-moment matrix and the weighting matrix. The calculations below treat the weight as fixed while the criterion is minimized; they do not address how a weight should be estimated or updated.

Definition: Let \(y\) be an \(n \times 1\) outcome vector, \(X\) an \(n \times k\) regressor matrix, and \(Z\) an \(n \times q\) instrument or moment-variable matrix. For a parameter vector \(\beta\), define the sample moment vector \( \bar g_n(\beta)=n^{-1}Z'(y-X\beta) \). Writing \( A=n^{-1}Z'X \) and \( b=n^{-1}Z'y \), the linear sample moments are \( \bar g_n(\beta)=b-A\beta \), where \(A\) is \(q \times k\) and \(b\) is \(q \times 1\).

The moment restrictions say that the population counterparts of these moments are zero at the target parameter. In a sample, the moments need not be exactly zero, particularly when there are more moment conditions than parameters. GMM selects the value of \(\beta\) that makes the sample moment vector small according to a chosen weight.

The Criterion and Its Closed-Form Minimizer

Let \(W\) be a fixed, symmetric, positive-definite \(q \times q\) weighting matrix. The sample GMM criterion is the weighted squared length of the moment vector:

Q_n(\beta)=(b-A\beta)'W(b-A\beta).

Positive definiteness makes this a genuine squared length: for any nonzero \(q \times 1\) vector \(v\), \(v'Wv>0\). To obtain a unique minimizer, also require that \(A\) have full column rank \(k\). This condition means the \(k\) parameter directions are distinguishable through the linear sample moments. Since \(W\) is positive definite, full column rank of \(A\) implies that \(A'WA\) is invertible.

Theorem: If \(W\) is symmetric positive definite and \(A\) has full column rank, the unique minimizer of \(Q_n(\beta)\) is \( \hat\beta_{\mathrm{GMM}}=(A'WA)^{-1}A'Wb \). Here \(A'WA\) is \(k \times k\), and \(A'Wb\) is \(k \times 1\), so the result is a \(k \times 1\) parameter vector.

The formula follows from the first-order condition, and the rank and weight assumptions ensure that this condition identifies the unique minimum.

Differentiate the criterion.
Because \(W\) is symmetric, expanding the quadratic and differentiating with respect to the \(k \times 1\) vector \(\beta\) gives the gradient \(-2A'W(b-A\beta)\).

Set the gradient to zero.
The first-order condition is \(A'W(b-A\hat\beta)=0\), or equivalently \(A'WA\hat\beta=A'Wb\).

Solve and verify a minimum.
Full column rank and positive definiteness make \(A'WA\) invertible, yielding the stated formula. For any nonzero \(k \times 1\) vector \(u\), \(u'A'WAu=(Au)'W(Au)>0\), since \(Au\ne 0\). Thus the Hessian \(2A'WA\) is positive definite and the stationary point is the unique minimum.

This derivation also shows what each matrix does. The matrix \(A=n^{-1}Z'X\) summarizes how the instruments or moment variables co-vary with the regressors. It connects changes in \(\beta\) to changes in the moment vector. The weight \(W\) determines how discrepancies across the \(q\) moments are combined in the criterion. It changes the minimizer when the moments are overidentifying, but it cannot create identification if the columns of \(A\) are linearly dependent.

Identification and the Role of the Weight

When \(q=k\) and \(A\) is nonsingular, the system is exactly identified: there is a parameter value that sets all \(q\) linear sample moments to zero. The formula then reduces to \(\hat\beta=A^{-1}b\), regardless of the positive-definite choice of \(W\). When \(q>k\), there are more moment conditions than parameters. In general, no single \(\beta\) sets every sample moment to zero, so \(W\) determines the weighted compromise.

The criterion is a weighted projection problem: it selects the parameter whose fitted moment vector \(A\beta\) is closest to \(b\) in the metric defined by \(W\). This is a finite-sample algebraic description. The formula alone does not establish that the moments are substantively valid, that the estimator has desirable large-sample properties, or that a particular weight is preferable.

Worked Example: Exactly Identified Scalar GMM

Suppose a hypothetical sample has \(n=3\), one regressor, and one instrument. The instrument values are \(z=(1,0,1)'\), the regressor values are \(x=(1,2,1)'\), and the outcomes are \(y=(2,5,4)'\). Use the single sample moment \(n^{-1}z'(y-x\beta)\). The one moment and one parameter make this exactly identified.

The cross-products are \(z'x=1(1)+0(2)+1(1)=2\) and \(z'y=1(2)+0(5)+1(4)=6\). Therefore, \( A=z'x/3=2/3 \) and \( b=z'y/3=2 \). Both are scalars, and \(A\ne0\), so the rank condition holds. Take \(W=1\), which is positive definite in this one-dimensional case.

The closed-form estimate is \( \hat\beta=(A'WA)^{-1}A'Wb \). Substituting the values, \( \hat\beta=((2/3)^2)^{-1}(2/3)(2)=(4/9)^{-1}(4/3)=3 \). The sample moment at this estimate is \(b-A\hat\beta=2-(2/3)(3)=0\), as expected for an exactly identified scalar linear moment.

Worked Example: Overidentified GMM and a Corrected Cross-Product

Consider a hypothetical sample with \(n=2\), one regressor, and two moment variables. Let \(Z=I_2\), \(X=(6,4)'\), and \(y=(4,10)'\), where \(I_2\) is the \(2 \times 2\) identity matrix. There are two moment conditions for one parameter, so the system is overidentified. Use \(W=I_2\).

The matrices and vectors are \( A=Z'X/2=(3,2)' \) and \( b=Z'y/2=(2,5)' \). In particular, the first component of \(A\) is \((2+4)/2=3\), not \(1\); the second is \((2+2)/2=2\). Thus \(A\) is \(2 \times 1\), \(b\) is \(2 \times 1\), and \(W\) is \(2 \times 2\). The single column of \(A\) is nonzero, so \(A\) has full column rank.

Calculate the scalar pieces of the estimator: \( A'WA=3^2+2^2=13 \), and \( A'Wb=3(2)+2(5)=16 \). Hence \( \hat\beta_{\mathrm{GMM}}=13^{-1}(16)=16/13\approx1.231 \). For a direct check, \(A\hat\beta=(48/13,32/13)'\), so the residual moment vector is \(b-A\hat\beta=(-22/13,33/13)'\). Its product with \(A'\) is \(3(-22/13)+2(33/13)=0\), which verifies the first-order condition for \(W=I_2\).

Worked Example: Multiple Parameters and a Nonidentity Weight

Suppose a hypothetical sample has \(n=3\), \(k=2\), and \(q=3\). Its moment matrices are \( A=\begin{pmatrix}1&0\\0&1\\1&1\end{pmatrix} \) and \( b=\begin{pmatrix}1\\2\\4\end{pmatrix} \). For example, these values result from \(Z=I_3\), \(X=3A\), and \(y=3b\), using \(A=Z'X/3\) and \(b=Z'y/3\). Take the fixed weighting matrix \(W=\operatorname{diag}(1,1,2)\), a symmetric positive-definite \(3 \times 3\) matrix.

The \(3 \times 2\) matrix \(A\) has rank 2 because its first two rows are linearly independent, so the required full-column-rank condition holds. The dimensions also line up: \(A'WA\) is \(2 \times 2\), \(A'Wb\) is \(2 \times 1\), and the estimate will have two components. Direct multiplication gives \( A'WA=\begin{pmatrix}3&2\\2&3\end{pmatrix} \) and \( A'Wb=\begin{pmatrix}9\\10\end{pmatrix} \).

The inverse of the first matrix is \( (A'WA)^{-1}=\frac{1}{5}\begin{pmatrix}3&-2\\-2&3\end{pmatrix} \). Therefore, \( \hat\beta_{\mathrm{GMM}}=\frac{1}{5}\begin{pmatrix}3&-2\\-2&3\end{pmatrix}\begin{pmatrix}9\\10\end{pmatrix} =\begin{pmatrix}7/5\\12/5\end{pmatrix} =\begin{pmatrix}1.400\\2.400\end{pmatrix} \).

The fitted moments are \(A\hat\beta=(1.4,2.4,3.8)'\), giving residual moments \((-0.4,-0.4,0.2)'\). Their weighted normal-equation check is \(A'W(b-A\hat\beta)=(-0.4+2(0.2),-0.4+2(0.2))'=(0,0)'\). The weighted criterion at the estimate is \(0.4^2+0.4^2+2(0.2^2)=0.400\). This nonzero residual is consistent with overidentification: the estimate balances the moments rather than setting each one to zero.

Common Mistakes and Practical Checks

A frequent error is to treat the number of moments \(q\) as the dimension of the parameter vector. The parameter dimension is \(k\), so \(A'WA\), not \(W\), is the matrix that must be inverted in the closed-form solution. Another error is to use the formula without checking rank: if \(A\) lacks full column rank, then \(A'WA\) is singular under a positive-definite \(W\), and this expression does not define a unique estimator.

A third mistake is to think that changing \(W\) must change every GMM estimate. With exactly identified linear moments and nonsingular \(A\), the solution is \(A^{-1}b\) for any positive-definite \(W\). Weight choice affects the solution in the overidentified case because it changes the relative contribution of moment discrepancies. It does not change the number of parameters, the dimensions of the moment vector, or the rank of \(A\).

For a calculation, check the cross-products before using the estimator, then check dimensions at every multiplication. In particular, \(A'W\) has dimensions \(k \times q\), so \(A'Wb\) is \(k \times 1\); \(A'WA\) is \(k \times k\). These checks often expose a transposition error or an arithmetic mistake before it propagates into the estimate.

Key takeaway: With linear sample moments \(b-A\beta\), a fixed symmetric positive-definite \(W\), and full column rank of \(A\), linear GMM has the unique minimizer \((A'WA)^{-1}A'Wb\). The matrix \(A\) links parameters to moments; \(W\) determines how moment discrepancies are weighted.

Check Your Understanding

Use the dimensions and conditions in the tutorial to answer the following.

If \(n=80\), \(k=3\), and \(q=5\), what are the dimensions of \(A\), \(b\), \(W\), \(A'WA\), and the GMM estimate?
Why does a symmetric positive-definite \(W\) and full column rank of \(A\) ensure that \(A'WA\) is invertible?
In an exactly identified linear system with nonsingular \(A\), why does the estimate not depend on the choice of positive-definite \(W\)?
For an overidentified system, what role does \(W\) play when no parameter value makes every sample moment zero?
Suppose \(A\) has two identical columns. What problem does this create for the stated closed-form estimator?

What You'll Learn