From Moment Conditions to Numerical Optimization
In linear GMM, the sample moments are linear in the parameters, which can make the minimization problem solvable in closed form. In nonlinear GMM, the moments may contain powers, ratios, exponentials, or other nonlinear functions of the unknown parameters. The estimator is then usually computed by numerical minimization. Numerical software can perform that minimization, but it cannot determine whether the moments were coded correctly, whether the parameters are admissible, or whether the reported solution is credible. Those checks are part of the statistical work.
Suppose there are \(n\) observations, a \(p\)-dimensional parameter vector \(\theta\), and \(q\) moment functions. The vector-valued function \(g_i(\theta)\) gives the moments for observation \(i\). The sample average of these moment contributions is the sample moment vector. In this tutorial, the weight matrix is treated as specified and fixed during minimization; the task is to find the parameter value that makes the weighted sample moments as small as possible.
A moment condition specifies a population restriction such as \(E[g_i(\theta_0)]=0\) at the parameter value of interest \(\theta_0\). The estimator uses the sample analogue: it searches for parameter values that make \(\bar g_n(\theta)\) close to zero, with closeness measured by \(W\). If there are more moments than parameters, the moments will not generally all equal zero at the minimizer. The criterion summarizes their weighted discrepancy.
The factor \(n\) is sometimes included in the objective, giving \(nQ_n(\theta)\). For a fixed \(n\) and fixed \(W\), multiplying the criterion by the positive constant \(n\) does not change its minimizer. It does change the scale of the objective values, so implementations and reported criterion values should state which convention is used.
A Derivative Check for the Minimizer
Let \(G_n(\theta)\) be the \(q\times p\) Jacobian matrix whose \((j,k)\) entry is the derivative of the \(j\)th sample moment with respect to the \(k\)th component of \(\theta\). When the moments are differentiable and \(\theta\) is an interior point of the parameter space, differentiating the criterion gives a useful first-order condition.
The derivative follows by applying the product rule to the quadratic form; symmetry of \(W\) makes the two resulting terms equal. At an unconstrained interior minimum, the gradient must be zero. This is a necessary condition, not a guarantee of a minimum: a stationary point may be a maximum or a saddle point, and a nonlinear criterion may have several local minima. Constraints also change the first-order conditions at a boundary.
Worked Example: A Nonlinear Criterion and Its Numerical Minimum
Consider a hypothetical sample whose two empirical moment functions are \(\bar g_n(\theta)=(\theta-1,\theta^2-4)'\), with scalar \(\theta\) and \(W=I_2\). The parameter space is the real line. Find and check the GMM minimizer.
Criterion. With the identity weight matrix, the two squared moment discrepancies receive equal weight:
First-order condition. Differentiating each squared term gives
Setting this derivative to zero and dividing by 2 yields \(2\theta^3-7\theta-1=0\). A numerical root-finding procedure gives the stationary point \(\hat\theta\approx1.9385\). The second derivative is \(Q_n''(\theta)=2+12\theta^2-16=12\theta^2-14\), which is positive at this point, so it is a local minimum.
Evaluate the moments and criterion. At \(\theta=1.9385\), the moments are \(\bar g_n(\theta)=(0.9385, -0.24221775)'\), because \(1.9385^2-4=-0.24221775\). Therefore,
The reported parameter value is rounded, so it need not make the derivative exactly zero. A numerical implementation should evaluate the criterion and gradient at its full-precision solution. To establish that this is the global minimum rather than just a local one, compare the criterion at all stationary points; here the other local minimum has a substantially larger criterion, and \(Q_n(\theta)\) tends to infinity as \(|\theta|\) increases.
Constraints and Feasible Parameter Values
The parameter space \(\Theta\) is part of the model, not an optional setting for the optimizer. A parameter may be required to be positive, to lie between specified bounds, or to satisfy a more complicated restriction. The optimizer must search only among feasible values. A convenient implementation sometimes transforms an unconstrained numerical variable into a feasible parameter—for example, setting \(\theta=\exp(\eta)\) enforces \(\theta>0\)—but the transformation must be incorporated consistently into the moments and derivatives.
At an interior solution, a zero gradient is a relevant necessary condition. At a boundary solution, the gradient need not be zero: moving in the direction that would lower the criterion may violate the constraint. Thus, checking only whether the unconstrained gradient is near zero can incorrectly reject a valid constrained minimum. The criterion should be compared over feasible alternatives, and the optimizer's boundary or constraint diagnostics should be examined.
Worked Example: A Minimum at a Parameter Boundary
Suppose four hypothetical observations are \(y=(-0.6,-0.2,-0.5,-0.3)\), and the moment for observation \(i\) is \(g_i(\theta)=y_i-\theta\). The parameter is constrained to \(\Theta=[0,\infty)\), and \(W=1\). Find the constrained GMM estimate.
The sample mean of the observations is \((-0.6-0.2-0.5-0.3)/4=-0.4\). Hence the sample moment and criterion are
Without the constraint, the criterion is zero at \(\theta=-0.4\), but that value is infeasible. For every feasible \(\theta\geq0\), the quantity \(0.4+\theta\) is at least \(0.4\), so \(Q_n(\theta)\geq0.4^2=0.16\). Equality occurs at the boundary, giving \(\hat\theta=0\) and \(Q_n(0)=0.16\). The derivative at the solution is \(Q_n'(0)=2(0.4)=0.8\), not zero. That positive derivative means increasing the feasible parameter from zero raises the criterion; moving in the decreasing direction would be infeasible.
The example illustrates why a boundary solution is not automatically an optimization failure. The relevant question is whether any feasible move can lower the objective, not whether an unconstrained first-order equation holds.
Scaling, Weighting, and Starting Values
Numerical optimization can be sensitive to the relative scales of the moments and parameters. If one moment is numerically much larger than another, an identity weight matrix makes the large-scale moment contribute much more to the criterion. That may reflect the intended weighting, but it can also result simply from units—for example, recording a variable in grams rather than kilograms. Rescaling can improve numerical conditioning, but it must not silently change the intended GMM objective. If the moments are transformed, the weight matrix must be adjusted to preserve the same quadratic criterion.
Parameters also need sensible numerical scales. An optimizer that changes one parameter by steps of size \(0.01\) may make little progress if that parameter is naturally around \(10^5\), while the same step may be too large for a parameter near \(10^{-6}\). Centering or rescaling parameters, choosing appropriate step sizes, and supplying derivatives where reliable can help. After any transformation, confirm that the solution maps back to the intended parameter values and constraints.
Worked Example: Different Moment Scales Change the Objective
Suppose a hypothetical problem has scalar sample moments \(\bar g_n(\theta)=(\theta-1,100(\theta-2))'\). Compare the minimizers using an identity weight and using \(W=\operatorname{diag}(1,0.0001)\). Let \(\theta\) range over the real line.
With \(W=I_2\), the criterion is \((\theta-1)^2+10000(\theta-2)^2\). Differentiating and setting the result to zero gives
The second moment's factor of 100 is squared in the criterion, so it dominates this minimization. With \(W=\operatorname{diag}(1,0.0001)\), the criterion instead becomes
Its derivative is \(2(\theta-1)+2(\theta-2)\), which is zero at \(\hat\theta=1.5\). Both calculations are correct for their stated weight matrices; they solve different weighted problems. If the factor of 100 arose only from expressing a moment in different units, the weight should be transformed correspondingly rather than left unchanged. Numerical scaling is therefore a modeling and implementation decision, not just a cosmetic adjustment.
Starting values matter because a nonlinear criterion may have multiple local minima or regions where the numerical search behaves poorly. A defensible workflow uses values informed by the parameter interpretation, simpler estimates, or a coarse exploration of the feasible space. It then repeats the optimization from several dispersed feasible starting values. Similar solutions and criterion values across starts are reassuring; they do not by themselves prove that the global minimum has been found.
Convergence Checks and a Practical Workflow
An optimizer's success flag is useful but not sufficient. Algorithms may stop because a step became small, an iteration limit was reached, or a numerical tolerance was met, and these stopping conditions do not all mean the same thing. A credible result should be supported by multiple checks: the parameters are feasible, the objective is finite, the gradient or constrained optimality diagnostics are satisfactory, and reasonable alternative starts lead to comparable solutions. Re-evaluating the moments and criterion independently also helps detect coding errors.
Define each \(g_i(\theta)\), the parameter vector, its admissible space, and the fixed weight matrix. Check dimensions and evaluate the moments at test parameter values.
Compute \(\bar g_n(\theta)\) as the sample average, then evaluate \(\bar g_n(\theta)'W\bar g_n(\theta)\). Confirm that \(W\) is symmetric positive definite and that the weighting and units are intentional.
Use an algorithm appropriate to the parameter space, provide analytic or carefully checked numerical derivatives when available, and try multiple starting values.
Recompute the moments and objective at full precision, inspect feasibility and convergence diagnostics, and assess the gradient or boundary optimality. Compare objective values across starting points and other plausible candidates.
Common Pitfalls
- Treating any optimizer output as the GMM estimate. A returned parameter vector may be infeasible, at a local rather than global minimum, or based on a failed convergence. Inspect the objective, constraints, and diagnostics.
- Demanding a zero gradient at a boundary. Constrained minima can have a nonzero gradient when all improving directions are infeasible. Check feasible directions and constraint-specific diagnostics.
- Rescaling moments but keeping the same weights automatically. This generally changes the criterion and can change the minimizer. Transform the weight matrix as needed to preserve the intended weighting.
- Using only one starting value. A single successful run cannot reveal sensitivity to local minima. Use several feasible starts and compare their solutions and objective values.
- Confusing a small criterion with a verified minimum. The value must be interpreted relative to the specified moments and weights; also check optimization and implementation independently.
Check Your Understanding
Answer these questions using the definitions and procedures developed above.
- For \(q\) sample moments and a \(q\times q\) fixed weight matrix, write the GMM criterion and state the dimensions of the sample moment vector.
- Why is a zero gradient not required at every constrained minimum?
- If one sample moment is multiplied by 10 but the identity weight matrix is retained, what happens to that moment's contribution to the criterion?
- Name two checks, beyond an optimizer's success flag, that help assess a numerical solution.
- Why should an analyst use multiple starting values when minimizing a nonlinear GMM criterion?