Continuously Updated GMM

Why Update the Weight at Every Candidate?

GMM chooses parameter values that make a collection of sample moments close to zero. When the number of moments exceeds the number of parameters, the moments may not all be set to zero simultaneously, so an estimator needs a rule for combining their discrepancies. The weight matrix determines how strongly each direction of discrepancy contributes to the criterion.

Two-step GMM estimates a weight matrix in an initial step and then holds it fixed during the second optimization. Continuously updated GMM (CU-GMM) instead recalculates the moment second-moment matrix at every candidate parameter value. This makes the objective a different function of the parameter, even though its large-sample behavior under correct specification is the same as that of optimally weighted GMM.

Definition: Let $g_i(\theta)$ be a $q$-dimensional vector of moment functions for observation $i$, where $\theta$ is a $k$-dimensional parameter and $q\geq k$. Define the sample mean of the moments and their uncentered sample second-moment matrix by $$ \bar{g}_n(\theta)=\frac{1}{n}\sum_{i=1}^n g_i(\theta), \qquad \widehat{\Omega}_n(\theta)=\frac{1}{n}\sum_{i=1}^n g_i(\theta)g_i(\theta)'. $$ When $\widehat{\Omega}_n(\theta)$ is invertible, the CU-GMM criterion is $$ Q_{CU,n}(\theta)=\bar{g}_n(\theta)'\widehat{\Omega}_n(\theta)^{-1}\bar{g}_n(\theta). $$ A CU-GMM estimator $\hat{\theta}_{CU}$ is a minimizer of this criterion over the parameter space.

The matrix $\widehat{\Omega}_n(\theta)$ depends on $\theta$, so it must be evaluated again for each candidate parameter value. The criterion is therefore not generally a quadratic function of $\theta$, even if the moments themselves are linear in $\theta$. Multiplying the criterion by $n$, a common alternative convention, does not change its minimizer.

What the Weight Matrix Estimates

The population moment is $m(\theta)=E[g_i(\theta)]$. Under correct specification, there is a true value $\theta_0$ satisfying $m(\theta_0)=0$. The population second-moment matrix is $$ \Omega(\theta)=E[g_i(\theta)g_i(\theta)']. $$ At $\theta_0$, correct specification makes this population second moment equal to the moment covariance: $$ \Omega(\theta_0) =E[g_i(\theta_0)g_i(\theta_0)'] =\operatorname{Var}(g_i(\theta_0)), $$ because $E[g_i(\theta_0)]=0$. The sample second-moment matrix $\widehat{\Omega}_n(\theta_0)$ estimates this population quantity and, under suitable regularity conditions, converges to it. It is not generally equal to the population covariance in a finite sample.

Away from $\theta_0$, the population moment need not be zero. In that case, $\Omega(\theta)$ includes the outer product of the nonzero population mean with itself, so it is not generally the covariance matrix of $g_i(\theta)$. This distinction matters: CU-GMM uses the uncentered second-moment matrix at each candidate, not a centered covariance matrix. Under independent and identically distributed sampling, this is the relevant matrix for the criterion and, at the correctly specified value, for efficient weighting.

Worked Example: Computing a CU-GMM Criterion

Suppose that at one candidate value $\theta$, the moment vectors from three hypothetical observations are $g_1=(1,0)'$, $g_2=(0,1)'$, and $g_3=(-1,1)'$. Calculate the CU-GMM criterion at this candidate.

The sample mean is $$ \bar{g}_n(\theta)=\frac{1}{3} \begin{pmatrix}1+0-1\\0+1+1\end{pmatrix} =\begin{pmatrix}0\\2/3\end{pmatrix}. $$ The uncentered second-moment matrix is $$ \widehat{\Omega}_n(\theta) =\frac{1}{3} \left[ \begin{pmatrix}1&0\\0&0\end{pmatrix} +\begin{pmatrix}0&0\\0&1\end{pmatrix} +\begin{pmatrix}1&-1\\-1&1\end{pmatrix} \right] =\begin{pmatrix}2/3&-1/3\\-1/3&2/3\end{pmatrix}. $$ Its inverse is $\begin{pmatrix}2&1\\1&2\end{pmatrix}$. Therefore, $$ Q_{CU,n}(\theta) =\begin{pmatrix}0&2/3\end{pmatrix} \begin{pmatrix}2&1\\1&2\end{pmatrix} \begin{pmatrix}0\\2/3\end{pmatrix} =\frac{8}{9}\approx 0.889. $$ This is the criterion value at this candidate, not a test statistic or a measure with a universal reference scale. To obtain the estimator, the criterion must be minimized over candidate values.

Optimization with a Parameter-Dependent Weight

At each trial value of $\theta$, an optimizer evaluates both $\bar{g}_n(\theta)$ and $\widehat{\Omega}_n(\theta)$, checks that the latter can be inverted, and then evaluates the criterion. The parameter-dependent weight can alter the ranking of candidates relative to a criterion using a fixed matrix. A numerical routine should therefore optimize the full CU-GMM criterion rather than compute a weight once and leave it unchanged.

For a numerical implementation, it is usually better to solve a linear system involving $\widehat{\Omega}_n(\theta)$ than to form its inverse explicitly. The sample matrix must be sufficiently well-conditioned as well as invertible: near-singularity can make the criterion highly sensitive to small changes in the data or candidate parameter. The parameter space, starting values, and convergence checks also matter when the objective is nonlinear or has multiple local minima.

Worked Example: The Criterion Changes with the Candidate

Consider a scalar parameter restricted to $0\leq\theta\leq1$, with four hypothetical moment vectors $$ g_i(\theta)= \begin{pmatrix}\theta+u_i\\1-\theta+v_i\end{pmatrix}, $$ where the four $(u_i,v_i)$ pairs are $(1,1),(1,-1),(-1,1),(-1,-1)$. Find the CU-GMM minimizer.

The sample means of both $u_i$ and $v_i$ are zero, so $\bar{g}_n(\theta)=m=(\theta,1-\theta)'$. Each component has sample second moment 1, and the sample cross-product of $u_i$ and $v_i$ is zero. Thus $$ \widehat{\Omega}_n(\theta)=I+mm' =\begin{pmatrix} 1+\theta^2&\theta(1-\theta)\\ \theta(1-\theta)&1+(1-\theta)^2 \end{pmatrix}. $$ For any vector $m$, $(I+mm')^{-1}m=m/(1+m'm)$, as is verified by multiplying the left side by $I+mm'$. Consequently, $$ Q_{CU,n}(\theta)=m'(I+mm')^{-1}m =\frac{r(\theta)}{1+r(\theta)},\qquad r(\theta)=\theta^2+(1-\theta)^2. $$ Since $r(\theta)=2(\theta-\tfrac12)^2+\tfrac12$, it is minimized at $\hat{\theta}_{CU}=1/2$. The function $r/(1+r)$ is increasing for $r\geq0$, so this also minimizes the CU-GMM criterion. At $\theta=1/2$, $Q_{CU,n}=1/3$; at either endpoint, $Q_{CU,n}=1/2$. The example shows explicitly that the weight matrix, as well as the moment mean, changes across candidate values.

Consistency and Asymptotic Behavior

The standard correct-specification result can be stated under independent and identically distributed sampling. Assume that $\theta_0$ is an interior point of a compact parameter space, $m(\theta)=E[g_i(\theta)]$ is uniquely zero at $\theta_0$, and $\Omega(\theta)$ is positive definite and well behaved over the relevant parameter space. Assume also that sample moments and second-moment matrices converge uniformly to their population counterparts, and that the moments have sufficient finite moments for the required laws of large numbers and central limit theorem. For asymptotic normality, assume $g_i(\theta)$ is differentiable near $\theta_0$, with $$ G=E\left[\frac{\partial g_i(\theta_0)}{\partial\theta'}\right] $$ of full column rank, and that the usual regularity conditions permit local expansions and stable matrix inverses.

Under these conditions, the population CU criterion is $Q_{CU}(\theta)=m(\theta)'\Omega(\theta)^{-1}m(\theta)$. It is nonnegative, and it equals zero at $\theta_0$. Positive definiteness of $\Omega(\theta)$ means it is zero only where $m(\theta)=0$; the unique-zero condition therefore makes $\theta_0$ its unique minimizer. Uniform convergence of the sample criterion to this population criterion, together with the stated compactness and identification conditions, gives consistency: $\hat{\theta}_{CU}$ converges in probability to $\theta_0$.

Theorem: Under correct specification and the regularity conditions above, CU-GMM is asymptotically normal with the efficient GMM covariance: $$ \sqrt{n}(\hat{\theta}_{CU}-\theta_0) \ \xrightarrow{d}\ N\left(0,\left(G'\Omega_0^{-1}G\right)^{-1}\right), \qquad \Omega_0=\operatorname{Var}(g_i(\theta_0)). $$ Here “efficient” refers to the smallest asymptotic covariance in the standard class of GMM estimators using the same correctly specified moments and regularity conditions.

Why the Efficient Covariance Appears

The key local argument is that the changing weight does not alter the leading first-order behavior at the true parameter. Let $h$ be a fixed $k$-vector and consider $\theta=\theta_0+h/\sqrt{n}$. Differentiability of the moments gives $$ \sqrt{n}\,\bar{g}_n(\theta_0+h/\sqrt{n}) =\sqrt{n}\,\bar{g}_n(\theta_0)+Gh+o_p(1). $$ At these local candidate values, $\widehat{\Omega}_n(\theta)$ converges to $\Omega_0$. Hence, up to a term that vanishes in probability, the locally rescaled criterion is $$ nQ_{CU,n}(\theta_0+h/\sqrt{n}) = \left(\sqrt{n}\bar{g}_n(\theta_0)+Gh\right)' \Omega_0^{-1} \left(\sqrt{n}\bar{g}_n(\theta_0)+Gh\right)+o_p(1). $$ The leading expression is a quadratic in $h$. Differentiating it and setting the derivative to zero yields $$ h= -\left(G'\Omega_0^{-1}G\right)^{-1} G'\Omega_0^{-1}\sqrt{n}\bar{g}_n(\theta_0)+o_p(1). $$ By the central limit theorem, $\sqrt{n}\bar{g}_n(\theta_0)$ converges in distribution to a mean-zero normal vector with covariance $\Omega_0$. Substituting this covariance into the expression for $h$ gives the theorem’s covariance matrix. The weight’s local variation contributes only higher-order terms: the moment mean is of order $n^{-1/2}$ in this neighborhood, so changes in the weight do not change this leading quadratic approximation.

Worked Example: Calculating the Asymptotic Variance

Suppose a correctly specified model has one parameter and two moments, with derivative vector $G=(1,2)'$ and moment covariance $$ \Omega_0=\begin{pmatrix}4&1\\1&2\end{pmatrix}. $$ Calculate the asymptotic variance of $\sqrt{n}(\hat{\theta}_{CU}-\theta_0)$.

The determinant of $\Omega_0$ is $4(2)-1(1)=7$, so $$ \Omega_0^{-1}=\frac{1}{7}\begin{pmatrix}2&-1\\-1&4\end{pmatrix}. $$ Multiplying by $G$ gives $\Omega_0^{-1}G=(0,1)'$, and therefore $$ G'\Omega_0^{-1}G=(1,2)\begin{pmatrix}0\\1\end{pmatrix}=2. $$ The asymptotic variance is the inverse of this scalar, $1/2$. Thus the asymptotic standard deviation of $\hat{\theta}_{CU}$ is approximately $\sqrt{1/(2n)}$, in the units of $\theta$. This is an asymptotic approximation, not an exact finite-sample standard error.

Practical Interpretation and Common Pitfalls

CU-GMM is useful when the moment covariance depends on the parameter and one wants to use the corresponding candidate-specific weighting directly in the objective. Under correct specification, its parameter-dependent optimization still delivers the standard efficient-GMM first-order limit. That asymptotic result does not guarantee that the numerical objective is easy to optimize or that a small-sample estimate will behave well.

Confusing second moments and covariance. The matrix used in the criterion is uncentered. It equals the population covariance at $\theta_0$ under correct specification, but a finite-sample matrix does not exactly equal that population quantity.
Holding the weight fixed by accident. Computing $\widehat{\Omega}_n$ at an initial estimate and using it throughout defines a different, fixed-weight optimization, not CU-GMM.
Ignoring singularity. If the sample second-moment matrix is singular at a candidate, the stated inverse-based criterion is undefined there. A numerical routine needs a defensible way to handle the parameter region and data configurations involved.
Overstating the asymptotic result. Efficient asymptotic covariance is a large-sample, correctly specified result under regularity conditions; it is not a guarantee of finite-sample superiority.

A related distinction is between correct and misspecified moments. The result derived here relies on $E[g_i(\theta_0)]=0$. If no parameter makes the population moments zero, the population criterion and the estimator’s limiting behavior require a separate analysis; the correct-specification covariance formula should not simply be carried over.

Key takeaway: CU-GMM minimizes the sample moment discrepancy weighted by the inverse of the candidate-specific uncentered moment second-moment matrix. Under correct specification, that matrix estimates the moment covariance at the truth, and the estimator has the efficient GMM asymptotic distribution.

Check Your Understanding

Use the definitions and results above to answer the following questions.

What changes in the CU-GMM criterion when the candidate parameter changes?
Why does the population second moment equal the moment covariance at the correctly specified parameter?
In what sense is $\widehat{\Omega}_n(\theta_0)$ an estimate of a population quantity rather than that quantity itself?
Which identification and rank conditions support consistency and asymptotic normality, respectively?
For the worked asymptotic-variance example, what is the approximate standard deviation of the estimator when $n=200$?

What You'll Learn