Why GMM Consistency Requires More Than a Small Sample Criterion
A GMM estimator chooses a parameter value that makes a collection of sample moments small. But a small sample criterion does not, by itself, imply that the selected value is close to the truth. To establish consistency, we need conditions connecting the sample criterion to its population counterpart, and we need the population criterion to distinguish the true parameter from all values far from it.
The essential argument has three parts. The moment restrictions identify a unique population minimizer; the sample criterion converges uniformly to the population criterion; and compactness ensures that parameters outside any neighborhood of the truth have a strictly worse population criterion. Uniform convergence then prevents a sample minimizer from remaining far away.
Population and Sample Criteria
Let \(O_1,\ldots,O_n\) be observations, and let \(\theta\) belong to a parameter space \(\Theta\subset\mathbb{R}^p\). For each observation, \(g(O_i,\theta)\) is a \(q\)-dimensional vector of moment functions, where \(q\geq p\). Write the sample moment vector as \(\bar g_n(\theta)\), and its population counterpart as \(m(\theta)\).
The population moment restriction at the true parameter \(\theta_0\) is \(m(\theta_0)=0\). Let \(A_n\) be the sample weighting matrix and \(A\) its probability limit. The GMM sample criterion and limiting population criterion are
The prime denotes transpose. We use a symmetric positive-definite limiting matrix \(A\), so \(v'Av>0\) for every nonzero vector \(v\). Thus \(Q(\theta)\geq 0\), and \(Q(\theta)=0\) exactly when \(m(\theta)=0\). More generally, the consistency argument also applies when the population criterion has a unique minimum above zero; the zero-moment formulation is the usual correctly specified case.
An approximate minimizer allows for numerical optimization error, provided that error vanishes in probability. It is not necessary for the finite-sample criterion to attain its infimum exactly. The essential requirement is that the optimization error becomes negligible relative to the fixed population criterion gap separating \(\theta_0\) from distant parameter values.
Worked Example: A Scalar GMM Estimator
Suppose \(O_i=Y_i\) are i.i.d. with \(E[Y_i]=\theta_0=2\) and finite variance, and consider \(g(Y_i,\theta)=Y_i-\theta\) on \(\Theta=[0,4]\). Use the scalar weight \(A_n=A=1\). A hypothetical sample is \(1,2,2,3\).
The sample mean is \(\bar Y=(1+2+2+3)/4=2\), so the sample moment and criterion are
The criterion is minimized at \(\hat\theta_4=2\), with \(Q_4(2)=0\). In the population, \(m(\theta)=2-\theta\), so \(Q(\theta)=(2-\theta)^2\) has the unique minimum at \(\theta_0=2\). The sample result is illustrative, not a consistency proof: consistency follows because the observations are i.i.d. with finite mean, so the law of large numbers gives \(\bar Y\to_p 2\). In fact, for every \(\theta\in[0,4]\), \(|\bar g_n(\theta)-m(\theta)|=|\bar Y-2|\), which directly gives uniform convergence of the sample moments in this example.
Identification and Separation
Identification requires that no other parameter value satisfies the population moment restrictions. In the zero-moment setup, this means \(\theta_0\) is the unique element of \(\Theta\) for which \(m(\theta)=0\). Because \(A\) is positive definite, this is equivalent to \(\theta_0\) being the unique minimizer of \(Q(\theta)\), with \(Q(\theta_0)=0\).
Uniqueness alone is not the whole argument. For consistency, we need a positive criterion gap outside each fixed neighborhood of \(\theta_0\). If \(\Theta\) is compact and \(Q\) is continuous, that gap follows from uniqueness. For any \(\varepsilon>0\) for which the set is nonempty, define \(\Theta_\varepsilon=\{\theta\in\Theta:\|\theta-\theta_0\|\geq\varepsilon\}\). This is a closed subset of a compact set, hence compact. A continuous function attains its minimum on it. Since it excludes the unique minimizer \(\theta_0\), that minimum must be strictly positive:
This property is called separation of the population minimum. It is what turns a qualitative statement, “the minimizer is unique,” into a usable bound against parameters at least \(\varepsilon\) away.
Worked Example: Verifying Separation in an Overidentified Model
Suppose \(O_i=(Y_i,Z_i)\) are i.i.d. with \(E[Y_i]=2\) and \(E[Z_i]=4\). On \(\Theta=[0,3]\), define \(g(O_i,\theta)=(Y_i-\theta,\; Z_i-\theta^2)'\), and use \(A=I_2\), the two-dimensional identity matrix. Then
The first component of \(m(\theta)\) is zero only at \(\theta=2\), so the two moments together have a unique zero at \(\theta_0=2\). For any \(\theta\in[0,3]\), \(Q(\theta)\geq(2-\theta)^2\). Therefore, if \(|\theta-2|\geq 0.5\), then \(Q(\theta)\geq 0.25\). This supplies an explicit separation bound for that neighborhood.
For a hypothetical sample of four pairs with \(\bar Y=1.9\) and \(\bar Z=4.3\), the sample criterion is \(Q_4(\theta)=(1.9-\theta)^2+(4.3-\theta^2)^2\). At the true value it is \(Q_4(2)=(-0.1)^2+(0.3)^2=0.10\); at \(\theta=1.9\), it is \(Q_4(1.9)=0^2+(4.3-3.61)^2=0.4761\). The nonzero criterion at the true value is consistent with sampling variation: in an overidentified model, all sample moments need not equal zero. The example illustrates why consistency concerns convergence of the criterion and its minimizer, not exact satisfaction of every sample moment.
Uniform Convergence and a Sufficient Set of Conditions
The required convergence is uniform over the parameter space:
Uniformity matters because the estimated parameter is chosen by minimizing \(Q_n\); it is random and may be any point in \(\Theta\). Convergence at each fixed \(\theta\) alone does not guarantee that the sample criterion is close to its population version at the selected value.
Here is one standard set of sufficient conditions for the zero-moment setup. Assume that \(O_i\) are i.i.d.; \(\Theta\) is compact; \(g(O,\theta)\) is continuous in \(\theta\) with probability one; and \(E[\sup_{\theta\in\Theta}\|g(O,\theta)\|^2]<\infty\). Assume also that \(A_n\to_p A\), where \(A\) is symmetric positive definite. Under these conditions a uniform law of large numbers gives \(\sup_{\theta\in\Theta}\|\bar g_n(\theta)-m(\theta)\|\to_p0\). The moment bound and continuity ensure the relevant uniform law; independence without identical distribution or suitable uniform moment controls would not suffice by themselves.
These conditions imply uniform convergence of the criteria. To see why, the envelope condition bounds \(m(\theta)\) uniformly, and uniform convergence of \(\bar g_n\) then implies \(\sup_\theta\|\bar g_n(\theta)\|=O_p(1)\). For every \(\theta\),
Taking the supremum over \(\Theta\), the first term converges in probability to zero because \(A_n\) is bounded in probability and the moment vectors are uniformly bounded in probability. The second term also converges to zero because \(A_n\to_p A\) and \(m\) is uniformly bounded. Hence the sample criterion converges uniformly to \(Q\).
Consistency Proof
The following argument establishes consistency for exact or approximate minimizers. It uses the separated population minimum and uniform convergence directly; no asymptotic distribution or standard-error calculation is needed.
Choose \(\varepsilon>0\). By compactness, continuity, and the unique population minimum, every \(\theta\in\Theta_\varepsilon\) has \(Q(\theta)\geq\delta_\varepsilon>0\), while \(Q(\theta_0)=0\).
With probability tending to one, \(\sup_{\theta\in\Theta}|Q_n(\theta)-Q(\theta)|<\delta_\varepsilon/3\). On this event, \(Q_n(\theta)\geq2\delta_\varepsilon/3\) for every \(\theta\in\Theta_\varepsilon\), whereas \(Q_n(\theta_0)<\delta_\varepsilon/3\).
An exact minimizer cannot lie in \(\Theta_\varepsilon\), since its criterion would exceed \(Q_n(\theta_0)\). For an approximate minimizer, the same conclusion holds whenever its optimization error is less than \(\delta_\varepsilon/3\): its criterion would otherwise be at least \(2\delta_\varepsilon/3\), although a value below \(\delta_\varepsilon/3\) is available at \(\theta_0\).
The probability that \(\hat\theta_n\) lies at least \(\varepsilon\) from \(\theta_0\) tends to zero. Since this holds for every \(\varepsilon>0\), \(\hat\theta_n\to_p\theta_0\).
The constants in this comparison are not special; they simply leave room for the uniform criterion error and, for an approximate minimizer, the optimization error. The decisive feature is that the population gap \(\delta_\varepsilon\) is fixed and positive for each fixed \(\varepsilon\), while both errors vanish in probability.
When the Argument Can Fail
Each condition plays a distinct role. Without identification, a sample minimizer may settle on a different population minimizer. Without uniform convergence, a sample criterion can behave poorly at data-dependent parameter values even if it converges at every fixed value. Without compactness, uniqueness and continuity need not yield a positive gap away from the truth: a sequence of parameter values can escape the parameter space or approach the infimum without attaining it. Other assumptions can replace compactness, but they must still provide an appropriate separation condition.
Worked Example: Failure When the Moments Do Not Identify the Truth
Let \(\Theta=[-2,2]\), let the true parameter be \(\theta_0=1\), and define the scalar moment \(g(O_i,\theta)=\theta^2-1\), which does not depend on the observation. The population moment is \(m(\theta)=\theta^2-1\), and with weight one the sample and population criteria are both
The criterion is zero at both \(\theta=1\) and \(\theta=-1\). If a minimization rule selects \(\hat\theta_n=-1\) for every \(n\), then \(P(|\hat\theta_n-\theta_0|>1)=1\), so this sequence of minimizers is not consistent for \(\theta_0=1\). The data criterion already equals the population criterion exactly; uniform convergence cannot repair the failure of unique identification.
A common mistake is to reason that a small value of \(Q_n(\hat\theta_n)\) proves that \(\hat\theta_n\) is close to \(\theta_0\). That does not follow without comparison to the population criterion and a positive gap. In the nonidentified example, the criterion is exactly zero at a parameter one unit from the truth. The correction is to verify all three links: identification gives a unique population minimizer, compactness and continuity give separation, and a suitable law of large numbers gives uniform convergence of the sample criterion.
Check Your Understanding
Use the definitions and consistency argument above to answer the following.
- Why is convergence of \(Q_n(\theta)\) at each fixed \(\theta\) weaker than uniform convergence for a minimization argument?
- State the role of positive definiteness of \(A\) when \(Q(\theta)=m(\theta)'A\,m(\theta)\).
- Which properties of \(\Theta\) and \(Q\) ensure a positive population criterion gap outside an \(\varepsilon\)-neighborhood of \(\theta_0\)?
- Why does a nonzero sample criterion at \(\theta_0\) not necessarily contradict consistency in an overidentified model?
- Give one way the consistency conclusion can fail if the population moments have two distinct zeros.