Why Small-Sample GMM Needs Extra Diagnostics
GMM can behave poorly in finite samples even when the population moment conditions are appropriate. The risk is especially visible in two-step or iterated procedures: the estimated covariance of the moments is inverted to form a weight matrix, so sampling noise in that covariance can strongly affect the objective function. A few influential observations or moments may then dominate the estimate, and the usual large-sample calibration of Hansen’s \(J\) test may be unreliable.
These problems are not evidence, by themselves, that GMM is invalid. They are reasons to examine how the estimate and inference respond to plausible changes in the estimated weight, individual observations, and moment set. The diagnostics below are sensitivity checks, not automatic tests that certify an estimator as sound.
For the standard asymptotic results, the moment conditions are correctly specified at a parameter value \(\theta_0\), so \(E[g_i(\theta_0)]=0\); the moments and their derivatives satisfy suitable regularity conditions; and the relevant population covariance matrix \(S\) is finite and positive definite. The usual \(J\)-test reference additionally relies on valid overidentifying restrictions and large-sample conditions, including a fixed number \(q\) of moments. Those conditions support asymptotic claims; they do not guarantee that a particular small sample gives a stable covariance estimate or an accurate finite-sample approximation.
Noisy and Nearly Singular Estimated Weights
The efficient population weight is \(S^{-1}\). If one direction in the moment vector has very small estimated variance, inversion assigns that direction a very large weight. This can be appropriate when the covariance is estimated precisely. In a small sample, however, a small estimated eigenvalue may partly reflect noise. Inverting it can then magnify estimation error and make the fitted parameter highly sensitive to minor changes in the data.
A useful diagnostic is to examine the eigenvalues of \(\hat S\), not just whether software successfully computed its inverse. For a symmetric positive definite matrix, the ratio of its largest to smallest eigenvalue is its condition number. A large condition number signals that small perturbations to the matrix may produce large changes in its inverse. It is a warning, not a universal pass/fail threshold: the scale and consequences of the problem depend on the application.
Worked Example: A Noisy Low-Variance Direction
Suppose a hypothetical two-moment analysis has an estimated covariance matrix whose eigenvalues are \(0.005\) and \(1.995\). In the corresponding eigenvector coordinates, suppose the sample mean moment vector at parameter value \(\theta\) is \(z(\theta)=(\theta-0.40,\ \theta-0.50)'\). Compare the efficient-weight objective with an identity-weight objective. All moment coordinates are in standardized units, and \(\theta\) is measured in the model’s parameter units.
The inverse covariance matrix assigns weights \(1/0.005=200\) and \(1/1.995\approx0.5013\) to the two eigenvector directions. Thus the objective, omitting the common positive factor \(n\), is
Differentiating and setting the derivative to zero gives
With identity weighting, the minimizer is \((0.40+0.50)/2=0.45\). The estimated efficient weight puts much more emphasis on the first coordinate, so the estimate is pulled close to \(0.40\). The covariance condition number is \(1.995/0.005=399\). This calculation does not show that either estimate is generally preferable; it demonstrates why the covariance estimate and resulting weight deserve scrutiny.
The source of the sensitivity can be seen from a first-order matrix expansion. Write the estimated covariance as \(\hat S=S+E\), where \(E\) is estimation error. When the perturbation is sufficiently small for the expansion to apply,
The error term contains \(S^{-1}\) on both sides of \(E\). Consequently, directions with large inverse eigenvalues can magnify covariance estimation error. If \(\hat S\) is singular, its ordinary inverse does not exist; if it is nearly singular, numerical computability alone does not make the resulting weight trustworthy.
Influential Observations and Moments
A moment condition can have substantial influence if a small number of observations contribute unusually large values, if its scale differs greatly from other moments, or if the fitted estimate changes markedly when that moment is removed or downweighted. These are different issues: scale can often be handled by sensible normalization, while unusual observations or redundant moments may require investigation of the data and model. A large contribution is not automatically erroneous; it may be genuine information or evidence of model tension.
Worked Example: Leave-One-Out Sensitivity
Consider a hypothetical sample of \(n=8\) observations and one parameter \(\theta\). Use two moment functions \(g_{i1}(\theta)=x_i-\theta\) and \(g_{i2}(\theta)=z_i-\theta\), with identity weighting for this diagnostic. The \(x_i\) values are seven zeros and one value of \(8\); all eight \(z_i\) values equal \(0.20\). The observations and moment values are in the same units as \(\theta\).
The sample mean of the \(x_i\) values is \(8/8=1\), and the mean of the \(z_i\) values is \(8(0.20)/8=0.20\). Therefore,
The identity-weight criterion is \(8\{(1-\theta)^2+(0.20-\theta)^2\}\). Its derivative is \(8\{-2(1-\theta)-2(0.20-\theta)\}\), which is zero at \(\hat\theta=(1+0.20)/2=0.60\). At that estimate the mean moments are \((0.40,-0.40)'\), and the criterion equals \(8(0.40^2+(-0.40)^2)=2.56\).
Now remove the observation with \(x_i=8\) and recompute the estimate using the remaining seven observations. The mean \(x_i\) is \(0\), while the mean \(z_i\) remains \(0.20\). The identity-weight estimate becomes \((0+0.20)/2=0.10\). A change from \(0.60\) to \(0.10\) is a strong influence warning in this illustration. It is not, by itself, a formal test or a reason to discard the observation: the observation’s validity and the model’s sensitivity should be investigated and reported.
A related check is to refit the model after removing one moment at a time, where that comparison is substantively meaningful. A moment that barely changes the estimate may still affect the estimated covariance or test statistic; conversely, a large change may be expected if the moment carries unique identifying information. Report which comparison was made and what was held fixed. A sensitivity result should not be mistaken for a new estimator with automatically valid standard errors.
Why the Hansen J Test Can Mislead in a Small Sample
With \(q\) moment conditions and \(k\) estimated parameters, the Hansen \(J\) statistic for efficient GMM is the minimized weighted discrepancy, conventionally written \(J=n\bar g_n(\hat\theta)'\hat S^{-1}\bar g_n(\hat\theta)\). Under the null that the overidentifying restrictions are valid and under the relevant regularity conditions, its large-sample reference distribution is chi-square with \(q-k\) degrees of freedom. The test assesses the joint restrictions; a rejection does not identify which moment condition is responsible.
In a small sample, the estimated weight can be noisy, the fitted moments need not have their asymptotic distribution, and the chi-square approximation may be poor. The resulting test can reject too often or too rarely. A near-singular \(\hat S\) is especially concerning because its inverse may make a small, noisy component of \(\bar g_n(\hat\theta)\) dominate \(J\). A non-rejection is not proof that all moments are valid, and a rejection is not a diagnosis of a specific invalid moment.
Worked Example: A J Statistic Sensitive to the Estimated Weight
Suppose a hypothetical analysis has \(n=20\), \(q=2\) moments, and \(k=1\) parameter. At the fitted estimate, the mean moments in the eigenvector coordinates of \(\hat S\) are \((0.03,0.04)'\). Suppose the estimated covariance eigenvalues in those directions are \(0.005\) and \(1.995\). These are illustrative summary values, not data from a real study.
Using the estimated efficient weight, the statistic is
The asymptotic reference has \(q-k=1\) degree of freedom. Comparing \(3.616\) with that chi-square distribution gives an approximate \(p\)-value of \(0.0573\). This is not quite below a \(0.05\) significance level, but the value should not be treated as a reliable finite-sample calibration merely because it can be computed.
For comparison, using identity weighting in the same coordinates would give the quadratic form \(20(0.03^2+0.04^2)=0.0500\). That is not the efficient-weight Hansen \(J\) statistic and its chi-square calibration cannot simply be borrowed. The large numerical difference illustrates how heavily the near-zero estimated eigenvalue affects the weighted discrepancy; it does not establish that identity weighting provides a valid replacement test.
Practical Remedies and Reporting Checks
A sound response to instability is to diagnose its source and show how consequential it is. Remedies involve trade-offs: reducing the moment set may improve stability but discard useful restrictions; changing the weight may improve numerical behavior but change efficiency or the test’s reference distribution. No adjustment should be presented as a universal finite-sample correction.
Report its eigenvalue range or condition number, check that it is positive definite, and document any numerical warning or generalized inverse used.
Compare estimates under reasonable alternative weight constructions and moment subsets; examine whether influential observations account for major changes.
State which estimator and weight produced each standard error and test. Do not attach the standard Hansen \(J\) reference distribution to an arbitrary modified-weight statistic without justification.
Give the sample size, number of parameters and moments, covariance and weight method, relevant diagnostics, and sensitivity results. Explain any removed moments, regularization, or numerical modifications and their inferential consequences.
Possible responses include simplifying a redundant or weakly motivated moment set, improving the scaling of moments, and using a more stable covariance estimation strategy justified for the application. If a regularized or truncated inverse is used, its tuning choice and effect on estimates and inference should be disclosed. Such modifications can stabilize calculations, but they generally alter the weight and may alter standard asymptotic test results. A fixed positive definite weight can yield a consistent estimator under standard identification and moment conditions, but it is not automatically as efficient as optimal GMM, and its minimized criterion is not automatically Hansen’s \(J\) test.
Common Mistake
Wrong reasoning: “The covariance matrix is positive definite and the software returned an inverse, so the efficient GMM estimate and \(J\)-test \(p\)-value are dependable.” Positive definiteness only ensures that the inverse exists; it does not show that its entries are estimated precisely or that the finite-sample chi-square approximation is accurate.
Correction: Check the eigenvalues and sensitivity of the estimate, inspect influential observations and moments, and state the assumptions behind any test reference distribution. Treat \(J\)-test results as asymptotic evidence whose small-sample reliability must be assessed, not as a mechanical verdict on moment validity.
Check Your Understanding
Use the definitions and diagnostics in this tutorial to answer the following.
- Why can a small estimated eigenvalue of the moment covariance matrix make a two-step GMM estimate unstable?
- A covariance estimate has eigenvalues \(0.02\) and \(2.00\). What are the corresponding inverse weights, and what does their ratio suggest?
- In a leave-one-out analysis, the parameter estimate changes substantially after one observation is removed. What does this establish, and what does it not establish?
- State the usual degrees of freedom for Hansen’s \(J\) test when there are \(q\) moments and \(k\) parameters, and give one reason its small-sample \(p\)-value may be unreliable.
- Why should a researcher disclose the use of a regularized inverse rather than report only the resulting GMM estimate?