Bootstrap Inference For GMM

Why Bootstrap Inference for GMM?

GMM inference often relies on a large-sample approximation to the estimator’s sampling distribution. In a limited sample, that approximation may not represent the estimator’s skewness or other finite-sample features well. The bootstrap offers a way to approximate sampling variation by repeatedly resampling the observed data and re-estimating the model. Its usefulness depends on more than the number of replications: the resampling scheme must reflect how the observations were generated, and the estimation procedure must be repeated faithfully.

This tutorial develops bootstrap confidence intervals and tests for GMM, distinguishes resampling for independent and dependent observations, and explains conditions that make the approximation credible. A bootstrap is not an automatic repair for weak identification, invalid moments, or an inappropriate dependence assumption.

What the Bootstrap Approximates

For independent, identically distributed observations, the usual nonparametric bootstrap draws \(n\) rows with replacement from the observed rows. Each row must retain all variables used to construct its moment contributions. In each replication, the estimator’s sampling error is approximated by the difference \(\hat\theta^*-\hat\theta\), conditional on the observed sample. Here \(\hat\theta\) is the estimate from the original data.

Under correct moment conditions, identification, smoothness, and suitable finite-moment conditions, GMM has a locally linear approximation. Write \(D=E[\partial g_i(\theta_0)/\partial\theta']\), where \(\theta_0\) is the true parameter, and suppose \(D'WD\) is nonsingular for the limiting weight matrix \(W\). To first order, the estimation error is a linear transformation of the sample average of the moment contributions:

The iid bootstrap replaces the empirical average with an average drawn from the empirical distribution. When the regularity conditions hold, the conditional distribution of \(\sqrt{n}(\hat\theta^*-\hat\theta)\) then approximates the sampling distribution of \(\sqrt{n}(\hat\theta-\theta_0)\). This is the central justification for using bootstrap quantiles or standard errors. It is an approximation as sample size grows, not a guarantee of accurate coverage in every finite sample.

Bootstrap Confidence Intervals

For a scalar component \(\theta_j\), generate \(B\) bootstrap estimates \(\hat\theta_1^*,\ldots,\hat\theta_B^*\). A percentile interval takes the lower and upper quantiles of these estimates. A basic interval instead reflects the quantiles of the centered errors \(\hat\theta_b^*-\hat\theta\) around the original estimate. A studentized interval uses bootstrap estimates divided by their bootstrap standard errors; it can account for changing scale, but requires a reliable standard error in every replication.

Here \(Q_u\) is the empirical \(u\)-quantile of the bootstrap values. The confidence level is a repeated-sampling target: under the stated conditions, the procedure is intended to cover the fixed parameter in the stated proportion of repeated samples. It does not mean that, after observing the interval, the parameter has a corresponding probability of lying inside it.

Repeating the GMM Procedure

In overidentified or multi-step GMM, the bootstrap estimate is generally not obtained by holding every intermediate estimate fixed. For example, two-step GMM estimates a weight matrix from data and then minimizes the resulting criterion. If that is the procedure used on the original sample, the weight matrix should normally be estimated anew in each resample. This carries the sampling variation of the full estimator into the bootstrap distribution. A deliberately fixed-weight procedure is different, and its bootstrap should hold the weight fixed if that is part of the defined estimator.

Bootstrap Tests

A bootstrap test compares an observed statistic with a bootstrap approximation to its distribution under the null hypothesis. For a scalar restriction \(H_0:r(\theta)=0\), one approach uses a studentized statistic. In each ordinary bootstrap replication, compute \(T_b^*=[r(\hat\theta_b^*)-r(\hat\theta)]/SE_b^*\), where \(SE_b^*\) is the bootstrap-sample standard error for the restriction. Compare it with the observed \(T_{\mathrm{obs}}=r(\hat\theta)/SE\). This centered bootstrap statistic approximates estimation error around the fitted value; under the null and regularity conditions, it can approximate the null distribution of the observed statistic. A restricted or null-imposed bootstrap is another option, particularly when the null changes how the data or nuisance parameters should be generated. The chosen construction must be stated and justified for the model.

For a two-sided test with \(B\) replications, a common finite-replication estimate of the bootstrap p-value is \((1+\#\{|T_b^*|\ge |T_{\mathrm{obs}}|\})/(B+1)\). It is an approximation to the probability, under the null approximation, of a statistic at least as extreme as the observed one. It is not the probability that \(H_0\) is true.

Choose Resampling to Match Dependence

Resampling individual rows independently is appropriate only when the rows are independent draws from the relevant population. If observations are dependent over time, iid resampling destroys serial dependence and can give an incorrect standard error and interval. For a stationary time series with sufficiently weak dependence, block bootstrap methods resample consecutive blocks so that nearby observations remain together. Moving-block and circular-block schemes are common choices; block length must grow with sample size while remaining small relative to it, under standard asymptotic designs. In practice, the block length affects the bias-variance trade-off and should be examined rather than selected mechanically.

For clustered data, resample whole independent clusters, not individual observations within clusters, when dependence may occur inside a cluster. For panel data, the appropriate scheme depends on which dimension is independent and on the intended sampling population. The dependence structure of the moment contributions matters: the bootstrap should preserve the structure relevant to the GMM estimator and its long-run variance.

When Is the Bootstrap Credible?

Bootstrap validity is a sampling statement. For standard GMM bootstrap inference, a useful checklist is:

• Identification: the parameter is locally identified, and the relevant derivative matrix has full column rank. The estimator should not be near a singular identification problem.
• Regularity: moment functions are sufficiently smooth near the target, and the sampling distribution has finite moments adequate for the asymptotic approximation being used.
• Sampling design: resampling reflects independent sampling, clustering, or time dependence as appropriate. A large number of replications cannot fix a mismatch here.
• Estimator construction: the bootstrap repeats weight estimation, nuisance estimation, and other data-dependent steps that form part of the original estimator.
• Stable computation: optimization converges in the resamples, and enough replications are used to estimate the relevant quantiles or tail probabilities with tolerable Monte Carlo error.

Ordinary bootstrap approximations may fail or perform poorly under weak identification, parameter boundaries, nonsmooth estimators, severe small-sample distortions, or a growing number of moment conditions without suitable theory. Moment misspecification also changes the target and the relevant sampling approximation; one should not presume that a conventional correctly specified-moment bootstrap remains valid. The bootstrap is a method of approximation, not a remedy for a model or design that does not support the intended inference.

Common Mistake: Resampling the Wrong Units

A common error is to resample every observation independently because the data appear in rows. That reasoning confuses a data layout with a sampling design. If rows are repeated measurements within people or adjacent observations in a time series, independent row resampling breaks the dependence and generally misrepresents uncertainty. The correction is to identify the independent sampling units first, then resample clusters or blocks accordingly. Another error is to bootstrap only the final coefficient while leaving an estimated weight matrix unchanged, even though it was estimated as part of the stated procedure. Repeat all data-dependent estimation steps, or clearly define an estimator that holds them fixed.