Simulation Error In Indirect Inference

Why Simulation Error Matters

Indirect inference compares an auxiliary statistic estimated from observed data with the same statistic produced by a structural model at candidate parameter values. In practice, the model-generated statistic is often approximated using random draws. Those draws make the simulated statistic noisy, so an optimizer can select a parameter partly because of a favorable simulation realization rather than because the structural parameter truly matches the data.

This creates a second source of uncertainty in addition to sampling variation in the observed data. Increasing the number of independent simulation draws reduces this Monte Carlo error, but costs computation. The relevant question is not simply whether the simulation error is nonzero; it is how strongly it affects the estimate and whether it is small enough for the inferential task.

Here, \(B\) means the effective number of independent simulated draws contributing to the statistic. If a procedure averages \(R\) independent simulated auxiliary estimates, each based on \(m\) independent observations, then often \(B=Rm\). The relevant count may be smaller than the raw number of generated values when draws are dependent. For example, a dependent simulation sequence can contain less information than the same number of independent draws.

How Simulation Error Enters the Estimate

Suppose the true parameter is \(\theta_0\), with \(k\) components, and the auxiliary statistic has a smooth population mapping \(a(\theta)\). At the truth, write the observed statistic and simulated statistic as

The term \(u_n\) is sampling error in the observed auxiliary statistic; \(v_B(\theta)\) is simulation error. For a local approximation, assume the model is correctly specified for these auxiliary targets, the mapping is differentiable, the relevant Jacobian has full column rank, and the simulation approximation is sufficiently smooth near \(\theta_0\). Suppose also that observed-data and simulation draws are independent. Their leading covariance matrices are written as

The matrices \(\Omega_d\) and \(\Omega_s\) describe the variability of the observed and simulated auxiliary statistics, respectively. These expressions are large-sample approximations, not claims that every finite-sample auxiliary statistic has exactly that variance.

For a fixed positive-definite weighting matrix \(W\), the estimator minimizes the weighted squared discrepancy. Let \(G\) denote the \(q\)-by-\(k\) Jacobian of \(a(\theta)\) at \(\theta_0\). The next calculation shows how the two error sources perturb the estimate.

A Local Derivation

Let \(\delta=\hat\theta-\theta_0\). A first-order expansion of the population mapping gives \(a(\hat\theta)\approx a(\theta_0)+G\delta\). To first order, the discrepancy at \(\hat\theta\) is therefore \(u_n-v_B(\theta_0)-G\delta\). Minimizing its weighted squared length is equivalent, locally, to minimizing

Differentiating this quadratic expression with respect to \(\delta\) and setting the result to zero gives

When \(G'WG\) is nonsingular, solving the equation yields the local approximation

This is the key propagation result: auxiliary-statistic error is translated into parameter error by \(A\). In particular, directions in the auxiliary statistics to which the parameter estimate is highly sensitive can magnify simulation noise. Under the independence and covariance conditions above, the corresponding leading covariance is

The first term is sampling uncertainty; the second is simulation uncertainty. If observed and simulated errors are not independent, their covariance terms must also be included. In applications, standard errors that ignore a non-negligible simulation term can understate the uncertainty of the estimate.

Simulation Effort and Standard Errors

The scalar example illustrates a general rate: when simulation variance is proportional to \(1/B\), simulation-induced standard errors are proportional to \(1/\sqrt{B}\). Quadrupling the number of effective draws halves that component of the standard error. It does not halve the sampling component, which is controlled primarily by the observed sample size and the variability of the data.

This distinction matters when reporting uncertainty. With a fixed sample, increasing simulation effort can make the computed estimate less sensitive to the particular simulation run. But it does not create additional observed information, and it cannot reduce sampling uncertainty. A practical choice of \(B\) compares the propagated simulation standard error with the data-based standard error in the parameter or contrast of interest.

How Draws Should Grow With the Data

The covariance approximation also clarifies how simulation effort relates to sample size. For the simulation contribution to vanish as an estimator is studied across increasing samples, \(B\) must increase. For simulation error to be negligible at the same scale as sampling error, which is typically \(1/\sqrt{n}\), the usual sufficient rate is \(B/n\to\infty\). If instead \(B/n\to\kappa\), where \(0<\kappa<\infty\), simulation error remains part of the leading asymptotic uncertainty. If \(B\) grows but more slowly than \(n\), the estimate may still become consistent under regularity conditions, but simulation error can dominate its sampling-scale variation.

These comparisons are rate statements, not a substitute for checking the actual variability and sensitivity of the auxiliary statistics. The matrices \(\Omega_s\) and \(A\) determine how much simulation error reaches each parameter. A large draw count may still be inadequate for a particularly noisy statistic or a parameter that responds sharply to small changes in the simulated targets.

There is also a distinction between the number of draws and the number of simulated datasets. If the procedure averages auxiliary estimates from \(R\) independent simulated datasets, each based on \(m\) independent observations, the variance can often decrease in proportion to \(1/(Rm)\). But if datasets share random inputs or contain dependent draws, simply multiplying their counts may overstate the effective simulation effort. The variance reduction must follow from the actual simulation design.

Repeated Runs and Common Random Numbers

One direct diagnostic is to rerun the estimator with independent simulation draws while holding the observed data and the estimation procedure fixed. Variation across those estimates reflects simulation sensitivity. It is not a replacement for sampling uncertainty: each run uses the same observed sample, so it does not reveal how estimates would vary across new observed samples.

When comparing candidate parameter values, using the same underlying random numbers at each value is often called common random numbers. If simulation outcomes at nearby parameter values move together, shared draws can reduce the noise in their difference. This can make an objective function or its numerical comparison smoother and help the optimizer distinguish candidate values. However, shared draws do not eliminate simulation error in the statistic's level. The resulting estimate and its uncertainty can still depend on the shared simulation sample.

Common Pitfalls and Practical Checks

• Treating a simulated target as exact. The simulated statistic is an estimate based on random draws. If its variation is appreciable, the parameter estimate and standard errors can both be affected.
• Assuming more draws reduce every source of uncertainty. More draws reduce simulation error, not sampling error in the observed data.
• Using the raw draw count as the effective count. Dependence or shared simulation inputs can make the effective number of independent draws smaller.
• Confusing smoother comparisons with zero error. Common random numbers can reduce noise in differences between candidate values, but do not automatically remove level error or the need to assess simulation uncertainty.
• Ignoring nonlinear effects. The local derivation describes the leading perturbation under smoothness and regularity conditions. With a nonlinear mapping, a zero-mean simulation error can still produce finite-simulation bias through curvature and optimization; the first-order variance formula does not rule that out.

A useful practice is to check the estimate across larger simulation budgets and, when feasible, across independent simulation runs. Stability should be assessed in the parameters or contrasts that matter, not only in the overall objective value. If simulation error is material, increase \(B\), account for its contribution to uncertainty, or both. If it is already negligible compared with sampling uncertainty, allocating substantially more computation to simulation may not improve the statistical conclusion.