Tutorials › Econometrics (Graduate) › Numerical Integration and Simulation Error in Demand Estimation

Demand and Discrete Choice · Tutorial 736 of 952

Numerical Integration and Simulation Error in Demand Estimation

Compare integration methods for simulated demand probabilities and learn to recognize, quantify, and reduce simulation noise during estimation.

Advanced 11 min read

What You'll Learn

  • Express random-coefficient demand shares as integrals of conditional choice probabilities.
  • Distinguish deterministic quadrature error from Monte Carlo simulation error.
  • Calculate a Monte Carlo standard error and interpret its dependence on draw count.
  • Explain how common random numbers improve comparisons across parameter values.
  • Diagnose noisy estimates using draw-count checks, repeated simulations, and parameter stability.

Why Integration Accuracy Matters

In a random-coefficient demand model, consumers differ in their preferences, and market shares average choice probabilities across that heterogeneity. Those probabilities are usually integrals rather than simple closed-form expressions. Estimation therefore depends on numerical approximations: even when the economic model is correctly specified, inaccurate integration can make the computed shares—and the parameter estimates based on them—unreliable.

Two common approaches are quadrature, which evaluates the integrand at a carefully chosen set of points with fixed weights, and Monte Carlo integration, which averages evaluations at random draws. They have different error properties. Quadrature is deterministic for a given rule and number of nodes; Monte Carlo error is random and can be estimated from the variation across draws.

The Demand Integral and Its Approximations

Consider market \(t\), with inside products \(j=1,\ldots,J_t\) and an outside option \(0\). Let \(\nu\) denote a consumer's random taste draw, distributed according to \(F\). Given model parameters \(\theta\), let \(p_{jt}(\nu;\theta)\) be that consumer's conditional probability of choosing product \(j\). A population market share is the average of this probability over the distribution of tastes:

$$ s_{jt}(\theta)=\int p_{jt}(\nu;\theta)\,dF(\nu). $$

For example, in a random-coefficient logit model, the conditional probability is the product's exponentiated utility divided by the sum of exponentiated utilities for all inside products and the outside option. It varies with \(\nu\), so averaging it over \(F\) is necessary to obtain the market share. The approximation methods below apply to this integral; they do not change the underlying choice model.

Definition: Numerical integration error is the difference between an integral and its computed approximation. Quadrature error is the discrepancy from replacing the integral with a weighted sum at selected nodes. Monte Carlo error is the random discrepancy from approximating the integral with an average over random draws.

With \(M\) draws \(\nu^{(1)},\ldots,\nu^{(M)}\) from \(F\), the Monte Carlo approximation is

$$ \widehat{s}_{jt,M}(\theta) = \frac{1}{M}\sum_{m=1}^{M}p_{jt}(\nu^{(m)};\theta). $$

If the draws are independent and identically distributed from \(F\), and the conditional probabilities have finite variance, this average is unbiased for \(s_{jt}(\theta)\). Its standard error, conditional on \(\theta\), is estimated by

$$ \widehat{SE}\!\left(\widehat{s}_{jt,M}(\theta)\right) = \frac{s_p}{\sqrt{M}}, \qquad s_p^2= \frac{1}{M-1}\sum_{m=1}^{M} \left(p_{jt}(\nu^{(m)};\theta)-\widehat{s}_{jt,M}(\theta)\right)^2. $$

Here \(s_p\) is the sample standard deviation of the simulated probabilities. This standard error describes Monte Carlo variation in the approximated integral at the stated parameter value. It is not a standard error for the estimated economic parameter, and it does not include uncertainty from sampling the market data.

Quadrature: Fixed Nodes and Weights

A quadrature rule replaces integration by a weighted sum. For a one-dimensional random variable with density \(f\), a rule with nodes \(a_1,\ldots,a_K\) and weights \(w_1,\ldots,w_K\) approximates the integral by \(\sum_{k=1}^K w_k p(a_k;\theta)\). The nodes and weights are chosen to approximate integrals accurately for a class of functions. For a normally distributed taste draw, Gaussian quadrature rules are often constructed for the normal distribution, with the nodes and weights scaled to match its mean and variance.

For multiple taste dimensions, one can use a product rule that combines one-dimensional nodes. The number of evaluations can grow rapidly with the number of dimensions: using \(K\) nodes in each of \(d\) dimensions requires \(K^d\) evaluations for a full product rule. This is one reason Monte Carlo methods can be attractive in models with many random coefficients.

Worked Example: A Three-Node Normal Quadrature Rule

Suppose a single inside product competes with an outside option, and its conditional choice probability for a consumer with taste draw \(\nu\) is \(g(\nu)=1/(1+\exp(-\nu))\). Let \(\nu\sim N(0,1)\). The population share is \(E[g(\nu)]\). A three-node Gauss-Hermite rule for a standard normal uses nodes \(-\sqrt{3},0,\sqrt{3}\), with respective weights \(1/6,2/3,1/6\). These weights sum to one.

Evaluating the conditional probability gives \(g(-\sqrt{3})\approx0.1503\), \(g(0)=0.5000\), and \(g(\sqrt{3})\approx0.8497\). Thus the quadrature approximation is

$$ \widehat{s}_{\mathrm{quad}} = \frac{1}{6}(0.1503)+\frac{2}{3}(0.5000)+\frac{1}{6}(0.8497) \approx 0.5000. $$

In this particular example the integral is exactly \(0.5\), not merely approximately so: symmetry of the standard normal and the identity \(g(-x)+g(x)=1\) imply \(E[g(\nu)]=0.5\). The symmetric quadrature nodes and weights also reproduce that symmetry. This is a useful check on the arithmetic, but it should not be mistaken for evidence that three nodes approximate every demand integral accurately. An asymmetric integrand or a higher-dimensional model may need many more nodes.

Monte Carlo Error and the Role of Draw Counts

For independent draws, the variance of the Monte Carlo average is the variance of one draw's integrand divided by \(M\). This gives the familiar inverse-square-root relationship between draw count and standard error. Quadrupling \(M\) halves the standard error in expectation, provided the distribution of draws and the integrand remain the same. It does not guarantee that every realized approximation will be closer to the integral.

The following justification also explains why the sample standard deviation of the simulated probabilities is relevant. Write \(Y_m=p_{jt}(\nu^{(m)};\theta)\), with \(E[Y_m]=s_{jt}(\theta)\) and \(\operatorname{Var}(Y_m)=\sigma_p^2\). Independence makes the covariance terms in the variance of the average zero:

$$ \operatorname{Var}\!\left(\frac{1}{M}\sum_{m=1}^M Y_m\right) = \frac{1}{M^2}\sum_{m=1}^M\operatorname{Var}(Y_m) = \frac{\sigma_p^2}{M}. $$

Replacing the unknown \(\sigma_p\) with the sample standard deviation \(s_p\) gives the estimated standard error \(s_p/\sqrt{M}\). For bounded choice probabilities, the variance is finite. The calculation still relies on independent draws; correlated draws require accounting for their dependence rather than using the independent-draw formula uncritically.

Worked Example: Diagnosing a Small Monte Carlo Approximation

Return to \(g(\nu)=1/(1+\exp(-\nu))\), with \(\nu\sim N(0,1)\), whose integral is \(0.5000\). Suppose a hypothetical simulation uses these eight independent standard-normal draws: \(-1.5,-0.8,-0.2,0.1,0.5,0.9,1.4,2.0\). The resulting probabilities, rounded to four decimal places, are \(0.1824,0.3100,0.4502,0.5250,0.6225,0.7109,0.8022,0.8808\).

The Monte Carlo average is

$$ \widehat{s}_{\mathrm{MC}} = \frac{0.1824+0.3100+0.4502+0.5250+0.6225+0.7109+0.8022+0.8808}{8} = \frac{4.4840}{8} = 0.5605. $$

The sample standard deviation of these eight probabilities is approximately \(s_p=0.2410\). Therefore the estimated Monte Carlo standard error is

$$ \widehat{SE}(\widehat{s}_{\mathrm{MC}}) = \frac{0.2410}{\sqrt{8}} \approx 0.0852. $$

The difference between this simulation estimate and the quadrature value \(0.5000\) is \(0.0605\), smaller than the estimated Monte Carlo standard error. That is consistent with ordinary simulation variation, though a single comparison is not a formal test of equality. If the draw count increased from 8 to 32 while the integrand's variance stayed similar, the standard error would be expected to fall by a factor of \(\sqrt{8/32}=1/2\), to about \(0.0426\). The actual error in a particular new simulation need not be exactly half as large.

Common Random Numbers in Estimation

Estimation evaluates model predictions at many parameter values. If each evaluation uses newly generated draws, the change in a simulated share or objective function combines two things: the effect of changing the parameter and the effect of changing the draw set. This simulation noise can make comparisons difficult and can interfere with numerical optimization.

Common random numbers address this problem by using the same underlying draws at each parameter value. The simulated values then move together when the parameter changes. When their simulation errors are positively correlated, the error in their difference is smaller than it would be with independent draw sets. Common draws do not make either simulated integral exact, but they can make differences between parameter evaluations more stable.

Result: If the paired evaluations are independent and identically distributed across \(m\), the variance of their Monte Carlo difference is \(\operatorname{Var}(A_m-B_m)/M\). Since \(\operatorname{Var}(A_m-B_m)=\operatorname{Var}(A_m)+\operatorname{Var}(B_m)-2\operatorname{Cov}(A_m,B_m)\), positive covariance reduces the variance of the difference.

Worked Example: Paired Draws Stabilize a Parameter Comparison

Suppose four common taste draws produce simulated probabilities at a baseline parameter value \(\theta_0\) of \(0.20,0.40,0.60,0.80\), and at a nearby value \(\theta_1\) of \(0.25,0.48,0.68,0.84\). These are hypothetical paired evaluations: each position uses the same draw at both parameter values.

The average at \(\theta_0\) is \(2.00/4=0.5000\); at \(\theta_1\), it is \(2.25/4=0.5625\). The paired draw-by-draw differences are \(0.05,0.08,0.08,0.04\). Their mean is \(0.25/4=0.0625\), equal to the difference between the two averages. The sample variance of the paired differences is \(0.000425\), so their estimated standard error is

$$ \frac{\sqrt{0.000425}}{\sqrt{4}} \approx 0.0103. $$

For comparison, if the two sets of four evaluations were independent, the sample variances of the values at \(\theta_0\) and \(\theta_1\) would be \(0.06667\) and \(0.06509\). The estimated standard error of their difference would then be

$$ \sqrt{\frac{0.06667}{4}+\frac{0.06509}{4}} \approx 0.1815. $$

This large contrast is a small-sample illustration, not a general guarantee of a particular reduction. It shows the mechanism: the paired values vary similarly across draws, so the draw-specific fluctuations largely cancel in their differences.

Practical Checks and a Common Pitfall

In simulated demand estimation, numerical integration is part of the computation used to fit the model. It is therefore important to check that results are not artifacts of a particular approximation. Useful checks include increasing the number of quadrature nodes or Monte Carlo draws, repeating Monte Carlo calculations with independent draw sets, and checking whether the estimated parameters and fitted shares remain stable. For an optimization routine, holding a draw set fixed across parameter evaluations can reduce objective-function noise; a larger or independent draw set can then be used to check the final result.

These checks address different issues. Agreement across repeated Monte Carlo draw sets is evidence that random integration noise is small, while stability as the quadrature rule is refined is evidence that the chosen nodes and weights are adequate. Neither check by itself establishes that the economic model is correctly specified. Nor should simulation error be confused with statistical uncertainty due to finite market data.

A common mistake is to treat a computed share as exact because a numerical routine returned many decimal places. For instance, reporting a Monte Carlo share as \(0.560500\) does not mean it is accurate to six decimal places; in the example, its estimated simulation standard error was about \(0.0852\). The correction is to report precision in light of the integration method and draw count, and to test whether the substantive estimates are stable when the approximation is improved.

A second mistake is to redraw tastes at every parameter evaluation and interpret every change in the objective as an economic response to the parameter. With fresh draws, some of that change may be simulation noise. Use common random numbers for comparisons where appropriate, and assess integration accuracy separately by raising the draw count or refining quadrature.

Key takeaway: Quadrature trades a deterministic weighted approximation for selected nodes; Monte Carlo trades a random average for flexibility in higher dimensions. Draw counts govern Monte Carlo precision at an inverse-square-root rate, and common random numbers can stabilize comparisons across parameter values without eliminating integration error.

Check Your Understanding

Answer each question by distinguishing integration accuracy from uncertainty in the market data.

  1. Write the integral defining a random-coefficient market share in terms of conditional choice probabilities and the taste distribution.
  2. If the Monte Carlo draw count is multiplied by nine and the integrand variance is unchanged, by what factor does the standard error change?
  3. Why does a symmetric quadrature rule give \(0.5\) for the integral of \(g(\nu)=1/(1+\exp(-\nu))\) under a standard normal distribution?
  4. What assumption about the draws supports the standard error formula \(s_p/\sqrt{M}\)?
  5. Explain how using common random numbers can reduce noise in the difference between predictions at two parameter values.
  6. Name one practical check for quadrature accuracy and one for Monte Carlo simulation noise.
Clinical Biostats