Numerical Error and Monte Carlo Error in Simulation Reports

Why a Simulation Result Can Be Imprecise for More Than One Reason

A simulation report may give a precise-looking number without having computed the intended quantity precisely. Random samples create Monte Carlo error; numerical methods such as discretization or numerical integration create approximation error; and finite-precision arithmetic or rounding creates computational error. These sources have different causes and are not interchangeable. In particular, increasing the number of simulations reduces Monte Carlo error but does not necessarily reduce a numerical approximation error.

The practical task is to identify which source matters most for the reported result. A Monte Carlo standard error describes random variation under the simulation design. A numerical error bound describes how far a computational approximation may be from its target. Rounding error is controlled by the calculation and reporting precision. Comparing these quantities helps determine whether additional simulations would meaningfully improve the answer.

Definition: Let $\theta$ be the quantity of interest. Suppose a numerical method targets an approximation $\theta_h$, a simulation produces an estimate $\hat{\theta}_h$, and the displayed result is $r$. Then the total error has the exact decomposition $$ r-\theta = (\theta_h-\theta) + (\hat{\theta}_h-\theta_h) + (r-\hat{\theta}_h). $$ The first term is numerical approximation error, the second is Monte Carlo sampling error relative to the method's target, and the third is reporting or arithmetic error. The decomposition does not imply that the terms are independent or that their magnitudes should automatically be added as standard errors.

This separation is important because the Monte Carlo standard error is calculated for the quantity the simulation actually estimates. If a discretized calculation targets $\theta_h$ rather than $\theta$, a small standard error around $\theta_h$ does not establish that $\theta_h$ is close to $\theta$. Likewise, a rounding interval is not a confidence interval: it describes the effect of rounding, not the randomness of the simulated sample.

Monte Carlo Error and Numerical Error

For independent, identically distributed simulation outputs $Z_1,\ldots,Z_n$ with finite variance, the simulation estimate of their mean is $\bar Z$. If $s$ is the sample standard deviation, the estimated Monte Carlo standard error is

\widehat{\operatorname{SE}}_{\mathrm{MC}}(\bar Z)=\frac{s}{\sqrt{n}}.

This measures sampling variability in $\bar Z$, not the discrepancy between the simulation model and reality, nor an error introduced by a numerical approximation used inside each replication. A normal approximation often gives a 95% interval of $\bar Z \pm 1.96s/\sqrt n$ when the sampling distribution is adequately approximated by a normal distribution. That interval concerns Monte Carlo error only unless other error sources are explicitly incorporated.

Numerical error arises when a calculation replaces the desired quantity with an approximation. Examples include replacing an integral with a finite grid, truncating an infinite sum, or stopping an iterative calculation at a tolerance. It may be estimated by comparison with an exact result, by a convergence study using finer grids, or by a valid analytical error bound. Numerical error can be systematic: repeating the same calculation with new random numbers does not make the same discretization bias disappear.

Rounding error is different again. When a value is rounded to the nearest unit $q$, the absolute rounding error is at most $q/2$, assuming ordinary round-to-nearest rules. Arithmetic at finite precision can also introduce roundoff before the final reporting step; this is usually negligible in well-conditioned calculations, but can matter when subtracting nearly equal quantities or iterating many operations.

Worked Example: Choosing Precision for a Simulated Mean

A hypothetical portfolio simulation estimates an expected annual loss, measured in thousands of dollars. The simulation uses $n=400$ independent outputs. The sample mean is $125.37$, and the sample standard deviation is $20.00$. Each output also uses a numerical approximation whose error in the estimated mean is known to be at most $0.08$ thousand dollars. The reported mean is to be rounded to the nearest $0.1$ thousand dollars. Identify the important error scales and give a sensible interpretation.

Method. Calculate the Monte Carlo standard error from the sample standard deviation, then compare it with the numerical bound and the actual reporting-rounding error.

First, the Monte Carlo standard error is

\frac{20.00}{\sqrt{400}} = \frac{20.00}{20} = 1.00 \text{ thousand dollars}.

An approximate 95% Monte Carlo margin is

1.96(1.00)=1.96 \text{ thousand dollars}.

Thus, considering Monte Carlo error alone, the approximate interval is $(125.37-1.96,\ 125.37+1.96)=(123.41,\ 127.33)$ thousand dollars. The stated numerical error bound, $0.08$, is much smaller than the Monte Carlo standard error of $1.00$. Rounding $125.37$ to the nearest $0.1$ gives $125.4$, an actual rounding change of $0.03$ thousand dollars, also much smaller than the Monte Carlo standard error. The sampling error is the main limitation on precision in this example.

The rounded value can be reported as approximately $125.4$ thousand dollars, but its tenths digit should not be mistaken for high certainty. The uncertainty from sampling is on the scale of about one thousand dollars, not one tenth of a thousand dollars. The numerical bound is an additional qualification, not another Monte Carlo standard error.

Check. The approximate interval is centered on $125.37$, and its half-width is $1.96$, as required by the stated 95% normal approximation. Common mistake avoided: treating the displayed tenths digit as evidence that the expected loss is known to the nearest hundred dollars.

How to Compare Error Sources

A useful comparison puts each error on the scale of the reported quantity. For a Monte Carlo standard error, a common scale is the standard error itself or a multiple such as $1.96$ standard errors for an approximate 95% margin. For a numerical approximation, use a justified error bound or a comparison with a more accurate calculation. For rounding, use the actual change caused by rounding or the maximum half-unit bound. These are different kinds of quantities, so state what each number means before drawing a conclusion.

The following derivation explains why a reported result can be close to its computational target yet still miss the intended target. Add and subtract $\theta_h$ and $\hat{\theta}_h$ from $r-\theta$:

r-\theta = (r-\hat{\theta}_h) + (\hat{\theta}_h-\theta_h) + (\theta_h-\theta).

Each term corresponds to a separate stage: display and arithmetic, random estimation, and numerical approximation. This is an algebraic identity, so it holds whether the terms are positive or negative. However, the terms can offset one another. For a conservative worst-case bound, if the absolute numerical error is at most $b$, the absolute Monte Carlo deviation is at most $m$, and the absolute rounding error is at most $q/2$, the triangle inequality gives

|r-\theta| \leq b+m+\frac{q}{2}.

This is a worst-case bound only when each component has the stated bound. A Monte Carlo standard error is not a guaranteed bound on the realized sampling error, so it cannot be substituted for $m$ as though it were one. Instead, use a confidence interval or another probabilistic statement for the Monte Carlo component.

Worked Example: Numerical Integration Error Exceeds Sampling Error

A hypothetical payment is $Y=\min(X,6)$, where $X$ is uniform on $[0,10]$. The target is $E[Y]$. A midpoint integration routine uses intervals $[0,4]$, $[4,8]$, and $[8,10]$. Separately, a simulation with $n=40{,}000$ reports sample mean $4.19$ and sample standard deviation $1.99$. Compare the integration approximation error with the Monte Carlo standard error.

Method. Calculate the exact expectation, evaluate the midpoint approximation, and then calculate the simulation standard error.

Since $Y=x$ for $0\leq x\leq 6$, and $Y=6$ for $6<x\leq10$, the exact expectation is

E[Y] = \frac{1}{10}\left(\int_0^6 x\,dx+\int_6^{10}6\,dx\right) = \frac{1}{10}\left(18+24\right) = 4.2.

For the midpoint approximation, the interval widths are $4,4,2$, and the midpoints are $2,6,9$. The payment values at those midpoints are $2,6,6$. Therefore,

\frac{1}{10}\bigl(4(2)+4(6)+2(6)\bigr) = \frac{44}{10} = 4.4.

The absolute midpoint integration error is $|4.4-4.2|=0.2$. The simulation's estimated Monte Carlo standard error is

\frac{1.99}{\sqrt{40{,}000}} = \frac{1.99}{200} = 0.00995.

The integration error is about $0.2/0.00995\approx20.1$ times the Monte Carlo standard error. For comparison, the simulation's approximate 95% Monte Carlo margin is $1.96(0.00995)=0.0195$, still far below the integration discrepancy. In this case, refining the numerical integration grid would matter much more than increasing the simulation count. The simulated mean $4.19$ is close to the exact value $4.2$, but that observed closeness does not remove the midpoint method's $0.2$ error.

Check. The midpoint result is above the exact expectation because the grid interval $[4,8]$ uses a midpoint payment of $6$ even though the payment increases from $4$ to $6$ over part of that interval. Common mistake avoided: assuming the simulation standard error measures the error of the separate integration routine.

Rounding, Approximation, and the Precision You Report

A simulation can produce many digits because software stores and prints them, not because those digits are stable. The appropriate number of reported digits depends on the uncertainty and the purpose of the report. If a Monte Carlo standard error is several times larger than the final reported unit, extra digits may still be useful for reproducibility, but they should not be interpreted as meaningful certainty. Reporting the estimate alongside its standard error or interval makes the distinction clear.

Probability estimates provide a simple illustration. If $K$ simulated trials out of $n$ meet an event condition, then $\hat p=K/n$. Under independent Bernoulli trials, its estimated standard error is $\sqrt{\hat p(1-\hat p)/n}$. The fraction $K/n$ may be computed exactly from the count, apart from arithmetic and display rounding, but its sampling uncertainty can be much larger than the last displayed digit.

Worked Example: A Precisely Computed Probability With Sampling Uncertainty

A hypothetical simulation runs $10{,}000$ independent trials of a claim event and records $2{,}374$ events. The report displays the estimated probability to four decimal places. Assess whether those digits indicate comparable statistical precision.

Method. Compute the event fraction and its Bernoulli Monte Carlo standard error.

The estimated probability is

\hat p=\frac{2{,}374}{10{,}000}=0.2374.

The estimated standard error is

\sqrt{\frac{0.2374(1-0.2374)}{10{,}000}} = \sqrt{\frac{0.2374(0.7626)}{10{,}000}} = \sqrt{0.0000181041} \approx 0.0043.

The last calculation is rounded to four decimal places. The approximate 95% Monte Carlo margin is $1.96(0.0042549)\approx0.0083$, so an approximate interval is $(0.2291,\ 0.2457)$. The estimate $0.2374$ is arithmetically correct to four decimal places, but the Monte Carlo uncertainty is about $0.0043$, far larger than $0.0001$, the place value of the fourth decimal. A concise report might give $0.237$ with a standard error of about $0.0043$, or provide the interval, depending on the reporting convention.

Check. The approximate interval contains the estimate and has a half-width of approximately $0.0083$. Common mistake avoided: interpreting four calculated decimal places as four statistically reliable decimal places.

Common Traps and Reporting Practice

Increasing $n$ to fix every error. More replications generally reduce Monte Carlo standard error at a rate proportional to $1/\sqrt n$, but do not automatically reduce discretization bias, truncation error, or a fixed stopping tolerance.
Calling a standard error an error bound. A standard error describes the scale of sampling variation; it does not guarantee that a particular estimate is within one standard error of the target.
Combining unlike quantities as if they were standard errors. A deterministic error bound and a Monte Carlo standard error have different interpretations. State them separately unless a justified procedure combines them.
Confusing computational precision with inferential precision. A value can be calculated to many decimal places while its sampling uncertainty is much larger.
Ignoring what the simulation targets. If each simulated output uses an approximation, the Monte Carlo standard error applies to the approximated target unless the approximation is separately shown to be adequate.

The simulation standard error also does not account for uncertainty in estimated model parameters. Parameter uncertainty is a separate source of uncertainty, and a simulation may need a separate method to reflect it. Nor does the usual standard error automatically capture model misspecification or bias in a random-number generator. These issues should not be hidden inside a statement about Monte Carlo precision.

Key takeaway: First identify the target, then separate random sampling variation from deterministic numerical approximation and rounding. Compare the error scales using their proper interpretations. Report enough digits to make the calculation transparent, but use a standard error or interval to communicate how much the simulation itself supports.

Check Your Understanding

For each situation, identify the dominant error source or calculate the requested quantity. Treat any stated Monte Carlo standard error and numerical error bound as applying to the same reported target unless noted otherwise.

A simulation has $n=625$ independent outputs and sample standard deviation $5$. Its reported mean uses a numerical method with error bound $0.03$, and the final display is rounded to the nearest $0.1$. Which error scale is largest? (Answer: Monte Carlo standard error $=5/\sqrt{625}=0.20$, larger than the $0.03$ numerical bound and the maximum $0.05$ rounding error.)
In $2{,}500$ independent Bernoulli trials, an event occurs $100$ times. Calculate the estimated probability and its Monte Carlo standard error. (Answer: $\hat p=0.04$, with standard error $\sqrt{0.04(0.96)/2{,}500}\approx0.0039$.)
For $X$ uniform on $[0,8]$, approximate $E[\min(X,5)]$ by midpoint integration over $[0,4]$ and $[4,8]$. Find the exact expectation and the absolute integration error. (Answer: midpoint estimate $3.5$, exact expectation $3.4375$, absolute error $0.0625$.)
A simulated estimate is $74.286$, its Monte Carlo standard error is $0.12$, the numerical error is bounded by $0.02$, and the display is rounded to the nearest $0.01$. Which source limits the estimate's precision, and what value is displayed? (Answer: Monte Carlo sampling error is largest; the displayed value is $74.29$, with maximum rounding error $0.005$.)
A simulation reports an event probability of $0.245$ from $1{,}960$ events in $8{,}000$ independent trials. Calculate its estimated standard error and state whether rounding to three decimal places indicates precision of $0.001$. (Answer: standard error $\sqrt{0.245(0.755)/8{,}000}\approx0.0048$; no, sampling uncertainty is substantially larger than $0.001$.)

What You'll Learn