Monte Carlo Precision and Simulation Error

Why a Simulated VaR Is Not Exact

Monte Carlo VaR estimation uses simulated losses to approximate a population quantile. Even when the loss model is fixed, a different set of random draws will generally produce a different empirical VaR. That variation is simulation error: it arises from finite sampling, not from a change in the risk model or the underlying loss distribution.

A reported VaR estimate should therefore be read together with an assessment of its precision. An estimate of $500,000, for example, may be stable to a few hundred dollars or may vary by tens of thousands across independent simulation runs. The number of simulations alone does not tell us which; the confidence level and the shape of the loss distribution around the target quantile matter too.

Definition: Let $X$ be a loss with cumulative distribution function $F_X$. For confidence level $p$, define the population VaR as $q_p=\inf\{x:F_X(x)\ge p\}$. Given $n$ independent simulated losses $X_1,\ldots,X_n$, an empirical VaR estimate is a sample quantile, commonly the order statistic $X_{(\lceil np\rceil)}$, where $X_{(1)}\le\cdots\le X_{(n)}$. Simulation error is the sampling variation in this estimate when the loss model and $n$ are held fixed.

The stated order-statistic convention is used here to make calculations definite. Other empirical quantile conventions can give slightly different finite-sample answers, particularly when $np$ is not an integer. The large-sample precision ideas below apply to standard sample-quantile conventions under the stated regularity conditions.

Quantile Standard Error

Suppose simulated losses are independent and identically distributed from a continuous distribution, and the density $f_X(q_p)$ exists and is positive at the population quantile. For a sufficiently large simulation sample, the empirical quantile is approximately normal around $q_p$, with variance and standard error

\operatorname{Var}(\widehat q_p)\approx \frac{p(1-p)}{n f_X(q_p)^2}, \qquad \operatorname{SE}(\widehat q_p)\approx \frac{\sqrt{p(1-p)}}{\sqrt{n}\,f_X(q_p)}.

Here $\widehat q_p$ denotes the simulated VaR estimate, and $f_X(q_p)$ is the loss density at the population quantile. The standard error is in the same units as the loss. This approximation requires a continuous distribution with a positive, sufficiently well-behaved density near the quantile and a large enough $n$. It is not automatically reliable for discrete losses, a density that is zero or irregular at the quantile, or a simulation sample too small for the approximation to be credible.

A short derivation explains the roles of $p$, $n$, and the density. At the true quantile, each simulated loss is below $q_p$ with probability $p$. Thus the empirical fraction below $q_p$, $F_n(q_p)$, is an average of $n$ Bernoulli indicators, with approximate variance $p(1-p)/n$. Near $q_p$, a small change in cumulative probability corresponds to a loss change of approximately that amount divided by $f_X(q_p)$, because $F_X(q_p+\Delta)-F_X(q_p)\approx f_X(q_p)\Delta$. The empirical quantile adjusts the loss threshold to offset the sampling fluctuation in the empirical fraction. Consequently,

\widehat q_p-q_p\approx -\frac{F_n(q_p)-p}{f_X(q_p)}, \qquad \operatorname{Var}(\widehat q_p)\approx \frac{p(1-p)}{n f_X(q_p)^2}.

This also shows why a low density around the quantile makes VaR harder to estimate: a small probability-scale fluctuation translates into a larger loss-scale fluctuation. Increasing $n$ by a factor of four reduces the approximate standard error by a factor of two, not four, because it is proportional to $1/\sqrt n$.

Worked Example: Precision of a 99% Normal-Loss VaR

A hypothetical one-year portfolio loss is simulated from $X\sim N(100,20^2)$, with losses measured in thousands of dollars. The mean is $100,000 and the standard deviation is $20,000. Estimate the approximate standard error of a 99% Monte Carlo VaR based on $n=10{,}000$ independent simulations, and give an approximate 95% simulation-error interval around the population VaR.

Method. Use the normal quantile and density in the asymptotic sample-quantile standard-error formula. The 99th percentile of the standard normal is $z_{0.99}\approx2.3263$. The population VaR is

q_{0.99}=100+2.3263(20)=146.526

Thus $q_{0.99}$ is about $146,526. The normal density of $X$ at this value is the standard normal density at $2.3263$, divided by the standard deviation $20$:

f_X(q_{0.99})=\frac{\phi(2.3263)}{20} \approx\frac{0.02665}{20}=0.0013325

The density is expressed per thousand dollars. Substituting into the standard-error formula gives

\operatorname{SE}(\widehat q_{0.99}) \approx\frac{\sqrt{0.99(0.01)}}{\sqrt{10{,}000}(0.0013325)} =\frac{0.09950}{100(0.0013325)} \approx0.7467

The standard error is approximately 0.7467 thousand dollars, or $746.70. For a normal-approximation 95% interval, use $1.96$ standard errors on either side of the population VaR:

146.526\pm1.96(0.7467)=146.526\pm1.4635

The interval is approximately $145,063 to $147,990, after rounding to the nearest dollar. This is an approximate interval for the variation of a simulated quantile under the assumed model, not a statement that the model itself is correct. Common mistake avoided: treating 1% of 10,000, or 100, as the standard error. That is the expected number of losses above the VaR, not a dollar measure of quantile uncertainty.

Why Tail Quantiles Need Many Simulations

At confidence level $p$, the expected number of simulated losses above the population VaR is $n(1-p)$. At a 99% level, 10,000 simulations yield an expected 100 exceedances; at a 99.9% level, the same number yield only 10. A small number of tail observations makes the empirical tail information especially sensitive to which random losses happen to appear.

The expected exceedance count is a useful warning signal, but it is not itself a complete precision calculation. The standard error also depends on $f_X(q_p)$. Even with the same $n$ and $p$, two loss distributions can have different densities at their quantiles and therefore different VaR standard errors. A high confidence level tends to make estimation more demanding because fewer observations lie beyond the target quantile; the exact dollar uncertainty still depends on the distribution near that point.

Worked Example: Sample Size for an Exponential-Loss VaR

Consider a hypothetical annual loss $X$ with an exponential distribution and mean $100, so its density is $f_X(x)=0.01e^{-x/100}$ for $x\ge0$. Compare the approximate standard error of its 99% VaR using $n=100{,}000$ and $n=400{,}000$ independent simulations.

Method. For an exponential loss with mean 100, the cumulative distribution is $F_X(x)=1-e^{-x/100}$. Solving $F_X(q_{0.99})=0.99$ gives

q_{0.99}=-100\ln(1-0.99)=100\ln(100)\approx460.517

At this quantile, $e^{-q_{0.99}/100}=0.01$, so the density is

f_X(q_{0.99})=0.01(0.01)=0.0001

For $n=100{,}000$, the standard error is

\frac{\sqrt{0.99(0.01)}}{\sqrt{100{,}000}(0.0001)} =\frac{0.09950}{316.2278(0.0001)} \approx3.146

This is approximately $3.15. For $n=400{,}000$, the sample size is four times larger, so the standard error is half as large: approximately $1.57. The expected number of exceedances is 1,000 in the first simulation and 4,000 in the second. The standard-error result confirms the square-root sample-size relationship. Common mistake avoided: assuming that quadrupling the number of simulations cuts the error to one-quarter. It cuts the approximate standard error in half.

Assessing Precision with Repeated Runs

When the density at the quantile is not available or is difficult to estimate reliably, independent repeated simulation runs provide a practical diagnostic. Run the same model with the same $n$, using independent random draws each time, and calculate a VaR estimate for each run. The sample standard deviation of those estimates approximates the run-to-run simulation standard error for that simulation design.

The runs must use the same model, confidence level, simulation size, and quantile convention for this comparison to be interpretable. Their random draws should be independent, or any dependence between runs must be accounted for. A small number of repeated runs gives only a rough estimate of variability; if precision matters, use enough replications to assess whether the estimated spread itself is stable.

Worked Example: Reading Variation Across Independent Runs

A hypothetical portfolio has one-year loss $X\sim N(500,20^2)$, measured in thousands of dollars. Five independent runs each use 20,000 simulated losses and the same 99% empirical-quantile convention. Their reported 99% VaRs, in thousands of dollars, are 544, 546, 548, 545, and 547. Estimate the run-to-run simulation standard error.

Method. Calculate the sample standard deviation of the five estimates. Their mean is

\bar q=\frac{544+546+548+545+547}{5} =\frac{2730}{5}=546

The deviations from the mean are $-2,0,2,-1,1$, and their squared deviations sum to

$$ (-2)^2+0^2+2^2+(-1)^2+1^2=10 $$

Because these are five observations of the run-level estimate, divide by $5-1=4$ to obtain the sample variance:

s^2=\frac{10}{4}=2.5, \qquad s=\sqrt{2.5}\approx1.581

The estimated run-to-run standard deviation is 1.581 thousand dollars, or about $1,581. It estimates how much a single 20,000-loss run’s VaR may vary under repeated runs, subject to the uncertainty from using only five replications. The standard error of the mean of these five estimates would instead be $1.581/\sqrt5\approx0.707$ thousand dollars; that describes the average of five run estimates, not the variation of an individual run’s VaR. Common mistake avoided: dividing by $\sqrt5$ and then reporting that smaller number as the simulation error of each run.

Quantile Ranks and Uncertainty

A quantile is determined by the rank of losses in the ordered sample. This provides a second way to understand uncertainty: even if the loss values are not known in advance, the rank of the population quantile in a simulation sample is random. If the distribution is continuous and $q_p$ is the population $p$-quantile, the number $K$ of simulated losses below $q_p$ has a binomial distribution:

K\sim\operatorname{Binomial}(n,p)

This follows because each independent simulated loss has probability $p$ of being below $q_p$. Binomial probabilities can be used to select order-statistic ranks for an interval for the quantile. In particular, an interval from $X_{(r_L)}$ to $X_{(r_U)}$ covers $q_p$ when $r_L\le K<r_U$, apart from ties, which have probability zero under continuity. Choose the ranks so that this event has the desired probability. The resulting interval uses the simulated losses directly and does not require inserting a density estimate into the asymptotic standard-error formula.

Rank intervals and asymptotic standard errors answer related but distinct practical needs. The standard error summarizes the approximate spread of the estimator around the population quantile. A rank interval gives a range of loss values supported by the sample at a selected coverage level. Both rely on the simulation representing independent draws from the specified distribution; neither accounts for uncertainty in choosing that distribution or its parameters.

Interpreting and Reporting Simulation Error

Simulation precision matters because a small reported difference in VaR may be no larger than the random variation of the estimates. If two estimates differ by $1,000 while each has simulation error of roughly $5,000, the apparent ranking is not well supported by those simulation runs alone. More simulations may help resolve the comparison, but they do not address differences in model assumptions or parameter choices.

A clear report states the confidence level, number of simulated losses, quantile convention, and precision method. If a standard error or interval is reported, identify the distributional assumptions and whether the calculation is an asymptotic approximation, a repeated-run estimate, or a rank-based interval. For a useful first diagnostic, compare $n(1-p)$ with the desired tail resolution and assess whether the density near the quantile is low. A large total simulation count can still leave very few observations beyond an extreme quantile.

Simulation error is finite-sample variation conditional on the model and its parameters.
Model uncertainty concerns whether the chosen distribution and assumptions represent the risk adequately. More draws from the same model do not remove it.
Parameter uncertainty concerns uncertainty in fitted model parameters. It is not included in the basic quantile standard-error formula when parameters are treated as fixed.

Key takeaway: Under regular continuous-loss assumptions, Monte Carlo VaR standard error is approximately $\sqrt{p(1-p)}/(\sqrt n f_X(q_p))$. More simulations reduce this error at the square-root rate, while a low density near the target quantile increases it. Repeated runs and binomial rank reasoning offer complementary ways to assess precision.

Check Your Understanding

For each problem, treat the simulation draws as independent and use the asymptotic quantile standard error when the required density is given or can be derived.

A hypothetical uniform loss is distributed over $0 to $1,000. For a 99% VaR estimated from 10,000 simulations, find the population VaR, the density at that quantile, the expected number of exceedances, and the approximate standard error. (Answer: VaR = $990; density = 0.001 per dollar; expected exceedances = 100; standard error ≈ $0.9950.)
A hypothetical exponential loss has mean $50. For a 95% VaR based on 2,500 simulations, calculate the population VaR and approximate standard error. (Answer: VaR ≈ $149.79; standard error ≈ $4.3589.)
Under the same continuous loss model, confidence level, and density at the VaR, a simulation standard error is estimated as $8 with 25,000 runs. Approximately how many runs are needed to reduce it to $2? (Answer: 400,000 runs.)
A hypothetical exponential loss has mean $200. For a 99.5% VaR, use the asymptotic formula to find the minimum simulation size needed for standard error no greater than $2. (Answer: 1,990,000 simulations.)
Five independent simulation runs produce 99% VaRs, in thousands of dollars, of 490, 505, 500, 495, and 510. Calculate their mean and sample standard deviation, and state which one estimates the run-to-run variation of an individual run’s VaR. (Answer: mean = $500,000; sample standard deviation ≈ $7,905.69; the sample standard deviation estimates individual-run variation.)

What You'll Learn