Asymptotic Uncertainty of Simulated Value-at-Risk

Why a Simulated Value-at-Risk Estimate Has Uncertainty

A simulation can estimate a loss quantile, but a finite run will generally not reproduce the population quantile exactly. The resulting difference is Monte Carlo error: it arises because only a finite number of random losses were generated. It is distinct from uncertainty about model parameters or whether the chosen loss model represents a real portfolio adequately.

For a loss $X$ with cumulative distribution function $F_X$, the value-at-risk at probability level $p$ is the $p$-quantile. In this tutorial, $p$ is a probability such as $0.90$ or $0.99$, not a percentage written as 90 or 99. Suppose $X_1,\ldots,X_n$ are independent simulated losses from the same distribution. Their empirical distribution function is $F_n(x)=n^{-1}\sum_{j=1}^n I(X_j\leq x)$, and the simulated quantile is an empirical $p$-quantile, denoted $\hat q_p$. A common definition is the order statistic $X_{(\lceil np\rceil)}$; standard interpolation conventions can differ in finite samples but have the same large-sample result under the regularity conditions below.

Definition: The population value-at-risk at level $p$ is $q_p=\inf\{x:F_X(x)\geq p\}$. When $F_X$ is continuous and strictly increasing near $q_p$, $F_X(q_p)=p$. The estimated value-at-risk $\hat q_p$ is the corresponding empirical quantile of the simulated losses.

Asymptotic Standard Error

The key feature of quantile uncertainty is its dependence on the density at the quantile. For a smooth distribution, a steep cumulative distribution function corresponds to a relatively high density, so a small change in cumulative probability requires only a small change in loss. A low density means the same probability fluctuation can move the quantile farther.

Theorem: Suppose $X_1,\ldots,X_n$ are independent and identically distributed, $F_X$ is continuous near $q_p$, and it has a finite, positive, continuous density $f_X(q_p)>0$ there. For a fixed $0<p<1$, the empirical quantile satisfies

\sqrt{n}(\hat q_p-q_p)\ \xrightarrow{d}\ N\!\left(0,\frac{p(1-p)}{f_X(q_p)^2}\right).

Thus, for a sufficiently large simulation, the approximate standard error is

\operatorname{SE}(\hat q_p)\ \approx\ \frac{\sqrt{p(1-p)/n}}{f_X(q_p)}.

In practice, $q_p$ and usually $f_X(q_p)$ are unknown. Replace them with $\hat q_p$ and a density estimate $\hat f_X(\hat q_p)$. This gives the plug-in estimate

\widehat{\operatorname{SE}}(\hat q_p) = \frac{\sqrt{p(1-p)/n}}{\hat f_X(\hat q_p)}.

The density can come from a fitted parametric model or a suitable smooth estimate based on the simulated losses. It must be expressed in units inverse to the loss: if losses are in dollars, density has units per dollar, and the resulting standard error is in dollars.

Why the Formula Holds

The derivation links the empirical quantile to the empirical distribution function at the true quantile. At $q_p$, each indicator $I(X_j\leq q_p)$ is a Bernoulli random variable with probability $p$. Therefore $F_n(q_p)$ has mean $p$ and variance $p(1-p)/n$. For a large sample, the central limit theorem gives

\sqrt{n}\bigl(F_n(q_p)-p\bigr)\ \xrightarrow{d}\ N(0,p(1-p)).

Since $F_n(\hat q_p)$ is close to $p$, expand the population distribution function around $q_p$. Differentiability with positive derivative $f_X(q_p)$ gives $F_X(\hat q_p)-F_X(q_p)\approx f_X(q_p)(\hat q_p-q_p)$. Accounting for the empirical fluctuation at $q_p$, the first-order relation is

0\ \approx\ F_n(\hat q_p)-p \ \approx\ F_n(q_p)-p+f_X(q_p)(\hat q_p-q_p).

Rearranging shows that the quantile error is approximately the negative empirical probability error divided by the density:

\sqrt{n}(\hat q_p-q_p) \ \approx\ -\frac{\sqrt{n}(F_n(q_p)-p)}{f_X(q_p)}.

The minus sign does not affect the variance. Dividing the Bernoulli variance $p(1-p)$ by $f_X(q_p)^2$ yields the stated asymptotic variance. This argument also explains why a density close to zero makes the standard error large.

Constructing an Approximate Confidence Interval

Under the theorem’s conditions, a large-sample two-sided confidence interval with confidence level approximately 95% is

\hat q_p\ \pm\ 1.96\,\widehat{\operatorname{SE}}(\hat q_p).

For another confidence level, replace 1.96 with the corresponding standard normal critical value. This interval describes Monte Carlo uncertainty in estimating the quantile under the assumed simulation distribution. It does not include model or parameter uncertainty unless those sources are separately incorporated.

Worked Example: A Capped Hypothetical Benefit

A hypothetical benefit amount $X$, in dollars, is modeled as uniform on $[0,100]$, so $X\sim U(0,100)$. A simulation uses $n=2{,}500$ independent losses and produces an empirical 80th percentile of $\hat q_{0.80}=\$80.00$. Use the model density to estimate the standard error and form an approximate 95% confidence interval.

Method. For $0<x<100$, the model density is $f_X(x)=1/100=0.01$ per dollar. Use $p=0.80$, $n=2{,}500$, and evaluate the density at the estimated quantile. The uniform model has positive density at this interior quantile, as required.

First calculate the probability-scale standard error:

\sqrt{\frac{p(1-p)}{n}} = \sqrt{\frac{(0.80)(0.20)}{2{,}500}} = \sqrt{0.000064} = 0.008.

Convert this to loss units by dividing by the density:

\widehat{\operatorname{SE}}(\hat q_{0.80}) = \frac{0.008}{0.01} = \$0.80.

The 95% margin is $1.96(\$0.80)=\$1.568$. The interval endpoints before rounding are $80.00-1.568=\$78.432$ and $80.00+1.568=\$81.568$. Rounded only at the final step to the nearest cent, the interval is $[\$78.43,\$81.57]$.

Check and trap avoided. The standard error is in dollars, as it should be. Using $p(1-p)/n$ divided directly by the density would give $0.000064/0.01=0.0064$, which is not the standard error: the square root of the probability-scale variance is required before dividing by the density.

Worked Example: A High Quantile From an Exponential Loss Model

A hypothetical simulated loss, measured in thousands of dollars, follows an exponential distribution with rate $\lambda=0.02$ per thousand dollars. In $n=6{,}400$ independent simulations, the empirical 90th percentile is $\hat q_{0.90}=115$ thousand dollars. Treat the stated exponential model as fixed and use its density at the empirical quantile to form an approximate 95% confidence interval.

Method. For an exponential distribution with rate $\lambda$, $f_X(x)=\lambda e^{-\lambda x}$ for $x\geq0$. Thus the plug-in density at 115 is

\hat f_X(115) = 0.02e^{-(0.02)(115)} = 0.02e^{-2.3} \approx 0.00200518

per thousand dollars. The probability-scale standard error is

\sqrt{\frac{(0.90)(0.10)}{6{,}400}} = \sqrt{0.0000140625} = 0.00375.

Therefore,

\widehat{\operatorname{SE}}(\hat q_{0.90}) = \frac{0.00375}{0.02e^{-2.3}} \approx 1.87016

thousand dollars. The 95% margin is $1.96(1.87016)\approx3.66551$ thousand dollars. The unrounded endpoints are approximately $115-3.66551=111.33449$ and $115+3.66551=118.66551$ thousand dollars. Rounded only at the final step to two decimal places in thousands, the interval is $[111.33,118.67]$ thousand dollars.

Check and trap avoided. The estimated density is per thousand dollars, so dividing by it gives a standard error in thousands of dollars. The calculation uses the density at the estimated quantile, not the exponential rate by itself; the rate $0.02$ and the density $0.02e^{-2.3}$ are different quantities.

Worked Example: A Beta-Distributed Simulated Severity

A hypothetical normalized severity $X$ is modeled as a beta distribution with parameters $\alpha=2$ and $\beta=1$, with density $f_X(x)=2x$ for $0<x<1$. An independent simulation of $n=400$ losses gives $\hat q_{0.75}=0.87$. Estimate the standard error using the model density at 0.87 and calculate an approximate 95% interval.

Method. The model density at the estimated quantile is $f_X(0.87)=2(0.87)=1.74$ per unit of normalized severity. The probability-scale standard error is

\sqrt{\frac{(0.75)(0.25)}{400}} = \sqrt{0.00046875} \approx 0.0216506.

Hence

\widehat{\operatorname{SE}}(\hat q_{0.75}) = \frac{0.0216506}{1.74} \approx 0.012443.

The 95% margin is $1.96(0.012443)\approx0.024388$. The interval endpoints are $0.87-0.024388=0.845612$ and $0.87+0.024388=0.894388$. Rounded to four decimal places only at the final step, the interval is $[0.8456,0.8944]$.

Check and trap avoided. The density is evaluated at the simulated estimate $0.87$, as specified, rather than at a rounded or separately calculated population quantile. The resulting standard error is on the same normalized scale as $X$.

Assumptions, Limitations, and Exam Traps

The formula is an asymptotic result, not an exact finite-sample standard error. It is most defensible when the simulation count is large enough for the normal approximation, the simulated losses are independent and identically distributed, and the density is positive and reasonably smooth near the target quantile. A density estimate that is unstable can make the reported interval unstable as well.

Atoms or discrete losses: If the distribution has a jump at the quantile, the positive-density derivation does not apply as stated. Ties can also be common, so a continuous-quantile approximation may be inappropriate.
Very small density: The standard error grows as $1/f_X(q_p)$. A smooth-looking quantile estimate does not guarantee precision if the fitted density in that region is tiny.
Dependent simulation output: The formula assumes independent draws. If outputs are dependent, the Bernoulli indicators at the quantile are also dependent, and the independent-sample variance $p(1-p)/n$ cannot automatically be used.
Density units: A density measured per dollar cannot be combined with a quantile reported in thousands of dollars without converting units consistently.
Meaning of the interval: This is a confidence interval for the simulated estimate under a fixed loss distribution. It is not, by itself, an interval for future losses or for uncertainty in estimated model parameters.

A practical report should identify the probability level, simulation count, empirical quantile, density method and value at the quantile, estimated standard error, and confidence level. Those details allow a reader to assess whether the density estimate and normal approximation are credible for the intended use.

Key takeaway: For independent simulations from a smooth continuous loss distribution, the asymptotic standard error of an empirical $p$-quantile is $\sqrt{p(1-p)/n}/f_X(q_p)$. Estimate it by substituting the empirical quantile and a density estimate at that point, while checking the assumptions and keeping loss units consistent.

Check Your Understanding

For each problem, use a normal critical value where specified and round only the final interval endpoints.

A hypothetical capped loss is uniform on $[0,200]$ dollars. With $n=10{,}000$, a simulated 75th percentile is $\$151.00$. Use the model density and a 95% confidence level to calculate the estimated standard error and interval. (Answer: standard error $\$0.8660$; interval $[\$149.30,\$152.70]$.)
A hypothetical continuous severity model has estimated density $0.004$ per dollar at the empirical 95th percentile of $\$80$. For $n=2{,}500$, calculate the standard error and a 90% interval using critical value 1.645. (Answer: standard error $\$1.0897$; interval $[\$78.21,\$81.79]$.)
A simulated continuous loss has empirical 99th percentile $500$ units, density estimate $0.002$ per unit there, and $n=10{,}000$. Calculate the estimated standard error and 95% interval. (Answer: standard error $0.4975$ units; interval $[499.02,500.98]$ units.)
A smooth hypothetical loss model has $p=0.90$, density $0.02$ per thousand dollars at the target quantile, and an estimated quantile of 40 thousand dollars. What minimum simulation count makes the approximate 95% margin of error no more than 0.25 thousand dollars? (Answer: $n\geq 13{,}830$ independent simulations.)
For independent simulations, $p=0.90$, $n=3{,}600$, estimated quantile $\$40$, and estimated density $0.015$ per dollar at that quantile. Calculate the standard error and 95% interval. (Answer: standard error $\$0.3333$; interval $[\$39.35,\$40.65]$.)

What You'll Learn