Delta-Method Error Propagation for Simulated Risk Measures

Why Derived Risk Measures Have Simulation Error

A simulation study may estimate several quantities from the same simulated losses and then combine them into a single decision measure. For example, a risk-adjusted amount might combine an estimated mean and standard deviation, or a tail measure might be expressed as a ratio of two simulated averages. The uncertainty of the final quantity depends not only on the uncertainty of each input but also on how the inputs vary together.

The multivariate delta method provides a first-order approximation to that uncertainty. It is useful when the derived quantity is a smooth function of estimated statistics and those statistics have a joint covariance matrix. The method carries the full covariance matrix through the function, rather than treating its inputs as independent by default.

Definition: Let $\hat{\boldsymbol{\theta}}$ be a vector of simulated estimators for a parameter vector $\boldsymbol{\theta}$, with covariance matrix $\boldsymbol{\Sigma}$. For a scalar function $g$ that is differentiable near $\boldsymbol{\theta}$, the delta method approximates the variance of $g(\hat{\boldsymbol{\theta}})$ by $\nabla g(\boldsymbol{\theta})^\mathsf{T}\boldsymbol{\Sigma}\nabla g(\boldsymbol{\theta})$. In practice, unknown quantities are replaced by their estimates, giving a plug-in estimate of the variance.

The method is an approximation, not an exact finite-sample variance formula. It is generally justified when the estimator vector is approximately jointly normal, as can follow from a central limit theorem, and when $g$ is smooth in the region where the estimates lie. The relevant variances must be finite, and the gradient should not be unstable. A ratio, for instance, can be poorly approximated if its estimated denominator is near zero.

The Covariance Propagation Formula

Suppose $\hat{\boldsymbol{\theta}}=(\hat{\theta}_1,\ldots,\hat{\theta}_k)^\mathsf{T}$, and write $\boldsymbol{\Sigma}$ for its covariance matrix. The diagonal entries are the variances of the estimated inputs; the off-diagonal entries are their covariances. Define the gradient vector using the partial derivatives of $g$. The estimated simulation variance and standard error of the derived measure are then

\widehat{\operatorname{Var}}\{g(\hat{\boldsymbol{\theta}})\} \approx \nabla g(\hat{\boldsymbol{\theta}})^\mathsf{T} \widehat{\boldsymbol{\Sigma}} \nabla g(\hat{\boldsymbol{\theta}}), \qquad \widehat{\operatorname{SE}}\{g(\hat{\boldsymbol{\theta}})\} = \sqrt{\widehat{\operatorname{Var}}\{g(\hat{\boldsymbol{\theta}})\}}.

If $\hat{\theta}_1$ and $\hat{\theta}_2$ are the two inputs and $g$ is scalar, the calculation expands to

\widehat{\operatorname{Var}}\{g(\hat{\theta}_1,\hat{\theta}_2)\} \approx g_1^2\widehat{\operatorname{Var}}(\hat{\theta}_1) +g_2^2\widehat{\operatorname{Var}}(\hat{\theta}_2) +2g_1g_2\widehat{\operatorname{Cov}}(\hat{\theta}_1,\hat{\theta}_2),

where $g_j$ is the partial derivative with respect to input $j$, evaluated at the estimated inputs. The covariance term can increase or decrease the final variance, depending on the signs of the derivatives and the covariance. Omitting it is justified only when the inputs are uncorrelated or when the product $g_1g_2$ is zero.

Why the Formula Holds

The justification is a first-order Taylor expansion around the true input vector. Locally, a small change in the inputs changes the function by approximately the gradient dotted with that change:

g(\hat{\boldsymbol{\theta}}) \approx g(\boldsymbol{\theta})+ \nabla g(\boldsymbol{\theta})^\mathsf{T} (\hat{\boldsymbol{\theta}}-\boldsymbol{\theta}).

The constant $g(\boldsymbol{\theta})$ contributes no variance. For any fixed vector $\boldsymbol{a}$, the variance of the linear combination $\boldsymbol{a}^\mathsf{T}\hat{\boldsymbol{\theta}}$ is $\boldsymbol{a}^\mathsf{T}\boldsymbol{\Sigma}\boldsymbol{a}$. Taking $\boldsymbol{a}=\nabla g(\boldsymbol{\theta})$ gives the delta-method expression. Replacing the unknown gradient and covariance matrix by their estimated counterparts gives the practical plug-in calculation.

This argument also shows the limitation: curvature in $g$ is ignored at first order. The approximation can be weak for small simulation samples, highly skewed estimates, or functions that change rapidly near the estimate. A narrow standard error is not evidence that the approximation is reliable if those conditions are problematic.

Worked Example: A Risk-Adjusted Loss Estimate

A hypothetical insurer simulates annual claim losses, in dollars. No interest or mortality assumption is involved, and no parametric loss distribution is imposed. The same simulation replications produce an estimated mean loss $\hat{\mu}=\$100$ and estimated standard deviation $\hat{s}=\$20$. The estimated covariance matrix of these two simulated statistics is

\widehat{\boldsymbol{\Sigma}}= \begin{pmatrix} 0.04 & 0.06\\ 0.06 & 0.25 \end{pmatrix},

where the first coordinate is $\hat{\mu}$ and the second is $\hat{s}$. Calculate the estimate and simulation standard error of the risk-adjusted amount $R=\mu+0.5s$.

Method. Apply the multivariate delta method to $g(\mu,s)=\mu+0.5s$. The gradient is $\nabla g=(1,0.5)^\mathsf{T}$.

Point estimate. Substitute the simulated estimates:

\hat{R}=100+0.5(20)=110.

Variance estimate. Use both variances and the covariance:

\widehat{\operatorname{Var}}(\hat{R}) =1^2(0.04)+(0.5)^2(0.25) +2(1)(0.5)(0.06) =0.04+0.0625+0.06=0.1625.

Therefore, the simulation standard error is $\sqrt{0.1625}=0.4031$ dollars, rounded to four decimal places. The estimated risk-adjusted amount is $\$110.00$, with an approximate simulation standard error of $\$0.40$. The method avoids treating the mean and standard deviation estimates as independent merely because they are different statistics.

Ratios of Simulated Tail Quantities

A common derived measure is a ratio of two sample averages. For a fixed threshold $c$, define $A=I(X>c)$ and $B=(X-c)I(X>c)$. If $P(X>c)>0$, then

E[X\mid X>c] = c+\frac{E[B]}{E[A]}.

The identity follows because $B=(X-c)I(X>c)$, so $E[B]=P(X>c)E[X-c\mid X>c]$, while $E[A]=P(X>c)$. In simulation, estimate the two expectations by sample averages and use the delta method for their ratio. The threshold is fixed here; this calculation does not itself account for uncertainty from estimating a threshold.

Worked Example: Conditional Mean Above a Fixed Threshold

Consider a hypothetical annual loss $X$ with the discrete distribution $P(X=0)=0.50$, $P(X=12)=0.25$, and $P(X=16)=0.25$, with amounts in thousands of dollars. There is no interest or mortality assumption. In 1,000 independent simulations, suppose the estimated averages are $\bar{A}=0.50$ and $\bar{B}=2.00$, for threshold $c=10$, where $A=I(X>10)$ and $B=(X-10)I(X>10)$. Estimate the conditional mean above 10 and its simulation standard error.

Method. Estimate $E[X\mid X>10]$ by $10+\bar{B}/\bar{A}$. First calculate the covariance matrix of the sample averages. Under the stated distribution, $A$ equals 1 for losses 12 and 16, so $E[A]=0.50$ and $\operatorname{Var}(A)=0.50(0.50)=0.25$. The variable $B$ takes values 0, 2, and 6 with probabilities 0.50, 0.25, and 0.25. Thus $E[B]=2$, $E[B^2]=0.25(4)+0.25(36)=10$, and $\operatorname{Var}(B)=10-2^2=6$. Also $E[AB]=E[B]=2$, so $\operatorname{Cov}(A,B)=2-(0.50)(2)=1$.

For 1,000 independent simulations, the covariance matrix of the two sample averages is the single-simulation covariance matrix divided by 1,000:

\widehat{\boldsymbol{\Sigma}}= \begin{pmatrix} 0.25/1000 & 1/1000\\ 1/1000 & 6/1000 \end{pmatrix} = \begin{pmatrix} 0.00025 & 0.001\\ 0.001 & 0.006 \end{pmatrix}.

Point estimate. With $g(b,a)=10+b/a$, the estimate is $10+2/0.50=14$ thousand dollars. The derivatives with respect to $b$ and $a$, evaluated at $b=2$ and $a=0.50$, are $g_b=1/a=2$ and $g_a=-b/a^2=-8$.

Variance estimate. Applying the covariance formula gives

$$ (2)^2(0.006)+(-8)^2(0.00025) +2(2)(-8)(0.001) =0.024+0.016-0.032=0.008. $$

The standard error is $\sqrt{0.008}=0.0894$ thousand dollars, rounded to four decimal places. The estimated conditional mean is $\$14{,}000$, with a simulation standard error of about $\$89.44$. The negative covariance contribution matters: omitting it would overstate the variance as $0.040$, rather than $0.008$, in thousands of dollars squared.

Ratios and Confidence Intervals for Risk Estimates

The same method applies when simulation methods have already supplied estimates and standard errors for two risk measures. For a ratio, both the denominator and the covariance between numerator and denominator matter. If an approximate normal interval is appropriate, a two-sided 95% interval is the estimate plus or minus $1.96$ times its estimated standard error. This interval reflects simulation error under the assumed model; it does not automatically include uncertainty in the model or its parameters.

Worked Example: A Tail-to-Quantile Risk Ratio

A hypothetical portfolio simulation gives a 99% value-at-risk estimate $\widehat{V}=\$120$ thousand and a 99% tail value-at-risk estimate $\widehat{T}=\$160$ thousand. Their simulation standard errors are $\$2$ thousand and $\$3$ thousand, respectively, and their estimated correlation, based on the same replications, is 0.60. No interest or mortality assumption is involved; the estimates come from an empirical simulation distribution. Estimate $Q=T/V$, its standard error, and an approximate 95% interval.

Method. The estimated covariance is $0.60(2)(3)=3.6$ in squared-thousand-dollar units. Thus the covariance matrix is

\widehat{\boldsymbol{\Sigma}}= \begin{pmatrix} 4 & 3.6\\ 3.6 & 9 \end{pmatrix},

with coordinates $V,T$. The point estimate is $\hat{Q}=160/120=1.3333$, rounded to four decimal places. For $g(V,T)=T/V$, the gradient at the estimates is

\nabla g= \begin{pmatrix} -T/V^2\\ 1/V \end{pmatrix} = \begin{pmatrix} -160/120^2\\ 1/120 \end{pmatrix} = \begin{pmatrix} -0.0111111\\ 0.0083333 \end{pmatrix}.

Variance and standard error. Using the two variance terms and the covariance term,

\widehat{\operatorname{Var}}(\hat{Q}) \approx (-0.0111111)^2(4)+(0.0083333)^2(9) +2(-0.0111111)(0.0083333)(3.6) \approx 0.0004522.

Therefore, $\widehat{\operatorname{SE}}(\hat{Q})=\sqrt{0.0004522}\approx0.0213$, rounded to four decimal places. The approximate 95% interval is $1.3333\pm1.96(0.0213)$, or approximately $(1.2916,1.3750)$, rounded to four decimal places. This is an interval for the dimensionless ratio, not for either dollar-valued risk estimate alone.

Common Exam Trap: Dropping Dependence

A frequent error is to combine estimated standard errors as if the inputs were independent, even though they were computed from the same simulated losses. In the tail-to-quantile example, dropping the covariance term would give

(-0.0111111)^2(4)+(0.0083333)^2(9) \approx 0.001119.

That would imply a standard error of about $0.0335$, rather than $0.0213$. The difference arises because the covariance is positive while the two derivatives have opposite signs, making the covariance contribution negative. The correction is to use the joint covariance of the input estimates, not just their separate standard errors. Conversely, a covariance term can increase the propagated variance if its sign and the gradient signs make the product positive.

Key takeaway: Identify the estimated input vector, write the function explicitly, calculate its gradient, and propagate the full covariance matrix. Check that the function is smooth near the estimates and that any ratio denominator is safely away from zero before relying on the approximation.

Check Your Understanding

For each problem, use the plug-in delta method and include covariance terms where provided.

A hypothetical risk score is $R=3\mu+2s$, with $\hat{\mu}=50$, $\hat{s}=8$, $\operatorname{Var}(\hat{\mu})=0.25$, $\operatorname{Var}(\hat{s})=0.16$, and $\operatorname{Cov}(\hat{\mu},\hat{s})=0.06$. Find the estimate and standard error. (Answer: $\hat{R}=166$, standard error $1.90$.)
For a fixed loss threshold, simulated estimates of $e=E[(X-c)I(X>c)]$ and $p=P(X>c)$ are $0.90$ and $0.30$. Their variances are $0.0009$ and $0.0001$, and their covariance is $0.00012$. For the derived measure $e/p$, find the estimate and standard error. (Answer: $3.000$, standard error approximately $0.1095$.)
Simulated estimates $A=7$ and $B=11$ have variances 1 and 4 and covariance $-0.5$. For $R=2A+B$, find the estimate and its standard error. (Answer: $25$, standard error $\sqrt{6}\approx2.4495$.)
Estimated tail value-at-risk $T=100$ and value-at-risk $V=80$ have standard errors 2.5 and 1.5, respectively, and correlation 0.40. For $Q=T/V$, calculate its estimate and delta-method standard error. (Answer: $1.2500$, standard error approximately $0.0307$.)

What You'll Learn