Importance Sampling Weights and Estimator Precision

Why Importance Sampling Requires Careful Weighting

A simulation under the probability model of interest can be inefficient when the outcomes that matter are rare. Importance sampling addresses this by drawing outcomes from a different distribution that visits important regions more often, then correcting for the change in sampling distribution with weights. The correction preserves the target quantity in expectation, but it can also make estimates noisy when a few draws receive very large weights.

For actuarial work, this distinction matters: observing many severe losses in a deliberately tilted simulation does not mean severe losses are equally common under the original model. The likelihood ratio adjusts their contributions. This tutorial develops weighted estimates, their standard errors, and two diagnostics for uneven weights. No interest-rate or mortality assumption is involved; each example states its probability model and parameters explicitly. The methods are aligned with the SOA Exam ASTAM (Advanced Short-Term Actuarial Mathematics) syllabus.

Definition: Let $P$ be the target distribution and $Q$ the proposal distribution used to generate independent simulation draws $X_1,\ldots,X_n$. If the target distribution is absolutely continuous with respect to the proposal distribution, define the likelihood-ratio weight $w(x)=dP/dQ(x)$. In a discrete model, $w(x)=P(X=x)/Q(X=x)$; for densities, $w(x)=f_P(x)/f_Q(x)$. The support condition requires $Q$ to assign positive probability or density wherever the target assigns positive probability or density. For a quantity $h(X)$, the ordinary importance-sampling estimator is $\hat{\theta}_{IS}=n^{-1}\sum_{i=1}^n w(X_i)h(X_i)$.

Here $h$ is the output of interest: it might be a loss, a loss indicator, or another function of the simulated outcome. For an event $A$, set $h(X)=I(X\in A)$, where the indicator is one when the event occurs and zero otherwise. The proposal distribution should make informative outcomes more likely, but the weights must use the ratio of target probability to proposal probability—not its reciprocal.

Ordinary Estimates and Their Standard Errors

Suppose the likelihood ratios are known exactly and the draws are independent from $Q$. Write $Z_i=w(X_i)h(X_i)$. The ordinary estimator is the sample mean of the $Z_i$. Its standard error is estimated by the sample standard deviation of those contributions divided by $\sqrt{n}$:

\hat{\theta}_{IS}=\bar{Z},\qquad \widehat{\operatorname{SE}}(\hat{\theta}_{IS}) =\frac{s_Z}{\sqrt{n}},\qquad s_Z^2=\frac{1}{n-1}\sum_{i=1}^n(Z_i-\bar{Z})^2.

The calculation uses the variation in the weighted contributions, not the variation in the raw values $h(X_i)$ alone. It requires independent proposal draws and a finite variance for $w(X)h(X)$ under $Q$. When that variance is infinite, the usual sample-standard-deviation formula may not provide a reliable approximation to Monte Carlo error.

Why the ordinary estimator works: By the definition of the likelihood ratio, $E_Q[w(X)h(X)]=E_P[h(X)]$, provided the expectation exists. Thus each weighted contribution has the target mean, and the average of independent contributions is unbiased for that mean. Its variance is $\operatorname{Var}_Q(w(X)h(X))/n$, which explains why the sample variance of the weighted contributions estimates the squared standard error.

Worked Example: A Weighted Estimate of a Discrete Expected Loss

A hypothetical loss outcome $X$ takes values $0,1,2$, measured in thousands of dollars. Under the target model, $P(X=0)=0.60$, $P(X=1)=0.30$, and $P(X=2)=0.10$. To simulate higher losses more often, a proposal model uses $Q(X=0)=0.20$, $Q(X=1)=0.40$, and $Q(X=2)=0.40$. Eight independent proposal draws produce two zeros, four ones, and two twos. Estimate the target expected loss and its standard error. There is no interest or mortality assumption.

Method and weights. For $h(X)=X$, the likelihood-ratio weights are $w(0)=0.60/0.20=3$, $w(1)=0.30/0.40=0.75$, and $w(2)=0.10/0.40=0.25$. The weighted contributions $Z=w(X)X$ are therefore $0$, $0.75$, and $0.50$, respectively.

Estimate. The eight contributions are two zeros, four $0.75$'s, and two $0.50$'s. Their sum is $2(0)+4(0.75)+2(0.50)=4$. Thus $\hat{\theta}_{IS}=4/8=0.5$ thousand dollars.

Standard error. The contribution mean is $0.5$. The sum of squared deviations from this mean is $2(0-0.5)^2+4(0.75-0.5)^2+2(0.50-0.5)^2 =0.50+0.25+0=0.75$. So $s_Z^2=0.75/7=0.1071429$, and $\widehat{\operatorname{SE}}=\sqrt{0.1071429/8}=0.1157$ thousand dollars, or about $\$115.73$.

Check and trap avoided. Directly under the target model, $E_P[X]=0(0.60)+1(0.30)+2(0.10)=0.50$ thousand dollars, consistent with the estimate. The estimate need not equal the target expectation in a finite simulation; its equality here is a coincidence. Using the unweighted proposal mean, $(2(0)+4(1)+2(2))/8=1$, would estimate the proposal mean instead. The method is ordinary importance sampling, and the correction avoids treating proposal draws as if they came from the target model.

Rare-Event Estimates: Weight the Event Indicator

For a target event probability, the target quantity is $P(A)=E_P[I(X\in A)]$. An ordinary importance-sampling estimate averages the weighted indicators. A non-event contributes zero, but an event contributes its likelihood ratio; it is not enough simply to count events and divide by the number of draws.

Worked Example: Estimating a Rare Probability Under a Tilted Model

In a hypothetical binary loss model, the target event $A$ has probability $P(A)=0.01$, while the proposal assigns $Q(A)=0.10$. Ten independent proposal draws contain three events and seven non-events. Estimate $P(A)$, its standard error, and the effective sample size of the likelihood-ratio weights. There is no interest or mortality assumption.

Method and event estimate. The event weight is $0.01/0.10=0.10$. The non-event weight is $0.99/0.90=1.10$. For the event probability, the contributions $wI(A)$ are three $0.10$'s and seven zeros. Therefore, $\hat{P}_{IS}(A)=3(0.10)/10=0.0300$.

Standard error. The contribution mean is $0.03$. The sum of squared deviations is $3(0.10-0.03)^2+7(0-0.03)^2 =3(0.0049)+7(0.0009)=0.021$. Thus $s_Z^2=0.021/9=0.0023333$, and $\widehat{\operatorname{SE}}=\sqrt{0.0023333/10}=0.0153$. The estimated standard error is large relative to the estimate, so the simulation is not very precise despite having observed three events.

Check and trap avoided. The direct target probability is $0.0100$; a finite-sample estimate of $0.0300$ is possible, not a reason to omit the weights. The ordinary proposal event frequency is $3/10=0.3000$, which is not the target estimate. Also, the weighted estimate is $3(0.10)/10=0.0300$, not $0.0100$: multiplying the event weight by the three observed events and dividing by all ten draws is essential. The method is the ordinary weighted-indicator estimator.

Self-Normalized Estimates

Sometimes the target density is known only up to a multiplicative constant. Then the absolute likelihood ratios are unavailable, but ratios of weights can still be used. With unnormalized weights $r_i$ proportional to the target-to-proposal density ratio, define normalized weights $\tilde{w}_i=r_i/\sum_j r_j$. The self-normalized estimate is a weighted average. Unlike the ordinary estimator, it is generally not exactly unbiased in a finite sample, but under standard support and integrability conditions it is consistent as the number of independent draws grows.

\hat{\theta}_{SN}= \frac{\sum_{i=1}^n r_i h(X_i)}{\sum_{i=1}^n r_i} =\sum_{i=1}^n \tilde{w}_i h(X_i), \qquad \widehat{\operatorname{SE}}(\hat{\theta}_{SN}) = \sqrt{\frac{n}{n-1} \sum_{i=1}^n \tilde{w}_i^2 \bigl(h(X_i)-\hat{\theta}_{SN}\bigr)^2}.

The standard-error formula is a large-sample ratio-estimator approximation. To see its structure, let $\theta=E_P[h(X)]$ and suppose the unnormalized weight is proportional to the likelihood ratio. The numerator and denominator of the estimate are sample means. Linearizing their ratio around their expectations gives contributions proportional to $r_i(h(X_i)-\theta)$. Replacing $\theta$ by $\hat{\theta}_{SN}$, estimating the variance of these contributions from the sample, and dividing by the squared sample mean weight yields the displayed expression. It is an approximation, especially with small samples or highly concentrated weights; it is not an exact finite-sample confidence guarantee.

Worked Example: A Self-Normalized Estimate and Its Error

A hypothetical simulation has four independent proposal draws. The target density is available only up to a constant, and the corresponding unnormalized weights are $1,2,3,4$. The observed output values $h(X_i)$ are $2,4,1,5$. Estimate the target mean using normalized weights, calculate the ratio-based standard error, and find the effective sample size. There is no interest or mortality assumption.

Method and estimate. The total weight is $1+2+3+4=10$, so the normalized weights are $0.1,0.2,0.3,0.4$. The weighted output sum is $1(2)+2(4)+3(1)+4(5)=33$. Hence $\hat{\theta}_{SN}=33/10=3.3$.

Standard error. The output deviations from $3.3$ are $-1.3,0.7,-2.3,1.7$. The terms $\tilde{w}_i^2(h(X_i)-3.3)^2$ are $0.01(1.69)=0.0169$, $0.04(0.49)=0.0196$, $0.09(5.29)=0.4761$, and $0.16(2.89)=0.4624$. Their sum is $0.9750$. Therefore the estimated standard error is $\sqrt{(4/3)(0.9750)}=\sqrt{1.3}=1.1402$.

Check and trap avoided. The normalized weights sum to $1$, and the estimate equals $33/10$, as calculated from the unnormalized weights. The ratio-based standard error is approximate, not a claim of exact finite-sample coverage. The method is self-normalized importance sampling. A common error is to use $n^{-1}\sum r_i h(X_i)$ when only proportional weights are known; that expression depends on the arbitrary scaling of $r_i$ and is not the self-normalized estimate.

Weight Variability and Effective Sample Size

An estimate can be dominated by a small number of draws with large weights. Effective sample size (ESS) summarizes this weight concentration by comparing the observed normalized weights with equal weights. It does not replace a standard error: it ignores the particular output values $h(X_i)$, and therefore cannot by itself determine the precision of a specific estimate.

Definition: For nonnegative weights $w_1,\ldots,w_n$, the weight-based effective sample size is $\operatorname{ESS}=(\sum_i w_i)^2/\sum_i w_i^2=1/\sum_i\tilde{w}_i^2$. It lies between $1$ and $n$ when the weights are not all zero. It equals $n$ when all weights are equal and approaches $1$ when one draw carries nearly all the weight. Multiplying every weight by the same positive constant leaves ESS unchanged.

The ESS is best read as a warning signal: an ESS far below the number of draws indicates that the weighted estimate relies heavily on a limited subset of them. A large ESS does not guarantee a small standard error, because $h(X_i)$ itself may vary greatly or may be associated with the weights. For ordinary importance sampling, calculate the standard error from $w_i h(X_i)$; for a self-normalized estimate, use the ratio-based formula. Always inspect those output-specific contributions as well as the weights.

Worked Example: Diagnosing Uneven Weights

In a hypothetical proposal simulation, five draws have weights $1,1,1,1,6$. Calculate the effective sample size and describe what it says about the simulation. The weights are proportional target-to-proposal ratios; no interest or mortality assumption is involved.

Method. The sum of weights is $1+1+1+1+6=10$. The sum of squared weights is $1+1+1+1+36=40$. Thus $\operatorname{ESS}=10^2/40=2.5$.

Check and trap avoided. Five draws were generated, but the weight concentration corresponds to an ESS of only $2.5$ by this diagnostic; the draw with weight $6$ contributes $0.6$ of the normalized weight. ESS need not be an integer. The method diagnoses weight concentration, not the standard error of every possible output. Reporting “five effective observations” merely because five proposals were drawn ignores the unequal weights.

A Practical Calculation Sequence

Specify target and proposal.
Write down their probabilities or densities and verify that the proposal covers the target support.

Compute the weights.
Use target divided by proposal. For ordinary estimates the likelihood ratios must be correctly normalized; for self-normalized estimates a common proportionality constant cancels.

Form the right contributions.
For an ordinary estimate use $w_i h(X_i)$. For a self-normalized estimate divide the weighted output sum by the sum of weights.

Quantify and diagnose uncertainty.
Calculate the standard error from the appropriate weighted contributions, then inspect the weight-based ESS and the output-specific contribution variability.

Common Exam Traps

Reversing the likelihood ratio. The weight is target probability or density divided by proposal probability or density. Reversing it generally estimates a different quantity.
Forgetting the zero contributions. For an event probability, non-events contribute zero, but the estimate still divides the total weighted indicator by the full number of proposal draws.
Using the raw sample standard deviation. The ordinary estimator's standard error comes from the weighted contributions, not from $h(X_i)$ alone.
Using unnormalized weights as though they were exact likelihood ratios. An arbitrary proportionality constant is acceptable for the self-normalized ratio, but not for the ordinary estimator.
Treating ESS as a standard error or a literal count. ESS is a weight-concentration diagnostic. Precision remains specific to the output and must be assessed from its weighted contributions.

Key takeaway: Importance sampling changes where outcomes are generated, not the target quantity. The likelihood ratio corrects each contribution; the standard error must reflect the variability of those corrected contributions. Use ESS to flag weight concentration, then judge precision using the estimator's own standard error.

Check Your Understanding

For each problem, use the stated target and proposal models and show the weighting step before calculating.

A hypothetical binary event has target probability $0.02$ and proposal probability $0.10$. In 20 independent proposal draws, the event occurs twice. Find the ordinary importance-sampling estimate of its target probability. (Answer: $0.0200$)
A discrete target model has probabilities $P(X=0)=0.50$, $P(X=1)=0.30$, and $P(X=2)=0.20$; the proposal probabilities are $0.25,0.50,0.25$, respectively. Five proposal draws are $0,1,1,2,0$. Estimate $P(X\geq 1)$ by ordinary importance sampling. (Answer: $0.4000$)
Three draws have unnormalized weights $1,1,4$, and output values $0,2,3$. Calculate the self-normalized estimate of the target mean. (Answer: $2.3333$)
Four draws have weights $1,2,2,5$. Calculate their weight-based effective sample size to four decimal places. (Answer: $2.9412$)
The ordinary importance-sampling contributions for four independent draws are $0,0,1,1$. Calculate the estimate and its estimated standard error using the sample standard deviation of the contributions. (Answer: estimate $0.5000$; standard error $0.2887$)

What You'll Learn