Planning Simulations for Rare Events

Why Rare Events Need Deliberate Simulation Planning

A simulation can generate many observations and still provide little information about a rare event. If the event probability is \(0.0002\), for example, 10,000 independent runs produce only two events on average. A simulation that observes zero or one event may be consistent with a range of probabilities, so the number of runs should be planned around the precision required—not chosen merely because it seems large.

This tutorial focuses on planning a fixed number of independent runs for a Bernoulli event. It uses expected event counts, relative error targets, and an upper confidence limit when no event is observed. The techniques are aligned with the SOA Exam ASTAM (Advanced Short-Term Actuarial Mathematics) syllabus.

The assumptions matter: the runs must be independent, and each must represent the same event probability \(p\). If simulation outcomes are dependent or probabilities differ across runs, the binomial calculations below do not directly apply. The event probability is a model quantity; the simulation error discussed here is the variability in \(\hat p\) across repeated sets of runs.

Expected Counts and the Chance of Seeing No Events

The expected number of observed events is \(E[K]=np\). This is a useful first planning check, but it is not itself a precision guarantee. The probability of observing no events is

For small \(p\), \((1-p)^n\) is approximately \(e^{-np}\). This approximation shows why expected counts are informative: when \(np\) is small, zero events are quite plausible; when \(np\) is larger, a zero-event outcome becomes less likely. For instance, about three expected events give roughly a 95% chance of at least one event, since \(e^{-3}\) is about \(0.05\). That is enough to make an event visible, but generally not enough for a stable probability estimate.

Relative Precision for a Simulated Probability

For the Bernoulli estimator, \(E[\hat p]=p\) and \(\operatorname{Var}(\hat p)=p(1-p)/n\). Its standard error is therefore \(\sqrt{p(1-p)/n}\). Dividing this standard error by \(p\) gives the relative standard error, which measures uncertainty as a proportion of the probability being estimated.

If a target relative standard error is \(r\), planning with a specified value of \(p\) requires

For small \(p\), \(1-p\) is close to 1, so this is often approximated by \(n\geq 1/(p r^2)\). The expected event count at this approximate run size is about \(1/r^2\): a 10% relative standard error corresponds to roughly 100 expected events, while a 5% target corresponds to roughly 400.

Relative Confidence-Interval Width

A relative standard error target is not the same as a confidence-interval target. For a sufficiently well-behaved estimate, an approximate two-sided 95% normal interval is \(\hat p\pm1.96\operatorname{SE}(\hat p)\). Its approximate half-width relative to \(p\) is \(1.96\operatorname{SE}(\hat p)/p\). To plan for a relative half-width no greater than \(h\), use

This is a planning approximation, not a promise that every simulated interval will achieve the target. It uses a planning value for \(p\) and a normal approximation; that approximation can be poor when very few events are expected or observed. For small event counts, do not interpret a normal interval as reliable simply because the planned run count is large. The zero-event case requires particular care, as discussed below.

Zero Observed Events: Report an Upper Limit

If \(K=0\), then \(\hat p=0\), but this does not show that the underlying probability is zero. A relative standard error calculated by dividing by \(\hat p\) is undefined, and a relative error target cannot be assessed from that estimate. Instead, report an upper confidence limit that reflects how large \(p\) could plausibly be given the zero-event outcome.

For a one-sided confidence level \(1-\alpha\), the exact binomial upper limit \(p_U\) after zero events is found by setting the probability of zero events equal to \(\alpha\):

This is exact for independent Bernoulli runs with a fixed probability \(p\). For small \(p_U\), use \(\ln(1-p_U)\approx-p_U\) to obtain the useful approximation

At 95% one-sided confidence, \(-\ln(0.05)\approx2.996\), giving the familiar rule of three, \(p_U\approx3/n\). The limit is an upper bound, not a point estimate. Its confidence interpretation is a repeated-sampling statement: this procedure covers the fixed probability at least \(1-\alpha\) of the time under the model assumptions.

Putting the Planning Decisions Together

A useful planning sequence is to state the event and its probability scale, choose an initial planning value, specify the required precision or probability of observing events, and then check the expected count and the zero-event risk. Each calculation answers a different question:

The main exam trap is to treat the expected event count as though it were a guarantee. The count \(np\) is an average over repeated simulations; it does not specify the count in a particular simulation. A related trap is to apply a normal relative-error calculation after observing zero events. Relative error around an estimate of zero is not meaningful, so use an upper confidence limit instead. These distinctions are especially important when a simulation budget is limited and an apparently large \(n\) may still yield only a few events.