Why Rare Events Are Hard to Estimate by Simulation

Why Rare Events Challenge Simulation

A simulation can be useful for estimating a probability only if it produces enough information about the event of interest. When that event is rare, an ordinary-sized run may contain just a handful of occurrences—or none. In that situation, the simulated proportion can change substantially with only one or two additional events, so a numerical estimate may look more precise than it really is.

The key quantity is not just the total number of simulation runs. It is also the expected number of times the event occurs. A million runs sounds large, but if an event has probability one in ten million, that run is still expected to contain only one tenth of an occurrence. The chance of seeing no occurrences would be high.

These calculations assume each run is generated independently, with the same event probability \(p\), and that the event is defined and recorded consistently. If those assumptions fail, the formulas below need not apply. Also, simulation precision concerns random variation among runs; it does not by itself establish that the probability model represents reality well.

Expected Counts and the Chance of Seeing Nothing

To justify the expected count, define \(I_i\) to equal 1 when the event occurs on run \(i\), and 0 otherwise. The total event count \(K\) is the sum \(I_1+\cdots+I_n\). Each indicator has expected value \(p\), so by additivity of expectation, \(E(K)=np\). This is an average over repeated batches of \(n\) runs, not a promise about the count in any particular batch.

If the runs are independent, the chance that the event does not occur on any one run is \(1-p\). For no occurrences in all \(n\) runs, it must be absent on every run. Independence therefore gives:

This formula is useful for planning. If \((1-p)^n\) is large, seeing zero events is unsurprising even when \(p\) is positive. Conversely, an expected count of one does not mean exactly one event will appear: the count varies from run to run of the simulation.

How Much Monte Carlo Error Should We Expect?

An event estimate is a proportion: if \(K\) of the \(n\) runs contain the event, then \(\hat{p}=K/n\). For independent runs with event probability \(p\), the standard error of this proportion is:

This standard error describes the typical random variation of the simulated proportion across repeated runs of size \(n\), when \(p\) is the true model probability. Because \(p\) is usually unknown, a reported estimate commonly uses \(\hat{p}\) in its place:

For a rare event, the relative error matters especially. An absolute error of 0.0002 is small on a probability scale from 0 to 1, but it is 40% of a probability of 0.0005. Dividing the standard error by \(p\) gives:

When \(p\) is very small, \(1-p\) is close to 1. The relative standard error is then approximately \(1/\sqrt{np}\). This shows why the expected event count \(np\), not just \(n\), is a useful guide: a small expected count implies a large relative error.

Zero Occurrences Do Not Mean Zero Probability

If a simulation produces no events, the observed proportion is \(\hat{p}=0\). That is the correct description of the simulated runs, but it does not show that the model probability \(p\) is zero. In particular, substituting \(\hat{p}=0\) into the estimated standard-error formula produces zero. Treating that result as proof of perfect precision is a serious mistake: the plug-in calculation cannot reveal uncertainty about an event that was never observed.

One way to judge a zero count is to ask what event probability would make zero occurrences fairly unlikely. For example, an upper probability \(p_U\) that gives a 5% chance of zero events solves:

Solving this equation gives \(p_U=1-0.05^{1/n}\). For a large \(n\), this is approximately \(3/n\), because \(-\ln(0.05)\approx 2.996\). This is the basis of the rough “rule of three”: after zero occurrences in \(n\) independent runs, probabilities around \(3/n\) or larger would produce zero occurrences only about 5% of the time or less. The rule is an uncertainty diagnostic, not an estimate that the event occurred at that rate.

Planning a Run Size for Relative Precision

A simulation plan should start with a precision goal, not just an appealingly large run count. If the anticipated event probability is \(p\), and the desired relative standard error is at most \(r\), use:

This calculation requires a planning value for \(p\), perhaps from a trusted exact calculation or a preliminary run. If that value is very uncertain, the chosen run size is uncertain too. A preliminary simulation with zero or only a few events does not provide a dependable estimate of \(p\) for this purpose.

When Is a Rare-Event Estimate Too Imprecise to Trust?

A report should be viewed cautiously when the expected or observed event count is small relative to the precision being claimed. For example, an estimated probability based on three events should not be presented with many meaningful digits. A zero count is also a warning sign: it supports reporting that no events appeared in the run, but not that the probability is exactly zero.

Before relying on a simulated rare-event probability, check that the report makes clear:

• the number of independent simulation runs and the number of observed events;
• the estimated proportion and an appropriate measure of Monte Carlo uncertainty;
• whether the uncertainty is large relative to the estimated probability; and
• that the event and simulation model are held consistent across runs.

If the estimated relative error is too large, increasing the number of runs can help, provided the assumptions remain appropriate. A large run count alone is not persuasive if it still yields very few events. Also keep two questions separate: Monte Carlo precision asks how much the answer varies because of random simulation, while model adequacy asks whether the simulation represents the probability question of interest.

Common Mistake