Sample Size for Absolute Precision

Why Plan the Simulation Count?

A simulation estimate is subject to Monte Carlo error: repeating the simulation with new random draws would generally produce a different estimate. Before running a costly simulation, an analyst can choose a target absolute precision, stated as a confidence-interval half-width $h$, and calculate how many independent replications are needed to reach it. Absolute precision means that the tolerated error is expressed in the same units as the quantity being estimated: for example, dollars for a mean cost or percentage points for a probability.

The calculation depends on the output being estimated and its variability. For a mean, the relevant quantity is the output variance. For an event probability, each replication produces an indicator with variance $p(1-p)$. A pilot run can estimate these quantities, while known bounds can provide more conservative planning values. The resulting count is a planning estimate under stated assumptions, not a guarantee that every realized interval will have the target width.

Definition: For an estimator $\hat{\theta}$, an absolute confidence-interval half-width $h$ is the distance from the estimate to either endpoint of a symmetric confidence interval, so the full interval width is $2h$. A target such as $h=50$ dollars is not a target full width of 50 dollars.

Sample Size for a Simulated Mean

Suppose $X_1,\ldots,X_n$ are independent, identically distributed simulation outputs with mean $\mu$ and finite variance $\sigma^2$. The Monte Carlo estimator is $\bar{X}$, with standard error $\sigma/\sqrt{n}$. For a two-sided confidence level $1-\alpha$, a normal-approximation interval has half-width approximately $z_{1-\alpha/2}\sigma/\sqrt{n}$, where $z_{1-\alpha/2}$ is the corresponding standard normal quantile.

h \approx z_{1-\alpha/2}\frac{\sigma}{\sqrt{n}} \quad\Longrightarrow\quad n \approx \left(\frac{z_{1-\alpha/2}\sigma}{h}\right)^2.

In practice, $\sigma$ is often unknown. A pilot run of $m$ independent replications gives a sample standard deviation $s$, which can be substituted as a planning estimate. Then round the calculated count up to the next integer. The formula uses $n$ as the total number of replications in the final estimate. If the pilot observations are retained and meet the same simulation design, the additional number required is $n-m$, or zero if $m$ already exceeds the planned total.

Planning rule: With a pilot estimate $s$ of the output standard deviation, plan approximately $$ n=\left\lceil\left(\frac{z_{1-\alpha/2}s}{h}\right)^2\right\rceil. $$ This normal-approximation rule presumes independent, identically distributed outputs and a sample size for which the normal approximation is suitable. It uses $s$ as an estimate of $\sigma$; uncertainty in the pilot estimate means the achieved half-width is not assured.

Worked Example: Mean Simulated Claim Cost

A hypothetical insurer simulates the annual covered claim cost for one policy under a fixed model. A pilot of 400 independent policy-years gives a sample standard deviation of $1,200. How many total policy-year simulations are needed for an approximate 95% confidence interval for the mean cost to have half-width no greater than $50? State any additional simulations if the pilot is retained.

Method. Use the normal-approximation planning rule with $z_{0.975}=1.96$, $s=1200$, and $h=50$. The pilot estimate is used as the planning value for the unknown standard deviation.

Substitute the values: $$ n \approx \left(\frac{1.96(1200)}{50}\right)^2 =\left(\frac{2352}{50}\right)^2 =(47.04)^2 =2212.7616. $$

Rounding up gives a planned total of $n=2213$ simulations. Since 400 pilot simulations are already available and are retained, the additional number is $2213-400=1813$.

Check. At the planned total, the estimated half-width is $1.96(1200)/\sqrt{2213}$, which is approximately $49.997, below the $50 target. With one fewer replication, the planning expression is just above the target because the unrounded requirement was 2212.7616.

Common mistake avoided: Treating $50 as the desired full interval width would use $h=25$ and overstate the required count by a factor of four. Here $50 is explicitly the half-width.

Sample Size for an Event Probability

To estimate the probability $p=P(A)$ of an event $A$, record $I_j=1$ when event $A$ occurs in replication $j$, and $I_j=0$ otherwise. The estimator is the sample proportion $\hat p=\bar I$. Since each $I_j$ is Bernoulli with variance $p(1-p)$, the approximate standard error of $\hat p$ is $\sqrt{p(1-p)/n}$. A normal-approximation half-width target $h$ therefore gives:

h \approx z_{1-\alpha/2}\sqrt{\frac{p(1-p)}{n}} \quad\Longrightarrow\quad n \approx \frac{z_{1-\alpha/2}^{\,2}p(1-p)}{h^2}.

A pilot run can supply $\hat p$ in place of $p$. If the pilot has $x$ event occurrences in $m$ replications, then $\hat p=x/m$. A small pilot can be unstable, particularly when the event is uncommon. If the pilot has no events, substituting $\hat p=0$ would incorrectly suggest zero variance and no simulations are needed. In that situation, a conservative planning value is $p(1-p)\leq 1/4$, which holds for every probability $p$.

Conservative variance bound: Because $p(1-p)=\tfrac14-(p-\tfrac12)^2\leq\tfrac14$, a conservative normal-approximation count for a probability is $$ n=\left\lceil\frac{z_{1-\alpha/2}^{\,2}}{4h^2}\right\rceil. $$ This protects the planning calculation against choosing a pilot proportion with a smaller variance than the true one. It does not turn the normal-approximation interval into an exact finite-sample interval.

Worked Example: Probability of Exceeding a Policy Threshold

A hypothetical model estimates the probability that a policy's annual loss exceeds a specified threshold. In 2,000 independent pilot simulations, 18 losses exceed the threshold. Plan the total number of simulations for an approximate 95% confidence-interval half-width of 0.001. Also find the additional simulations if the pilot is retained.

Method. Use the pilot proportion in the Bernoulli normal-approximation formula. The pilot estimate is $\hat p=18/2000=0.009$, so $1-\hat p=0.991$, and $\hat p(1-\hat p)=0.008919$.

With $z_{0.975}=1.96$ and $h=0.001$: $$ n \approx \frac{1.96^2(0.008919)}{0.001^2} = \frac{3.8416(0.008919)}{0.000001} = \frac{0.0342632304}{0.000001} =34263.2304. $$

Round up to $n=34264$ total simulations. Retaining the pilot requires $34264-2000=32264$ additional simulations.

Check. The planned half-width using the pilot variance estimate is $1.96\sqrt{0.008919/34264}$, approximately 0.001000, just below the target. The estimate $0.009$ is a planning input, not a claim that the true probability is known exactly.

Common mistake avoided: The count uses the variance of the indicator, $\hat p(1-\hat p)$, rather than $\hat p$ alone. Replacing the variance by $\hat p$ would overstate the variance for this pilot and produce a different, unjustified calculation.

Using a Known Range for a Conservative Plan

Sometimes a simulation output is known to lie in a finite interval, even when its variance is poorly estimated. If every output lies in $[a,b]$, its variance is at most $(b-a)^2/4$. One can combine this bound with Chebyshev's inequality to obtain a distribution-free, though often very conservative, sample-size plan.

Bound the variance.
For $a\leq X_j\leq b$, use $\operatorname{Var}(X_j)\leq(b-a)^2/4$.

Apply the variance of the mean.
For independent outputs, $\operatorname{Var}(\bar X)=\operatorname{Var}(X_j)/n$.

Require the desired coverage bound.
Chebyshev's inequality gives $P(|\bar X-\mu|\geq h)\leq \operatorname{Var}(\bar X)/h^2$. Set the right side no greater than $\alpha$, then round $n$ up.

Combining these steps gives $P(|\bar X-\mu|\geq h)\leq(b-a)^2/(4nh^2)$. Thus a sufficient count for confidence at least $1-\alpha$ is:

n\geq \frac{(b-a)^2}{4\alpha h^2}.

This is a genuine probability bound under the stated independence and range assumptions, but it can require far more replications than a normal-approximation calculation based on a pilot standard deviation. It is useful when conservatism matters more than simulation cost and a valid output range is known.

Worked Example: Bounded Simulated Payment

A hypothetical policy payment $X$ is known to be between $0 and $10,000 for every simulated policy-year. Determine a sufficient number of independent simulations to ensure, using Chebyshev's inequality, that the sample mean is within $250 of its true mean with probability at least 95%. A pilot standard deviation is $1,800; compare the range-based count with the usual pilot-based normal plan.

Method. First use the known range for a conservative distribution-free guarantee. Here $b-a=10000$, $h=250$, and $\alpha=0.05$: $$ n\geq\frac{(10000)^2}{4(0.05)(250)^2} =\frac{100000000}{0.20(62500)} =\frac{100000000}{12500} =8000. $$

Thus 8,000 simulations suffice for the stated Chebyshev bound. The bound is $100000000/(4(8000)(62500))=0.05$, so the probability of an absolute error at least $250 is at most 0.05.

For comparison, the 95% normal-approximation plan using the pilot standard deviation is $$ n\approx\left(\frac{1.96(1800)}{250}\right)^2 =(14.112)^2 =199.148544, $$ which rounds up to 200. This much smaller value is a model-based planning estimate, not the same distribution-free guarantee as the range calculation.

Check. The two counts answer different questions: 8,000 follows from the known range and Chebyshev's inequality; 200 follows from a normal approximation and the pilot estimate. They should not be presented as interchangeable guarantees.

Common mistake avoided: A bounded output does not make the pilot-based normal count a distribution-free guarantee. To claim the stated Chebyshev coverage, use the range-based count.

Practical Checks Before Running the Simulation

A sample-size calculation is only as good as its inputs and assumptions. Before committing to a run, check that the confidence level, target half-width, and output units are consistent. For a probability, $h=0.005$ means half a percentage point, not 0.5 as a probability. For a mean measured in dollars, both $s$ and $h$ should be in dollars.

The count should be based on the intended final estimator. If simulation replications are dependent, the standard error is not generally the independent-replication formula used here. Likewise, if the model or assumptions themselves are uncertain, increasing the replication count reduces Monte Carlo error but does not eliminate model uncertainty. When a pilot-based count is used, it can be prudent to update the standard error after an initial larger run and assess whether the target is being met; a fixed initial count remains a planning choice, not a promise about the realized interval.

Key takeaway: For a mean, plan with $z^2s^2/h^2$; for a probability, plan with $z^2\hat p(1-\hat p)/h^2$, or use the conservative variance bound $1/4$. Use a range and Chebyshev's inequality when a distribution-free guarantee is needed. In every case, calculate the total count, round up, and distinguish that total from any additional simulations after a retained pilot.

Check Your Understanding

For each problem, treat normal-approximation counts as planning calculations unless a Chebyshev bound is explicitly requested. Round required counts up to whole simulations.

A pilot gives a standard deviation of $400 for an independent simulated mean. What total count is needed for a 90% normal-approximation half-width of $20, using $z_{0.95}=1.645$? (Answer: 1,083 simulations.)
In 600 pilot replications, an event occurs 72 times. Using a 95% normal-approximation interval, what total count is planned for half-width 0.02, using $z_{0.975}=1.96$? (Answer: 1,015 simulations.)
For a 95% normal-approximation probability interval with target half-width 0.01, use the conservative variance bound $p(1-p)\leq 1/4$. What total count is planned? (Answer: 9,604 simulations.)
A simulated payment is known to lie in $[0,6000]$. Using Chebyshev's inequality, how many independent simulations suffice for a 90% probability that the sample mean is within $150 of its true mean? (Answer: 4,000 simulations.)
A pilot has 300 replications and no occurrences of an event. For a 95% normal-approximation half-width of 0.03, use the conservative probability variance bound rather than substituting a zero pilot proportion. What total count and, if the pilot is retained, how many additional simulations are needed? (Answer: 1,068 total simulations; 768 additional.)

What You'll Learn