Fixed-Width Stopping Rules for Monte Carlo Estimates

Why Stop by Precision?

A simulation budget is often chosen in advance, but its required size depends on the variability of the simulated output. A fixed-width stopping rule reverses that approach: continue generating results until the estimated confidence-interval half-width is no greater than a specified tolerance. This directly ties the stopping condition to the precision required for the estimate.

The rule is useful only when the reported standard error matches the simulation design. For independent, identically distributed outputs, the usual standard error of a sample mean is straightforward. For paired comparisons, stratified simulation, or other variance-reduction designs, use the standard error for that estimator rather than treating the raw outputs as an ordinary independent sample.

Definition: Let $Z_1,Z_2,\ldots$ be independent, identically distributed simulation outputs with mean $\theta=E[Z_i]$ and finite variance $\sigma^2$. At sample size $n$, the sample mean is $\bar Z_n=\frac{1}{n}\sum_{i=1}^n Z_i$, and the sample standard deviation is $S_n=\sqrt{\frac{1}{n-1}\sum_{i=1}^n(Z_i-\bar Z_n)^2}$. A fixed-width rule stops at the first eligible sample size $N$ for which the estimated confidence-interval half-width is at most a chosen tolerance $\varepsilon$.

The Fixed-Width Rule

For a two-sided confidence interval with confidence level $1-\alpha$, a common large-sample estimate of the half-width is

H_n=z_{1-\alpha/2}\frac{S_n}{\sqrt{n}}, \qquad N=\min\{n\geq n_{\min}:H_n\leq\varepsilon\}.

Here $z_{1-\alpha/2}$ is the corresponding standard normal quantile, $n_{\min}$ is a minimum run size, and $\varepsilon$ is expressed in the same units as the simulated output. The minimum prevents stopping after only a few observations, when the estimated variability may be especially unstable. The inequality is checked using the current estimate $S_n$; it is not a guarantee that the true standard deviation is known.

If $\sigma$ is known, replace $S_n$ with $\sigma$. The resulting half-width decreases deterministically with $n$, so the required sample size follows directly. If $\sigma$ is estimated, the rule must be checked against the accumulating simulation results. A pilot estimate can suggest a useful workload, but it does not determine the final stopping time.

Why the Sample-Size Formula Works

For independent outputs with finite variance, the central limit theorem says that, for sufficiently large $n$, the standardized sample mean is approximately standard normal. Thus an approximate two-sided interval for $\theta$ has the form $\bar Z_n\pm z_{1-\alpha/2}\sigma/\sqrt n$. Its half-width is the margin $z_{1-\alpha/2}\sigma/\sqrt n$. To require that margin to be no greater than $\varepsilon$, solve the inequality:

z_{1-\alpha/2}\frac{\sigma}{\sqrt n}\leq\varepsilon \quad\Longleftrightarrow\quad n\geq\left(\frac{z_{1-\alpha/2}\sigma}{\varepsilon}\right)^2.

Since $n$ must be an integer, round the right-hand side up. This is a planning result under the stated variance and interval approximation; rounding down would fail the specified width. When the variance is unknown, substituting a pilot estimate $s_0$ gives a plug-in planning size, not a certainty that the eventual estimated half-width will meet the target.

Specify precision.
Choose the confidence level and a half-width tolerance $\varepsilon$ in meaningful output units.

Estimate variability.
Use an analytic standard deviation if available; otherwise use a pilot run that follows the same simulation design as the main run.

Plan and simulate.
Calculate a provisional total sample size, then run the simulation and recompute the estimated half-width at the chosen checking points.

Stop only when the condition holds.
End the run when the current estimated half-width is at most $\varepsilon$, subject to any minimum sample size.

Worked Applications

Worked Example: Known Output Standard Deviation

A hypothetical simulation produces independent outputs $Z_i$ following the normal location model $N(\theta,100)$, where the second parameter is the variance and the target $\theta$ is unknown. The output is measured in thousands of dollars. There is no discounting or mortality component in this simulation. The standard deviation is known to be $10$ thousand dollars. Find the smallest sample size giving an approximate 95% confidence-interval half-width of at most $0.40$ thousand dollars.

Method. Use the known-standard-deviation formula with $z_{0.975}=1.96$, $\sigma=10$, and $\varepsilon=0.40$. The calculation is in thousands of dollars throughout:

n\geq\left(\frac{1.96(10)}{0.40}\right)^2 =\left(\frac{19.6}{0.40}\right)^2 =49^2 =2401.

Thus, the smallest eligible size is $n=2401$. Check the half-width:

H_{2401}=1.96\frac{10}{\sqrt{2401}} =1.96\frac{10}{49} =\frac{19.6}{49} =0.40 \text{ thousand dollars}.

The target is met exactly. The result is $2401$ independent simulation runs. The common mistake avoided here is rounding a noninteger sample-size calculation down; the general rule is to round up, because a smaller sample could exceed the target.

Worked Example: Planning From a Pilot and Rechecking

A hypothetical simulated cost $Z_i$, measured in dollars, is modeled as independent $N(\theta,\sigma^2)$ output with both $\theta$ and $\sigma$ unknown. A pilot of 25 runs gives sample standard deviation $s_0=\$8$. Plan for an approximate 95% half-width of at most $\$0.50$, then assess what happens if the observed variability changes.

Method. Use the pilot standard deviation in the planning formula. The pilot is included in the total:

n_{\text{plan}} =\left\lceil\left(\frac{1.96(8)}{0.50}\right)^2\right\rceil =\left\lceil 31.36^2\right\rceil =\left\lceil 983.4496\right\rceil =984.

The planned total is 984 runs, so 959 more are needed after the initial 25. At $n=984$, suppose the actual accumulated sample standard deviation is $S_{984}=\$8.2$. The estimated half-width is

H_{984}=1.96\frac{8.2}{\sqrt{984}} \approx \frac{16.072}{31.3688} \approx \$0.5124.

This exceeds $\$0.50$, so the stopping condition is not met. After continuing to $n=1100$, suppose the updated sample standard deviation is $S_{1100}=\$8.1$. Then

H_{1100}=1.96\frac{8.1}{\sqrt{1100}} \approx \frac{15.876}{33.1662} \approx \$0.4787.

The condition now holds, so the rule permits stopping at this checking point, with 1100 runs in total. The pilot-based size was useful for budgeting, but it was not a substitute for checking the actual estimated width. The common mistake avoided is treating a pilot-based sample-size calculation as proof that the final run will meet the precision target.

Worked Example: A Probability Estimate and a Zero-Event Pilot

A hypothetical simulation estimates the probability $p$ of an event using independent Bernoulli indicators $I_i$, where $I_i=1$ if the event occurs and $I_i=0$ otherwise. Thus $I_i\sim\operatorname{Bernoulli}(p)$, with unknown $p$, and $\hat p_n=\bar I_n$. A 100-run pilot has no events. Plan for a 95% Wald half-width of $0.01$ without treating the zero-event pilot as evidence that the variance is zero.

Method. For a Bernoulli probability estimate, the plug-in Wald half-width is $1.96\sqrt{\hat p_n(1-\hat p_n)/n}$. The pilot gives $\hat p=0$, which would yield a zero estimated width and a nonsensical plug-in size of zero. Instead, use $p(1-p)\leq 0.25$ for all $0\leq p\leq1$ as a conservative planning variance bound:

n\geq\left(\frac{1.96\sqrt{0.25}}{0.01}\right)^2 =\left(\frac{1.96(0.5)}{0.01}\right)^2 =98^2 =9604.

The conservative planning total is 9604, including the 100 pilot indicators, so 9504 further indicators are required to reach that planned size. The bound ensures that the estimated Wald half-width cannot exceed $0.01$ at that size, since $\hat p_n(1-\hat p_n)\leq0.25$ as well. This does not resolve limitations of the Wald interval for rare events; it only demonstrates a way to avoid declaring success solely because a small pilot observed no events. The common mistake avoided is interpreting a zero pilot variance as proof of zero population variance.

Practical Conditions and Exam Traps

A fixed-width rule is meaningful only when its inputs and interpretation are clear. In particular, distinguish the half-width from the full interval width: a target half-width of $\varepsilon$ corresponds to total interval width $2\varepsilon$. Also state the output units before calculating. A half-width of $0.5$ dollars is not the same precision target as $0.5$ thousand dollars.

A pilot should represent the same output definition, model, and simulation design planned for the main run. If the pilot uses common random numbers, stratification, or another variance-reduction technique, the planning estimate must be based on the variance of the resulting estimator. A standard deviation computed from a different output or an incompatible design does not provide a sound size estimate. For probability estimates, zero or very few observed events can produce a severely misleading variance estimate, as the preceding example shows.

The usual normal half-width is an approximation. With unknown variance, a fixed-sample Student $t$ interval can use $t_{n-1,1-\alpha/2}S_n/\sqrt n$; because its critical value depends on $n$, plan by checking candidate sample sizes until the inequality holds. The normal critical value is often convenient for large simulation runs, but the chosen method should match the intended interval and sample size.

Important distinction: A fixed-width rule controls the estimated interval width; it does not by itself guarantee that an interval calculated at a data-dependent stopping time has its nominal coverage probability. Repeatedly checking an ordinary fixed-sample interval can affect coverage. Precision planning and confidence coverage are separate requirements.

The method is valuable when computation is costly and an arbitrary fixed number of runs might be either wasteful or inadequate. The pilot provides a first workload estimate, while the running half-width provides an operational precision check. Report the confidence level, target half-width, standard-error method, minimum sample size, and final number of runs so the stopping decision can be understood and reproduced.

Check Your Understanding

Use a 95% normal critical value of $1.96$, except where another confidence level is stated. For probability estimates, use the Wald half-width.

A simulated output has known standard deviation $6$ units. What is the minimum sample size for half-width at most $0.30$ units? (Answer: 1537 runs)
A pilot estimates the standard deviation of an independent normal simulation output as $4$. Find the planned total sample size for a 90% half-width of $0.20$, using $z_{0.95}=1.645$. (Answer: 1083 runs)
A Bernoulli pilot of 100 runs estimates an event probability as $0.16$. Using that pilot proportion, what is the approximate planned total for a 95% Wald half-width of $0.04$? (Answer: 323 runs)
A pilot of 200 Bernoulli trials has zero events. For a conservative 95% Wald-width plan with half-width $0.02$, use $p(1-p)\leq0.25$. How many total trials should be planned? (Answer: 2401 trials)
A normal-output pilot of 40 runs gives $s_0=5$. For a 95% half-width target of $0.25$, find the pilot-based total plan and the additional runs required. (Answer: 1537 total runs; 1497 additional runs)

What You'll Learn