Why Replicate Count Is a Design Choice
A simulated exposure distribution is an approximation to the distribution implied by the model, covariate representation, dose regimen, and simulation procedure. With a finite virtual cohort, the estimated mean, quantiles, and fraction meeting an exposure target can change if the simulation is repeated. Choosing the number of simulated subjects is therefore a numerical design decision: too few can make summaries unstable, while far more than needed consumes time without materially improving the answer.
The objective is not to make Monte Carlo error zero; that would require infinitely many draws. Instead, choose a replicate count that makes numerical error acceptably small for the decision at hand, then check that the selected size behaves as expected. Reproducible Monte Carlo Simulation establishes how to make a run reproducible. Here the focus is how many draws the run needs.
For a fixed model and regimen, individual simulated exposures vary because the virtual cohort includes variability in random effects, covariates, or other modeled inputs. That variation is part of the exposure distribution being estimated. Monte Carlo error concerns how accurately a finite sample represents that distribution. Increasing the number of simulated subjects reduces the latter, but it does not make the former smaller or resolve uncertainty in model structure or parameter estimates.
Precision Goals and the Basic Calculation
Start by deciding which summary matters and how much numerical imprecision is tolerable in its units. A mean exposure may need to be stable within a specified number of exposure units; a quantile may need to be stable within a clinically interpretable range; and a target-attainment estimate may need a sufficiently narrow numerical margin. A single replicate count need not provide equally good precision for every summary.
For independent, identically distributed replicates \(X_1,\ldots,X_N\) with finite variance \(\sigma^2\), the sample mean estimates the model-implied mean \(\mu\). Its standard error is \(\sigma/\sqrt{N}\), or is estimated using the sample standard deviation \(s\). Thus, if a pilot run estimates the standard deviation as \(s\), and the desired approximate 95% margin of error for the mean is at most \(d\), a planning calculation is:
This calculation assumes independent draws from the same specified simulation distribution, a finite variance, and a sufficiently large sample for the normal approximation to the sample mean to be useful. The pilot estimate of \(s\) is itself uncertain, so the result is a planning estimate rather than a guarantee. Rounding the required count upward is essential. If draws are correlated, for example because a simulation design reuses dependent draws, the simple formula understates the required size.
Worked Example: Sizing a Simulation for a Mean Exposure
Suppose a hypothetical pilot simulation produces \(N=10{,}000\) independent AUC values, with sample standard deviation \(s=30\) mg·h/L. The desired approximate 95% margin for the estimated mean AUC is no more than \(0.5\) mg·h/L. The pilot is assumed to represent the intended model and regimen, and the number of draws is large enough for the mean approximation to be reasonable.
The estimated standard error in the pilot is \(30/\sqrt{10{,}000}=30/100=0.300\) mg·h/L. For the desired precision, the planning formula gives:
Round up to \(N=13{,}830\) total independent replicates. At this count, the estimated standard error using the pilot \(s\) is \(30/\sqrt{13{,}830}\approx0.2551\) mg·h/L, giving an approximate 95% margin, calculated without rounding the intermediate value, of \(1.96\times30/\sqrt{13{,}830}\approx0.5000\) mg·h/L, slightly less than \(0.5\). This count is a starting point, not proof that all exposure summaries are equally precise.
Planning for a Target-Attainment Proportion
For a specified target, each simulated subject can be represented by an indicator \(I_i\), equal to 1 if the subject meets the target and 0 otherwise. If the model-implied probability of meeting it is \(p\), then \(I_i\) has a Bernoulli distribution with variance \(p(1-p)\). The average of \(N\) independent indicators estimates \(p\), and its standard error is approximately \(\sqrt{p(1-p)/N}\), with \(p\) replaced in practice by a pilot estimate \(\hat p\).
If no reliable pilot estimate is available, using \(p=0.5\) is conservative for this variance calculation because \(p(1-p)\leq0.25\) for \(0\leq p\leq1\). For a desired approximate 95% margin \(d\) in probability units, the corresponding planning rule is:
For example, a margin of \(0.01\) means one percentage point, not one percent of the estimated proportion. This calculation concerns Monte Carlo precision for a fixed target and regimen. It does not address whether the target is appropriate, whether a dose should be selected, or uncertainty in model parameters.
Worked Example: Planning a Replicate Count for a Target Proportion
In a hypothetical pilot simulation, the estimated probability of meeting a prespecified exposure target is \(\hat p=0.82\). The aim is an approximate 95% Monte Carlo margin no larger than \(0.01\), or one percentage point. Assume independent virtual subjects and use the pilot estimate as a planning value.
The required count is:
Round up to \(N=5{,}671\). This count targets the stated precision under the assumed proportion; if the true probability is closer to 0.5, the actual standard error will be larger than the pilot-based estimate. A conservative calculation instead uses \(p(1-p)=0.25\): \(N\geq3.8416(0.25)/0.0001=9{,}604\). The choice between a pilot-based and conservative count should be reported, along with the precision goal.
Exposure Distributions Need More Than a Mean Check
A simulation may be intended to estimate a percentile or describe the shape of an exposure distribution, not just its mean. The mean-based formula does not automatically control the precision of a quantile. Quantile stability depends on the distribution near that quantile: a relatively small change in probability can correspond to a large change in exposure units where the distribution is sparse. Consequently, use a precision goal in exposure units and assess the quantile directly across independent batches or increasing cumulative runs.
For a distribution summary, practical checks include comparing the empirical cumulative distribution at clinically relevant exposure values, comparing selected quantiles across batches, and tracking whether the summary changes materially as the simulation grows. Define tolerances before interpreting the checks. A tolerance should reflect the use of the summary; it should not be chosen after seeing only a favorable run.
Worked Example: Checking Stability of an Exposure Quantile
A hypothetical simulation is intended to estimate the 10th percentile of steady-state AUC. Four independent batches of 5,000 subjects yield 10th-percentile estimates of 82, 83, 82.5, and 84 mg·h/L. The analyst prespecifies that the batch estimates should span no more than 3 mg·h/L before treating the quantile as adequately stable for this use.
The observed range is \(84-82=2\) mg·h/L, which is within the prespecified 3 mg·h/L tolerance. The average of the four batch quantiles is \((82+83+82.5+84)/4=82.875\) mg·h/L, approximately 82.9 mg·h/L. This average is a descriptive stability check, not a universal unbiased estimator of the pooled-sample quantile. The result supports the chosen tolerance in these batches, but does not establish precision for other quantiles or guarantee that a different random run will pass. A useful follow-up is to compare the pooled estimate at cumulative sizes such as 5,000, 10,000, 15,000, and 20,000.
A Practical Convergence Workflow
A good replicate-count decision combines an analytic planning calculation with empirical checks. The calculation gives a rational initial size; convergence checks reveal whether the desired summaries behave acceptably in the actual simulation. The steps below assume that the simulation algorithm and input distributions are fixed, and that independent batches can be generated reproducibly.
Specify the exposure metrics, distribution summaries, or target probabilities that drive the decision. Set precision in meaningful units, such as mg·h/L or probability points.
Use a pilot large enough to give a credible estimate of the spread and, where relevant, the approximate target probability. Preserve the simulation settings intended for the full run.
Apply the mean or proportion planning formula when its assumptions fit. For quantiles and other distribution features, treat the formula as indirect guidance and plan direct stability checks.
Check the decision-relevant summaries across batches and as the total count increases. If changes exceed the tolerances, simulate more and repeat the checks.
Report the final count, precision goals, pilot assumptions, and checks used. Preserve the random-number settings so the run can be reproduced.
Diagnostics, Alternatives, and Limitations
A stable-looking estimate can be misleading if the simulation has not sampled an important part of the distribution. Inspect exposure summaries and, where useful, plots or empirical distribution comparisons for rare tails, multiple modes, extreme values, or implausible gaps. If different independent batches disagree substantially, first investigate whether draws are genuinely independent and whether the batches use the same model inputs and settings. A larger run will not repair a coding error, a misrepresented covariate distribution, or an incorrect exposure calculation.
The main alternatives are complementary rather than interchangeable. A formula-based count is efficient when its assumptions and target summary are clear, as for a mean or proportion. Batch stability is broadly useful and directly evaluates the summaries being reported, but can miss a feature that is rare or not monitored. Sequentially doubling the cumulative count is convenient, though successive estimates are dependent because each includes earlier draws. Independent batches make between-run comparisons easier, but do not themselves give a guarantee of precision. Use more than one check when the decision is sensitive to the result.
| Approach | Useful for | Main limitation |
|---|---|---|
| Analytic planning formula | Means and proportions with suitable independent draws | Depends on variance estimates and approximation assumptions |
| Independent batch comparison | Checking stability of several summaries | May not reveal rare or unmonitored distribution features |
| Increasing cumulative counts | Observing how estimates change as computation grows | Successive estimates are dependent and can appear stable temporarily |
No finite simulation demonstrates exact convergence. Precision can also vary across dose levels: an allocation adequate for one regimen or target may be inadequate for another. Where parameter uncertainty is deliberately propagated, distinguish the variability from parameter draws from Monte Carlo error in estimating the resulting distribution. Increasing the number of virtual subjects reduces finite-draw error, but does not eliminate uncertainty in the estimated model or the choice of assumptions.
- Mistake: treating subject variability as Monte Carlo error. A wide exposure distribution does not by itself imply that more draws are needed; the relevant question is how precisely its summaries are estimated.
- Mistake: choosing a count because it is conventional. The same count can be ample for a mean and inadequate for a tail quantile. Set tolerances for the summaries that matter.
- Mistake: stopping after one stable comparison. A finite run may look stable by chance. Use prespecified checks, inspect more than one relevant summary, and investigate batches that disagree.
- Mistake: assuming more replicates fix a wrong model. More draws reduce numerical error around the specified simulation distribution; they do not correct its assumptions.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- A pilot estimates \(s=20\) mg·h/L for an exposure mean. What count is needed for an approximate 95% margin of \(0.4\) mg·h/L?
- Why can a count adequate for estimating a mean be inadequate for estimating a lower-tail quantile?
- For a target-proportion estimate, why is \(p=0.5\) conservative when no pilot estimate is available?
- Four batches give noticeably different summaries. Name two checks to make before simply increasing the simulation size.
Solutions. (1) \(N\geq[1.96(20)/0.4]^2=98^2=9{,}604\) independent replicates. (2) Quantile precision depends on the distribution near that quantile; the mean’s standard error formula does not control it, so assess the quantile directly against a tolerance. (3) Since \(p(1-p)\leq0.25\), it gives the largest Bernoulli variance and hence a conservative standard-error planning value. (4) Confirm the batches use the same model, inputs, and exposure calculation, and verify independence of draws; then inspect the summaries and distributions for rare tails or other features that could explain the disagreement.