Reproducible Monte Carlo Simulation

Why Reproducibility Matters

A Monte Carlo simulation represents uncertainty by repeatedly drawing values and passing them through a model. In population pharmacokinetics, these draws might represent virtual subjects’ random effects or covariates, and the model may translate them into exposure under a specified dose history. The simulation result is therefore shaped by both the model and the particular random draws used in that run.

Reproducibility is essential when checking a simulation, comparing candidate regimens, or reviewing a dose justification. If a result changes, the analyst needs to know whether the cause was different random draws, a changed model or input, or a change in the computational environment. Setting a seed helps, but a seed alone does not document or guarantee an exactly repeatable analysis.

A seed does not make the output inherently more valid, representative, or unbiased. It makes a particular pseudorandom sequence repeatable under specified conditions. A simulation result remains an approximation based on the model and the draws it used; reproducibility does not establish that the model is appropriate for its intended use.

What Must Be Held Fixed

A rerun will generally match exactly only when the relevant computational details match. These include the seed and PRNG algorithm, the code and its order of random-number calls, the simulation inputs, and the procedures used to transform generator output into model inputs. Software versions and settings can also matter. For example, a change in a random-number library or a parallel-processing method can change the sequence or its assignment to virtual subjects.

Document the simulation as a connected set of choices, not as a seed written in isolation. A useful record includes the model specification and parameter values; dose history and covariate inputs or their sampling distributions; the number of simulated subjects; the PRNG and seed; the software and relevant package versions; and the code or workflow version. Also record how simulated quantities are summarized. This helps distinguish a changed result due to a changed input from a changed result due to a different random sequence.

The key computation can be written generally as an average of a function of simulated draws. Let \(Z_i\) denote the random inputs for simulated subject \(i\), and let \(\theta\) collect fixed model and design inputs. A simulation may calculate:

Here, \(M\) is the number of simulated subjects and \(h\) is the chosen model output or summary contribution. For fixed \(\theta\), changing the sequence \(Z_1,\ldots,Z_M\) can change \(\hat{\psi}_M\). Changing \(\theta\), by contrast, changes what is being simulated, even if the same \(Z_i\) values are reused.

Replay a Simulation and Check Its Inputs

Consider a deliberately simple hypothetical example. Assume a one-compartment exposure calculation in which \(AUC = \text{Dose}/CL\), and suppose clearance for a virtual subject is \(CL = 5\exp(\eta)\) L/h, with \(\eta\) drawn from a normal distribution having mean 0 and standard deviation 0.1. A 100 mg dose is used. The calculation illustrates repeatability; it is not a clinical study or a claim about a real drug.

A practical workflow is to set the seed once at the start of a run, then allow the generator to advance as the simulation requests draws. Saving the code and an input record alongside the seed makes it possible to identify what must be restored for a rerun. Saving the generated random inputs or the PRNG state can provide an additional audit trail, particularly for long or computationally expensive runs.

Why the Order of Random Calls Matters

A seed initializes a sequence; it does not assign particular draws to particular variables or subjects. That assignment depends on the order in which the code asks the generator for values. If a code change inserts a new random draw before existing draws, the sequence can shift: later values may now be assigned to different subjects or variables. The seed may be unchanged even though the simulation output differs.

Parallel simulation requires special care. Workers should use a documented approach for generating separate random-number streams, rather than each worker starting from the same seed and unintentionally generating duplicate sequences. The specific mechanism depends on the software. Record the method and worker settings, and verify that the intended streams are distinct. Do not assume that adding more workers preserves the same assignment of draws as a sequential run.

Separate Model Changes From Random Variation

When comparing two model configurations, using the same underlying random inputs can make the comparison easier to interpret. This is often called using common random numbers. For each simulated subject, the same draw is passed through both configurations; the paired difference then reflects the contrast between model configurations for that draw, rather than also reflecting a different draw for each configuration.

This does not make the comparison definitive or remove uncertainty. It is a computational comparison under the chosen model and draws. But it is a useful diagnostic for determining whether a changed result follows from a deliberate input change.

If separate random sequences are used for the baseline and changed configuration, the observed difference combines the model contrast with variation between the two sets of draws. Using common draws can reduce that extra variation, especially when outputs from the two configurations are correlated. If the comparison instead changes the random-effect distribution, covariate sampling, or dose history, record that as an input change as well. A difference should not be labelled “random noise” or “a model effect” without checking what was held fixed.

Diagnostics, Alternatives, and Limitations

When a rerun does not match, investigate the computational chain systematically. Check that the same seed and PRNG were used, that the code and input values are unchanged, and that the random-number calls occur in the same order. Confirm that parallel settings, software versions, and transformations of generated values have not changed. Differences limited to small numerical rounding may reflect hardware or numerical-library details; larger differences more often suggest changed inputs, streams, or call order. Comparing saved random inputs can help separate a draw-generation problem from a model-calculation problem.

• Reseeding inside a loop: resetting to the same seed for every subject can repeat the same draw and create unintended duplicate virtual subjects. Initialize the generator once for the run unless a documented stream design calls for otherwise.
• Unrecorded input edits: a changed parameter or covariate distribution can alter outputs even when the seed is unchanged. Keep a dated, versioned record of simulation inputs.
• Assuming a seed travels across software: the same numeric seed does not guarantee the same sequence across different PRNG algorithms or implementations. Record the generator and relevant software versions.
• Choosing a seed because it gives a preferred result: trying many seeds and reporting only a favorable run can misrepresent simulation variability. Choose and record the seed before inspecting results, and report the simulation design transparently.

A fixed seed is useful for debugging, review, and exact replay under a specified environment. Using a fresh seed for a separate run can reveal how sensitive a finite simulation result is to its random draws, but it does not by itself identify a model change. Analytical calculations or deterministic numerical integration may be preferable when they are available and address the same quantity reliably. Monte Carlo methods are flexible for complex models, but their results depend on simulated draws and remain conditional on the model, assumptions, and implementation.

Reproducibility is also not the same as independent replication. Replaying the same seed and code reproduces a computational run; it does not provide a new independent realization of the simulation. Conversely, another analyst may independently implement the same scientific specification but obtain slightly different numerical results because the software or PRNG differs. Clear documentation makes both kinds of work interpretable.