Why Define a Run?
A reported estimate is meaningful only when it can be connected to the computation that produced it. In pharmacometrics, the same named population model can be estimated using different data extracts, initial estimates, optimizer settings, or software versions. The name of the model alone cannot say which computation produced a particular result.
A run is the unit of execution: one occasion on which a specified model is applied to a specified dataset under a specified estimation configuration and software environment. Making this distinction is important when diagnosing numerical behavior, reproducing estimates, or interpreting whether two results came from the same computation or from different computations.
The Parts of a Run Specification
It is useful to separate the run specification from the execution occurrence. The specification describes what is to be executed; the occurrence is the particular execution event. A compact representation is:
Here, \(M\) is the identified model version, \(D\) is the identified dataset version, \(C\) is the estimation configuration, \(E\) is the software environment, and \(q\) distinguishes one execution occurrence from another. This is a conceptual definition, not a prescribed file format or identifier-generation method.
Each component must be specific enough for the question being asked. A model name such as “two-compartment model” may identify a model family, but not the exact equations, parameterization, or source version used. The data component should distinguish the actual input snapshot and any transformations that determine the observations being analyzed. The estimation configuration includes choices such as the estimation method, initial estimates, parameter bounds, tolerances, and other options that can affect the fit. The software environment includes the relevant software and version information, such as the estimation program and numerical libraries.
These categories are conceptually distinct even when a workflow stores them together. For example, an optimizer tolerance belongs to the estimation configuration, while the version of the optimizer software belongs to the environment. A random seed is part of the run specification when it controls a stochastic algorithm; if a seed is not fixed or does not fully determine stochastic behavior, the execution occurrence still matters.
When Are Two Runs the Same?
A practical comparison asks two questions. First, do the executions have the same run identity? Second, do they have the same specified inputs and conditions? The answers are related but not interchangeable. A run identity must distinguish execution occurrences, while the specification explains what was executed.
If any specified component differs, the executions are different runs. For example, changing the dataset snapshot or estimation tolerance changes the specification. If every specified component matches but the model was launched twice, there are still two runs: matching specifications do not turn separate executions into one event. This distinction allows repeated fits to be compared without pretending they were a single computation.
Worked Example: Same Model Name, Different Data
Suppose a hypothetical team estimates a model named “PK-oral-2c.” Run A uses a dataset containing 120 participants, with concentrations measured through 24 hours. Run B uses a revised dataset containing 124 participants, with the same nominal model, estimation method, and software version.
The shared model name does not make these the same run. The datasets differ: \(D_A \ne D_B\). Therefore the specifications differ, \(S_A = (M,D_A,C,E)\) and \(S_B = (M,D_B,C,E)\), so the executions must have distinct run identities. Even if the estimated clearance values happen to be equal, that numerical coincidence does not make the runs identical.
A clear description would identify the model version and the two dataset versions, not merely report “PK-oral-2c.” This makes the change in input data explicit without implying that the model itself changed.
Worked Example: Two Executions With the Same Specification
Consider a hypothetical model, dataset, estimation configuration, and software environment that are held fixed. The estimation is launched once in the morning and once again later that day. Both executions use the same initial estimates, tolerances, and fixed random seed.
The two executions have the same specification \(S\). However, they occurred at separate times, so they have distinct occurrence labels \(q_1\) and \(q_2\). Their run identities are \(R_1=(q_1,S)\) and \(R_2=(q_2,S)\), with \(R_1 \ne R_2\).
If the outputs match exactly, that is evidence that the executions produced matching results under the specified conditions; it does not collapse two execution events into one run. If the outputs differ, the separate identities allow the discrepancy to be investigated rather than obscured by treating the executions as interchangeable.
Why Execution Occurrence Belongs in the Definition
The need for an execution component follows from a simple collision argument. Suppose run identity were defined only by the specification \(S=(M,D,C,E)\). If the same specification were executed twice, both events would map to the same identity. The identity would therefore fail to distinguish two executions.
Including an execution-distinguishing component prevents this collision. The principle is not that every workflow must use a particular kind of label. It is that, whatever convention is used, it must distinguish separate executions within the scope where uniqueness is required. If the scope is one project, uniqueness must hold within that project; if the scope is broader, the identifier must distinguish runs across that broader scope.
Worked Example: A Changed Estimation Setting
A hypothetical concentration-time model is fitted to the same data using the same model version and software environment. In Run C, the optimizer tolerance is \(10^{-6}\); in Run D, it is \(10^{-8}\). All other stated settings are unchanged.
Because the tolerance is part of the estimation configuration, \(C_C \ne C_D\). Thus \(S_C \ne S_D\), and the two executions have distinct run identities even if they converge to estimates that agree to the reported precision. The equality of rounded parameter estimates is not a substitute for equality of the estimation conditions.
A suitable interpretation is that the estimates are numerically similar at the displayed precision under two different configurations. It would be inaccurate to call the runs identical or to conclude from this comparison alone that the optimizer settings can never affect the result.
Alternative Labels and Common Failure Modes
Several labels are useful in workflow management but do not, on their own, define a run. A model name identifies a candidate or scientific formulation. A job number may identify a scheduler task, but can be reused or may fail to capture the model and inputs. An output folder name may be descriptive but ambiguous. An estimate or objective-function value describes an outcome, not the execution that produced it.
| Label or description | What it can identify | Why it is not sufficient alone |
|---|---|---|
| Model name | A model family or candidate | Does not specify data, settings, environment, or execution occurrence |
| Job number | A computing task in a particular system | May not uniquely identify the full specification or remain unique outside that system |
| Output value | A reported result, such as an estimate | Different runs can yield the same value; one run can also produce multiple outputs |
| Run identity | One execution in its defined scope | Must be linked to a sufficiently precise specification to be interpretable |
The most common conceptual errors are to equate a model label with a run, to treat matching outputs as proof of identical execution, or to assume that a repeated execution is the same run because all settings were intended to be unchanged. The correction is to distinguish the scientific object (the model), the conditions of computation (the specification), the execution event, and its outcome.
A run definition also has limits. It supports clear attribution and comparison, but it does not prove that the model is scientifically appropriate, that the data are valid, or that the numerical method succeeded. Nor does a run identity by itself make an execution reproducible. Reproduction additionally depends on sufficiently describing and retaining the relevant inputs and computational conditions.
Checks Before Calling Something a Run
When naming or comparing a run, check that the description answers these questions:
- Which exact model version was executed, rather than only which model family was intended?
- Which dataset version and relevant preprocessing define the analysis inputs?
- Which estimation choices could affect the fit, including the method, starts, bounds, and tolerances?
- Which software environment was used, at a level adequate for interpreting the computation?
- Does the identity distinguish this execution from every other execution in the stated scope?
If one of these answers is unknown, the execution may still have occurred, but its run description is incomplete for a comparison that depends on that information. State the uncertainty rather than silently treating an unrecorded component as identical.
Check Your Understanding
For each item, decide what the run definition implies. Solutions follow the questions.
- Two fits use the same model name, but one uses a newer dataset. Are they the same run? Identify the relevant reason.
- Two separate launches use the same model version, data, settings, and software environment. Are they one run or two?
- A team changes only the optimizer tolerance. Which component changes, and what follows for run identity?
- Two runs report exactly the same clearance estimate to three significant figures. Does that establish that they are the same execution?
- Why is a scheduler job number not necessarily a complete run identity?
Solutions. 1. They are different runs because the dataset version differs, even though the model name is shared. 2. They are two runs because there are two execution occurrences; matching specifications do not merge them. 3. The estimation configuration changes, so the specifications and run identities differ. 4. No. Matching rounded estimates describe an outcome and do not establish identical inputs, conditions, or execution occurrence. 5. A job number may be unique only within a particular scheduler or time period and may not identify the model, dataset, configuration, and environment; its scope and connection to the full specification matter.