Tutorials › Quantitative Systems Pharmacology (QSP) › A Reproducible QSP Project Layout

Reproducibility and Model Sharing · Tutorial 772 of 799

A Reproducible QSP Project Layout

Learn a practical QSP project structure and trace a hypothetical simulation from its equations and inputs through its script to its generated results.

Intermediate 10 min read

What You'll Learn

  • Separate model equations, parameter values, simulation inputs, scripts, and generated outputs.
  • Define the role of each project folder and file before running a simulation.
  • Trace a hypothetical QSP simulation from source inputs to reported predictions.
  • Check that output files can be linked to the inputs and model used to generate them.
  • Recognize layout choices that make reruns ambiguous or risk overwriting source data.
  • Understand what a clear project layout supports—and what it cannot guarantee by itself.

Why Project Layout Matters

A QSP result is not just a table of predicted concentrations. It is the consequence of a model, a parameter set, specified inputs, and a simulation procedure working together. If those components are mixed together or their roles are unclear, a reader may be unable to determine which equations or values generated a result. A reproducible project layout makes those relationships visible and gives each component a predictable place.

Here, reproducible means that a person can identify the files used for a run, understand how they contribute to the calculation, and rerun the specified workflow to obtain the corresponding results, subject to the computational details being held sufficiently consistent. A directory structure is a practical aid to this goal, not proof that a model is correct or that a prediction is qualified.

Definition: A QSP project layout is an organized arrangement of source files, inputs, scripts, and generated outputs in which each file has a clear role and the flow from model specification to result can be followed. Source files are the maintained materials needed to define and run the analysis; generated outputs are created by executing that analysis and should not be mistaken for its inputs.

The layout should distinguish five essential roles: model equations, parameter sets, input data, simulation scripts, and outputs. Supporting files, such as a short project guide, can explain how to use these components without duplicating their contents.

A Practical Separation of Project Components

The following is one workable structure for a small QSP simulation. It is not a universal standard; the important feature is that the role of each directory is unambiguous.

LocationRoleExample contents
model/Defines the mathematical model implemented for simulation.equations.py: state equations and a function that evaluates their rates of change.
parameters/Stores parameter values separately from the equations.reference.csv: parameter names and values for a specified run.
input/Contains inputs that specify the scenario or data supplied to the workflow.scenario.csv: treatment condition and requested observation times.
scripts/Contains the executable workflow that reads the model, parameters, and inputs and runs the simulation.run_simulation.py: loads source files, calls the model, and writes results.
outputs/Receives generated results rather than maintained source inputs.predictions.csv: simulated states at the requested times.
README.mdExplains the project’s purpose and the basic run sequence.Brief instructions identifying the script to run and the expected output location.

The main operational rule is that the script reads maintained inputs and writes generated results to a distinct output location. It should not silently modify the equation or parameter files while running. Keeping outputs separate also makes it easier to notice when an expected result is missing or when an old result is being inspected in place of a new one.

A parameter file belongs apart from the equation implementation because the same equations may be evaluated with different parameter values. Similarly, an input file specifies the simulated scenario, while the script describes the operations that connect that scenario to model predictions. These roles are conceptually different even if a very small demonstration could fit in one file.

Worked Example: Assigning Files to Clear Roles

A hypothetical team has five files for a simple cytokine simulation: a function defining the change in cytokine concentration, a table of parameter values, a table of requested time points, a program that performs the simulation, and a table of predicted concentrations. Assign each file a project location.

The function defining the rate of change belongs in model/, for example model/equations.py. The parameter table belongs in parameters/, such as parameters/reference.csv. Requested time points belong in input/, for example input/scenario.csv. The executable program belongs in scripts/, and its generated predictions belong in outputs/.

This arrangement distinguishes the model definition from the values used in that model, and both from the procedure used to produce a result. If the team later runs the same equations with another parameter set, it can make the change explicit in the parameter input rather than editing the equations to hide new values.

Trace a Simulation from Source to Output

A useful way to check a project layout is to follow one run in order: identify the equation, identify the parameter values, identify the scenario inputs, identify the script that combines them, and inspect the generated result. The example below uses a deliberately simple one-state model so that every numerical step can be checked by hand.

Model and assumptions: Let \(C(t)\) be a hypothetical cytokine concentration in nM. For the example, \(C(0)=0\), production is constant at rate \(k_{\mathrm{in}}\), and removal is proportional to concentration with rate constant \(k_{\mathrm{out}}\). The model is $$ \frac{dC}{dt}=k_{\mathrm{in}}-k_{\mathrm{out}}C. $$ Assume \(k_{\mathrm{in}}=2\ \mathrm{nM/h}\), \(k_{\mathrm{out}}=0.5\ \mathrm{h}^{-1}\), and that the simulation uses an explicit Euler step of \(h=1\ \mathrm{h}\). The scenario requests predictions at integer times from 0 to 4 h. This is a hypothetical computational example, not data from a study.

For an explicit Euler step, the script updates the state by multiplying the rate at the start of the step by the step length. Thus, for time index \(j\),

$$ C_{j+1}=C_j+h\left(k_{\mathrm{in}}-k_{\mathrm{out}}C_j\right). $$

The equation implementation supplies the rate expression, the parameter file supplies \(2\ \mathrm{nM/h}\) and \(0.5\ \mathrm{h}^{-1}\), and the scenario file supplies the time grid. The script applies the update and writes the resulting values to the output table. With this separation, the numerical calculation can be traced back to its inputs rather than inferred from the output alone.

Worked Example: From Parameter File to Prediction Table

Suppose parameters/reference.csv contains \(k_{\mathrm{in}}=2\ \mathrm{nM/h}\) and \(k_{\mathrm{out}}=0.5\ \mathrm{h}^{-1}\), while input/scenario.csv requests times \(0,1,2,3,4\ \mathrm{h}\). Starting from \(C_0=0\ \mathrm{nM}\), calculate the predictions that the script should write.

At \(t=0\), the initial condition gives \(C_0=0\ \mathrm{nM}\). For the first step, $$ C_1=0+1\left(2-0.5(0)\right)=2\ \mathrm{nM}. $$ For the second step, $$ C_2=2+1\left(2-0.5(2)\right)=3\ \mathrm{nM}. $$ For the third step, $$ C_3=3+1\left(2-0.5(3)\right)=3.5\ \mathrm{nM}. $$ For the fourth step, $$ C_4=3.5+1\left(2-0.5(3.5)\right)=3.75\ \mathrm{nM}. $$

The expected generated table, before any formatting or rounding, is therefore:

Time (h)Predicted concentration (nM)
00
12
23
33.5
43.75

The table is a generated result, not an independent source of the simulation specification. If the reported value at 2 h were 2.8 nM, for example, the trace identifies places to inspect: the parameter values read by the script, the implementation of the rate equation, the initial condition, the update rule, and the selected time step. The arithmetic above gives a simple check on the expected output for this particular run.

Project Boundaries and a Second Run

A project layout is most useful when one can change a single component deliberately and understand the consequence. For instance, changing the parameter file while leaving the equation implementation and scenario unchanged defines a different parameterized run. Changing the scenario changes the requested inputs. Changing the model implementation changes the mathematical system being evaluated. Keeping these changes in their appropriate places helps avoid accidental changes to more than one part of the analysis.

Worked Example: Comparing Two Parameter Sets

Use the same hypothetical model, initial condition, scenario, and Euler step as above, but create a second parameter file with \(k_{\mathrm{in}}=1\ \mathrm{nM/h}\) and \(k_{\mathrm{out}}=0.5\ \mathrm{h}^{-1}\). Compute the first two updates.

At \(t=1\ \mathrm{h}\), $$ C_1=0+1\left(1-0.5(0)\right)=1\ \mathrm{nM}. $$ At \(t=2\ \mathrm{h}\), $$ C_2=1+1\left(1-0.5(1)\right)=1.5\ \mathrm{nM}. $$

The script can produce this run by reading the second parameter file and writing to a clearly distinguishable output file, such as outputs/predictions_alternative.csv. If both parameter sets are used but the output is always written to the same filename, the second run may overwrite the first. The file location and name should therefore make it possible to distinguish the generated results that are intended to be retained.

A short run description can record which model, parameter file, and scenario were used. This is not a substitute for keeping the actual source files; it is a convenient pointer between a result and the materials that produced it. Avoid names such as final.csv or final_new.csv when they do not reveal which run they contain.

Checks, Alternatives, and Limitations

Before treating an output as the intended result, check that the workflow read the expected files, that each role is represented once and clearly, and that the output is written where anticipated. For a small deterministic simulation, compare a few output values with hand calculations, as in the Euler example. Also check that the output has the requested time points and that its values use the expected state and units. These checks can reveal a mistaken path, a stale output, or a mismatch between the scenario and the result.

A common mistake is to put equations, parameter values, scenario data, and results together in one spreadsheet or script because that is convenient for an initial calculation. The reasoning is that a single file is automatically easier to reproduce. The correction is that convenience does not make the roles of its contents clear: a changed constant might be an equation change or a parameter update, and a result may have overwritten an input. For a small, one-off illustration, a compact file may be adequate; for a shared workflow or repeated runs, separate files with explicit roles make the calculation easier to inspect.

The opposite extreme is also unhelpful. Splitting every short expression into its own file can make it harder to follow the workflow without adding meaningful clarity. Choose boundaries that match distinct responsibilities: equations, parameter values, scenario inputs, execution, and generated results. A project guide should explain the run sequence, not repeat entire files in prose.

A clean layout does not establish that the equations represent the biology adequately, that the parameters are supported by evidence, or that the simulation predicts a relevant outcome. Nor does the layout alone ensure identical results on every computer. Its contribution is narrower and valuable: it makes the computational ingredients and their connection to a result easier to inspect and rerun.

1
Identify the run inputs.
Locate the equation implementation, selected parameter file, and scenario file that the run is intended to use.
2
Follow the execution path.
Confirm which script reads those inputs, combines them, and writes generated results.
3
Inspect the result.
Check that the output contains the expected quantities and requested times, then verify selected values when a hand calculation is feasible.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

  1. Where should the model’s rate equations, parameter values, requested simulation times, executable workflow, and predicted concentrations be stored?
  2. In the one-state example, calculate the prediction at \(t=3\ \mathrm{h}\) when \(k_{\mathrm{in}}=2\ \mathrm{nM/h}\), \(k_{\mathrm{out}}=0.5\ \mathrm{h}^{-1}\), \(h=1\ \mathrm{h}\), and the values at 1 h and 2 h are 2 nM and 3 nM.
  3. A script reads a new parameter file but writes to the same output filename used for an earlier run. What is the main layout risk, and how can it be reduced?
  4. Why is a project directory structure not, by itself, evidence that a QSP model is biologically valid?

Solutions. (1) Put the equations in model/, parameter values in parameters/, requested times in input/, the executable workflow in scripts/, and predictions in outputs/. (2) Apply the stated update: \(C_3=3+1(2-0.5(3))=3.5\ \mathrm{nM}\). (3) The new run can overwrite the earlier result, making it unclear which parameter set generated a retained file. Use distinct, informative output names or locations for results that must be kept separate. (4) Organization helps identify and rerun computational components, but it does not show that the equations, parameter values, or predictions adequately represent the biology.

Clinical Biostats