Why Modularize an Analysis Script?
A long pharmacometric script may read files, check records, derive analysis variables, prepare a model dataset, fit a model, and produce summaries in one continuous sequence. Such a script can run successfully and still be difficult to review: it may be unclear which section depends on which input, where a value was changed, or whether a transformation can be checked separately from the full analysis.
Modularization addresses those problems by separating work into functions with clear responsibilities and scripts that coordinate particular tasks. The aim is not to create the largest possible number of files. It is to make each transformation understandable, reusable where appropriate, and testable without running unrelated parts of the workflow. This builds on the earlier material on Script Entry Points and Execution Order: an entry script establishes how work begins, while modular functions make the work it coordinates easier to inspect.
A function is easier to reason about when its result depends on its declared inputs. For example, a preparation function might receive a longitudinal dataset and a dose-unit conversion factor, then return a prepared dataset. If it instead reads an undeclared file, retrieves a conversion factor from a global variable, and overwrites an output file, the apparent interface does not describe what the function actually needs or does.
Choose Boundaries by Responsibility and Data Flow
A useful first step is to annotate the existing script by responsibility: input reading, validation, transformation, model-data preparation, model execution, and reporting. Then identify which blocks produce data used by later blocks. A block is a candidate function when its task can be named precisely and its required inputs and returned result can be stated.
For instance, “prepare concentration records” is more informative than “run lines 80–130.” A preparation function might take records and assay metadata and return records with a standardized concentration field. A validation function might take those records and return a clear pass result or a useful error. The function should not silently proceed with an invalid input merely because later code expects a certain column.
The following interface makes data flow explicit:
| Component | Inputs that should be visible | Output or effect |
|---|---|---|
| Record-standardization function | Records and the relevant assay metadata | Standardized records returned to the caller |
| Analysis-dataset function | Standardized records and specified derivation rules | Analysis dataset returned to the caller |
| Task-specific preparation script | Declared project inputs and the functions it calls | Coordinates preparation and makes its result available to the next task |
| Model-execution script | Prepared dataset and model specification | Coordinates the model run and subsequent review outputs |
The table describes responsibilities, not a requirement that every row become a separate file. A small, stable transformation may live beside related functions. The important test is whether a reviewer can follow the inputs and results without searching through unrelated work.
Worked Example: Splitting a Concentration-Data Preparation Block
Consider a hypothetical script that reads six assay records, converts reported concentrations from nanograms per milliliter to milligrams per liter, removes no records, and writes a prepared table. The original block uses a global variable called conversion_factor, modifies the input table in place, and writes a file before the transformation can be inspected.
The conversion is known for this example: 1 ng/mL equals 0.001 mg/L. The hypothetical input concentrations are 120, 80, 250, 40, 310, and 100 ng/mL. The corresponding converted values are 0.120, 0.080, 0.250, 0.040, 0.310, and 0.100 mg/L. For example, 120 × 0.001 = 0.120 mg/L.
A more inspectable design separates two responsibilities. A function such as standardize_concentration(records, factor) takes the records and the factor as arguments and returns a new table with a concentration_mg_L field. A task-specific preparation script obtains the records and factor from the declared project inputs, calls the function, and passes the returned table to the next step. The function does not choose a file path or write an output file.
Now a reviewer can see that the transformation depends on exactly two inputs. The function can be checked using a small table containing a single concentration of 120 ng/mL and factor 0.001; the expected returned value is 0.120 mg/L. The original six records can also be checked against the six values calculated above. The returned table has six rows because this transformation changes units but does not remove records. That row-count expectation is an explicit check, not an assumption hidden in the code.
If an assay-specific convention means some records require a different treatment, the relevant metadata or rule must also be an explicit input. It should not be retrieved from a global object under an undocumented name. Making that dependency visible helps distinguish a correct conversion from a function that happens to work only in one session.
Make Functions Testable Through Explicit Interfaces
A function’s interface is the practical contract between it and the rest of the workflow. State what inputs it accepts, what conditions must hold for those inputs, what it returns, and what happens when those conditions fail. For a data transformation, useful conditions might include required columns being present and a conversion factor being positive. Such checks belong close to the function that relies on them, even if a broader project validation step also checks the dataset.
A function that returns a transformed object is usually easier to test than one that changes a file or an object in the surrounding environment. This does not mean that writing files is never appropriate. It means that transformation logic and file-writing effects should not be coupled by default. The coordinating script can call a transformation, inspect or validate its result, and then decide when to write or pass that result onward.
For a deterministic function, a simple test compares its result on a small, specified input with an expected result. A test should check more than whether the function completes: it should verify meaningful values, row counts, types, or other required properties. If a function’s result changes because a global variable or ambient session state changed, the test has exposed an implicit dependency that should be made explicit.
Worked Example: Testing a Derived Analysis Field
Suppose a hypothetical analysis-dataset function derives time_after_dose_h from two numeric columns: sample_time_h and dose_time_h. It should return sample_time_h minus dose_time_h for each row, without changing the input. Assume the function’s interface requires both columns and returns the original fields plus the derived field.
Use a three-row test table with sample times 2, 8, and 14 hours and dose times 0, 6, and 12 hours. The expected derived values are 2 − 0 = 2 hours, 8 − 6 = 2 hours, and 14 − 12 = 2 hours. Thus the expected returned vector is (2, 2, 2), and the returned table should still have three rows.
The test procedure is to pass this table to the function, compare the returned derived vector with (2, 2, 2), confirm that its units are hours, and confirm that the row count remains three. Also compare the original input table with a saved copy: they should match, because this function is specified to return a result rather than mutate its input. Each check targets a stated part of the contract.
A second test checks a boundary condition: if sample_time_h is missing, the function should either report a clear error or follow a documented missing-value rule. It must not silently substitute zero, since zero would imply a specific time rather than an unknown one. The expected behavior should be chosen according to the analysis specification and tested accordingly.
Task-Specific Scripts Coordinate; Functions Do the Reusable Work
A task-specific script should be readable as a sequence of meaningful actions. It may call a function to read or access declared inputs, call validation functions, call transformation functions, and hand their returned objects to later steps. Reviewers should be able to tell which data object flows into each function and which result is used next.
Avoid two opposite designs. In a monolithic script, every transformation is entangled with execution and output effects, so a small change can require running and reviewing the entire workflow. In an excessively fragmented design, each trivial operation has its own wrapper and file, making the actual sequence harder to follow. A function boundary is useful when it gives a coherent task a clear interface or permits meaningful independent checking, not merely because a block has a few lines.
Worked Example: Organizing a Preparation Task and a Modeling Task
Imagine a hypothetical project with a raw longitudinal dataset containing 12 participants and 36 concentration records. The preparation task standardizes units, checks required fields, derives time after dose, and returns a model-ready dataset. The modeling task accepts that dataset and a model specification, then coordinates model execution and review of the resulting output.
A preparation script can be read in this order: obtain the declared raw dataset and assay metadata; call the record-standardization function; call the required-field validation; call the analysis-dataset function; confirm that the returned dataset has 36 records when no filtering rule applies; and make that prepared object available to the modeling task. Each call names its input and receives a result. If the validation fails, later steps should not proceed as if preparation succeeded.
The model-execution script begins with the prepared dataset and model specification, then calls the model-running function and the relevant review functions. The 36-record count is not asserted if a documented exclusion rule changes the population; in that case, the expected count must reflect the declared rule. This distinction keeps a check from becoming an incorrect hard-coded expectation.
The functions that standardize units or derive time may be reused by another task if the same rules apply. The two scripts remain task-specific: one coordinates dataset preparation, and the other coordinates modeling. A reviewer can inspect the dependency chain without needing one all-purpose script that mixes data preparation, model execution, and reporting.
Justification: Why Explicit Inputs Improve Repeatability
The reason for exposing dependencies can be stated directly. If a transformation is fully described by an input object \(x\) and a parameter \(\theta\), its intended result is \(y=f(x,\theta)\). Given the same \(x\), the same \(\theta\), and deterministic behavior, calling the function again should produce the same \(y\). This lets a reviewer reproduce the transformation using only the declared inputs.
Suppose instead the function also reads an undeclared global value \(g\), so its actual behavior is \(y=f(x,\theta,g)\), although the interface suggests only \(x\) and \(\theta\). Two runs with identical declared inputs but different global values \(g_1\) and \(g_2\) can produce different results: \(f(x,\theta,g_1) \ne f(x,\theta,g_2)\). Therefore, the declared interface is insufficient to reproduce the result. Passing \(g\) explicitly, or making its role otherwise visible in the calling task, restores an inspectable dependency boundary.
This argument does not claim every function must be mathematically pure. File access, model execution, and reporting can require side effects. Rather, it shows why separating the transformation from those effects, when practical, makes its behavior easier to test and review.
Diagnostics, Failure Modes, and Limits
After refactoring, review both the code boundaries and the behavior. A script that runs without errors is not enough evidence that its data flow is correct. Check that function arguments correspond to genuine dependencies, returned objects are actually used, and required checks can stop downstream work when they fail. Small tests should exercise ordinary cases and important boundary cases, including missing fields or missing values where relevant.
- Hidden state remains. A function accepts a dataset but reads a global setting or file. Correction: make the dependency explicit and test the function with that input supplied.
- Inputs are modified unexpectedly. Later steps may receive changed data without a visible assignment. Correction: specify whether the function returns a new object or intentionally mutates one, and test that contract.
- A function does too much. It transforms data, runs a model, and writes several outputs. Correction: separate distinct responsibilities so each result and effect can be reviewed.
- Functions are too small or numerous. A script becomes a trail of trivial wrappers with little meaningful reuse or independent test value. Correction: combine tightly related operations when that makes the task clearer.
- Tests check only successful completion. A wrong conversion or dropped row may go unnoticed. Correction: assert specific expected values and structural properties, such as units, required columns, and row counts.
Modularization cannot establish that the scientific assumptions, dataset definitions, or model specification are appropriate. It also does not by itself guarantee that a run is reproducible across different software environments. Its narrower contribution is to expose how a workflow is organized and how each step receives and returns information. Clear boundaries make errors easier to localize; they do not make an incorrect rule correct.
A practical review asks whether each function has a clear purpose, whether its inputs and outputs are visible, whether failure behavior is defined, and whether the coordinating scripts tell a comprehensible story. If those questions cannot be answered, the refactoring is not yet complete.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- A transformation reads a dataset argument but also uses a global conversion factor. What dependency is hidden, and how should the interface change?
- A function converts 75 ng/mL to mg/L using a factor of 0.001. What value should a test expect, and what unit should it report?
- A function derives a time difference for four input records and applies no filtering. What row-count property should a test check?
- What is the main difference between a task-specific script and a reusable transformation function?
- A refactored project has a separate function for every single arithmetic operation. Why might this be a poor modular design?
Solutions. (1) The conversion factor is an undeclared dependency. Pass it as an explicit argument or otherwise make its source and use visible in the calling task. (2) \(75 \times 0.001 = 0.075\) mg/L, so the test should expect 0.075 mg/L. (3) The returned dataset should still contain four records, since the function is specified not to filter. (4) A function performs a defined operation on explicit inputs and returns a result; a task-specific script coordinates functions in the order needed for a particular workflow task. (5) Excessive fragmentation can obscure the actual workflow and add wrappers without useful independent responsibility or test value; combine tightly related operations where that improves clarity.