From Scattered Files to a Reviewable Project
A project can contain correct scripts and still be difficult to review. Files may be spread across a desktop folder, an email attachment, and a shared drive; settings may exist in both a script and a separate note; and an output may have a filename that does not reveal which run produced it. In that state, a reviewer cannot reliably tell what to run, which files are authoritative, or how a reported result was generated.
This workshop uses a hypothetical pharmacometric analysis project to show how to turn a fragmented collection into a navigable repository. The goal is not to make every project use the same directory names. It is to make the responsibilities of the files, the intended execution order, and the route from a result to its recorded provenance apparent and checkable.
The restructuring assumes that the team can identify the intended analysis files and has permission to place each item in the proposed location. If the source of a script, input, or output is uncertain, do not resolve that uncertainty by guessing which copy is current. Record the uncertainty and ask the responsible analyst. Likewise, first check data-handling requirements: sensitive data should not be copied into a repository merely to make the project appear self-contained.
Inventory Before Moving Files
Begin with an inventory, not a mass rename. Consider this hypothetical project, assembled from a working directory, a shared folder, and an analyst’s notes:
- Working directory: fit_model.R, summarize_fit.R, helpers.R, and config-final.yml.
- Shared folder: analysis-data.csv, model-results.csv, and a folder named output2.
- Desktop: config-final-v3.yml and notes.txt, including a command used to run the model.
- output2: an execution log and several tables, but no record identifying which configuration produced them.
The filenames suggest that some files have distinct roles, but they do not establish which configuration is authoritative or whether the output folder contains the intended run. List candidate files with their location, role, owner or source, and disposition. Disposition means a specific decision: keep as a maintained project file, provide as a controlled input, move to a run record or artifact store, or exclude. Resolve conflicting versions before changing the execution path.
Worked Example: Classifying a Fragmented File Set
Suppose the inventory contains the four working-directory scripts, two YAML files, one data file, one results table, one log, and two notes files. That is \(4 + 2 + 1 + 1 + 1 + 2 = 11\) candidate items. The count is only an inventory check; it does not imply that all 11 belong in version control.
The team confirms that fit_model.R and summarize_fit.R are the intended analysis scripts, helpers.R contains functions they both use, and config-final-v3.yml is the configuration used for the run discussed in the notes. The older config-final.yml is retained temporarily outside the active project for comparison, rather than silently treated as current. The data file is confirmed to be a controlled, non-sensitive input available to authorized analysts; if that confirmation failed, the project would document how to obtain it without committing it.
The results table and log are generated run artifacts, not maintained source files. The team preserves the relevant run records in an approved artifact location and records a stable reference to them in the project. The notes are reviewed: the exact run command and an explanation of the input belong in maintained documentation; personal reminders and unverified claims do not. Every item now has a disposition, so moving files does not erase the distinction between source, input, and output.
Choose Structure to Expose Responsibilities
A practical structure for this hypothetical project might look like this:
README.md explains prerequisites, input access, the supported run command, and where outputs are written.
config/base.yml holds the maintained run settings; scripts/ contains the ordered scripts; R/ contains reusable functions called by those scripts.
decisions/ holds the decision log; the environment lock file records the pinned software environment.
data/README.md explains controlled input access; runs/ holds run-specific records or stable pointers, with large or restricted artifacts stored according to project policy.
The directory names are less important than the boundaries. A reviewer should be able to distinguish maintained source from run-specific output and understand how configuration is supplied. A README should describe the actual structure and execution command, not a hoped-for workflow. The earlier material on modular scripts, externalized run configuration, README documentation, and run folders provides the relevant conventions; this workshop applies them to one assembled project rather than redefining them.
Avoid duplicating an authoritative setting in several places. If a configuration value is intentionally overridden for a particular run, the run record should identify that override. Otherwise, two apparently valid settings files create ambiguity about which value the scripts consumed. Similarly, do not move every historical output into the repository simply because it is available. Keep records that support review, but store bulky, restricted, or replaceable generated files in an approved location and make their relationship to the run explicit.
Worked Example: Mapping Files to Their New Roles
The team places fit_model.R at scripts/02_fit_model.R and summarize_fit.R at scripts/03_summarize_fit.R, making the order visible in the names. helpers.R moves to R/helpers.R. The confirmed configuration becomes config/base.yml. README.md documents that the controlled input is obtained through the approved data location and is read as data/analysis-data.csv during execution.
The project entry point is a documented command, for example, “Rscript scripts/run_analysis.R --config config/base.yml --run-id run-2026-04-12-001”. The wrapper loads the configuration, checks that the expected input exists, calls the preparation and fitting steps in their documented order, then writes summaries and a run manifest under the specified run identifier. If no preparation step exists in the project, the wrapper must not pretend that one does: its real sequence should be documented instead.
Before moving on, the analyst checks that every path used by the scripts refers to the new structure, that the command does not depend on a desktop working directory, and that generated output goes to the designated run location. A directory tree alone is not evidence of a working project; the references inside the code and configuration must agree with it.
Make the Execution Path Explicit
An execution path is the ordered sequence that takes a stated input and configuration to a stated output. A simple dependency representation is a directed graph: each file or step is a node, and an arrow \(A \rightarrow B\) means that step B uses or depends on A. A valid run order must respect every such dependency. If the fitting script calls a function in R/helpers.R, for example, the function must be available before or during that fitting step.
For the example project, the path might be: obtain the controlled input; load config/base.yml; validate required settings and input paths; load R/helpers.R; prepare the analysis data if that step is part of the project; fit the model; write the model output and log; summarize the result; and write the run manifest. The README should say what command starts this sequence, while the scripts should implement it. A command copied into documentation but not tested against the reorganized files is only an assertion, not a verified entry point.
Worked Example: Checking a Run Path From a Clean Checkout
A reviewer checks out the hypothetical repository into a new directory and runs the documented command with run identifier run-2026-04-12-001. The configuration points to the approved input path, the wrapper reports that the input is present, and the run creates a log, a model output, a summary table, and a manifest in the corresponding run folder.
The reviewer follows the execution chain rather than judging success from the existence of a final table. The log records the command and the order of steps. The summary script reads the output created by the fitting step, not a stale table in the original shared folder. The manifest records the configuration and code revision. The reviewer also checks that output files are newly created for this run identifier and are not silently read back as inputs.
If the command fails because the input is unavailable, the result is not a successful clean-checkout reproduction. The project may still be reviewable if the input is controlled and the access procedure is documented, but the reviewer must record that the run depends on an external input. If the command succeeds only after manually editing an absolute path, that manual edit is an undocumented dependency and must be removed or documented and controlled before claiming the workflow is portable.
Trace a Result to Its Run Record
A result is not traceable merely because it appears in a table with a plausible filename. The record should let a reviewer connect the reported value to a specific output and then identify the conditions that produced that output. At minimum, record a unique run identifier, execution time, code revision, configuration reference, input reference, software environment, command or entry point, and relevant log or artifact locations. The earlier material on recording run provenance and repository traceability establishes these elements; here they are checked as a linked chain.
Worked Example: Tracing a Hypothetical Clearance Estimate
Suppose the hypothetical summary file runs/2026-04-12-001/summary/parameter_estimates.csv contains the row “CL, 4.82, L/h”. The report cites this value as the estimate from run-2026-04-12-001. The reviewer first checks that the cited run identifier matches the summary file’s run folder and that the row and units are present exactly as reported.
The reviewer then opens runs/2026-04-12-001/manifest.yml. In this hypothetical record, it identifies the summary file, the model output used by the summary script, the command that started the workflow, commit 7ac91ef, config/base.yml, the controlled input’s recorded checksum, the locked software environment, and the log location. The commit identifies the code revision; the configuration and input references identify the run settings and data; the environment record identifies the software specification. The log shows the fitting step completed before the summary step.
The trace is therefore: reported CL value and units → summary table and run identifier → manifest and log → model output, configuration, input reference, code revision, and environment record. The reviewer can verify that chain without claiming that the estimate is scientifically appropriate; scientific interpretation is a separate question. In a real project, abbreviated commit identifiers or checksums are useful only when they resolve unambiguously to the full recorded value.
Diagnostics, Alternatives, and Limits
Several checks reveal whether the restructuring has gone wrong. A script that still refers to an old folder, a configuration that disagrees with the README, or an output that lacks a matching run identifier indicates a broken boundary or execution path. A manifest with no input reference, an environment record that does not match the lock file, or a log that ends before the summary step leaves a gap in the provenance chain. A clean checkout that requires undocumented manual edits is evidence that the workflow is not yet portable as described.
A common alternative is to organize everything by file type: all scripts in one folder, all tables in another, and all notes in a third, without distinguishing maintained files from run-specific files. That can be convenient for browsing, but it often hides the relationship between one run’s configuration, output, and log. Another approach is to create a separate, self-contained folder for every run. That makes run-level grouping obvious, but it can duplicate scripts and settings and make it unclear which copy is maintained. A repository with shared maintained code and configuration plus run-specific records often makes the responsibilities clearer, provided each run points back to the exact code and settings it used.
There is no single directory layout that fits every team. Storage constraints, access controls, external modeling software, and controlled data systems may require some inputs or outputs to remain outside Git. In that case, record stable identifiers and access instructions rather than copying restricted files. A recorded environment specification supports reproducibility but does not guarantee that external services, licenses, hardware, or operating-system behavior remain available. Likewise, a successful rerun does not establish that the underlying model or analysis choices are scientifically suitable.
The main pitfall is treating a tidy folder tree as proof of reproducibility. The wrong reasoning is, “The scripts, config, and output are all here, so the result can be reproduced.” The correction is to test the documented command from a clean checkout, inspect the actual input and output references, and follow the manifest back to the recorded code, configuration, and environment. Structure makes review possible; execution and trace checks provide evidence about what the project can actually reproduce.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- A configuration file in a desktop folder conflicts with one in the project. What should the analyst do before choosing the active file?
- A clean checkout runs only after a reviewer edits an absolute input path in a script. What does this show, and what should be changed?
- A report cites a value in a summary table, but the table has no run identifier and no manifest points to it. Is the value traceable? Explain.
- Why might a team store a generated model output outside the repository while keeping a run record in the project?
Solutions. (1) Compare the files’ source and use, establish which configuration produced the intended run, and resolve or record the discrepancy before restructuring; do not infer authority from a filename such as “final.” (2) The documented workflow has an uncontrolled path dependency and has not passed a clean-checkout test as written. Use a controlled, documented input path or access mechanism, update the project references, and rerun the documented command without an unrecorded edit. (3) No. The table’s value cannot be connected reliably to a particular run or to its input, configuration, code, and environment. Add a stable run identifier and a run record that links the output to those provenance elements. (4) Generated outputs may be large, sensitive, or governed by a separate artifact policy. A run record can preserve the output’s location and identity, together with the run conditions, without committing the artifact itself to version control.