Packaging an Auditable NLME Analysis

From Analysis Results to a Reviewable Package

A fitted NLME model is not, by itself, an auditable analysis. A reviewer also needs to know which data and code produced the fit, how the run was configured, where its outputs are stored, and how a reported number was taken from those outputs. Packaging makes those links visible and preserves the materials needed to inspect them.

An auditable package is not necessarily a guarantee that every computer will reproduce every output bit for bit. Software versions, numerical libraries, hardware, and computational settings can affect a rerun. The practical goal is to preserve the relevant artifacts and records, explain the environment and execution, and provide a traceable route from report claim to its source. The package should let a reviewer distinguish what was actually used from what is merely a later or similar version.

Definition: An auditable NLME analysis package is an organized set of identified and versioned analysis artifacts, execution records, results, and instructions that allows a reviewer to trace a reported result to its originating output and the data, code, and settings used to produce it. A package is traceable when each link in that path is explicit and checkable; it is not enough for files to be present without their roles and relationships being stated.

The package assembles materials whose individual content is covered in the earlier material on traceable datasets, model code, run registries, computational provenance, and reporting. Its distinct task is to make these components work together as one reviewable deliverable. A reviewer should not have to infer which of several data exports, model scripts, or output folders is authoritative.

Define the Package and Its Links

A useful package contains, at minimum, the following related components. Their exact file formats may vary, but each role should be identifiable.

Report: the approved or review-ready report, including its version and date. It should identify the final run and point to the relevant tables, figures, or output files.
Analysis dataset documentation: the dataset identifier, source and transformations, record structure, variable definitions, units, coding conventions, exclusions, and the exact data artifact used by the run. Include the dataset file or a controlled reference to it, subject to access and confidentiality requirements.
Model code and configuration: the exact model implementation and configuration files, with versions or immutable identifiers. Include any required supporting code that affects the run.
Run record: a unique run identifier and its links to data, code, settings, software environment, execution status, and outputs. Record failures and reruns distinctly rather than silently replacing them.
Outputs: the files used for assessment and reporting, such as parameter estimates, diagnostic outputs, run logs, and generated tables or figures. Identify which outputs are authoritative for each claim.
Review instructions: concise steps for locating artifacts, checking their identifiers, understanding the folder structure, and, where feasible, executing a rerun or inspecting the recorded outputs.

A manifest is a compact index of the package. For each artifact it can record a stable name, role, version or identifier, relative path, and checksum. A checksum is a value computed from a file's contents; comparing it with the recorded value can reveal whether the file has changed. A checksum does not establish that the file is correct, scientifically appropriate, or complete. Those questions require review of the contents and the links among artifacts.

\text{report claim} \longrightarrow \text{reported table or figure} \longrightarrow \text{run output} \longrightarrow \text{run record} \longrightarrow \text{data, code, and settings}

Each arrow must be supported by an identifier or an instruction that removes ambiguity. The relationship is broken if, for example, the report names a run but the run record does not identify the dataset version, or the record points to code that is no longer available. A package can still be useful when some artifacts cannot be distributed, but it must state what is withheld, why, and how an authorized reviewer can access or verify it.

Assemble and Check the Lineage

The following sequence is a practical assembly and review procedure. It emphasizes relationships rather than a particular directory layout.

Choose the deliverable boundary.
State which report version and final run the package supports. Identify any excluded or restricted materials and the route for authorized access.

Inventory the exact artifacts.
List the data, model code, configuration, run record, outputs, report, and instructions. Record identifiers and checksums where appropriate; do not substitute a “current” file for the version actually used.

Connect claims to sources.
For each important reported number, name its report location, source output, run identifier, and the data and code identifiers recorded for that run.

Check completeness and consistency.
Confirm that references resolve, identifiers agree across the manifest and run record, outputs belong to the named run, and instructions match the package as delivered.

Test the review path.
Have a reviewer who did not assemble the package follow a sample claim from the report back to its source. Record unresolved gaps rather than filling them by assumption.

Worked Example: Building a Package Inventory

Suppose a hypothetical analysis report is based on run R042. The team plans to deliver the report, one dataset artifact and its documentation, model code, a configuration file, a run record, outputs, and review instructions. The inventory should identify each item and its role rather than simply list filenames.

Artifact	Identifier	Role
Final report	REP-03	Contains reported results and source references
Analysis dataset	DATA-07	Exact data artifact used for R042
Dataset documentation	DOC-07	Defines fields, units, derivations, and exclusions
Model code and configuration	CODE-12, CFG-04	Implementation and run settings
Run record	R042	Links inputs, environment, execution, and outputs
Output bundle	OUT-042	Contains the outputs cited in REP-03
Review instructions	GUIDE-01	Explains how to trace and inspect the artifacts

There are seven inventory entries. If a completeness check finds only six, the observed inventory coverage is \(6/7=0.8571\), or 85.71%. That percentage is only a simple check against this stated list; it does not measure scientific quality. More importantly, the missing entry must be named. If it is the run record, a reviewer may be unable to verify that DATA-07 and CODE-12 were the inputs for the reported run, even if the other six artifacts are present.

For each entry, the manifest can also store a relative path and a checksum. During review, the checksum can be recomputed and compared with the recorded value to detect an altered file. The reviewer should still verify that the manifest's identifiers and the contents of the run record agree. An internally consistent list can faithfully describe an incorrect or unintended set of files.

Trace Reported Results to Their Source

A report-to-source map makes the package useful for substantive review. It should be specific enough that two reviewers looking at the same package select the same output and understand the same transformation. A concise entry might name the report section and table cell, the output file and field, the run ID, and any display transformation such as rounding or unit conversion. If a table combines values from multiple runs, the mapping should identify the source for each relevant value rather than cite one run for the whole table.

Worked Example: Tracing a Reported Parameter Estimate

In a hypothetical report, Table 4 displays typical clearance as 4.80 L/h. Its source map says that the number comes from the “typical clearance estimate” field in the parameter output for run R042. The run record identifies DATA-07, CODE-12, and CFG-04 as the inputs and OUT-042 as the output bundle. The output gives 4.803 L/h.

The report uses two decimal places. Applying that display rule gives \(4.803\ \text{L/h}\rightarrow4.80\ \text{L/h}\), since the third digit after the decimal point is 3 and does not increase the second digit. The units in the output and report agree, so no unit conversion is needed. A reviewer can now check the exact output field, confirm the run linkage, and verify the displayed value.

If the output instead recorded clearance in mL/h, the map would need to state the conversion and its arithmetic. For example, \(4{,}803\ \text{mL/h}\div1{,}000=4.803\ \text{L/h}\), which rounds to \(4.80\ \text{L/h}\). Omitting either the unit or the conversion would leave an important part of the reported result unexplained.

This procedure is a short derivation of the traceability requirement: a displayed claim can be checked only if its source value, source run, and display transformation are all identifiable. If any one is missing, the reviewer cannot determine from the package alone whether the report accurately represents the run output. Stating the chain makes the claim testable; it does not, by itself, establish that the model or interpretation is appropriate.

Worked Example: Detecting an Input-Version Mismatch

Consider a hypothetical run record for R042. It specifies dataset DATA-07 and code CODE-12. The package manifest includes DATA-07 and a newer dataset DATA-08, while the report's source note says only “final analysis dataset.” A reviewer finds that DATA-08 has a different checksum from DATA-07.

The record provides evidence that R042 used DATA-07, not DATA-08. The checksum difference confirms that the two files are not byte-for-byte identical; it does not explain the substantive differences between them. The report note is therefore too vague to distinguish the used data from the later file. The appropriate action is to clarify the report's source reference and preserve the documented R042 linkage, not to assume that DATA-08 was used because it is newer.

If the run record itself named DATA-08 while the execution log and archived input file identified DATA-07, the package would contain a provenance conflict. The reviewer should flag it for resolution and record the evidence and disposition. Choosing whichever version seems more plausible would create an undocumented decision and undermine the audit trail.

Diagnostics, Alternatives, and Limits

A package review should look for failures in the links as well as missing files. Common warning signs include report tables with no source references, run identifiers that differ between the report and output folder, a code or data version absent from the manifest, instructions that refer to paths not present in the delivered package, and checksums that do not match. Another warning sign is an output file whose creation date appears later than the report but whose relationship to the cited run is not explained. Dates can prompt investigation, but they are not a substitute for identifiers and records.

The main alternative to a packaged, indexed deliverable is an informal handoff: files attached to an email, a shared directory described as containing “the final analysis,” or a report accompanied by a single code file. Such a handoff may be sufficient for a narrow informal discussion, but ambiguity increases when there are multiple revisions or candidate runs. A package with a manifest and a claim-to-source map is more reviewable because it states which artifacts belong together and how to verify that relationship.

A second practical alternative is to provide a rerunnable computational environment, such as a container or a locked dependency specification. This can strengthen the package by documenting or preserving software dependencies, but it complements rather than replaces the run record and output archive. A rerun may differ because of hardware, stochastic procedures, numerical libraries, or other execution details. If exact byte-for-byte reproduction is not demonstrated, describe the intended rerun procedure and the expected level of agreement without promising more than has been tested.

Packaging also has limits. A complete, internally consistent package cannot prove that the analysis question was well chosen, that the model is scientifically adequate, or that an interpretation is justified. Those questions require substantive review. Confidentiality constraints may restrict data distribution; access controls or a secure review environment may be necessary. Any restriction should be explicit, since an unavailable artifact is a real limit on independent verification.

Mistake: “The folder contains a dataset, code, and output, so the result is auditable.”
Correction: Presence is not lineage. Identify which versions were used, connect them to the run, and connect that run's output to the report claim.
Mistake: “The checksum matches, so the dataset is correct.”
Correction: A matching checksum supports file identity relative to the recorded checksum. It does not validate the dataset's contents, provenance, or suitability.
Mistake: “The package is reproducible because it includes code.”
Correction: Code alone omits data, configuration, software environment, execution record, and often the output used in the report. State what was preserved and what level of rerun agreement has actually been checked.

Key takeaway: A review-ready NLME package is organized around explicit links. For each reported result, a reviewer should be able to identify its source output, the run that generated it, the exact inputs and settings recorded for that run, and any transformation used to display it.

Check Your Understanding

Answer each question, then compare with the solutions below.

A report cites run R021, but its run record identifies code version C05 and the package contains only C06. What can the reviewer conclude, and what should be resolved?
A parameter output contains an estimate of 2.746 mg/L, and the report displays 2.75 mg/L. Is the rounding consistent? What should the source map identify?
What does a matching checksum establish, and what does it not establish?
Why is a rerunnable environment useful but not a replacement for archived outputs and a run record?

Solutions. (1) The package does not contain the code version recorded for R021, so the reviewer cannot verify that input from the delivered files. The team should locate and identify C05 or document the discrepancy and its resolution; it should not silently substitute C06. (2) Yes. The third decimal digit is 6, so 2.746 mg/L rounds to 2.75 mg/L. The map should identify the report location, output file and field, run ID, units, and rounding rule. (3) A matching checksum supports that the file's contents match the file represented by the recorded checksum. It does not show that the file is correct, complete, or appropriate. (4) An environment can support an attempted rerun, but computational differences can affect results, and code execution alone does not show which output was used for the report. The run record and archived outputs preserve that evidence and enable comparison.