Why Reproducible Diagnostic Reporting Matters
A diagnostic plot can be informative when viewed beside the MICE run that produced it, yet difficult to verify months later if it has no run identifier, its source object is unclear, or the plotting code has been lost. Saving a picture alone does not establish which data, model settings, software, or run generated it. Nor does a directory full of images show which diagnostics someone actually examined.
The goal here is a reproducible record of diagnostic outputs, not a second lesson in how to interpret them. The earlier material on capturing and inspecting MICE run objects and reporting sample flow and imputation outputs provides context for what a run contains and how its results are described. Here, the practical task is to make diagnostic generation, saving, labeling, and review traceable to a particular run.
Define the Run Identity Before Saving Outputs
Assign a stable run identifier before exporting diagnostics. It should distinguish the run from other runs and remain attached to every output and review record. It may be a human-readable label, such as mice_main_20261024_01, or an opaque identifier linked to a separate manifest. Avoid identifiers that reveal confidential information.
The identifier is a label, not a substitute for provenance. The record should also identify the input-data version or checksum, relevant MICE settings, the random-number seed when available, software and package versions, and the date of the run. Include enough information to distinguish runs that used different data or settings. Do not assume that a seed alone guarantees identical results across different software versions, computational environments, or parallel execution settings.
For a MICE object, useful diagnostic outputs commonly include trace plots, density plots, strip plots, and numerical convergence summaries when those are produced by the workflow. For example, the R package mice provides plotting methods for a mids object, including plot(imp), densityplot(imp), and stripplot(imp). The exact objects returned and options available can depend on the installed package version. Record the function and relevant options actually used rather than relying on a remembered default.
A practical export pattern is to call the plotting method on the saved run object, write the result to a file named with the run identifier, and save a sidecar manifest that identifies the object, function, and software version. For lattice plots, explicitly printing the plot object while a graphics device is open helps ensure it is written to the file. Keep plotting code with the run artifacts, and test that each resulting file exists and can be opened.
Identify the saved MICE run object and its data and configuration provenance before generating diagnostic files.
Run the chosen plotting and summary functions against that object, using recorded options and the documented software environment.
Use filenames and captions that include the run identifier, and store a manifest linking each file to its source and generation method.
Check that expected files were created and open correctly, then separately record which outputs were examined and by whom or when, as appropriate.
Worked Examples of a Traceable Export
Worked Example: Naming a Diagnostic Set
Suppose a hypothetical analysis team has saved a MICE run object called imp_main. They plan to save four outputs: a trace plot, a density plot, a strip plot, and a numerical convergence summary. They assign the run identifier mice_main_20261024_01.
A consistent set of filenames could be mice_main_20261024_01_trace.pdf, mice_main_20261024_01_density.pdf, mice_main_20261024_01_strip.pdf, and mice_main_20261024_01_convergence.csv. The filename makes the run association visible, while the manifest records that the plots were generated from imp_main, names the plotting functions and options, and records the package version.
The team checks the expected count explicitly. There are three planned plot files and one planned summary file, so the expected total is \(3+1=4\) files. If only three are present, the set is incomplete; the missing file must be investigated rather than silently omitted from the report. This count checks file completeness only. It does not establish that the plots are correct or have been reviewed.
Worked Example: Recording What Was Examined
In a hypothetical review, an analyst generated six outputs for run mice_sensitivity_20261024_02: two trace plots, two density plots, one strip plot, and one convergence summary. The analyst opened and examined five outputs but did not examine the second density plot.
The examined-output fraction for this review is \(5/6\). Substituting the counts gives \(5\div6=0.8333\), or \(83.3\%\) to one decimal place. This is an administrative completeness measure, not a statistical diagnostic or a measure of imputation quality.
The review log should identify the five examined files and list the unexamined density plot explicitly, for example: “Not examined: mice_sensitivity_20261024_02_density_variableB.pdf.” It should also record the review date and analyst if those details are relevant to the workflow. Reporting “diagnostics reviewed” without specifying which outputs would incorrectly imply that all six were examined.
Worked Example: Distinguishing Two Runs
Suppose a hypothetical team reruns MICE after changing one model setting. The original run is mice_main_20261024_01; the rerun is mice_main_20261025_01. Both produce a file called trace.pdf if saved to the same directory.
Saving the second plot under the same name would overwrite the first, leaving no reliable way to tell which run it represents. Instead, save the outputs as mice_main_20261024_01_trace.pdf and mice_main_20261025_01_trace.pdf. The two manifests should separately record the setting used in each run and point to the corresponding saved run object.
The changed setting is part of the run provenance, not something to infer from the plot. A caption can say “Trace plot; run mice_main_20261025_01,” while the manifest supplies the setting, software version, input-data reference, and generation function. This combination preserves both an immediately readable label and the details needed to distinguish the runs.
What to Put in the Manifest
A compact manifest connects outputs to the run and supports later checks. It can be a text file, a table, or structured metadata stored beside the artifacts. For each output, record at least its filename, run identifier, output type, generation function, and generation date. For the run, record the saved-object reference, input-data version or checksum, relevant MICE configuration, seed if applicable, and software versions.
Separate generation records from review records. A file can be generated but not examined; a file can also be examined and then replaced by a later export. Record the reviewed filename or file checksum, reviewer or process identifier where appropriate, review date, and any note needed to explain exclusions. If outputs are regenerated, retain or clearly supersede the previous set instead of leaving two indistinguishable copies.
A checksum can help establish whether a file changed after it was generated. If a file has digest \(h_1\) when first recorded and digest \(h_2\) later, matching digests support the claim that the file contents are unchanged; different digests indicate that they differ. A matching checksum does not prove that the file was generated correctly, that it corresponds to the claimed run, or that anyone examined it. Those claims require the associated generation and review records.
Common Failure Modes and Alternatives
- Saving images without identifiers. “trace_final.pdf” is ambiguous when several runs exist. Include a stable run identifier in the filename and caption, and link it to a manifest.
- Treating file creation as review. An automated pipeline can export plots without a person ever opening them. Track generation and examination as separate events.
- Overwriting previous outputs. Reusing a generic filename can erase evidence of earlier runs. Use run-specific names or immutable, run-specific directories.
- Editing a plot without recording the change. Cropping or annotating an exported image can alter what a reviewer sees. Prefer regenerating the plot from code; if a manual change is necessary, preserve the original and document the modification.
- Assuming the default plot is self-documenting. A plot may not identify the object, function options, or package version used. Put that information in a caption or sidecar record.
- Reporting only selected outputs. A subset may be appropriate, but name the outputs examined and note any planned outputs that were not examined. Do not imply a complete review from a partial record.
A screenshot or manually assembled report can be convenient for discussion, but it is less reproducible if the underlying plot objects, source run, and generation settings are missing. A script that regenerates each file is generally a stronger alternative. Likewise, retaining only the script is not enough if the input object or environment needed to run it is unavailable. A robust record links saved artifacts, code, run provenance, and review documentation.
These practices have limits. A complete inventory cannot establish that the chosen diagnostics were sufficient for the analysis, and reproducible plotting does not guarantee valid imputation models. Diagnostic interpretation remains a separate statistical task. The purpose of the record is narrower: make clear what was generated, from which run, by what documented process, and what was examined.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- A run produces four planned diagnostic files, but only three exist. What should the analyst report or do?
- Why is a run identifier in a plot filename not a substitute for a manifest?
- An analyst generated five outputs and examined four. Calculate the examined-output fraction as a percentage, and state what it does and does not mean.
- Two runs save a plot under the same filename. What reproducibility problem can result, and how can it be avoided?
- Why should the review record distinguish outputs generated from outputs examined?
Solutions. (1) Report that the set is incomplete, identify the missing output, and investigate whether export failed or the plan changed; do not imply all four were saved. (2) The identifier distinguishes the run label, but the manifest records the source object, data reference, settings, software, and generation method needed to substantiate that association. (3) \(4/5=0.8\), so \(80\%\) were examined. This is a process-completeness count, not evidence of diagnostic adequacy or imputation quality. (4) The later file may overwrite the earlier one, obscuring which run produced the remaining file; use distinct run-specific filenames or directories. (5) Export can occur automatically without review, so treating generation as examination overstates what was checked.