Reproducibility Without Releasing Records
A reproducible analysis should make it possible to understand and, where authorized, rerun the workflow. It does not follow that every file used to produce the results should be publicly distributed. In a MICE analysis, the code may be shareable while the input records, completed datasets, diagnostic traces, or saved objects remain confidential. A release is safe only when its contents and the risks of combining them with other information have been considered.
The practical task is to separate the workflow from the confidential data it processes. Review not just the final tables, but also scripts, logs, saved objects, example datasets, temporary files, and file metadata. Imputation does not anonymize a dataset: observed values are generally retained, and completed datasets may contain a person's observed values alongside imputed values.
A useful review begins by specifying a threat model: what materials will be released, who can access them, what external information a recipient might have, and what consequences disclosure would have. Risk depends on this setting. A public code repository, a controlled-access data enclave, and a private project folder do not have the same audience or protections. No single screening rule establishes that every release is safe; applicable law, institutional policy, and data-use agreements still govern what may be shared.
Where Confidentiality Can Fail
A record can be exposed even when names have been removed. Combinations of quasi-identifiers may single out a person in a small cohort, and a recipient may link a rare combination to an external source. An unusual missingness pattern or a rare category can also be revealing. Removing one identifier is not equivalent to assessing the remaining information.
MICE-related files deserve particular attention. A saved object may retain the original incomplete data, completed values, model information, or other details depending on the software and how it was saved. Logs can capture file paths, error messages, record-level output, or values printed during debugging. An example dataset copied from the analysis data can expose real records even if it contains only a few rows. Tables and plots may disclose small groups or unusual values. Review the actual files and software behavior; do not assume that a file is harmless based on its extension or filename.
A simple calculation helps illustrate one narrow kind of risk. Suppose a recipient knows a person's quasi-identifier combination, and that combination appears in a released file. If it occurs for \(k\) records and the recipient has no further distinguishing information, a simplifying assumption is that each of those \(k\) records is equally plausible as the match. The chance of selecting the correct record under that assumption is \(1/k\). This is only an illustration of ambiguity, not a general privacy guarantee: auxiliary information, unequal plausibility, sensitive attributes shared by a group, or other disclosure routes can change the risk.
Worked Example: A Unique Combination in a Released Table
Hypothetical data: A project has 120 participants. A proposed example file contains age band and district. The combination “70–79 years, North District” occurs once in the file. A separate public directory is known to contain the age band and district of some residents.
The combination is a quasi-identifier because it may be linked to information outside the file. Its class size is \(k=1\). Under the simple matching calculation, the match probability conditional on a correct match being among the records is \(1/k=1/1=1\), or 100%. This does not establish that the public directory actually identifies this participant; it shows that the released combination provides no ambiguity within this file if the combination is known.
If a revised grouping makes the combination occur for 4 records, the simplified calculation becomes \(1/4=0.25\), or 25%, under the equal-plausibility assumption. That is more ambiguity in this narrow calculation, not proof of safety. A reviewer would still check whether the four participants share a sensitive attribute, whether another variable distinguishes one of them, and whether the combination is permitted for release. If not, remove or coarsen the variables, restrict access, or omit the file.
Review the Whole Release, Not Just the Script
A practical review inventories both the release package and the files produced while running the workflow. Check version-control changes and export directories, not only the intended final deliverables. Confirm that the proposed example data were generated for demonstration or otherwise approved for release; a small extract of real records is still real data. Check filenames, comments, console output, error messages, and metadata for names, record keys, paths that reveal sensitive project details, or values copied into debugging statements.
List exactly which scripts, documentation, tables, logs, objects, and data files recipients will receive, and identify the intended audience and access conditions.
Determine which files can contain raw records, completed values, identifiers, or record-level diagnostics. Inspect saved objects and logs using the tools and software behavior relevant to the workflow.
Check direct identifiers, combinations of quasi-identifiers, sensitive values, rare patterns, small groups, and information that may be inferred by linking files with external sources.
Keep confidential inputs in an approved location; provide shareable code and instructions that refer to a user-supplied protected input, and use an independently created, non-confidential fixture for tests and demonstrations.
Release only reviewed files. Rerun the workflow in the intended configuration, inspect the resulting files and package contents, and confirm that temporary or confidential artifacts have not been copied into the release.
Worked Example: Finding a Leak in a Workflow Log
Hypothetical situation: A team intends to share a log showing that a MICE run completed. During debugging, the script printed the first two rows of the input data. Those rows contain a participant's record number, age, and a rare diagnosis. The log also contains an error message with the full path to the confidential input file.
The log contains a direct identifier (the record number), a sensitive attribute (the diagnosis), and a path that reveals project details. Removing the record number alone would not resolve the problem: the diagnosis and accompanying age could still contribute to identification or disclose sensitive information. The intended purpose—documenting successful execution—does not require any of these record-level details.
The team should remove the exposed log from the release, including any copies or version-history entries that recipients could access, and revise the script so that routine logging reports non-record-level status only. For example, a log can state that the run completed and report an approved summary, without printing rows or sensitive paths. The revised workflow should then be rerun and its new log inspected before release. A summary is not automatically safe merely because it is aggregated; small groups and unusual values still require review.
Worked Example: Choosing What to Share From a MICE Run
Hypothetical situation: A project directory contains a script, an incomplete dataset, a saved MICE object, a diagnostic plot, a completed dataset, and a small demonstration file. The team wants others to inspect the method and run the code, but the project records are restricted.
The script and documentation may be candidates for release after review for embedded values, private paths, credentials, or other sensitive details. The incomplete dataset and completed dataset contain participant-level information and should not be placed in a public package without specific authorization and a suitable disclosure assessment. The saved MICE object also requires inspection: depending on its contents, it may retain original data or completed values, so treating it as “just a model” would be unsafe.
The diagnostic plot must be checked for record-level points, readable labels, rare values, and small groups. The demonstration file is suitable only if it was created without confidential records or separately cleared for release; renaming or sampling a confidential file does not make it safe. A shareable setup can instruct authorized users to provide the restricted input from an approved location, run the released script, and store outputs in a local protected directory. This preserves a reproducible workflow without bundling the confidential records.
Designing a Shareable and Reproducible Workflow
A sound release separates the analysis specification from the analysis records. Share the reviewed code, dependency and version information, instructions, and a safe fixture that exercises the expected workflow. Keep confidential input and record-level outputs out of the distribution. The fixture should be generated independently for software testing or use values that are demonstrably non-confidential; simulated data are useful for showing structure, but they do not reproduce the actual results from protected records.
Reproducibility can mean that another authorized analyst can rerun the code with access to the same protected data under approved conditions. It need not mean that every reader can download those data. State clearly which inputs are restricted, how an authorized user supplies them, and which outputs are safe to share. If an output is meant to be public, define a review and approval step for it rather than relying on a developer's intention or a script's default save behavior.
Data minimization is a useful design principle: avoid loading, copying, printing, or saving information that the shared workflow does not need. Use separate protected input and release directories, and explicitly select files for distribution rather than copying an entire working directory. Keep credentials and access tokens outside scripts. When diagnostics or summaries are to be released, inspect their granularity and consider whether small counts, rare categories, or combinations reveal information.
Common Mistakes and Limitations
- “The names are gone, so the data are anonymous.” Quasi-identifiers can support linkage, and unusual combinations may be distinctive. Review combinations and external information, not just names.
- “Imputation creates new values, so the completed data are safe.” MICE usually retains observed values as well as imputing missing ones. Completed datasets remain record-level data and require the same confidentiality controls unless an approved assessment establishes otherwise.
- “A saved object is not a dataset.” A software object can contain data or other sensitive state. Inspect its contents and verify what the chosen software saves.
- “A log is only technical output.” Logs can contain printed records, sensitive error context, or revealing file paths. Keep routine logs minimal and review them before sharing.
- “A synthetic example is automatically private.” Independently simulated data are often useful for demonstration, but data generated from confidential records can still disclose information. Their release needs its own assessment.
Simple uniqueness checks and small-cell reviews are screening tools, not comprehensive guarantees. Risk depends on the audience, access controls, external data, applicable rules, and the consequences of disclosure. Where the consequences are serious or the release is uncertain, consult the responsible privacy, security, or data-governance authority rather than inferring permission from a technical check.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- A released file has no names, but one row has a rare combination of age band and district. What kind of information could create a disclosure risk?
- Why should a saved MICE object be inspected before release rather than assumed to contain only model settings?
- A log prints a record number and diagnosis. What should be done before any log is shared?
- What is one way to support reproducibility when the input dataset must remain restricted?
- Does a uniqueness calculation by itself establish that a release is safe? Explain.
Solutions. 1. Age band and district are quasi-identifiers that may be linked with external information; a rare combination can make a participant easier to single out. 2. The object may retain original data, completed values, or other sensitive state, depending on how the software saves it. 3. Keep that log out of the release, remove accessible copies as appropriate, revise logging to avoid record-level output, and inspect a rerun before sharing. 4. Share reviewed code and instructions that let authorized users supply the restricted input locally, and provide a non-confidential fixture for demonstration or testing. 5. No. The calculation rests on simplifying assumptions and addresses only a narrow matching scenario; linkage, sensitive-attribute disclosure, and the release context also require review.