Why Check Defaults Before a Run?
A MICE run is controlled not only by the settings written into a script but also by settings supplied automatically by software. These defaults can determine what the run stores, what it reports while running, whether computation uses multiple workers, and how random-number generation is initialized. A run can complete successfully while retaining too little information to inspect or repeat it.
The aim is not to override every default. Some defaults are suitable and stable for a particular workflow. The aim is to identify consequential defaults, verify their behavior for the installed software, and make deliberate choices visible in the run configuration. In this tutorial, setting names such as “save trace” or “worker count” are generic descriptions, not claims about the exact names or defaults of any particular package.
For a particular MICE run, defaults matter only if they affect its behavior or the evidence retained about that behavior. A setting that controls storage may leave the imputations unchanged but make later checks impossible. A setting that controls random-number streams or parallel scheduling can affect the generated imputations themselves. These consequences should not be treated as interchangeable.
What to Inspect
Inspect the defaults that affect at least one of four areas: stored outputs, logging, parallel execution, and reproducibility. The installed package’s documentation and configuration interface are the starting points. Check the behavior for the package version and execution environment being used: a default described for a different release, wrapper, or platform may not apply. Where possible, confirm the resolved setting from the run object or a small test run rather than relying only on memory.
| Area | Questions to resolve | Why it matters |
|---|---|---|
| Stored outputs | Are completed imputations, chain-related summaries, diagnostics, warnings, and run metadata retained? Can results be overwritten? | A run may finish but leave insufficient material for review or downstream analysis. |
| Logging | What is recorded, at what level, and where? Are warnings and errors captured, or only progress messages? | A log can support investigation, but a progress log is not necessarily a record of settings or diagnostic results. |
| Parallel execution | How many workers are used? How are tasks assigned? Is random-number generation coordinated across workers? | Parallel work can change execution order and may affect repeatability if streams are not managed appropriately. |
| Reproducibility | What initializes random-number generation? Does the setting cover all workers? Which software and environment details are needed to interpret a rerun? | A seed alone may not reproduce results if other inputs or stream behavior differ. |
Keep the distinction between results and evidence about results clear. For example, not retaining a trace or log does not by itself prove that an imputation model was wrong. It does mean that some checks may be unavailable after the run. Conversely, retaining a log does not establish that the imputation process was statistically appropriate.
Turn Implicit Choices Into an Explicit Configuration
Before a run, list settings that affect saved results, inspection, or execution. For each, write down the intended value and whether it is explicitly supplied, deliberately accepted as a verified default, or still unknown. Then resolve unknowns before relying on the run. A concise configuration is more useful than a long list of settings when it makes consequential behavior unambiguous.
Include what is retained, what is logged, worker and scheduling choices, and random-number handling. Add any setting that could change the result or prevent later review.
Consult documentation or the package’s configuration interface for the installed release. Check wrapper behavior and environment-level settings where relevant.
Set options directly when the workflow needs a particular storage, logging, worker, or random-number behavior. Do not assume that an omitted setting means the same thing across software releases.
Confirm that the run used the intended values, and save the configuration alongside the results. A stated intention is not a substitute for checking what was actually applied.
This process complements the earlier material on Logging MICE Workflows for Auditing and Recording Provenance for Imputed Datasets. It does not require repeating those practices: the specific task here is to make hidden option values visible before they can silently shape a run.
Worked Examples
Worked Example: Choosing What to Retain
Suppose a hypothetical analysis will create \(m=5\) completed data sets, each with \(n=1{,}200\) rows and \(p=30\) variables. The team finds that the installed software’s default storage behavior retains the completed data sets but does not retain a requested chain summary. The team wants both the completed data and the summary for review.
First, verify the stated default in the installed software rather than assuming that all MICE implementations store the same objects. Given that verified behavior, the team explicitly selects storage of the summary as well as the completed data sets, and checks the resulting run object for both.
A rough lower-bound estimate for the numeric cell values in the completed data sets is \(mnp\) values. If each value occupied 8 bytes, the calculation would be
This estimate concerns only numeric cell storage under the stated 8-byte assumption. It excludes object overhead, character variables, metadata, chain summaries, and any additional copies. It helps anticipate storage needs, but it does not determine which outputs are worth retaining. The decision follows the analysis requirement: because review needs the chain summary, the team explicitly retains it and verifies that it is present.
Worked Example: Making Logging Behavior Testable
In a hypothetical run, the software’s verified default writes progress messages to the console but does not save them to a file. A team needs a persistent record of warnings and run completion. It chooses to direct a log to a designated file and to check that the file contains both the start and completion messages and any warnings emitted during the run.
Suppose the run produces 3 warnings. The team checks the saved log and finds 3 warning entries, along with the completion message. The arithmetic check is simple: the number of warning entries expected from the run is 3, and the number found is also 3. This supports the conclusion that those warnings were captured in this log. It does not show that no other issue occurred, that every diagnostic was logged, or that the imputation models were adequate.
The important choice is not “maximum logging” in every analysis. It is choosing a level and destination suited to review, then verifying that the chosen configuration captured what the team needs. If warnings are reported separately from progress messages, the team must check both channels or use a logging setup that captures both.
Worked Example: Checking Parallel and Random-Number Settings
Consider a hypothetical run with \(m=4\) imputations and 2 workers. The package documentation for the installed version states that this configuration uses independently managed random-number streams for the workers, and a test run confirms that setting the seed through the supported interface initializes those streams. The analyst records the seed, worker count, and stream-handling choice in the run configuration.
The analyst repeats the run with the same data, software, configuration, and environment. Under the stated assumption that this software’s documented stream behavior is deterministic for these fixed inputs, the two runs produce matching imputed values. That is evidence of repeatability under this setup; it is not a general guarantee that the same seed produces identical results under another release, worker count, or execution environment.
If the software does not document or confirm how random-number streams are assigned to workers, the analyst should not infer that one seed is sufficient. The safe next step is to resolve the behavior in documentation or a controlled test, or to use a verified serial configuration when exact repeatability is essential. Parallel execution itself is not a defect: it is an execution choice whose stream and scheduling behavior needs to be understood.
Why a Seed Is Not the Whole Reproducibility Record
A useful way to reason about repeatability is to represent the output of a run abstractly as a function of the data \(D\), effective configuration \(C\), software and execution environment \(V\), and random-number state \(R\):
Here, \(Y\) denotes the generated MICE results and any retained run outputs. This notation is a bookkeeping device, not a claim that every package exposes a single function \(F\). It makes the requirement for a controlled rerun explicit: matching the seed addresses part of \(R\), but identical output also depends on the relevant data, configuration, software behavior, and random-number stream handling being held fixed.
The justification follows directly from the representation. If \(D\), \(C\), \(V\), and \(R\) are all the same, and the software evaluates the run deterministically given those inputs, then the same \(F\) is applied to the same inputs and produces the same \(Y\). If any input differs, equality of outputs is not guaranteed. Parallel scheduling matters when it changes the assignment or order of random draws; worker count matters if the software’s stream construction depends on it. This is why a seed without the surrounding settings is an incomplete reproducibility record.
For the intended purpose of exact reruns, test whether repeated runs match under the chosen setup. For a purpose that only requires scientifically comparable results, bit-for-bit equality may not be necessary; the required level of reproducibility should be stated. Keep software and dependency details with the run record, consistent with the earlier material on Reporting Software Versions and Dependencies.
Diagnostics, Alternatives, and Limitations
Warning signs that defaults have not been adequately checked include a run object missing an expected summary, a log that contains progress but not warnings, output files overwritten by a later run, an unexpected worker count, or repeated runs that differ despite an unchanged seed. These signs call for investigation, not an automatic conclusion that the MICE analysis is invalid. Check the effective configuration, storage behavior, and random-number handling before interpreting the discrepancy.
There are three common approaches to defaults. Accepting defaults is efficient when the behavior has been verified and suits the analysis, but undocumented or changing defaults are hard to audit. Explicitly setting every consequential option makes intent clearer and reduces ambiguity, but adds configuration to maintain; explicit settings can also become invalid or behave differently after software changes. Using a wrapper or project configuration centralizes choices across runs, but the wrapper can itself introduce defaults or override package settings. Whichever approach is used, verify the effective configuration and the retained outputs.
No configuration can guarantee scientific reproducibility by itself. It cannot compensate for changes in the data, an unsuitable imputation model, or an unrecorded change in execution behavior. Nor should extra storage or logging be mistaken for a statistical diagnostic. Defaults are one part of workflow control, not evidence that the imputation assumptions hold.
A common mistake is to reason, “I fixed the seed, so the run is reproducible.” The correction is to record and check the seed and the settings that govern stream creation, worker execution, software behavior, and inputs. Another mistake is to assume that a completed run has retained everything needed for review. Confirm the actual contents of the saved results instead of inferring them from successful completion.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- A MICE run finishes, but its saved object lacks a summary needed for review. What should be checked before concluding that the analysis itself failed?
- Why does a log containing all progress messages not necessarily establish that all warnings were recorded?
- A run is repeated with the same seed but a different worker count. Can identical imputations be assumed? Explain.
- Name two kinds of information, in addition to a seed, that can matter when trying to reproduce a run.
Solutions. (1) Check the verified storage defaults and the effective configuration, then confirm whether the summary was requested, retained, or saved elsewhere. Its absence limits review but does not by itself show that the imputation failed. (2) Progress messages and warnings may be handled or stored separately, so the warning channel and saved log contents must be checked. (3) No. Stream assignment or scheduling may depend on worker count; identical output is not guaranteed unless the software’s behavior for both configurations establishes it. (4) Examples include the data, effective configuration, software and execution environment, and random-number stream handling.