Why Failure Handling Needs Its Own Workflow
A MICE run can stop for many reasons: a fitting error, a resource limit, a machine shutdown, or an interruption by a user. A saved object may still contain some imputations, but that alone does not show that the requested run finished. The practical risk is treating a partial result as final, or resuming from a state that does not contain enough information to continue reliably.
This tutorial focuses on operational control: how to recognize a failed or interrupted run, preserve useful state, and make a defensible resume-or-restart decision. A computationally complete run is not automatically statistically adequate. Completion means the requested computation finished and its outputs passed the planned checks; it does not establish that the imputation models are appropriate or that convergence has been achieved.
The planned run should specify, at minimum, the input-data identity, the imputation configuration, the requested \(m\), the iteration target, and the checks that define an acceptable output. These should agree with the reproducible project and configuration practices established earlier in the course. They let a reviewer distinguish “the process stopped” from “the requested computation finished.”
The workflow also depends on technical conditions. The saved state must be written completely, be readable, and correspond to the intended input and software environment. If continuation is planned, the software must support continuation from that state, and the saved object must contain the state needed by that continuation method. If those conditions are not met, restarting from the original inputs is safer than guessing how to reconstruct the run.
What Counts as Progress, and What Counts as Completion?
Track planned progress explicitly. For example, with \(m=5\) imputations and a target of 20 iterations per imputation, the plan comprises \(5\times20=100\) chain-iteration units. If all five chains have completed 12 iterations when the process stops, the recorded progress is \(5\times12=60\) of 100 units. This fraction describes computational progress only; it is not a measure of imputation quality.
A robust status record separates at least three states:
- Running: computation is underway, and the current output must not be used as final.
- Interrupted or failed: the requested work did not finish, or an unresolved error prevents accepting the output.
- Complete and checked: the requested work finished, the output can be read, and required checks passed.
A process exit code is useful evidence but is not sufficient by itself. A process can terminate normally after producing the wrong number of imputations, and a run can save a useful checkpoint before exiting with an error. Inspect both the process outcome and the saved object. Check the recorded iteration progress, the number of completed datasets, warnings or errors, and the structural checks required for the project. The earlier material on testing invariants in completed datasets is relevant here: a checkpoint should not be labelled checked merely because it can be loaded.
Worked Example: Classifying a Run Stopped Midway
Suppose a hypothetical analysis requests \(m=5\) completed datasets and 20 iterations per chain. A machine shutdown occurs after the saved state records 12 iterations for each chain. There are five imputations in the saved object, and they can be loaded.
The planned total is \(5\times20=100\) chain-iteration units. Recorded progress is \(5\times12=60\), so the fraction completed is
The object is recoverable work, not a computationally complete result: it falls 40 chain-iteration units short of the plan. The fact that five datasets are present does not change that conclusion. Record the status as interrupted, preserve the checkpoint and its context, then either resume using a verified continuation procedure or restart. Do not send these partial datasets into the planned final analysis as though they met the 20-iteration target.
Checkpointing Without Corrupting the Evidence
A checkpoint is most useful when it captures a consistent state, rather than a file written while the run is changing. Where possible, save at a defined boundary, such as after a full iteration has completed across the imputations, and include enough information to identify the run and its progress. A checkpoint record should link to the input-data version, configuration, software environment, run identifier, timestamp, and stated progress. Preserve relevant warnings and errors with it; a checkpoint without context can be difficult to interpret.
Use a write pattern that avoids presenting a partially written file as a valid checkpoint. One standard pattern is to write to a temporary file, verify that it can be read and has the expected identifying information, and then replace the designated checkpoint path with the verified file using an atomic rename where the file system supports that operation. Keep the last known-good checkpoint until the replacement has been verified. An atomic rename is useful because readers see either the older complete file or the newer complete file, rather than a file in the middle of being written.
A checkpoint can be valid as an archival snapshot without being suitable for exact continuation. In particular, a saved completed-dataset object may not retain every detail required to reproduce the next random draws or resume an interrupted iteration. If exact continuation matters, the saved state must include the state required by the software's supported continuation mechanism, including random-number state when required. A seed alone does not necessarily reproduce the same continuation: the sequence also depends on how many random draws have already occurred and on the software and execution details.
Worked Example: Deciding Whether a Checkpoint Can Be Resumed
In a hypothetical run, a checkpoint file is readable and reports 8 completed iterations out of a target of 16 for \(m=4\). Its metadata identifies the input and configuration. However, the file was created by a custom export that stores the completed datasets but not the sampler state or the random-number state required by the chosen continuation procedure.
The progress fraction is
The checkpoint is useful for diagnosing what was produced, but the missing continuation state means it has not been shown to support a valid continuation under the intended procedure. Do not simply add eight more iterations to a newly initialized run and call that an exact resume: those iterations would not necessarily continue the original chains. If the software offers a documented continuation method from this object, verify its requirements first. Otherwise, restart from the original inputs with the planned configuration, preserve the old checkpoint as an interrupted artifact, and record why it was not resumed.
A Safe Resume-or-Restart Decision
The decision should be made from evidence, not from the desire to save computation. The following sequence makes the decision explicit.
Record whether the run ended normally, raised an error, was interrupted, or lost its execution environment. Capture the last available message and progress state.
Confirm that it can be read, matches the intended input and configuration, and reports a coherent progress state. Keep an unmodified copy for recovery or inspection.
Confirm that the software supports continuation from this type of saved state and that all required sampler and random-number information is available.
Resume only when the documented method and saved state justify it. Otherwise, rerun from the preserved inputs and configuration rather than improvising a continuation.
After continuation or restart, confirm that the full requested run finished and that required output checks passed. Mark the result complete only then.
Why This Decision Is Defensible
The key distinction is between a partial history and a valid next state. A continuation procedure is justified only if its starting state represents the end of the computation so far in the way that procedure expects. If state is missing or inconsistent, the next generated values need not be the next values of the intended chains. Restarting avoids falsely claiming that a newly initialized computation is a continuation of the old one.
This is an operational justification, not a claim that every run must be bit-for-bit identical after resumption. Exact reproducibility may depend on software, hardware, parallel execution, and random-number handling. The practical requirement is to use a documented continuation method, preserve enough information to explain what was done, and avoid asserting exact continuation when it has not been established.
Worked Example: An Error After the Requested Work Finished
A hypothetical run requests \(m=3\) datasets and 10 iterations per chain. Its saved state reports all 10 iterations for each of the three imputations, the process log then records an error during a later export step, and the output contains three readable datasets.
The imputation progress is \(3\times10=30\) of \(3\times10=30\) chain-iteration units, or 100%. The error occurred after the requested imputation computation, so it does not by itself prove that the imputations are incomplete. But the run cannot yet be labelled complete and checked: the export error must be resolved, and the saved object must be verified against the planned checks. If the object passes those checks, it may be accepted as a completed imputation result while the failed export is rerun. If the error leaves the object incomplete or its state uncertain, treat the run as failed and recover or restart. The decision follows the error's location and verified output, not just the existence of an error message.
Diagnostics, Alternatives, and Limitations
Warning signs include a progress count below the target, fewer than \(m\) completed datasets, an unreadable or unexpectedly small file, mismatched input or configuration identifiers, and errors that occur during imputation rather than during later handling. A checkpoint whose recorded iteration count disagrees with the run record also needs investigation. Do not infer that all is well from a process finishing, a file existing, or a single summary field.
There are two main recovery options:
| Approach | When it is appropriate | Main caution |
|---|---|---|
| Resume from checkpoint | The checkpoint is verified and the software supports continuation from its saved state. | Missing state or incompatible software can make the purported continuation invalid. |
| Restart from original inputs | The checkpoint is incomplete, uncertain, or not supported for continuation. | It uses more computation, and the new run may not reproduce the old random sequence unless the full setup supports that. |
Neither option replaces statistical review. A fully completed run can still have problematic chains, unsuitable conditional models, or failed structural checks. Conversely, an interrupted run may contain useful diagnostic information, but partial output is not a substitute for the planned completed result. This workflow also cannot guarantee recovery from every hardware or file-system failure; keep original inputs and verified prior checkpoints safe, and record when recovery was not possible.
Common Mistake
Wrong reasoning: “The saved object contains all \(m\) datasets, so the MICE run must be complete.” The number of datasets does not establish that each has completed the planned iterations, that the run ended without a relevant error, or that the output passed its checks.
Correction: Check the requested target against recorded progress, inspect the stop and error information, verify the saved state and required checks, and label the result accordingly. A partial object can be preserved as a checkpoint, but it must not be confused with a completed imputation.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- A run requests \(m=6\) datasets and 15 iterations per chain. It stops after 9 complete iterations across all chains. What fraction of planned chain-iteration units is complete, and should the output be treated as final?
- A checkpoint loads successfully but its recorded input identifier does not match the current analysis input. What should happen before any attempt to resume?
- Why is a seed by itself not always enough to continue the exact random sequence of an interrupted run?
- A run reports all requested iterations, but an error occurs during output export. What evidence is needed before labelling the imputations complete and checked?
Solutions. (1) The plan is \(6\times15=90\) units; progress is \(6\times9=54\), so \(54/90=0.60\), or 60%. It is interrupted, not final. (2) Do not resume it as part of the current analysis. Investigate the mismatch, preserve the checkpoint, and locate the correct inputs or restart from the intended inputs. (3) The random-number state depends on how many draws have already occurred, not only the initial seed; continuation also depends on the saved state and supported software procedure. (4) Verify the imputation object's progress and integrity, determine whether the error occurred after imputation, and run the required structural checks. If those checks pass, the imputation may be complete even though export failed; otherwise do not label it complete and checked.