Running MICE Checks in Continuous Integration

Why Run MICE Checks in Continuous Integration?

A MICE workflow can run successfully on one analyst’s computer but fail in a clean environment because a dependency is missing, an input assumption has changed, or a required command never ran. Continuous integration (CI) provides a repeatable place to test a small, controlled version of the workflow whenever code changes. Its purpose here is operational: confirm that the environment can be prepared, a minimal MICE run can execute, and required output checks are completed.

A CI smoke check is not evidence that an imputation model is scientifically appropriate, that the imputation is adequate for a particular analysis, or that its statistical conclusions are valid. It is a fast, automated gate against selected failures. The check must be explicit about what counts as success; otherwise a job can finish with a misleading green status.

Definition: A MICE CI smoke check is a short automated run in a controlled software environment that executes a small MICE example and evaluates specified success conditions. The CI job passes only when environment setup, the run, and every required check complete successfully.

Specify the Gate Before Configuring It

A useful gate has three parts: environment setup, execution, and validation. Pin the R version and the R package versions used by the example. In an R project, a lockfile such as renv.lock records package versions; the CI job should restore from that lockfile rather than install whichever versions happen to be current. The workflow definition itself should also be reviewed and versioned. Where the CI platform permits it, pin third-party workflow actions to verified immutable references, not just movable version tags.

For example, configure a CI workflow to run on a proposed code change and on changes to the main branch. Its steps can check out the repository, install a specified R version, restore the project library from the lockfile, and run a dedicated smoke-check script. The exact commands depend on the chosen CI platform, but the important contract is the same: if a required step exits unsuccessfully, the job must be marked as failed. Also configure the repository’s required status checks so that a failed job cannot be ignored when merging.

Before relying on this procedure, verify several assumptions. The CI runner must be able to access the required package source, the lockfile must describe a compatible library, and the smoke-check script must use the same configuration and preprocessing entry points intended for the workflow. The example data must be synthetic or otherwise approved for automated use; confidential records do not belong in a public or broadly accessible CI job. A small run should be deterministic where practical, but a fixed seed does not guarantee identical results across all package versions or platforms. Pinning dependencies reduces that source of variation.

Define the gate as a conjunction. Let \(S\) mean that setup succeeded, \(R\) that the MICE run completed, and \(C_1,\ldots,C_k\) that each required check passed. Then the job passes exactly when

G = S \land R \land C_1 \land \cdots \land C_k.

This form clarifies the failure policy: one false required condition makes \(G\) false. A workflow that runs the imputation but skips a validation step has not passed this gate, because completion of the run alone is not the stated success criterion.

A Small, Checkable MICE Run

The following synthetic example uses two continuous variables, \(x\) and \(y\), with some deliberately introduced missing values. It is deliberately small and uses two imputations and two iterations to keep the CI job quick. Those settings are for a smoke check, not a recommendation for an analysis.

A script can create 20 rows, with \(x=1,2,\ldots,20\), and \(y=10+2x+e\), where the fixed offsets \(e\) repeat the sequence \(-1,0,1,2,-2\) four times. Set \(x\) missing in rows 3, 8, 13, and 18, and set \(y\) missing in rows 5, 10, 15, and 20. No row is missing both values. Use the default predictive mean matching method for these numeric variables, and set a seed for repeatability within the pinned environment.

A script may call mice::mice(data, m = 2, maxit = 2, printFlag = FALSE, seed = 814), then inspect the completed data from each imputation. It should stop with an error if the completed data have the wrong dimensions, any missing values remain, or an originally observed value has changed. In R, checks such as stopifnot() are useful because a false condition raises an error; when run by Rscript, that error produces a nonzero exit status for the CI job.

Worked Example: Determine Whether the Smoke Check Passes

Suppose the script completes both imputations and checks each completed data set. The input has 20 rows and two columns. Four \(x\) cells and four \(y\) cells were set to missing, so the input contains \(20 \times 2 = 40\) cells, of which \(40 - 8 = 32\) were observed.

For each completed data set, the required checks are: 20 rows, two columns, no missing values, and unchanged values wherever the input was observed. Suppose both completed data sets meet all four conditions. For either one, the check for unresolved missing values is \(0\) missing cells, and the check for preserved observed values compares 32 completed values with their corresponding 32 observed input values.

Thus \(S=1\), \(R=1\), and all four check indicators equal 1. Substitution into the gate gives \(G=1 \land 1 \land 1 \land 1 \land 1 \land 1=1\). The CI job passes. This result says only that the specified smoke check succeeded in this pinned environment; it does not establish that two imputations or two iterations are adequate for inferential analysis.

Making Failure Observable

The check script must not merely print a warning and continue when a required condition fails. Each required check should have a failure path that stops execution. For example, if a completed data set contains one unresolved missing cell, the condition “there are no missing values” is false. An error then makes the script’s exit status nonzero, which the CI runner must treat as a failed step. This chain from a failed condition to a failed job is part of the design, not an implementation detail.

A practical sequence is:

Prepare the environment.
Check out the code, install the specified R version, and restore the library using the committed lockfile. A restoration error must stop the job.

Run the smoke-check script.
Invoke the script as a required CI step, for example with Rscript scripts/ci_mice_smoke.R. Do not allow the workflow to continue as if this step succeeded when the command returns an error.

Evaluate required conditions.
Check the expected number of imputations and completed-data properties that are part of this smoke check. Stop on any failed condition.

Require the status check.
Configure the repository so the relevant CI status must pass before a change is accepted. A correctly failing job is not an effective gate if nobody is required to act on its result.

This sequence is a short justification for the conjunction rule: setup is a prerequisite for executing the intended code; execution is a prerequisite for checking its output; and the output checks define the stated acceptance conditions. If any prerequisite fails, later checks cannot establish that the complete procedure succeeded. A skipped step is therefore not equivalent to a passed step.

Worked Example: A Dependency Restoration Failure

Imagine that the lockfile requests a version of a package that cannot be installed on the runner, so library restoration ends with an error. The MICE script is not reached. In the gate notation, \(S=0\) and \(R\) is not established; it should not be recorded as a successful run.

The correct result is failure, even if the source code appears valid on a developer’s computer. The failure is evidence that the configured environment could not be prepared as specified. Inspect the package installation log, the lockfile, the selected R version, and whether the runner can reach the package repository. Do not “fix” the status by skipping restoration or falling back silently to unpinned current package versions: that would change the environment the gate is intended to check.

Worked Example: One Failed Output Condition

Suppose the MICE call returns, but inspection of the first completed data set finds one remaining missing value. There are 40 cells in total, and the check finds one missing cell, so the condition “zero missing cells” evaluates as false. Even if the other checks pass, the gate has at least one false term:

G = 1 \land 1 \land 1 \land 0 \land 1 \land 1 = 0.

The job must fail and identify which completed data set and variable triggered the check. Investigate whether the input contained an unsupported or unexpected pattern, whether the intended variables were actually included in the imputation, or whether the script inspected the wrong object. A printed warning without a nonzero exit status would be a misleading pass.

Diagnostics, Alternatives, and Limits

When CI fails, use the failing step and its log to separate environment problems from MICE-run problems and check failures. A failure during restoration points first to dependency availability or version compatibility. A failure inside the smoke-check command may reflect invalid inputs, an error in the call, or a problem in the script. A failure after the call means the run completed but did not meet at least one declared condition. Logs should make this distinction clear without recording sensitive data.

A green status also needs scrutiny. Confirm that the workflow actually ran on the intended changes, that the smoke-check step was not conditional in a way that skipped it, that errors were not suppressed, and that the status is required by the repository. Re-running an intermittent job may help diagnose a transient infrastructure problem, but repeated reruns should not be used to hide a reproducible failure.

The main alternative is relying on manual local execution. Local runs can be useful during development, but they may use different package versions and do not automatically gate changes. A CI job provides a consistent, visible check, at the cost of runner setup and maintenance. A larger end-to-end run can cover more of the workflow but is slower and more exposed to resource limits; the lightweight smoke check trades breadth for speed. Keep this tutorial’s gate narrowly focused on whether essential steps execute and selected output conditions hold. More extensive statistical diagnostics and software-update regression testing address different questions.

A smoke check has important limits. Small synthetic data can miss issues that arise with rare categories, separation, unusual missingness patterns, or realistic sample sizes. A successful run does not establish convergence or the validity of modeling choices. A fixed seed improves repeatability but cannot remove all differences due to operating systems, numerical libraries, or changed dependencies. Finally, CI only checks what its script asserts: an untested condition has no protection merely because the overall job is green.

Common Mistakes

Installing unpinned packages. A job that installs the latest available versions can change behavior without a code change. Restore from a committed lockfile and make version changes deliberate.
Allowing a failed command to look successful. Suppressed errors or warning-only checks can produce a green status despite a failed requirement. Make required failures stop the script and propagate to the CI exit status.
Treating a smoke check as statistical validation. A short run verifies selected software and workflow properties, not the adequacy of the imputation model for an analysis.
Not requiring the CI status. A workflow may correctly report failure but provide no protection if changes can be accepted without a passing required check.
Confusing a seed with a pinned environment. A seed controls random-number generation within a given setup; it does not pin R, packages, or the CI workflow.

Key takeaway: A reliable MICE CI check pins its environment, runs a deliberately small example, turns every required condition into a stopping check, and makes the resulting status a required gate. Its conclusion is operational success in that environment, not proof of statistical adequacy.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions below.

A job restores its locked library and runs MICE, but the completed data contain one missing value where none should remain. Should the job pass?
Why is installing the newest available version of every package on each CI run weaker than restoring a lockfile?
A smoke-check script prints “observed values preserved” but never stops if that condition is false. What is wrong with the gate?
What does a passing two-imputation, two-iteration smoke check establish, and what does it not establish?
Why might a correctly failing CI job still fail to protect a repository?

Solutions. (1) No. At least one required condition is false, so the conjunction defining the gate is false and the script should exit unsuccessfully. (2) Unpinned installation allows package versions to change between runs; a lockfile specifies the intended versions and makes the environment more reproducible. (3) The assertion is not connected to the exit status, so the workflow may report success despite a violated requirement. Make the check stop execution on failure. (4) It establishes that the specified small run and declared checks succeeded in that pinned environment; it does not establish model adequacy, convergence, or validity for a substantive analysis. (5) The repository may not require that CI status before accepting a change, so a failure can be ignored. Configure the relevant status as required.