From Diagnostic Plots to a Decision
A MICE run can produce many plots and summaries, but a collection of diagnostics is not yet a conclusion. A useful review asks, in order: do the chains appear to have settled into stable behavior; do they explore comparable regions; and do the imputed values look plausible when compared with observed data in relevant groups? The answers determine what to investigate next, not whether an imputation model is automatically “good” or “bad.”
This workflow assumes that the variables, conditional imputation models, predictor set, number of chains, and iteration count have been specified and recorded. Let \(m\) denote the number of chains, \(t\) the iteration, and \(j\) a variable. For each chain and iteration, retain summaries relevant to the variable type—for example, a mean and standard deviation for a continuous variable, or category proportions for a categorical variable. These summaries do not replace examination of the imputed values themselves.
The first two checks concern the behavior of the iterative procedure. The third concerns how the resulting imputations relate to observed data. These answer different questions: stable traces do not establish that the imputation models fit well, and plausible-looking observed–imputed comparisons do not establish that the chains have stabilized.
A Sequence for Reviewing a Completed Run
Record the variables imputed, their conditional models, predictor choices, chain count, iteration count, starting values or initialization approach, and any warnings or failed model fits. A plot cannot be interpreted reliably if the run configuration is unknown.
Plot each chain over iteration for summaries such as means, standard deviations, category proportions, or selected model parameters. Look for sustained drift, abrupt shifts, long periods at boundaries, and different behavior across chains. Choose summaries that could reveal problems for the variable and its use.
Check whether chains move through overlapping ranges rather than remaining separated or nearly fixed. Examine the traces at a scale that can show both overall movement and local variation; a smooth-looking average across chains can conceal a poorly moving individual chain.
For each variable, compare distributions using suitable summaries and plots, then investigate important predictor levels, missingness patterns, and joint relationships where relevant. A marginal comparison is a starting point, not a complete assessment.
Separate chain-behavior warnings from distributional or plausibility concerns. State which additional check is needed, whether a model revision and rerun is warranted, or why the evidence supports proceeding with a recorded limitation.
Before interpreting the plots, specify the assumptions behind the diagnostic reading. The chain traces should correspond to the recorded MICE run, and the plotted summaries should be computed consistently across chains and iterations. Comparisons across chains are most informative when chains use the same specified imputation procedure but have sufficiently different starting points to reveal sensitivity to initialization. These checks are descriptive: they cannot guarantee that the number of iterations is adequate for every relevant feature, or that the conditional models represent the data adequately.
Reading Trace and Mixing Evidence
A trace is a sequence of a summary against iteration. A trace that continues to move systematically late in the run suggests that the chain may not have reached stable behavior for that summary. A chain that appears nearly constant can also be concerning: it may reflect poor movement, a boundary or separation problem, or a summary that genuinely varies little. Conversely, visible fluctuation is not itself evidence of a problem; stochastic variation is expected.
Compare chains rather than relying only on their combined average. Persistent differences in level or range can indicate that chains have not explored comparable regions. If chains overlap but move very slowly, apparent overlap does not rule out poor mixing. Review multiple summaries for each important variable, including summaries that could reveal rare categories, boundary accumulation, or changing variability. A single stable mean can conceal a changing distribution.
Trace displays are not tests with universal pass/fail cutoffs. Their interpretation depends on the variable, model, scale, and purpose of the analysis. If a pattern is difficult to judge, plot a longer run or inspect additional summaries rather than declaring convergence from a short, visually quiet segment. A numerical convergence summary, if used, should support rather than replace the trace review; its assumptions and limitations matter, especially when applied to summaries from an iterative MICE procedure.
Worked Example: Stable Late Traces After Early Drift
Suppose a hypothetical run has three chains. The recorded trace is the imputed mean of a continuous variable, measured in years. During early iterations, the chain means are approximately 8.8, 9.4, and 10.0 years. In the final four iterations, the recorded means are:
- Chain 1: 10.1, 10.2, 10.0, 10.1 years.
- Chain 2: 10.0, 10.1, 10.2, 10.1 years.
- Chain 3: 10.2, 10.1, 10.1, 10.0 years.
The late mean for chain 1 is \((10.1+10.2+10.0+10.1)/4=10.10\) years. The corresponding means for chains 2 and 3 are also \(10.10\) years. Across all displayed late values, the minimum is 10.0 and the maximum is 10.2 years, a range of \(10.2-10.0=0.2\) years.
The early separation followed by closely overlapping late values is evidence that the early trace was still changing and that the displayed late summaries look more stable. It is not proof that every aspect of the chains has stabilized. The reviewer should inspect other relevant summaries and variables, and record whether the apparent late stability persists over a sufficiently informative portion of the run.
Interpreting Observed–Imputed Comparisons
Once chain behavior has been reviewed, compare observed and imputed values using summaries suited to the variable: distributions and spread for continuous variables, and category proportions for categorical variables. Where substantively important, repeat comparisons across predictor levels or missingness patterns and inspect joint behavior. Earlier material on observed–imputed comparisons establishes an important caution: the observed cases need not have the same distribution as the cases with missing values. A difference is therefore a reason to investigate, not a standalone rejection rule.
For a continuous variable, a standardized mean gap can describe the size of a marginal difference. If the observed group has mean \(\bar{x}_{obs}\) and standard deviation \(s_{obs}\), and the imputed group has mean \(\bar{x}_{imp}\) and standard deviation \(s_{imp}\), one descriptive scale is \(d=(\bar{x}_{imp}-\bar{x}_{obs})/\sqrt{(s_{imp}^2+s_{obs}^2)/2}\). This is a standardized descriptive contrast, not a test of model validity. Its interpretation should be paired with plots, subgroup checks, and knowledge of how cases came to have observed or missing values.
Worked Example: A Distribution Gap That Needs Context
In a hypothetical dataset, an observed laboratory measurement has mean 50 units and standard deviation 10 units. Its imputed values have mean 55 units and standard deviation 12 units. The standardized mean gap is:
The imputed mean is 5 units higher, or about 0.452 of the pooled descriptive scale used in this formula. This difference merits review, but it does not establish that the imputation is wrong: cases with missing measurements may differ systematically from those with observed measurements. The next checks are to examine distributions rather than means alone, compare relevant predictor subgroups and missingness patterns, and consider whether the values are plausible on the measurement scale. If the difference is concentrated in a particular subgroup, investigate that pattern rather than relying on the overall gap.
Worked Example: Combining Chain and Distribution Checks
Consider a hypothetical three-category clinical status variable. The chains’ category-proportion traces appear stable late in the run and overlap. In the observed cases, the proportions for categories A, B, and C are 0.50, 0.35, and 0.15. Among the imputed values, the corresponding proportions are 0.30, 0.45, and 0.25.
The absolute differences are \( |0.30-0.50|=0.20\), \( |0.45-0.35|=0.10\), and \( |0.25-0.15|=0.10\). Half their sum is \( (0.20+0.10+0.10)/2=0.20\). Thus, the total variation distance between these two sets of marginal proportions is 0.20. This records a notable marginal contrast; it is not a universal threshold for failure.
The stable traces reduce concern about obvious drift in these particular summaries, but they do not resolve the distributional difference. The reviewer should compare category proportions within important predictor levels and missingness patterns, and check for implausible relationships or sparse categories. If the difference is consistent with the observed-case selection pattern and the conditional comparisons are plausible, document that interpretation and its uncertainty. If it persists in ways the imputation model does not capture, investigate model specification and consider a revised run. Do not change the model solely to force the marginal proportions to match.
Making and Recording the Decision
A practical review distinguishes three outcomes. Proceed with documented caveats when the inspected traces show no material warning and the observed–imputed comparisons are plausible in context. Investigate further when evidence is ambiguous or a concerning pattern has not yet been checked within relevant groups. Revise and rerun when evidence points to a material problem that an adjustment to the specified imputation procedure could address. These are review decisions, not guarantees about the inferential validity of later analyses.
For each outcome, record the variable and summary examined, the chains and iterations covered, the pattern observed, the follow-up check, and the reason for the decision. For example: “Late category-proportion traces overlap across three chains; imputed category A is less common marginally than observed; subgroup comparisons are pending, so no model revision is made yet.” This is more informative than writing only “diagnostics passed.”
The main failure modes are equally important to record. A trace can conceal instability if only a pooled summary is shown. A marginal observed–imputed mismatch can be misread as model failure when it reflects selection into missingness. Conversely, a close marginal match can hide poor conditional or joint behavior. A diagnostic workflow helps prevent these errors by requiring the reviewer to say what each check can and cannot establish.
Check Your Understanding
For each question, decide what the diagnostic evidence supports and what additional step is appropriate.
- A variable’s trace mean is stable in one chain but continues to rise in another. What should the reviewer conclude?
- The observed and imputed means differ, but the variable has substantial missingness associated with a predictor. Is matching the two means a requirement?
- What is the standardized mean gap if the observed mean and standard deviation are 20 and 4 units, and the imputed mean and standard deviation are 22 and 5 units?
- All chain traces overlap, but one imputed category appears implausibly often within a predictor subgroup. Which part of the workflow should be revisited?
Solutions. (1) The trace evidence is mixed: do not declare stable behavior based on one chain. Inspect the other chain and additional summaries, and investigate whether longer runs or a model review are warranted. (2) No. Missingness related to the predictor can produce different observed and imputed marginals; examine relevant conditional comparisons and plausibility instead of forcing equality. (3) \(d=(22-20)/\sqrt{(4^2+5^2)/2}=2/\sqrt{20.5}\approx0.442\). This is a descriptive gap of about 0.442 on the stated scale, not a pass/fail result. (4) Revisit the observed–imputed comparison at the subgroup level and investigate the category pattern, even though the inspected traces overlap. Chain overlap does not rule out a distributional concern.