Reviewing Mixed-Type Diagnostics
A MICE diagnostic review must answer two different questions. First, do the iterative imputations behave stably across iterations and chains? Second, are the values imputed for missing entries plausible in relation to what is observed and to the other variables? Neither question has one summary that works equally well for every variable type. A mean trace can be informative for a continuous variable, but it cannot reveal whether a binary variable is stuck at one category or an ordinal variable has lost a rare, important level.
The aim here is a focused review across binary, ordinal, count, and continuous variables. The review uses trace summaries to look for drift, poor mixing, and disagreement between chains; observed–imputed plots to look for distributional or plausibility concerns; and a consistent way to decide which findings deserve attention first.
Conditions for Interpreting the Review
The review assumes that the chains were run using the intended MICE specification, that the plotted iterations come from the retained portion of the run, and that each chain’s summary is calculated consistently. Chains should be initialized so that between-chain agreement is informative; several chains that all start in nearly identical states provide a weaker check. The relevant summaries should be calculated from the imputed entries at each iteration, not inadvertently from a mixture of observed and imputed entries that changes the quantity being tracked.
A trace that wanders around a roughly stable level, with chains overlapping and no persistent trend, is reassuring about the behavior of that summary. It does not prove convergence of every feature of the imputation distribution, nor does it establish that the imputation model is suitable. Conversely, a visible trend, long periods with little movement, or persistent separation between chains is a reason to investigate. Monte Carlo variability can make traces look uneven, especially when few values are missing or a category is rare.
Observed and imputed values need not have the same marginal distribution. People or records with missing values can differ from those with observed values, and the imputation model conditions on available predictors. A visible gap is therefore a prompt to ask whether it is explainable by selection and predictors, whether it remains within relevant predictor groups, and whether it conflicts with substantive knowledge. The earlier material on stratified and conditional comparisons is useful when a marginal plot leaves those questions unresolved.
Choose Summaries That Respect the Variable Type
| Variable type | Useful trace summaries | Useful observed–imputed displays | Additional concern |
|---|---|---|---|
| Binary | Proportion in one specified category, separately by chain | Bar chart of category proportions | Boundary behavior or a category absent among imputations |
| Ordinal | Category proportions or cumulative proportions at each threshold | Ordered bar chart or cumulative-proportion plot | Category support and whether ordering is preserved |
| Count | Mean, zero proportion, and, when useful, upper quantiles | Count-frequency plot, with zero frequency visible | Excess zeros, implausible tail, or impossible values |
| Continuous | Mean and standard deviation; quantiles when skewed or bounded | Overlaid distributions, boxplots, or density plots | Outliers, boundary violations, or mismatched tail behavior |
For an ordinal variable, a numerical average is only an appropriate primary summary when the category scores have a meaningful quantitative interpretation. Otherwise, track category proportions or cumulative proportions. For example, the cumulative proportion at or below category \(k\) is the proportion assigned to one of the ordered categories \(1,\ldots,k\). This keeps the ordering visible without assuming equal distances between levels.
For a count variable, the mean alone can conceal a problematic distribution: many zeros and a few large values can yield the same mean as a more evenly spread set of counts. For a continuous variable, mean and standard deviation may likewise miss skew or a tail problem. In both cases, add summaries or plots that expose the feature most relevant to the data and the analysis.
Worked Example: Binary Traces and a Difference in Marginal Proportions
Suppose 20 entries for a hypothetical binary variable are missing. At one retained iteration, Chain 1 assigns the value “Yes” to 7 of those entries, and Chain 2 assigns “Yes” to 6. Calculate and interpret the trace summaries. In the observed records, 18 of 60 values are “Yes.”
For a binary variable, use the imputed proportion in the specified category. At this iteration the two chain summaries are
The observed proportion is
At this iteration, the chain summaries are close to each other, and both are near the observed proportion. This single iteration does not establish stable chains: inspect the sequence over retained iterations for trends, sticking, and recurring chain separation. Nor should the observed proportion of 0.300 be treated as a target that imputations must match. The missing records may have a different distribution, so interpretation also depends on predictors and the missingness pattern.
Worked Example: An Ordinal Trace Reveals Lost Category Support
In a hypothetical three-level ordinal variable, the levels are “low,” “moderate,” and “high.” At each iteration, 40 missing entries are imputed. Across several retained iterations, Chain 1 assigns 24, 13, and 3 entries to the levels, respectively. Chain 2 assigns 25, 12, and 3. In the observed records, 6 of 80 values are “high.” Compare the category proportions and identify what should be checked.
For Chain 1, the proportions are \(24/40=0.600\) low, \(13/40=0.325\) moderate, and \(3/40=0.0750\) high. For Chain 2, they are \(25/40=0.625\), \(12/40=0.300\), and \(3/40=0.0750\). The observed high-category proportion is
The chains agree closely on these category proportions, including the proportion assigned to “high.” That agreement is reassuring about these summaries, but does not prove convergence or adequate fit. Because “high” is uncommon, inspect its trace across iterations and check whether it disappears in any chain or important predictor subgroup. The similar marginal high proportions do not establish that the missing records have the same ordinal distribution as observed records.
Counts and Continuous Values Need More Than a Mean
Worked Example: A Count Trace With Stable Means but Different Zero Frequencies
Consider two chains imputing a nonnegative count for 30 missing entries. At one retained iteration, Chain 1 produces 18 zeros and a total count of 36. Chain 2 produces 9 zeros and a total count of 36. The observed records have 30 zeros among 100 values. Compare the means and zero proportions.
Both imputed means equal the total divided by the number of imputed entries:
But their zero proportions differ:
The matching means conceal a substantial chain difference in the zero frequency. This is a clearer concern than a small difference in means alone. Review zero proportions across all retained iterations, inspect the count distributions and relevant predictor groups, and confirm that the model and coding allow the observed count range. The observed zero proportion is context, not a required target, because missing records may differ from observed records.
For continuous variables, follow the same principle: inspect location and spread, then use shape-sensitive summaries or plots when the distribution is skewed, bounded, or heavy-tailed. A mean trace that is stable while an upper quantile drifts can signal a problem that the mean misses. An observed–imputed density or boxplot can reveal a tail mismatch, but apparent gaps should be assessed in context and, where important, within predictor strata.
Worked Example: Stable Continuous Trace, Questionable Imputed Tail
In a hypothetical biomarker measured in units per milliliter, 25 observed values have a mean of 12.0 and a standard deviation of 3.0. The imputed values have a mean of 12.4 and a standard deviation of 3.1 across retained iterations, with no clear trace trend. However, a distribution plot shows that 4 of 20 imputed values exceed 20, while none of the 25 observed values exceed 20. Calculate the means’ difference and explain why the review should not stop there.
The imputed-minus-observed mean difference is
The difference is small relative to the displayed standard deviations, and stable traces provide no obvious evidence of drift in these summaries. Yet the imputed upper tail merits review: \(4/20=0.200\), or 20.0%, of the imputed values exceed 20, compared with \(0/25=0\) in the observed group. This difference could reflect selection into missingness or sparse observed information rather than model failure. Check whether values above 20 are scientifically plausible, whether missingness is concentrated in predictor groups with higher biomarker levels, and whether the pattern is reproduced across chains and iterations.
Prioritize Findings Rather Than Counting Differences
A practical review gives greatest attention to concerns that are reproducible, consequential, and supported by more than one diagnostic view. A minor marginal shift that is stable, plausible, and explained by predictor composition is usually less urgent than a chain-specific failure to represent a rare but important category or repeated imputation of impossible values. Use the following sequence to organize the evidence.
Look for persistent trends, long periods of sticking, and disagreement in type-appropriate summaries. Ask whether the concern repeats over iterations rather than appearing at one isolated point.
For binary and ordinal variables, inspect category support and proportions. For counts, inspect zeros and the tail. For continuous variables, inspect spread, shape, and bounds as well as the mean.
Confirm that values obey coding rules and substantive limits. Determine whether a marginal observed–imputed gap could reflect differences in predictors or missingness patterns.
Prioritize findings that persist across chains, affect important values or subgroups, and are corroborated by traces and distributional displays. Record plausible explanations and what additional check would distinguish them.
This is a reasoned ranking, not a formal universal score. The most visually striking plot is not automatically the most consequential concern. A small mismatch in a clinically important rare category may warrant more attention than a larger difference in an unimportant tail region. Conversely, a difference that appears in only one sparse subgroup may be too unstable to interpret confidently.
Common Mistakes and Limits
- Using one summary for every type. A trace of the mean can miss binary category imbalance, ordinal support loss, or excess zeros. Add summaries that directly represent the feature of concern.
- Equating stable traces with good imputations. Stable chains indicate that the plotted summaries are not obviously drifting; they do not establish that the model captures the needed relationships or produces plausible distributions.
- Demanding identical observed and imputed marginals. The missing records may differ from observed records. Investigate predictor composition and conditional patterns before calling a marginal gap a defect.
- Overinterpreting a rare category. A few observations can make proportions unstable. Report the counts as well as proportions and consider whether the pattern persists across chains or relevant groups.
- Declaring a model adequate from these plots alone. These diagnostics reveal selected aspects of behavior and fit. They cannot prove that the imputation assumptions are correct or rule out bias from unmeasured structure.
When a concern remains, the next step is to sharpen the diagnostic question: identify the variable, the affected distributional feature, the chains or groups involved, and whether the issue concerns chain behavior, observed–imputed fit, or plausibility. That description is more useful than a general claim that “the imputations look different.”
Check Your Understanding
Answer each question, then compare with the brief solutions below.
- A binary trace alternates between 0.10 and 0.90 in one chain while another stays near 0.40. Name two features to investigate before concluding that the model failed.
- An ordinal variable has categories 1–4, but the category scores are labels rather than equally spaced measurements. Which trace summary is preferable to a mean?
- Two count-imputation chains have the same mean but very different proportions of zero. Why is the mean insufficient, and what should be checked next?
- Observed continuous values have a mean of 5.0, and imputed values have a mean of 5.8. Does this alone show poor imputation? Give one reason it may occur and one useful follow-up.
- A continuous mean trace is stable, but a few imputed values violate a known upper bound. Which finding should receive attention, and why?
Solutions. (1) Check whether the oscillation is persistent across iterations, whether the chain has poor mixing or a coding/model issue, and whether the summaries are based on the same imputed entries; one chain’s extreme swings are a concern but not a diagnosis by themselves. (2) Track category proportions or cumulative proportions at thresholds, since these preserve ordering without assuming equal spacing. (3) The same mean can arise from different distribution shapes; compare zero proportions and count-frequency or tail summaries across chains and review relevant predictors. (4) No. Missing records may differ from observed records; inspect distributions within important predictor groups and assess plausibility. (5) The bound violation deserves attention despite a stable mean, because a stable summary does not make impossible individual values acceptable.