Why Read Traces and Distribution Plots Together?
A continuous-variable imputation can show two quite different patterns of concern. The imputed values may still be moving systematically as the chained equations iterate, or they may have settled into a stable pattern whose distribution differs from the observed values. The first is a question about the behavior of the algorithm; the second is a comparison between two groups of values. A plot addressing one question cannot answer the other.
This distinction matters because observed values and values imputed for cases with missing data need not have the same marginal distribution. The people with observed values may differ from those with missing values, and the imputation model may condition on predictors related to the outcome. A difference between the groups is therefore a diagnostic clue to interpret, not proof by itself that the imputation is wrong. Conversely, a distribution plot that looks plausible does not establish that the algorithm has stabilized.
For example, the imputed mean in chain \(m\) at iteration \(t\) is
Here \(R_i=0\) indicates that \(Y_i\) was missing for case \(i\). Other useful trace summaries include the imputed standard deviation, median, or selected quantiles. The trace plot places the iteration on the horizontal axis and one such summary on the vertical axis, with a separate trajectory for each chain.
Assumptions for a Useful Diagnosis
A trace plot is interpretable only when the plotted quantity is defined consistently across iterations and chains. The same variable, units, missing cases, and summary statistic must be used throughout. Chains should be labelled separately, and their iteration indices should refer to comparable stages of the same imputation procedure. If the set of cases being summarized changes, apparent movement may reflect the changed summary rather than movement of the imputations.
Reading a trace for stabilization also relies on a working assumption: after any initial transient, the iterative procedure should fluctuate around a reasonably stable range for the summary being examined. A flat trace is evidence about that summary, not a mathematical guarantee that every feature of the imputed data has stabilized or that the imputation model is adequate. The summary may conceal changes elsewhere in the distribution.
For distribution plots, compare values on the same measurement scale and make clear which observations are observed and which are imputed. The comparison should use imputed values from a period judged sufficiently stable, rather than pooling an arbitrary mixture of early and late iterations. If observed cases and cases with missing \(Y\) differ in their predictors, their marginal distributions can differ even when the conditional imputation relationships are plausible. The earlier material on selection effects and observed-imputed comparisons is relevant to interpreting that gap.
Worked Example: Locating Drift in Three Chains
In a hypothetical data set, a laboratory measurement is recorded in mg/L. Forty values are missing. Three MICE chains are run, and the mean of the 40 imputed values is recorded at selected iterations:
| Iteration | Chain 1 mean (mg/L) | Chain 2 mean (mg/L) | Chain 3 mean (mg/L) |
|---|---|---|---|
| 1 | 42 | 45 | 40 |
| 5 | 49 | 50 | 46 |
| 10 | 54 | 53 | 52 |
| 15 | 57 | 56 | 55 |
| 20 | 56 | 57 | 56 |
| 25 | 57 | 56 | 57 |
The chains begin at different levels and all move upward during the early iterations. For chain 1, the endpoint change from iteration 1 to 25 is \(57-42=15\) mg/L over \(25-1=24\) iteration steps. Its descriptive endpoint slope is
This is not a formal test of convergence; it summarizes the conspicuous early movement. The late values, from iterations 15 to 25, are between 56 and 57 mg/L in chains 1 and 2 and between 55 and 57 mg/L in chain 3. The chains overlap and show small up-and-down changes rather than continued movement in one direction. The trace therefore suggests substantial early drift followed by stabilization of the imputed mean around 56 mg/L. It does not establish that all distributional features are stable.
Compare Distributions After Stabilization
Once the trace suggests a stable period, compare the observed values with imputed values from that period. A histogram or density overlay can reveal shifts, spread differences, skewness, or tail discrepancies. Quantiles and bin proportions give compact numerical checks alongside the plot. The comparison should not combine a changing early phase with the later phase: doing so can make the imputed distribution look more dispersed or shifted than the stabilized imputations themselves.
Worked Example: A Persistent Marginal Difference
Continue the hypothetical example. There are 120 observed measurements, with mean 50.0 mg/L and standard deviation 10.0 mg/L. After the trace has stabilized, the 40 imputed measurements have mean 56.5 mg/L and standard deviation 10.5 mg/L. A post-stabilization histogram groups both sets into the same four ranges:
| Measurement range (mg/L) | Observed count (out of 120) | Imputed count (out of 40) | Observed proportion | Imputed proportion |
|---|---|---|---|---|
| Below 40 | 24 | 4 | 24/120 = 0.200 | 4/40 = 0.100 |
| 40 to below 50 | 42 | 8 | 42/120 = 0.350 | 8/40 = 0.200 |
| 50 to below 60 | 36 | 16 | 36/120 = 0.300 | 16/40 = 0.400 |
| 60 or above | 18 | 12 | 18/120 = 0.150 | 12/40 = 0.300 |
The counts add to the stated group sizes: \(24+42+36+18=120\) observed values and \(4+8+16+12=40\) imputed values. The imputed group has a larger share at 50 mg/L or above: \(0.400+0.300=0.700\), compared with \(0.300+0.150=0.450\) among observed values. The mean difference is \(56.5-50.0=6.5\) mg/L. The higher imputed mean and the bin proportions both indicate a rightward marginal shift in the imputed group.
Because the trace stabilized near these imputed values before the distribution comparison was made, the observed-imputed gap is not explained simply by the early upward drift in the chains. It persists after stabilization. That makes it a reason to investigate the imputation model and the characteristics of cases with missing values, but it does not alone show that the imputation is defective: the two groups can legitimately differ marginally. A comparison within relevant predictor groups can help determine whether the overall gap reflects group composition or a mismatch within groups.
A Practical Diagnosis Sequence
Confirm the variable, units, chain labels, iteration index, and cases summarized. Plot comparable summaries for every chain.
Look for directional change over many iterations, long periods at distinct chain levels, or a late change in variability. Short irregular fluctuations alone do not establish drift.
Use a period in which traces overlap and fluctuate without an obvious sustained trend. Do not treat a chosen cutoff as proof of convergence.
Use a shared scale and report the sample sizes. Look at location, spread, shape, and tails rather than relying on the mean alone.
If a gap remains, examine whether it is expected from differences between cases with and without observed values, and whether it persists within relevant predictor groups. If drift remains, investigate the chain behavior before interpreting the distribution comparison.
Worked Example: Choosing the Next Diagnostic Step
In a second hypothetical run, the observed mean is 72.0 units. One chain’s imputed means at iterations 10, 20, 30, and 40 are 68.0, 70.5, 72.0, and 73.5; the other chains also rise through iteration 40. The post-iteration-40 imputed histogram is shifted above the observed histogram.
The rise from iteration 10 to 40 is \(73.5-68.0=5.5\) units, and it has not levelled off by the last recorded iteration. The immediate diagnosis is therefore continuing chain movement, not a settled marginal discrepancy. A distribution plot based only on late iterations may still be informative descriptively, but it cannot yet establish that the gap persists after stabilization. Simply extending the run may be useful, but the analyst should also inspect chain setup and the imputation specification if the directional movement continues.
Suppose instead that a longer run fluctuates around 73 units in every chain, and the imputed distribution remains shifted above the observed one. The computational-drift concern is then reduced for the displayed summaries, while the marginal discrepancy remains. The next question is whether that gap is consistent with differences in predictors or missingness patterns, or whether a relevant feature is not represented adequately in the imputation procedure. Matching the two marginal distributions by force is not a justified remedy: it could erase real differences relevant to the missing cases.
What Each Diagnostic Can and Cannot Show
Trace plots and observed-imputed plots answer complementary questions. A trace plot is ordered by iteration and helps reveal movement over the algorithm’s run. A distribution overlay summarizes the values in groups but generally does not show when those values arose. Numerical summaries such as means and standard deviations are easier to compare precisely, but can miss differences in shape or tails. A histogram or density plot adds shape information but can be sensitive to binning or smoothing choices.
Other diagnostics address different concerns. The earlier material on masked-value diagnostics uses values temporarily hidden from an analysis to assess predictive performance under that masking setup. Such checks do not replace inspection of chain behavior, and a stable trace alone does not establish predictive accuracy. Likewise, a trace that appears stable for the mean may conceal movement in spread or tails. Inspect more than one summary when those features matter, and consider relevant subgroup comparisons when an overall marginal gap may be driven by differing case composition.
These plots are diagnostic evidence, not pass-fail tests. Their conclusions depend on the amount of observed information, the number of missing values, the chosen summaries, and the display scale. A plot can fail to reveal slow movement or a localized problem. Strong evidence of drift warrants further investigation; absence of visible drift is not proof that the whole imputation process is correct.
Common Mistakes
- Calling every gap an algorithm failure. A stable imputed distribution can differ from the observed marginal distribution because the groups differ. Investigate the context and relevant predictors before judging the gap.
- Calling a flat mean trace full convergence. The mean is only one summary. Check other relevant features, and remember that a trace is evidence rather than a guarantee.
- Pooling early and late iterations for the distribution plot. If the chain drifts early, that mixture can misrepresent the stabilized imputations. Compare a defensible stable window and show how it was chosen.
- Forcing imputed values to match observed values. The observed cases are not automatically a valid marginal target for cases whose values are missing. Unexplained differences should prompt diagnosis, not mechanical distribution matching.
Check Your Understanding
Answer each question using the distinction between chain behavior and distributional fit.
- Three chains’ imputed means move steadily upward through the final displayed iteration. What can you conclude about a post-stabilization distribution comparison?
- Observed values have mean 30 units, while stabilized imputed values have mean 34 units. Calculate the difference and state what it does and does not establish.
- Why might a trace of the imputed mean look stable even if the imputed distribution is still changing?
- Observed and imputed values differ marginally, but the groups have different predictor distributions. What is a useful next comparison?
- A trace fluctuates irregularly around the same level in three chains after an early transient. Does this prove that the imputation procedure is correct? Explain.
Solutions. (1) The traces do not yet show stabilization of the displayed means, so the late distribution cannot be described as a post-stabilization comparison. Continue diagnosis of the drift; a distribution plot alone does not resolve it. (2) The difference is \(34-30=4\) units, with imputed values higher on average. It establishes a marginal mean difference in the displayed groups, not that the imputation is wrong or that the underlying population mean differs by 4 units. (3) Changes in spread, skewness, or tails can occur while the mean remains similar; inspect additional summaries or distribution plots. (4) Compare observed and imputed distributions within relevant predictor groups, so a difference in group composition is not mistaken for a within-group mismatch. (5) No. The pattern is evidence of stability for the plotted summary over the displayed iterations, not proof of correctness, adequate predictive performance, or stability of every feature.