Why Track Distributions Over Iterations?
A MICE run updates missing values through repeated conditional-model draws. Even if the observed data and model specification stay fixed, the imputed values can change from one iteration to the next. Tracking their marginal distributions asks a specific diagnostic question: does the shape of the imputed distribution for a variable continue to move as the run proceeds?
A stable marginal distribution is useful evidence that the run has stopped making substantial distributional changes for that variable. It is not, by itself, proof that the full chained system has converged, that different chains explore the same regions, or that the imputation model is appropriate. Marginal plots assess one variable at a time and can miss changes in relationships between variables.
The missing cases being summarized should stay fixed across the snapshots. Otherwise, a change in the displayed distribution could reflect a change in which values were included, rather than a change in the imputations. Compare completed sweeps, not partially updated states in which some variables have been updated and others have not.
Make Iteration Comparisons Comparable
A visual comparison is only interpretable when the displays are on a common footing. Use the same horizontal and vertical scales across snapshots, the same bin boundaries for histograms, and the same smoothing choices for density plots. If each plot rescales itself or chooses its own bins, real movement can look small, or an unchanged distribution can look different. The earlier material on choosing scales and bins gives the general plotting principles; here the essential requirement is to keep those choices fixed across iteration.
Choose snapshots that cover the run, for example an early, middle, and late iteration, but do not rely only on three isolated views when a longer sequence is available. A distribution may drift and later return to an earlier shape. Plotting a summary such as a selected quantile or a bin proportion against iteration can reveal such movement between snapshots.
For a continuous variable, useful summaries include selected quantiles, the interquartile range, and proportions of imputed values in fixed intervals. For a binary or categorical variable, track category proportions. No single summary describes an entire distribution: quantiles can miss changes in tails, while a mean and standard deviation can miss changes in modality or heaping. Combine a plot with summaries that reflect features important for the variable.
A simple way to summarize changes in a fixed-bin plot is the total variation distance between its bin-proportion vectors. If \(r_b^{(c,t)}\) is the proportion in bin \(b\), define
This quantity is zero when the displayed bin proportions agree exactly and is larger when more mass has shifted between bins. It summarizes the chosen binned display, not every possible difference between the underlying distributions. A different binning can yield a different value, so the bins must be fixed and reported. There is no universal cutoff at which a value of this distance establishes convergence.
Worked Example: Fixed-Bin Changes in a Continuous Imputation
Suppose one chain imputes a hypothetical continuous variable for 10 missing cases. At iteration 1 the values are \(2,3,4,4,5,5,6,7,8,10\). At iteration 10 they are \(1,3,4,4,5,5,6,7,9,14\), and at iteration 20 they are \(1,3,4,4,5,5,6,7,9,14\). Use the same four bins at every iteration: \([0,4)\), \([4,8)\), \([8,12)\), and \([12,16)\).
At iteration 1, the bin counts are \(2,6,2,0\). Dividing by 10 gives proportions \(0.20,0.60,0.20,0.00\). At iteration 10, the counts are \(2,6,1,1\), giving proportions \(0.20,0.60,0.10,0.10\). At iteration 20, the counts and proportions are the same as at iteration 10.
The distance between iterations 1 and 10 is
Thus, under this binning, 0.10 of the distributional mass has shifted between the displayed categories. Between iterations 10 and 20 the distance is zero. The snapshots suggest that this particular chain’s binned marginal has stopped changing by iteration 10 in this hypothetical run. They do not prove convergence: other iterations, other distributional features, and other chains still need consideration.
Check Chains Separately
MICE commonly uses multiple chains. Track the distribution within each chain before combining information across chains. Pooling can conceal chains that move in opposite directions. Conversely, two chains can each appear stable while remaining at different distributional levels. These patterns make a pooled marginal alone an incomplete stability check.
Worked Example: Pooling Hides Opposing Chain Movement
Consider a hypothetical binary variable with 20 missing cases in each of two chains. At iterations 1, 5, 10, and 20, chain A has 4, 7, 8, and 8 imputed ones; chain B has 14, 11, 10, and 10 imputed ones. The proportions in chain A are \(4/20=0.20\), \(7/20=0.35\), \(8/20=0.40\), and \(8/20=0.40\). In chain B they are \(14/20=0.70\), \(11/20=0.55\), \(10/20=0.50\), and \(10/20=0.50\).
At iterations 1 and 5, pooling the 40 imputed values gives \(18/40=0.45\) ones. At iterations 10 and 20, pooling gives \(18/40=0.45\) ones again. The pooled proportion is therefore constant at the four snapshots, despite chain A increasing from 0.20 to 0.40 and chain B decreasing from 0.70 to 0.50.
Within-chain tracking reveals substantial early movement that the pooled proportion hides. By iterations 10 and 20, both chains are stable across these snapshots, but they remain at different proportions, 0.40 and 0.50. The example therefore does not support declaring the chains interchangeable based on pooled stability. Inspect chain-specific plots and summaries, and treat remaining differences as a reason for further investigation.
Why One Summary Is Not Enough
Two distributions can share a mean while having very different shapes. The reason is that a mean compresses all values into one weighted average; different arrangements of values can produce the same average. Comparing quantiles, spreads, and fixed-bin proportions alongside a mean helps detect changes that a location summary alone cannot show.
Worked Example: A Stable Mean With a Changing Shape
Suppose a hypothetical chain imputes a variable for eight missing cases. At an early iteration the values are \(0,0,0,0,10,10,10,10\). At a later iteration they are \(4,4,4,4,6,6,6,6\). The early mean is
The later mean is also
Yet, with fixed bins \([0,3)\), \([3,7)\), and \([7,11)\), the proportions change from \(0.50,0.00,0.50\) to \(0.00,1.00,0.00\). The imputed distribution has changed markedly even though its mean is unchanged. A mean trace alone would fail to identify this shift. A fixed-scale plot or these bin proportions would show it clearly.
Interpreting Stability and Diagnosing Problems
Distributional stability is a diagnostic judgment, not an assumption that the imputed marginal must match the observed marginal. The observed and missing cases can differ in their distributions, so an iteration comparison should answer whether the imputations are still changing—not whether they reproduce the observed values exactly. Comparing observed and imputed distributions addresses a related but distinct question.
There is no single required numerical threshold for calling a distribution stable. Decide what changes would matter for the variable and analysis, examine the late-run behavior, and report the summaries or plots used. Small irregular fluctuations can occur in a stochastic run even when there is no sustained drift. A plot of summaries over many iterations helps distinguish such local variation from a persistent trend.
Warning signs include a continuing shift in quantiles or category proportions, changing tail frequencies, a trend in spread, or chains that settle at visibly different distributions. A display that changes because of automatic binning or rescaling is not evidence of movement in the imputations; remake it using consistent settings before interpreting it. If substantial changes remain late in the run, running more iterations may help diagnose whether the movement continues, but simply running longer does not repair a poorly specified imputation model.
A useful workflow is to state the variable and missing cells being monitored, create chain-specific snapshots or summary traces, use consistent display settings, and compare several aspects of shape. Record both apparent stability within chains and any differences between chains. This is complementary to scalar trace plots and other convergence diagnostics: scalar traces are easy to follow across iterations, while distributional comparisons can reveal changes that a selected scalar misses. Neither approach alone establishes that all features of the chained system have converged.
Identify the variable and keep the set of missing cells fixed across the iteration snapshots.
Use consistent scales, bins, and smoothing settings so differences reflect the imputations rather than plotting choices.
Review plots and suitable summaries, such as quantiles and bin proportions, across early, middle, and late iterations.
Look for sustained late-run movement and between-chain differences; describe the diagnostic evidence without treating it as a proof of convergence.
Common Mistakes and Limitations
- Calling a stable mean a stable distribution. The mean can remain unchanged while spread, tails, or modality change. Add distribution-sensitive summaries or plots.
- Pooling chains before checking them. Opposing chain movement can cancel in a pooled summary. Examine each chain first.
- Changing bins or scales between iterations. This makes visual differences hard to interpret. Hold plotting choices fixed.
- Equating stable marginals with convergence of the full system. Marginal stability does not guarantee stable joint relationships or adequate model specification.
- Demanding identical empirical distributions at every iteration. Imputation is stochastic, so minor fluctuations are expected. Look for meaningful sustained changes, not exact repetition.
These checks are limited by the features they display, the iterations selected, and the number of imputed values. Sparse information can make a marginal plot noisy, and a small number of snapshots can miss transient movement. Distributional stability is best interpreted as one component of a broader diagnostic assessment.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- A binary variable has 30 missing cases. At two iterations, a chain imputes 9 and 15 ones. Calculate the proportions and describe the change.
- Two chains each impute 10 values. At one iteration they contain 2 and 8 ones; at a later iteration they contain 5 and 5 ones. What pooled proportions result, and why should the chains still be checked separately?
- Two snapshots have the same mean but different proportions in fixed bins. Which conclusion is supported, and which is not?
- A density plot appears to become narrower across iterations, but each plot uses an automatically chosen horizontal scale. What should be changed before interpreting the apparent narrowing?
Solutions. For question 1, the proportions are \(9/30=0.30\) and \(15/30=0.50\), a rise of \(0.20\), or 20 percentage points. This shows a change in that chain’s binary imputed marginal at those snapshots; further iterations and diagnostics are needed to judge stability. For question 2, the pooled proportion is \((2+8)/20=0.50\) initially and \((5+5)/20=0.50\) later. The unchanged pooled value hides the chains’ different starting proportions and movement, so inspect each chain. For question 3, the fixed-bin proportions support the conclusion that the displayed distributions differ, despite equal means. They do not establish why the distributions differ or whether the run has converged. For question 4, redraw the plots using the same horizontal and vertical scales and the same smoothing choices. Only then can the apparent narrowing be assessed as a change in the imputations rather than a plotting artifact.