Why Shape and Boundaries Need Special Attention
A distribution can be poorly represented by a mean and standard deviation when it is strongly skewed: a small number of large values may dominate both. A variable with known limits presents a different challenge. Imputed values may cluster at a floor or ceiling, even when observed values do not, or may fall outside the variable’s possible range. Either problem can be hard to see in a single summary.
In MICE diagnostics, plots and summaries of observed and imputed values are evidence about the plausibility of the imputation model, not a test that the two groups must have the same distribution. Observed values are available for people with observed data; imputed values are generated for people whose values are missing. Those groups can differ in their predictors, and therefore can have different marginal distributions even when the imputation model is appropriate.
Choose Summaries That Match the Distribution
For a strongly right-skewed variable, useful summaries often include the median, a few quantiles, and proportions above a substantively meaningful threshold. These describe the center and selected parts of the distribution without allowing a handful of extreme values to determine the whole comparison. The mean and standard deviation can still be informative, especially when they are the quantities used in the substantive analysis, but they should not be the only summaries.
For a bounded variable, report the range and pay particular attention to the fractions at the lower and upper limits. It can also be useful to report a near-boundary fraction, defined in advance using a meaningful distance from the limit. For a variable on a 0-to-1 scale, for instance, the fractions exactly at 0 and within 0.05 of 0 answer different questions: one describes exact endpoints, while the other also captures values close to the floor.
Plots should respect the variable’s support. A histogram or density plot can show broad shape, while a quantile plot or empirical cumulative distribution function can help compare distributional positions. For bounded variables, make the limits visible and check whether any imputed values fall outside them. No single plot answers every question: a smooth density can obscure exact endpoints, so pair it with an explicit boundary count or proportion.
Use a consistent scale and comparable plotting choices for the observed and imputed values. If there are multiple completed data sets, do not simply stack all imputed values across them and treat the result as one larger sample: each missing record then appears repeatedly. Instead, examine the distribution in a completed data set or calculate the diagnostic separately in each completed data set and summarize its variation.
Worked Example: Comparing a Right-Skewed Duration
Suppose a hypothetical diagnostic data set contains 10 observed waiting times and 8 imputed waiting times, all measured in minutes. The sorted observed values are 1, 1, 2, 2, 3, 4, 5, 7, 10, 25. The sorted imputed values are 1, 2, 3, 4, 6, 9, 18, 30. Both sets have a long right tail.
The observed mean is \(60/10=6.00\) minutes, and its median is \((3+4)/2=3.50\) minutes. The imputed mean is \(73/8=9.125\) minutes, and its median is \((4+6)/2=5.00\) minutes. The mean difference is \(9.125-6.00=3.125\) minutes; the median difference is \(5.00-3.50=1.50\) minutes.
The mean is more strongly affected by the largest observations: 25 minutes among the observed values and 30 minutes among the imputed values. A plot and quantiles would help show whether the imputed upper tail is unusually extended. For a simple tail check, the proportion exceeding 10 minutes is \(1/10=10\%\) among observed values and \(2/8=25\%\) among imputed values. In this small illustration, that difference flags a feature to investigate; it does not establish that the imputation is wrong. The two groups may differ in predictors, and the small denominators make the proportions imprecise.
Check Limits and Boundary Rates Explicitly
A valid range should come from the variable’s definition, measurement protocol, or data-coding rules—not merely from the minimum and maximum values that happen to be observed. If a proportion is defined on \([0,1]\), an imputed value of 1.08 is outside its support. If an instrument records a score from 0 to 100, values above 100 require investigation. Possible explanations include a coding error, a scale mismatch, or an imputation model that does not respect the support.
Exact limits require a separate check. For lower limit \(L\) and upper limit \(U\), calculate the exact-boundary proportions among observed and imputed values using their respective denominators. If a near-boundary diagnostic is useful, define a tolerance \(\delta\) in the variable’s units and count values at most \(L+\delta\), or at least \(U-\delta\). The tolerance should have a substantive interpretation; choosing it after seeing the plot can make an apparent discrepancy seem larger or smaller than it is.
Worked Example: A Floor and Ceiling in a Bounded Score
Consider a hypothetical score with valid limits of 0 and 100. Among 200 observed scores, 8 equal 0 and 4 equal 100. Among 60 imputed scores, 15 equal 0 and 2 equal 100. Calculate the exact-boundary proportions separately for each limit.
At the floor, the observed proportion is \(8/200=0.0400\), or 4.00%. The imputed proportion is \(15/60=0.250\), or 25.0%. At the ceiling, the observed proportion is \(4/200=0.0200\), or 2.00%; the imputed proportion is \(2/60=0.0333\), or 3.33%.
The notable discrepancy is at the floor, not the ceiling. The imputed floor rate is 21.0 percentage points higher than the observed floor rate. This pattern warrants checking whether zero is a genuine score, whether the imputation model can represent floor values appropriately, and whether the people with missing scores differ in predictors associated with low scores. It does not justify concluding that the imputed values are invalid: observed and missing cases can have different covariate distributions, and the observed rate is not automatically the correct rate for missing cases.
Why Observed and Imputed Marginal Shapes May Differ
A useful justification comes from conditioning on predictors. Let \(Y\) be the incomplete variable, \(X\) the predictors used in its imputation model, and \(R\) indicate whether \(Y\) is observed. Under a missing-at-random assumption for this variable, the distribution of \(Y\) given \(X\) is the same in observed and missing cases. This does not imply that their marginal distributions are the same, because the distributions of \(X\) can differ.
Under the stated assumption, \(f(y\mid x,R=0)=f(y\mid x,R=1)\). But if \(f(x\mid R=0)\) differs from \(f(x\mid R=1)\), the integrals for \(r=0\) and \(r=1\) need not agree. Thus, even a correctly specified conditional imputation model can produce imputed values whose marginal shape differs from the observed values. A discrepancy is a prompt to investigate, not a demand that the two marginal plots match exactly.
Conversely, close marginal agreement does not prove that the conditional imputation model is adequate. Errors in different predictor groups can cancel when groups are combined. For this tutorial’s marginal checks, the practical conclusion is limited: assess whether shape and boundary patterns are plausible, then interpret them with knowledge of the variable and missingness process.
Worked Example: Interpreting Near-Zero Values on a Proportion Scale
A hypothetical variable is a proportion with valid limits 0 and 1. Among 80 observed values, 4 are exactly 0 and 10 are at most 0.05. Among 40 imputed values, 12 are exactly 0 and 20 are at most 0.05. Suppose the observed median is 0.42 and the imputed median is 0.31.
The exact-zero rates are \(4/80=0.0500\), or 5.00%, and \(12/40=0.300\), or 30.0%. The near-zero rates are \(10/80=0.125\), or 12.5%, and \(20/40=0.500\), or 50.0%. The imputed distribution is therefore more concentrated near the lower limit in this summary, and its median is also lower by \(0.42-0.31=0.11\) proportion units.
These checks reveal more than the median alone: they distinguish an exact-zero excess from a broader concentration near zero. The next investigation is contextual. If zero is a genuine and common outcome for cases with particular predictor values, the difference may be plausible. If zero is rare by definition and appears because predictions were clipped to the valid range, the pattern may signal a modelling or data-processing problem. The marginal summaries alone cannot distinguish those explanations.
Diagnostics, Alternatives, and Limits
A useful diagnostic sequence is to verify the variable’s units and valid range, compare plots on a common scale, calculate robust summaries and boundary rates, and then check whether visible discrepancies have a plausible explanation. For skewed variables, ask whether the center, upper quantiles, and tail proportions tell a consistent story. For bounded variables, check both support violations and endpoint concentrations. If a diagnostic changes sharply when the summary changes—for example, medians look similar but upper-tail proportions do not—describe the specific feature rather than declaring the distributions simply “the same” or “different.”
Establish the valid units and limits from the variable definition. Separate genuine endpoint values from missing-value codes or recording artifacts.
Use medians or quantiles and relevant tail proportions for skew; use exact and, when justified, near-boundary rates for bounded variables.
Use a plot suited to the distribution and make limits visible. Check for values outside support and for patterns that numerical summaries can conceal.
Consider differences in predictors and missingness before treating observed values as a benchmark for every missing case.
The mean and standard deviation are compact alternatives when they directly match a scientific question, but can be misleading as the sole description of a skewed distribution. A log transformation can make positive, right-skewed values easier to display or model, but it changes the scale; conclusions on the transformed scale are not automatically conclusions in the original units. Models designed for bounded outcomes may respect support better than an unconstrained model followed by truncation, but no model choice is validated by a marginal plot alone. A plot of observed and imputed values is a diagnostic, not a substitute for checking the conditional model and its assumptions.
Important failure modes include comparing values in inconsistent units, overlooking impossible imputed values, relying only on a smooth density that hides exact endpoints, and interpreting a boundary excess without checking whether the endpoint is substantively meaningful. A further limitation is that observed data cannot reveal the true distribution among missing cases without assumptions. When missingness depends on unobserved values, marginal agreement or disagreement cannot by itself resolve that uncertainty.
Common Mistake
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- For a strongly right-skewed positive variable, name two summaries that complement the mean and standard deviation.
- A bounded variable has 5 exact-zero values among 100 observed records and 9 among 30 imputed records. Calculate both zero rates and their difference in percentage points.
- Why can observed and imputed marginal distributions differ even under a suitable missing-at-random imputation model?
- An imputed value lies above a known upper limit. Give two checks to make before concluding why it occurred.
- Why should imputed values from multiple completed data sets not simply be stacked and treated as independent observations?
Solutions. (1) The median and a relevant upper quantile or tail proportion are useful; they are less dominated by a few extreme values. (2) The observed rate is \(5/100=5.00\%\), and the imputed rate is \(9/30=30.0\%\). The difference is \(30.0-5.00=25.0\) percentage points. (3) The predictor distributions can differ between observed and missing cases, so averaging the same conditional distribution over different predictor distributions can produce different marginals. (4) Verify the valid range and units, then check coding, preprocessing, and whether the imputation method respects the support. (5) Each missing record appears once in every completed data set; stacking repeats those records and falsely treats the repeated imputations as independent observations.