Why Overlay the Two Value Groups?
For a continuous variable with missing entries, an overlaid plot places its observed values and its imputed values on the same horizontal scale. The purpose is descriptive: to see whether the two groups show conspicuous differences in their marginal distributions, such as a shift in location or a different range. Such a plot is a useful diagnostic, but it does not by itself establish that an imputation model is valid or invalid.
In MICE, each missing value is imputed using a conditional model, typically involving other variables. Consequently, the imputed values need not have the same marginal distribution as the observed values. If the probability of a value being missing, or the conditional distribution used to impute it, varies with predictors, a marginal difference can be appropriate. The earlier material on chain summaries and stabilization is also relevant: distributional comparisons are more interpretable once the chain has moved beyond burn-in and its behavior is reasonably stable.
Constructing a Comparable Overlay
A histogram is a straightforward way to make the overlay. Divide the variable’s range into bins with shared boundaries for both groups. Then scale the heights so that each group is comparable even when the observed and missing groups have different sizes. Raw counts are usually a poor choice: a larger group can have taller bars simply because it contains more records.
One useful scale is histogram density. For group \(g\), let \(n_g\) be its number of values, \(c_{g,b}\) the number falling in bin \(b\), and \(h_b\) the width of that bin. The height is \(c_{g,b}/(n_g h_b)\). For equal-width bins of width \(h\), this is \(c_{g,b}/(n_g h)\). Multiplying each height by its bin width gives the proportion in that bin; the areas across all bins sum to one for each group.
The density scale is not a probability density model fitted to the data; it is a normalization of histogram counts. For a quick comparison, a relative-frequency histogram is also suitable: its height in bin \(b\) is \(c_{g,b}/n_g\), so heights sum to one. Whatever scale is used, label it clearly. A density histogram’s vertical axis has units reciprocal to the horizontal axis—for example, per milligram per deciliter if the measurements are in mg/dL.
A practical overlay uses distinct colors or outlines, a legend identifying “observed” and “imputed,” the original measurement units, and the same bin boundaries for both groups. If the imputed values are much fewer or more numerous than the observed values, normalization is especially important. If several imputations are available, overlaying one imputation at a time against the observed values makes it easier to see whether a pattern is consistent or varies across imputations. Pooling draws across imputations can give a compact summary, but hides that variation.
Use observed values for records where the variable was observed and imputed values for records where it was missing. State whether the display is for one imputation or summarizes several.
Use the same units, transformation, range, and bin boundaries for both groups. Include the ranges of both groups so an unusual tail or gap is not cropped away.
Use relative frequencies or density heights rather than unadjusted counts when group sizes differ. Label the groups and vertical-axis scale.
Look for conspicuous differences in location, spread, range, or shape, then interpret them in light of the predictors and imputation model. Treat the plot as a diagnostic prompt, not a pass-fail test.
Worked Comparisons
Worked Example: Similar Marginal Patterns on a Shared Scale
Suppose a hypothetical dataset records a continuous laboratory measurement in mg/dL. For one imputation, the ten observed values are 10, 11, 12, 12, 13, 13, 14, 15, 16, and 18. The six imputed values are 11, 12, 13, 14, 15, and 16. Use bins \([10,12), [12,14), [14,16)\), and \([16,18]\), with the final bin including its upper endpoint.
The observed bin counts are 2, 4, 2, and 2. The imputed bin counts are 1, 2, 2, and 1. Since the group sizes differ, compare density heights rather than counts. The bin width is 2 mg/dL, so the first observed height is \(2/(10\times2)=0.1000\) per mg/dL, while the first imputed height is \(1/(6\times2)=0.08333\) per mg/dL. Applying the same calculation to each bin gives observed heights \(0.1000, 0.2000, 0.1000, 0.1000\) and imputed heights \(0.08333, 0.1667, 0.1667, 0.08333\).
The observed mean is \(134/10=13.4\) mg/dL, and the imputed mean is \(81/6=13.5\) mg/dL. These calculations support the visual impression of broadly similar location and range in this small hypothetical example. They do not demonstrate that the imputation model is correct: the plot is a marginal comparison, and the data are too limited to rule out finer differences.
Worked Example: A Conspicuous Shift Toward Lower Imputed Values
Consider a second hypothetical dataset of a continuous measurement in mg/dL. Ten observed values are 90, 95, 100, 105, 110, 115, 120, 125, 130, and 135. Six imputed values for one imputation are 55, 60, 65, 70, 75, and 80. Use common bins \([40,60), [60,80), [80,100), [100,120)\), and \([120,140)\), each 20 mg/dL wide.
The observed counts are 0, 0, 2, 4, and 4. The imputed counts are 1, 4, 1, 0, and 0. For example, the observed height in \([100,120)\) is \(4/(10\times20)=0.02000\) per mg/dL. In that same bin, the imputed height is zero. The imputed height in \([60,80)\) is \(4/(6\times20)=0.03333\) per mg/dL, compared with zero observed values in that bin.
This overlay would show a conspicuous marginal shift toward lower values among the imputations. It is a reason to investigate, not proof of a specific error. Check the imputation model and predictors, the observed data patterns, the scale and units, and whether the displayed chain has stabilized. A lower marginal distribution could be consistent with the covariate profiles of records with missing values; alternatively, it could signal a model or implementation problem.
Worked Example: Why Raw Counts Can Conceal a Difference
In a hypothetical sample, 80 observed values and 20 imputed values are grouped into two equal-width bins. The observed counts are 40 in each bin. The imputed counts are 18 in the first bin and 2 in the second. If counts are overlaid directly, the first-bin bars look fairly similar: 40 observed versus 18 imputed. But these counts represent different group sizes.
The observed relative frequencies are \(40/80=0.5000\) in each bin. The imputed relative frequencies are \(18/20=0.9000\) and \(2/20=0.1000\). Thus, the imputed group is much more concentrated in the first bin than the observed group. If the bin width is 10 units, the corresponding density heights are \(0.5000/10=0.05000\) per unit for each observed bin and \(0.9000/10=0.09000\) and \(0.1000/10=0.01000\) per unit for the imputed bins.
The arithmetic shows why a shared scale alone is not enough: the vertical scale must also account for group size. A raw-count overlay could lead to an incorrect impression of similarity.
What the Plot Can and Cannot Diagnose
An overlay is most useful as a way to notice a pattern that deserves a closer look. A visible shift in the imputed group may prompt questions about predictors, conditional-model specification, or the records with missing values. A notably different range may prompt checks for impossible values, truncation, coding errors, or a model that does not respect the variable’s support. Differences that appear in one imputation but not others may suggest that the plot is sensitive to imputation-to-imputation variation.
There is no requirement that the marginal distribution of observed values equal that of imputed values. For example, if records with larger predictor values are more likely to have the target variable missing, their imputed values may reasonably be shifted relative to the observed group. The plot combines this pattern of missingness with the conditional imputation model; it does not isolate either one. Missingness assumptions cannot be verified from this overlay alone.
Nor is a histogram comparison a formal hypothesis test. Its appearance can depend on bin width and boundaries: a narrow feature may be hidden by wide bins, while small changes in boundaries can make a modest difference seem prominent. Use reasonable bins and, if a feature drives a substantive concern, check whether it persists under a small number of sensible choices. Do not keep changing the bins until a preferred conclusion appears.
When \(m\) imputations are available, there are two common display choices. One is to plot observed values against the imputed values from each completed dataset, repeating this for several imputations or showing them with light transparency. This reveals whether the visual pattern is consistent. Another is to pool the \(m\) sets of imputed values and make a single imputed histogram. A pooled display can summarize the average imputed marginal distribution, but the \(m\) draws for a given missing record are repeated draws for that record, not \(m\) new independent records. The pooled plot also hides between-imputation variation. It should not be interpreted as though its enlarged count were an enlarged sample of distinct people.
A density plot is another standard display for continuous distributions, but the focus here is on the directly binned overlay. Histograms make the role of bin boundaries explicit; they are less smooth and can be unstable in small samples. Whichever graphical approach is used, it should complement checks of chain behavior and examination of the imputation model rather than replace them.
Common Mistakes and Limitations
- Comparing raw counts. “The bars are similar in height, so the distributions are similar” ignores different group sizes. Normalize each group separately before comparing shapes.
- Using different bins. Bars from different boundaries cannot be compared directly. Choose shared bins before drawing either group.
- Treating a marginal difference as automatic failure. The missing records can have different predictor profiles. A discrepancy calls for contextual investigation, not immediate rejection of the imputations.
- Pooling all imputations without noting the consequence. A single pooled overlay hides variation across imputations and counts repeated draws for the same missing records. State how the imputed values were assembled.
- Reading too much into one plot. Sample size, bin choices, and chain instability can affect appearance. A histogram cannot establish that missingness assumptions hold, certify the conditional model, or diagnose every form of model failure.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- Why should observed and imputed histograms use the same bin boundaries?
- A bin contains 12 observed values among 60 observed records and 6 imputed values among 15 missing records. Compare their relative frequencies in that bin.
- Does a marginal shift in imputed values prove that an imputation model is wrong? Give one reason for your answer.
- What information can be lost by pooling imputed values from all imputations into one histogram?
- Name two checks to make if a conspicuous discrepancy appears in an overlay.
Solutions. (1) Shared boundaries ensure that each pair of bars represents the same interval of the variable; otherwise differences in bar heights may reflect binning rather than values. (2) The observed relative frequency is \(12/60=0.2000\); the imputed relative frequency is \(6/15=0.4000\). The imputed group has twice the proportion in that bin, despite having fewer records. (3) No. Records with missing values may have different predictor profiles, so their conditional imputed values can yield a different marginal distribution. The discrepancy warrants investigation but is not proof of failure. (4) Pooling hides variation across imputations and treats repeated draws for the same missing records as if they were additional distinct records if the pooled count is read naively. (5) Check that the chain has stabilized and inspect the predictors, missingness patterns, variable units and support, and imputation-model specification.