Quantile Plots for Observed and Imputed Values

Why Compare Quantiles?

A comparison of observed and imputed values should ask more than whether the two groups have similar means. Their distributions may differ in the center, in their spread, or only in a tail. A quantile plot places the two distributions on a common probability scale and makes these differences visible across the range of values. It is particularly useful when a discrepancy is localized—for example, when most quantiles agree but the largest imputed values are much higher.

The comparison is a diagnostic, not a test that observed and imputed values must have identical distributions. Observed values belong to cases whose variable was recorded; imputed values belong to cases whose value was missing. Those cases can differ in their covariates, even when the imputation model is appropriate. The plot can identify patterns that merit investigation, but it cannot by itself establish whether the imputation model is correct.

For a simple, transparent plotting convention, sort the values as \(x_{(1)}\le\cdots\le x_{(n)}\), and plot \(x_{(i)}\) at \(p_i=(i-0.5)/n\). These are plotting positions, not a claim that each order statistic is exactly the population quantile at that probability. With different sample sizes, compare the two sets of points at common probability levels by interpolating between adjacent order statistics, or use one documented sample-quantile convention for both groups.

Constructing and Reading a Quantile Plot

For a variable \(Y\), sort its observed values and its imputed values separately. Draw their quantiles against the same probability axis, \(p\). One display overlays the two quantile curves; another plots the difference \(Q_{\mathrm{imp}}(p)-Q_{\mathrm{obs}}(p)\) against \(p\). The difference display is often easier to read: values near zero indicate similar empirical quantiles, positive values indicate larger imputed quantiles, and negative values indicate smaller imputed quantiles. The vertical axis retains the variable’s units.

Use the probability axis to locate where a discrepancy occurs. A roughly constant difference across probabilities suggests a location shift. A difference that becomes more positive toward the upper end and more negative toward the lower end suggests greater imputed spread. A difference concentrated at one end suggests a tail discrepancy or a change in skewness. These are visual patterns to investigate, not diagnoses of a unique cause.

Why the Plotting Positions Are Useful

The plotting-position rule gives each ordered observation an increasing probability coordinate, centered within its rank interval. The positions are equally spaced by \(1/n\), and the first and last points remain inside \((0,1)\). This avoids assigning the smallest observation probability zero or the largest probability one, where endpoint quantiles can be especially unstable.

For example, with \(n=9\), the fifth observation is placed at \((5-0.5)/9=0.5\), the center of the probability scale. More generally, since \(p_i=(i-0.5)/n\), the point for rank \(n+1-i\) is at \(1-p_i\). Thus ranks equidistant from the center receive symmetric probability positions. This is a convenient display convention, not a theorem that the observed order statistics have no sampling variability.

Making a Fair Comparison

A quantile plot is meaningful only if the two sets of values are defined and displayed comparably. The observed group should contain the recorded values of the variable; the imputed group should contain the values imputed for its missing entries. Keep the same measurement units and use the same probability scale and quantile convention. Label the number of values in each group, since quantile estimates—especially near the extremes—are less stable with fewer observations.

In MICE, distributions can be examined at a selected iteration or after the chains have stabilized. A plot from an unstable iteration may reflect transient behavior rather than the distribution represented by later imputations. Compare more than one chain or iteration when needed, while keeping the comparison readable. The earlier material on trace plots and stabilization provides context for deciding whether an iteration is suitable for a distributional diagnostic.

When there are multiple completed datasets, do not simply stack all imputed values and treat them as independent observations. Repeated imputations reuse the same missing cases and reflect between-imputation variation; stacking can make the apparent sample size misleading and obscure chain-specific patterns. Inspect imputed values by imputation or chain, or summarize their quantile behavior across imputations without presenting the pooled values as one ordinary sample.

A marginal comparison can also be affected by case composition. If cases with missing \(Y\) tend to differ from cases with observed \(Y\) in predictors related to \(Y\), the imputed marginal distribution need not match the observed marginal distribution, even under a defensible imputation model. The comparison is therefore most informative when interpreted alongside the variables used to predict missingness and \(Y\), rather than as a requirement that the two curves coincide.

Diagnostics, Alternatives, and Limitations

Look first for broad shape. A nearly parallel separation across most probabilities suggests a location difference; increasing separation toward both ends suggests a spread difference; a departure at one end suggests a tail discrepancy. A curve that changes direction may indicate a more complex shape difference. These patterns can guide checks of variable coding, units, impossible values, predictor relationships, and the imputation model. They should not be assigned a single explanation from the plot alone.

Quantile plots complement the density plots for continuous observed-imputed comparisons: quantiles avoid dependence on histogram bins or density-smoothing choices and make tail positions explicit, while density displays can make overlapping modes and local peaks easier to see. A simple summary such as a mean or standard deviation is more compact but can conceal a difference confined to one tail. No display substitutes for checking chain behavior, model specification, and the plausibility of generated values.

The main limitations are finite-sample noise, unstable extreme quantiles, and dependence on the cases being compared. A few unusual observations can dominate the far ends, and an apparent difference may be due to a small observed or imputed sample. Quantile plots are descriptive; visual separation is not a p-value, and overlap is not evidence that the distributions are equivalent.