What a Scatterplot Can Diagnose
A scatterplot puts two variables on the same set of axes, making their joint pattern visible: its direction, shape, spread, clusters, and gaps. In MICE diagnostics, this can reveal a problem that separate plots of each variable might miss. An imputed variable can have a plausible range and distribution yet be paired with another variable in an implausible way.
Suppose \(X\) and \(Y\) are continuous variables. A useful plot marks a pair as observed when both values were recorded, and distinguishes pairs for which at least one coordinate was imputed. Depending on which variable is missing, it may be more informative to plot observed \(X\) with observed and imputed \(Y\), or observed \(Y\) with observed and imputed \(X\). The plot should make those statuses clear; otherwise, a cloud of points can conceal which values came from the data and which came from the imputation procedure.
The plot is a diagnostic, not a test of whether an imputation is correct. In particular, imputed values have no observed truth against which they can be directly checked. A striking contrast may be a model warning, a consequence of the missingness pattern, or both. Interpretation requires knowing which values are missing, which predictors informed the imputation, and whether the plotted groups have comparable observed information.
Assumptions and Fair Comparisons
Before interpreting a plot, verify that the variables use consistent units and transformations, the axes show the relevant range, and the plotted pairs correspond to the intended cases. Also identify which variables are observed for each pair. If both coordinates are imputed, the plotted association reflects the chained conditional models used for both variables, not just one conditional model.
A key qualification concerns missing at random (MAR). MAR says that, conditional on the observed information relevant to the missingness mechanism, the distribution of missing values agrees with the corresponding distribution in the data. This does not imply that missing and observed values must have the same scatterplot when conditioned only on \(X\), or that their unadjusted point clouds must match. For example, if missingness of \(Y\) depends on an observed variable \(W\), then the relevant comparison may need to account for \(W\), even when the plot shows \(X\) and \(Y\). An imputation model that omits relevant information or represents the conditional distribution poorly can fail to reproduce the needed relationship.
Consequently, assess a suspected difference within comparable groups of observed predictors or missingness patterns when possible. Check whether the imputation model includes the continuous predictors and relevant transformations or interactions needed to represent the relationship. Adequate overlap also matters: if few observed cases resemble the cases needing imputation, the plot cannot provide a strong empirical comparison in that region.
For a simple numerical check of a visible linear trend, the ordinary least-squares slope in a group of plotted pairs is \(b_1=S_{XY}/S_{XX}\), where \(S_{XY}=\sum_i(x_i-\bar{x})(y_i-\bar{y})\) and \(S_{XX}=\sum_i(x_i-\bar{x})^2\). This is a descriptive summary of those points, not a requirement that the relationship be linear. Curvature, clusters, and changes in spread can matter even when a single slope is unremarkable.
Worked Example: A Weakened Trend Among Imputed Values
Consider a hypothetical diagnostic plot of continuous predictor \(X\) and continuous variable \(Y\). Both groups contain five cases with \(X=-2,-1,0,1,2\). The observed \(Y\) values are \(6,8,10,12,14\); in a particular completed data set, the imputed \(Y\) values for five other cases are \(9,10,10,10,11\). The repeated \(X\) values here make the contrast easy to calculate; they are not intended to imply that the same individuals appear in both groups.
For the observed group, \(\bar{x}=0\), \(\bar{y}=10\), \(S_{XX}=(-2)^2+(-1)^2+0^2+1^2+2^2=10\), and \(S_{XY}=(-2)(6-10)+(-1)(8-10)+0+1(12-10)+2(14-10)=20\). Thus \(b_1=20/10=2\) units of \(Y\) per unit of \(X\).
For the imputed group, \(\bar{x}=0\) and \(\bar{y}=10\). Its cross-product sum is \(S_{XY}=(-2)(9-10)+(-1)(10-10)+0+1(10-10)+2(11-10)=4\), while \(S_{XX}=10\). Its descriptive slope is therefore \(b_1=4/10=0.4\) units of \(Y\) per unit of \(X\). The \(Y\) range is also narrower: \(11-9=2\) units for the imputed values, compared with \(14-6=8\) units for observed values.
The plot would show a much weaker, more compressed imputed pattern. That is a reason to investigate the conditional model and the cases’ observed predictors, not proof of failure. For example, a relevant observed variable could distinguish the cases in a way that makes this contrast legitimate. The numerical comparison is descriptive; it does not account for uncertainty across imputations or establish that the two groups should have matching patterns.
Patterns That Deserve Investigation
Look beyond whether two clouds seem to have different slopes. Several distinct visual features can indicate that a model is not reproducing important joint behavior:
- Unexpected clustering: imputed points concentrate in a small region or form separate clumps absent from comparable observed cases. Check whether the clusters correspond to observed subgroups or missingness patterns before attributing them to the imputation model.
- Range restriction: imputed coordinates occupy only a narrow part of the plausible range, even though the relevant observed predictors span a wider range. This can follow from a conditional distribution that is genuinely narrower for the missing cases, but can also flag inadequate predictors or an overly restrictive model.
- Weakened or reversed association: the imputed points have a visibly flatter trend, or a different direction, than comparable observed points. Consider whether conditioning on observed information changes the relationship before concluding that the model has lost it.
- Missing shape: observed points curve or change spread across \(X\), while imputed points follow a straight band or constant-spread cloud. This can suggest an omitted nonlinear term, interaction, or change in variance.
Worked Example: Restricted Range in an Imputed Predictor
Suppose a hypothetical plot uses continuous outcome \(Y\) on the vertical axis and predictor \(X\) on the horizontal axis. Among cases with observed \(X\), the displayed values range from 20 to 80 units. For cases with \(X\) imputed, the plotted values range from 48 to 52 units. The imputed range is \(52-48=4\) units, compared with an observed range of \(80-20=60\) units; the ratio is \(4/60=0.0667\), or about 6.67%.
This pronounced restriction is a warning to inspect the model, but the ratio alone does not show that the imputation is wrong. The cases with missing \(X\) might share observed characteristics associated with a narrow conditional distribution of \(X\). The plot should therefore be reviewed within relevant observed predictor groups and alongside the predictors used to impute \(X\). If the narrow band persists among comparable cases and the conditional model excludes useful information, the specification merits revision. If the relevant observed information explains the concentration, the marginal difference can be reasonable.
Worked Example: A Straight Imputation Pattern Hides Curvature
In a hypothetical data set, observed pairs follow \(Y=X^2\) at \(X=-2,-1,0,1,2\), giving \(Y=4,1,0,1,4\). A conditional imputation model for \(Y\) includes only a linear term in \(X\). Its fitted straight-line slope for these five points is zero: \(\bar{x}=0\), \(\bar{y}=2\), and
The fitted intercept is \(\bar{y}-b_1\bar{x}=2\). Thus a linear conditional mean predicts \(Y=2\) throughout this range of \(X\). If missing cases at \(X=-2,-1,1,2\) receive imputations near 2, their points form a horizontal band while the observed points follow a U-shaped pattern. The scatterplot exposes a possible functional-form problem: a linear term alone cannot represent this curvature. A suitable model could include nonlinear structure, but the plot by itself does not prescribe a unique model or prove which specification is correct.
How to Review the Plot Without Overreading It
Distinguish pairs observed on both variables from pairs with one or more imputed coordinates. State clearly which coordinates are imputed.
Describe the concern precisely: clustering, limited range, a changed trend, curvature, or a change in spread. Avoid relying only on a vague impression that the clouds “look different.”
Review observed predictors and missingness patterns that may explain why the plotted cases differ. Under MAR, comparisons should reflect the observed information relevant to missingness, not just a single displayed variable.
Inspect separate completed data sets or summarize the recurring visual pattern. Imputed draws vary, and combining many imputations into one opaque cloud can conceal that variation or create misleading visual density.
Check the conditional model, plausible transformations, and relevant predictors. Compare with suitable pairwise or subgroup displays and distributional diagnostics before deciding whether the pattern indicates a substantive problem.
A plot can mislead when many points overlap, groups have very different sample sizes, axes are truncated, or imputed values from several completed data sets are simply pooled. Use transparency or distinct plotting symbols to make density and status legible. Keep axis units and ranges consistent across displays. When a linear trend is not appropriate, a straight-line summary can hide the very curvature that the scatterplot is meant to reveal.
Alternatives, Limitations, and a Common Mistake
Scatterplots complement, rather than replace, other diagnostics. Distribution plots can reveal a marginal range or shape discrepancy but not how two variables pair. The earlier material on comparing correlations involving imputed variables provides a numerical association summary; a scatterplot adds information about nonlinear form, clusters, and range gaps that a single correlation cannot show. Pairwise plots can review several relationships efficiently, while subgroup displays can help distinguish a model concern from differences in observed case composition.
None of these displays establishes that missing values were imputed accurately. The unobserved values cannot be verified from the scatterplot, and a plausible-looking association does not establish that the conditional model is adequate. Conversely, a visible difference between observed and imputed points is not automatically a defect: the cases may have different observed predictors, and under MAR the appropriate conditional comparison may require those predictors. Use plots to generate and investigate specific concerns, not to certify an imputation procedure.
Common mistake: “The imputed points do not match the observed cloud, so the imputations violate MAR.” This conclusion is not justified. MAR concerns the distribution conditional on the observed information relevant to the missingness mechanism; it does not require unadjusted observed and imputed clouds to coincide. The correction is to identify that information, assess whether the imputation model represents the corresponding conditional distribution, and compare like with like. Even a conditional visual match is diagnostic evidence, not proof that MAR holds.
Check Your Understanding
Answer each question, then compare with the solutions below.
- Observed \(Y\) values rise with continuous \(X\), while imputed \(Y\) values show a nearly horizontal band. What are two explanations to investigate before calling this a model failure?
- Under MAR, must the unadjusted scatterplots of observed and imputed \(Y\) against \(X\) match? State why or why not.
- Observed \(X\) ranges from 10 to 70 units, but imputed \(X\) ranges from 30 to 40 units. Calculate the ratio of imputed to observed ranges and give one cautious interpretation.
- What feature of the observed points could be missed by an imputation model that includes only a linear term in \(X\)?
Solutions. (1) Check whether the cases differ in observed predictors relevant to missingness, and whether the imputation model omits nonlinear terms, interactions, or other useful predictors. (2) No. MAR concerns the distribution conditional on observed information relevant to missingness; unadjusted clouds can differ when case composition differs. (3) The observed range is \(70-10=60\) units, and the imputed range is \(40-30=10\) units, so the ratio is \(10/60=0.167\), about 16.7%. This suggests range restriction worth investigating, but relevant observed predictors might explain it. (4) Curvature, such as a U-shaped relationship, may be missed by a straight-line conditional mean.