Why Look at Two Variables Together?
A set of imputations can resemble the observed data when each variable is examined alone and still behave differently when two variables are considered together. For example, the observed values of two measurements may tend to increase together, while imputed pairs show little such pattern. Separate histograms would not reveal that change. Pairwise plots make the joint arrangement of values visible.
In MICE diagnostics, the comparison is descriptive rather than a direct test of whether an imputed value is correct. Observed pairs and pairs containing imputations may come from different groups of cases: a case with both values observed is not necessarily comparable to a case with one or both values missing. Plots can expose patterns that deserve investigation, but they cannot by themselves distinguish an imputation-model problem from a difference in the cases that could be observed.
For a quantitative \(X\) and quantitative \(Y\), a scatterplot is the natural display. If both are categorical, a two-way table or mosaic-style display is more appropriate; for a categorical and quantitative pair, use a distribution plot of the quantitative variable within categories. The focus here is on paired scatterplots and small multiples, especially for two quantitative variables.
Constructing the Comparison
For each case, retain the values of both variables and identify whether each was originally observed or imputed. When both variables have missing values, a single “imputed” group may hide important distinctions. It is often clearer to show four patterns: both observed, \(X\) imputed and \(Y\) observed, \(X\) observed and \(Y\) imputed, and both imputed. A plot can use separate panels for these patterns or symbols that identify them.
The axes, units, limits, and plotting scales should be the same across panels. Otherwise, an apparent difference in spread or position may be caused by the display rather than the data. When several MICE chains or completed data sets are shown, use consistent visual encodings and avoid treating repeated imputations of the same case as independent observations.
Select two variables whose joint behavior matters for the analysis or for the plausibility of the imputation model.
For each case, distinguish whether \(X\), \(Y\), or both values were imputed. Keep observed-observed pairs identifiable as a reference group.
Use a common coordinate system and, if needed, separate panels by missingness pattern or an important predictor.
Look for changes in direction, curvature, clustering, spread, gaps, or range. Then ask whether the groups being compared have similar case composition.
Check the relevant missingness patterns, predictor strata, and model specification. Treat the plot as a diagnostic clue, not a verdict about the unobserved values.
Why Matching Marginals Is Not Enough
A marginal distribution describes one variable at a time. It does not specify how values of \(X\) are paired with values of \(Y\). The following hypothetical construction makes this distinction explicit. Suppose four observed pairs lie along an increasing diagonal, while four imputed-containing pairs lie along a decreasing diagonal.
In both groups, the \(X\) values are \(1,2,3,4\), and the \(Y\) values are also \(1,2,3,4\). Thus, the two groups have exactly the same empirical marginal distributions for each coordinate. Yet their pairings are visibly different: one group increases together and the other decreases. The calculation is simply a comparison of the coordinate lists; equality of those lists does not imply equality of the plotted point arrangements.
Worked Examples
Worked Example: Marginals Match but the Joint Pattern Changes
Consider the hypothetical pairs in the display above. Assume the first four cases have both measurements observed, and the next four have at least one value imputed. Compare the groups using a scatterplot.
Observed group. The points are \((1,1),(2,2),(3,3),(4,4)\). Each increase in \(X\) is paired with an increase in \(Y\); all four points lie on the line \(Y=X\).
Imputed-containing group. The points are \((1,4),(2,3),(3,2),(4,1)\). As \(X\) increases, \(Y\) decreases; all four points lie on \(Y=5-X\).
Marginal check. Each group has \(X\) values \(1,2,3,4\) and \(Y\) values \(1,2,3,4\). Therefore, a plot of either variable alone would look identical for the two groups, but the pairwise scatterplot shows opposing joint patterns.
This is a visual warning that the relationship between the variables may have changed in imputed-containing cases. It is not proof that the imputed values are wrong: the two groups may differ in their missingness patterns or other characteristics. A useful follow-up is to separate the one-imputed and two-imputed pairs and examine relevant predictor strata.
Worked Example: A Pooled Plot Hides Subgroup Composition
Suppose a hypothetical dataset contains a continuous outcome \(Y\), a continuous predictor \(X\), and a binary age-group indicator. A pooled scatterplot shows that observed-observed pairs mostly have low \(X\) and low \(Y\), whereas imputed-containing pairs mostly have high \(X\) and high \(Y\). That separation might look like an imputation discrepancy.
Now consider two age-group panels. In the younger group, the observed pairs cluster near \((2,3)\) and the imputed-containing pairs near \((2.2,3.1)\). In the older group, observed pairs cluster near \((8,9)\) and imputed-containing pairs near \((8.2,9.1)\). Within each group, the two patterns are close; the pooled separation arises because more observed pairs came from the younger group and more imputed-containing pairs came from the older group.
The arithmetic of the comparison is qualitative but explicit: within the younger group, the centers differ by \(0.2\) units in \(X\) and \(0.1\) units in \(Y\); within the older group, they differ by the same amounts. Across groups, the coordinates differ by about \(6\) units on both axes. The large pooled gap therefore reflects group composition more than a within-group shift.
The plot should be stratified by age group, or otherwise show that group membership clearly. This does not establish that the imputations are adequate, but it prevents a difference in subgroup composition from being mistaken for a change in the within-group joint pattern. The comparison is informative only where both statuses have enough cases in a stratum to show their patterns.
Worked Example: One Variable Imputed, One Observed
Imagine a hypothetical diagnostic sample of four cases in which \(X\) is observed for everyone, while \(Y\) is observed for two cases and imputed for two. The observed pairs are \((10,4)\) and \((20,8)\); the imputed-containing pairs are \((15,6)\) and \((25,7)\). Both variables are measured in stated units: \(X\) in milligrams per litre and \(Y\) in milligrams per litre.
Plot all four pairs, marking the two imputed \(Y\) values with a different symbol. The observed points rise by \(4\) units in \(Y\) as \(X\) rises by \(10\). The imputed-containing points rise by only \(1\) unit in \(Y\) as \(X\) rises by \(10\). Thus the displayed trend is flatter among these two imputed cases.
This is a small-sample visual clue, not a stable estimate of a general relationship. There are only two points in each group, and the imputed cases might have different predictor or missingness-pattern profiles. The appropriate conclusion is that this pair deserves follow-up, not that the imputation procedure has definitively weakened the relationship. In particular, the unobserved \(Y\) values for those cases cannot be checked directly from this plot.
Reading the Plot and Diagnosing Problems
A useful interpretation names the feature seen and the group in which it appears. Ask whether the imputed-containing pairs show a different direction or curvature, whether a cluster present among observed pairs is absent, whether the spread changes at particular \(X\) values, and whether points appear outside plausible joint regions. The point is not to label every visual difference a failure. Sparse observed pairs, selective missingness, or a few extreme values can make patterns look different even when the imputation model is reasonable.
When overplotting obscures the point arrangement, use transparency, small symbols, or a modest amount of jitter only where values are discrete or rounded. Jitter is for display; it does not change the recorded data and should not be interpreted as additional precision. For dense data, small multiples by missingness pattern or predictor subgroup can be more legible than many overlapping symbols. Keep axis limits identical across panels so that visual comparisons remain valid.
The earlier material on Comparing Conditional Distributions Across Predictor Levels motivates examining patterns within relevant predictor groups. For pairwise plots, the same caution is essential: a pooled difference can reflect different subgroup composition. If a status group is absent or extremely sparse in a subgroup, the plot cannot support a reliable visual comparison there.
| Display | What it can show | Main caution |
|---|---|---|
| Separate one-variable distribution plots | Marginal shifts, range changes, or unusual shapes for each variable | They do not show which \(X\) and \(Y\) values occur together |
| Pairwise scatterplot | Joint clusters, direction, curvature, gaps, and changes in spread | Overplotting and group-composition differences can mislead |
| Small multiples by missingness pattern or predictor | Whether a visual contrast persists within comparable groups | Panels can become too sparse for useful comparison |
A common mistake is to reason, “The two variables have similar marginal plots, so their imputations preserve the relationship.” That conclusion does not follow: as the four-pair construction showed, the same values can be paired in very different ways. The correction is to inspect the joint plot and distinguish observation-status patterns. The opposite mistake is also possible: seeing different clouds and concluding that the imputation model is wrong. A plot cannot show the unobserved truth, and observed and imputed-containing cases may not be directly comparable.
Scope and Limitations
Pairwise plots are most useful as targeted diagnostics for pairs with substantive importance, strong expected relationships, or relevance to the analysis model. They do not establish that the imputation model is correctly specified, nor do they identify whether a discrepancy is caused by missingness selection, sparse data, model form, or chance. Missingness mechanisms can make observed cases systematically different from incomplete cases, so exact visual agreement is not a general requirement.
Use pairwise plots alongside marginal checks, missingness-pattern summaries, and subgroup comparisons rather than as a substitute for them. A numerical summary of association can complement a plot, but it can compress nonlinear or clustered patterns into a single value; quantitative comparison of correlations is a separate diagnostic. Most importantly, apparent agreement in observed regions does not validate predictions in regions with little or no observed support.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- What does an observed-observed pair mean, and how does it differ from an imputed-containing pair?
- Two groups have identical marginal distributions for \(X\) and \(Y\), but their scatterplots show opposite directions. Is this possible? Explain.
- A pooled plot separates observed and imputed-containing cases, but the separation largely disappears in panels by a relevant predictor. What should you conclude?
- Why is a pairwise plot not proof that imputed values are correct, even if its groups look similar?
- Name two display choices that help make small-multiple comparisons fair and readable.
Solutions. (1) An observed-observed pair has both values recorded before imputation; an imputed-containing pair has at least one value imputed. (2) Yes. The groups can contain the same \(X\) values and the same \(Y\) values but pair them differently, producing different joint patterns. (3) The pooled separation may be due mainly to different subgroup composition; the within-stratum plots provide the more relevant comparison, subject to adequate data in each panel. (4) The true missing values are unknown, and similar-looking groups do not establish model correctness or rule out differences outside observed support. (5) Use common axis limits and consistent symbols or colours across panels; transparency or small symbols can also reduce overplotting.