Why Check Marginals and Pairs Separately?
A two-variable MICE diagnostic has two distinct questions. First, do the imputed values for each variable look plausible when considered one variable at a time? Second, do the imputed values occur in plausible combinations with the other variable? A favorable answer to the first question does not guarantee a favorable answer to the second: two variables can each have the right marginal distribution while their pairing is wrong.
For variables \(X\) and \(Y\), marginal plots show the distribution of \(X\) and the distribution of \(Y\) separately. A pairwise plot places \(X\) on one axis and \(Y\) on the other, so the location and shape of the pairs become visible. A useful check therefore marks observed and imputed values distinctly, examines each marginal, and then examines the paired pattern. The conclusion should be about diagnostic evidence, not a claim that a plot proves the imputation model is correct or incorrect.
Conditions for Interpreting the Comparison
A plot compares the data it contains; interpretation depends on how those data were selected. A direct comparison of observed and imputed values as if they should have the same distribution is most defensible when the observed values are representative of the missing values for the comparison at hand. For example, under missing completely at random (MCAR), the chance a value is missing does not depend on observed or unobserved data, so the observed values can serve as an unadjusted reference for the missing values. Under other missingness processes, observed and missing values can differ even when the imputation procedure is appropriate. A discrepancy is then a prompt to investigate, not by itself evidence of model failure.
For a pairwise comparison, also check which values are observed and which are imputed. A row with observed \(X\) and imputed \(Y\) contributes a partly imputed pair; a row with both missing contributes a fully imputed pair. These cases should not be visually indistinguishable from fully observed pairs. In addition, show one completed data set or distinguish chains and iterations clearly. Treating all completed data sets as one enlarged sample can make repeated imputations of the same cases look like independent observations.
The examples below use hypothetical data and deliberately simple plots. They assume, where direct observed-versus-imputed comparison is interpreted as a fit check, that the comparison is meaningful for the stated missingness pattern. In an applied analysis, that assumption must be considered rather than inferred from a visually close match.
Worked Check: Matching Marginals, Conflicting Pairs
Worked Example: Identical One-Variable Distributions, Different Joint Pattern
Suppose eight hypothetical records have both \(X\) and \(Y\) observed, and eight other records have both variables missing. For this diagnostic illustration, compare the eight observed pairs with one set of eight imputed pairs. Assume the missing records are comparable to the observed records for this unadjusted comparison; MCAR would justify that assumption in expectation.
| Group | \(X\) values | \(Y\) values |
|---|---|---|
| Observed pairs | 1, 2, 3, 4, 5, 6, 7, 8 | 1, 2, 3, 4, 5, 6, 7, 8 |
| Imputed pairs | 1, 2, 3, 4, 5, 6, 7, 8 | 8, 7, 6, 5, 4, 3, 2, 1 |
Marginal check. The observed \(X\) values and imputed \(X\) values are both the list 1 through 8. The same is true for \(Y\). Their marginal distributions agree exactly in this constructed example: each value occurs once in each group. Any separate histogram or dot plot of either variable would show a match.
Pairwise check. In the observed pairs, low values of \(X\) occur with low values of \(Y\), and high values with high values. The scatterplot follows an increasing diagonal. In the imputed pairs, low \(X\) values occur with high \(Y\) values, and high \(X\) values with low \(Y\) values. The plot follows a decreasing diagonal. The pairings are plainly different even though both sets of marginals match.
The difference can also be checked by dividing each variable at 4.5. For the observed pairs, the four pairs with both values at most 4 are in the low-\(X\), low-\(Y\) region, and the four pairs with both values above 4.5 are in the high-\(X\), high-\(Y\) region. Thus the two same-side regions contain \(4+4=8\) pairs, and the two opposite-side regions contain none. For the imputed pairs, four are low-\(X\), high-\(Y\), and four are high-\(X\), low-\(Y\); the same-side regions contain none. This count confirms what the pairwise plot shows.
Conclusion. The marginal check gives no warning, but the pairwise check shows a strong joint-pattern discrepancy. If the observed increasing pattern is credible for records with both variables missing, this imputed pairing is concerning. It is not enough to report that the imputed distributions of \(X\) and \(Y\) separately look right.
A Short Justification: Why Marginals Cannot Settle the Joint Question
The example illustrates a general probability fact. Let \(L\) denote a low category and \(H\) a high category for each variable. In the observed group, suppose half the pairs are \((L,L)\) and half are \((H,H)\). In the imputed group, suppose half are \((L,H)\) and half are \((H,L)\). In both groups, half of the \(X\) values are low and half are high; the same is true of \(Y\). Thus the marginal probabilities agree:
But the joint probabilities differ. The observed group has \(P(X=L,Y=L)=1/2\), whereas the imputed group has \(P(X=L,Y=L)=0\). Matching the marginal probabilities places no requirement on this joint probability. A pairwise display is needed to see which combinations occur.
Worked Check: A Marginal Shift With a Similar Pairing
Worked Example: Separating a Shift From a Relationship Change
Consider six hypothetical observed pairs \((X,Y)\): \((1,2),(2,3),(3,4),(4,5),(5,6),(6,7)\). For six records with both values imputed, suppose the pairs are \((1,7),(2,8),(3,9),(4,10),(5,11),(6,12)\). As above, interpreting observed values as a direct reference requires that this comparison be appropriate for the missingness process.
The observed \(X\) values and imputed \(X\) values are both 1 through 6, so the \(X\) marginals match. The observed \(Y\) values are 2 through 7, while the imputed \(Y\) values are 7 through 12. Their means are:
The imputed \(Y\) values are shifted upward by 5 units. In the pairwise plot, both groups form increasing patterns with the same step from one pair to the next, but the imputed pattern lies above the observed pattern. The plot therefore suggests a marginal location discrepancy in \(Y\), rather than the reversal of pairing seen in the first example. The \(Y\) discrepancy and the pairwise pattern should both be reported; saying simply that the pairwise association looks similar would miss the shift.
The numerical comparison describes these particular values, not a formal test of fit. With a different missingness process, some or all of the shift could reflect which records have missing \(Y\), rather than a failure of imputation. The diagnostic conclusion is that the difference warrants investigation in context.
How to Make the Plot Check Informative
State which records contribute observed values, which contribute imputations, and whether one chain, one iteration, or a clearly identified collection is displayed. Avoid treating repeated completed data sets as independent records.
Compare the observed and imputed values of \(X\), then do the same for \(Y\). Note broad location, spread, gaps, and extremes without treating a visual resemblance as proof of equality.
Plot \(X\) against \(Y\), distinguish pair types by missingness or imputation status, and look for changes in direction, curvature, clusters, spread, or empty regions. Use comparable axis scales so visual differences are not created by plotting choices.
Describe separately what the marginals suggest and what the joint plot suggests. Identify assumptions or sparse regions that limit interpretation, and investigate a flagged pattern rather than declaring the method validated or invalid from one plot.
A useful display identifies whether a point is fully observed, has imputed \(X\), has imputed \(Y\), or has both values imputed. With many points, transparency or separate panels can reduce overplotting. If several chains are available, inspect them separately or mark them explicitly; pooling them without distinction can conceal a chain-specific pattern. A smooth-looking cloud is not automatically reassuring: excessive smoothing, overplotting, or a limited observed range can hide a discrepancy.
Worked Check: A Pairwise Warning Despite Similar Marginals
Worked Example: Counting Pair Regions in a Small Plot
Suppose six hypothetical observed pairs are \((1,1),(2,2),(3,3),(6,6),(7,7),(8,8)\). Six imputed pairs are \((1,8),(2,7),(3,6),(6,3),(7,2),(8,1)\). Each variable has the same marginal values in both groups: 1, 2, 3, 6, 7, and 8. Assume for this illustration that the observed pairs are a reasonable reference for the missing records.
Split each axis into low values (1, 2, 3) and high values (6, 7, 8). The observed pairs contribute three low-\(X\), low-\(Y\) points and three high-\(X\), high-\(Y\) points. The imputed pairs contribute three low-\(X\), high-\(Y\) points and three high-\(X\), low-\(Y\) points. The marginal counts are identical: each group has three low and three high values for each variable. Yet the pair-region counts are entirely different.
The practical reading is that the marginal plots alone would not flag these imputations, while the pairwise plot would show an opposite pattern. If the observed association is expected to persist in the missing records, the discrepancy merits review of the conditional imputation models and the variables used to predict each other. The plot does not identify the cause; it establishes what pattern needs explanation.
Common Pitfalls, Alternatives, and Limits
- Wrong: “The two histograms match, so the imputed data fit.” Correction: Histograms address one variable at a time. Check the pairwise pattern separately.
- Wrong: “Observed and imputed values differ, so MICE must have failed.” Correction: The missingness process can make missing and observed values differ. Assess whether the observed group is a defensible reference for the missing group.
- Wrong: “A pairwise plot proves the imputation model is correct because the clouds overlap.” Correction: Overlap is limited visual evidence. It may hide discrepancies, and a plot cannot verify unobserved values or all model assumptions.
- Wrong: “All imputed points can be pooled across chains as extra independent observations.” Correction: Imputations represent uncertainty about missing values, not new independent records. Preserve chain and record identities in the diagnostic display.
- Wrong: “A pairwise plot of observed complete cases is automatically representative of cases with missing values.” Correction: Complete cases may be selected differently. Describe the missingness pattern and avoid generalizing beyond the support of the observed comparisons.
Marginal plots are simpler and more readable for distributional features of one variable; pairwise plots are essential for combinations, but can become cluttered and are limited to two dimensions at a time. Numerical summaries can help describe a pattern, but a single summary may miss nonlinear structure or clusters. This check is diagnostic rather than a formal test with a universal cutoff. It cannot establish that missing values follow the observed pattern, and it may have little power when observed pairs are sparse or concentrated in a narrow region.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- In a hypothetical comparison, observed and imputed \(X\) values have the same distribution, and observed and imputed \(Y\) values have the same distribution. Does this establish that the pairs have the same joint pattern?
- Observed \(Y\) values are 2, 4, 6, and 8; imputed \(Y\) values are 4, 6, 8, and 10. Calculate both means and describe the marginal difference.
- Suppose observed pairs fall mainly in the low-low and high-high regions, while imputed pairs fall mainly in the low-high and high-low regions. What does this say about the marginal and joint checks if each group has the same number of low and high values for both variables?
- Why can an observed-versus-imputed discrepancy be misleading when values are not missing completely at random?
- Name two plotting choices that help prevent a pairwise diagnostic from obscuring which values were imputed.
Solutions. (1) No. Matching marginal distributions does not determine how values are paired; the joint patterns can differ. (2) The observed mean is \((2+4+6+8)/4=20/4=5\). The imputed mean is \((4+6+8+10)/4=28/4=7\). The imputed values have a mean 2 units higher in this example. (3) The marginal counts can agree while the pairwise pattern differs: the observed pairs concentrate in same-side regions and the imputed pairs in opposite-side regions. (4) Missingness can select values differently from those observed, so the observed distribution may not represent the missing values even under an adequate imputation procedure. (5) For example, use distinct symbols or colors for observed and imputed values, and use separate panels for different imputation-status groups or chains.