Why Compare Every Category?
For a nominal variable, category labels have no natural order. A comparison that reports only the most frequent category, or collapses several categories into a single “other” group, can therefore hide important changes in the distribution. This is especially consequential when an uncommon category is scientifically important: a small change in the overall percentage may correspond to a large relative change in that category.
In a MICE diagnostic, compare the distribution of values observed for a variable with the distribution of values imputed for its missing entries. The comparison is descriptive, not a test that the imputation model is correct. Observed cases and missing cases can have different predictor profiles, so their marginal category proportions may differ even when the conditional imputation model is appropriate.
The denominators matter. The observed proportions describe the records with observed values; the imputed proportions describe the missing records, completed using one particular imputation. Do not divide the imputed category counts by the total sample size when the aim is to compare the missing cases with the observed cases.
Constructing and Reading the Comparison
Use the same complete set of category labels in both distributions, including categories with zero counts in one group. For each category, report its observed count and proportion, its imputed count and proportion, and the difference \( \Delta_k^{(d)}=p_{\mathrm{imp},k}^{(d)}-p_{\mathrm{obs},k} \). A positive difference means the category has a larger proportion among imputed values; a negative difference means it has a smaller proportion. Since both distributions sum to one, the differences across categories sum to zero.
A grouped dot plot or grouped bar chart can make these category-by-category differences easy to see. Label the groups, show category counts when possible, and use a common proportion scale. A table is often helpful alongside a plot, particularly for rare categories: percentages alone can make a category based on only a few cases look more stable than it is. Because nominal categories have no meaningful order, use a consistent label order rather than implying that the categories form a numerical scale.
Worked Example: Finding a Shift in a Rare Category
Consider a hypothetical nominal variable with categories A, B, and C. Among 80 records with observed values, the counts are 48, 28, and 4. There are 20 missing values, and one completed dataset assigns 8, 9, and 3 of them to A, B, and C.
Observed proportions. Divide each observed count by 80: A is \(48/80=0.600\), B is \(28/80=0.350\), and C is \(4/80=0.050\).
Imputed proportions. Divide each imputed count by 20: A is \(8/20=0.400\), B is \(9/20=0.450\), and C is \(3/20=0.150\).
The differences, in category order A, B, C, are \(-0.200\), \(+0.100\), and \(+0.100\). Thus, category C rises from 5% among observed values to 15% among imputed values, a difference of 10 percentage points. Its imputed proportion is three times its observed proportion, but the counts are small, so the relative change should not be mistaken for a precise estimate. The plot and table should make both the percentage shift and the underlying counts visible.
A useful overall descriptive summary is total variation distance:
It ranges from 0, when the two distributions are identical, to 1, when they have no category in common with positive proportion. It summarizes the discrepancy but does not identify which categories contribute to it; always inspect the category-level differences as well.
Why Total Variation Uses One-Half
The factor of one-half follows from the fact that differences in proportions must balance. Write \(\Delta_k=p_{\mathrm{imp},k}-p_{\mathrm{obs},k}\). Because each distribution sums to one, \(\sum_k\Delta_k=0\). Consequently, the total positive difference equals the magnitude of the total negative difference. The sum of absolute differences counts both sides of the redistribution, so the amount of distributional mass that would need to be reassigned to turn one distribution into the other is half that sum.
For the preceding example, the absolute differences sum to \(0.200+0.100+0.100=0.400\), so \(\operatorname{TV}=0.400/2=0.200\). In this descriptive sense, 20% of the distribution’s mass would have to be shifted among categories to make the two distributions match. This is not a percentage of individual cases known to be incorrectly imputed.
Rare Categories and Misleading Aggregates
A category’s absolute difference in percentage points and its relative change answer different questions. When the observed proportion is near zero, a ratio such as \(p_{\mathrm{imp},k}/p_{\mathrm{obs},k}\) can be extremely large or undefined. It is therefore safer to display counts and percentage-point differences first. Ratios may add context when both proportions are nonzero, but they should not replace the category-level counts.
Worked Example: A Stable Common Summary Hides a Rare-Category Shift
In a hypothetical dataset, the observed counts for categories A, B, C, and D are 140, 40, 16, and 4 among 200 observed values. For 100 missing values in one completed dataset, the imputed counts are 70, 20, 2, and 8.
The observed proportions are \(140/200=0.700\), \(40/200=0.200\), \(16/200=0.080\), and \(4/200=0.020\). The imputed proportions are \(70/100=0.700\), \(20/100=0.200\), \(2/100=0.020\), and \(8/100=0.080\).
A summary that reports only the combined proportion in A and B gives 90% for both groups. It would miss the change in the less common categories: C falls by 6 percentage points, from 8% to 2%, while D rises by 6 percentage points, from 2% to 8%. The total variation is \( \frac{1}{2}(0+0+0.060+0.060)=0.060 \). Although the overall distance is modest, the shift may matter if C and D have distinct clinical meanings. D also has only four observed cases, so its observed proportion is based on limited information; the discrepancy merits inspection rather than an automatic conclusion that the imputation is wrong.
This example illustrates why a single aggregate number is not enough. A distance summary compresses the comparison; a collapsed category summary can erase it. Preserve every original category in the diagnostic display, especially when rare categories carry substantive meaning.
Comparing Across Completed Datasets
Imputed values vary across completed datasets because the MICE procedure draws plausible values rather than inserting one fixed prediction. For \(M\) completed datasets, calculate \(p_{\mathrm{imp},k}^{(d)}\) separately for each dataset \(d=1,\ldots,M\). A useful descriptive display shows the observed proportion once and the distribution of imputed proportions across completed datasets, for example with points and a range. The mean imputed proportion can summarize the center, but the spread across datasets is also informative.
Worked Example: Separating the Average Shift From Imputation Variation
Suppose the observed proportions for categories A, B, and C are 0.600, 0.300, and 0.100. In four hypothetical completed datasets, the imputed proportions are respectively (0.500, 0.400, 0.100), (0.450, 0.450, 0.100), (0.550, 0.350, 0.100), and (0.500, 0.400, 0.100).
For A, the mean imputed proportion is \((0.500+0.450+0.550+0.500)/4=0.500\), with a range from 0.450 to 0.550. For B, it is \((0.400+0.450+0.350+0.400)/4=0.400\), with a range from 0.350 to 0.450. For C, it is \((0.100+0.100+0.100+0.100)/4=0.100\), with no variation across these four datasets.
The mean imputed-minus-observed differences are \(-0.100\) for A, \(+0.100\) for B, and \(0\) for C. The display therefore shows both a consistent average shift between A and B and variation in the imputed proportions across datasets. Four datasets are used here only to demonstrate the calculation; in practice, the number of imputations should be chosen for the analysis rather than inferred from this example. Do not treat the \(M\) completed datasets as \(M\) independent samples of participants.
Assumptions, Interpretation, and Failure Modes
Before interpreting the display, check that observed and imputed values use the same category definitions and coding, that all possible categories are included, and that the denominators correspond to the observed and missing subsets. Observed values are assumed to be recorded correctly for this diagnostic. If category labels have been merged, recoded, or dropped differently in the two groups, the comparison is not meaningful until they are made consistent.
Marginal equality between observed and missing cases is not a general requirement of MICE. Under missing completely at random (MCAR), missingness is independent of the variable values, so the two groups have the same category distribution in the population, although their sample proportions can differ by chance. Under missing at random (MAR), missingness may depend on observed variables. The observed and missing groups can then have different marginal category proportions because their predictor compositions differ. A marginal discrepancy is a prompt to investigate, not by itself evidence of model failure. Even close marginal proportions do not establish that the conditional imputation model is adequate or resolve missing-not-at-random (MNAR) concerns.
A practical diagnostic sequence is:
Use identical labels and show zero counts explicitly. Check that a category has not disappeared through inconsistent recoding.
Inspect each category’s percentage-point difference alongside its observed and imputed counts. Mark rare categories clearly.
Look for large differences among imputed proportions across datasets, not only the mean difference from the observed distribution.
Review the model specification and relevant predictors, then assess whether differences could reflect the missingness pattern rather than a fitting problem.
There is no universal percentage-point or total-variation cutoff that declares a categorical imputation acceptable. A discrepancy’s importance depends on sample size, category frequency, scientific meaning, and the intended analysis. A small absolute difference can be consequential for a rare but important category, while a larger marginal difference can be understandable if observed and missing records differ in covariates.
Alternatives and Common Mistakes
| Approach | What it contributes | Limitation for this diagnostic |
|---|---|---|
| Category-by-category plot and table | Shows which categories rise or fall and gives counts and proportions. | Requires attention to scale and rare-category counts. |
| Total variation distance | Provides one bounded summary of the overall marginal difference. | Does not identify the categories responsible or explain the cause. |
| Collapsed-category summary | Can answer a prespecified question about a substantively defined group of categories. | May conceal shifts among categories within the group. |
| Goodness-of-fit test | Can test a specified distributional hypothesis under suitable sampling assumptions. | Does not by itself address the distinct observed and missing groups or dependence across imputations. |
The earlier material on comparing observed and imputed binary proportions applies the same basic observed-versus-imputed idea to two categories. For a nominal variable with several categories, do not reduce the comparison to a yes/no contrast unless that grouping answers a specific substantive question. Nor should a goodness-of-fit p-value replace inspection of the full display: a test’s assumptions and target need to match the data structure, and statistical significance is not a measure of practical importance.
Common mistake: “The observed and imputed proportions of the most common category are nearly equal, so the nominal distribution matches.” This reasoning checks only one category. The other category proportions can shift while the common category stays constant, as in the example where A and B together remained at 90% while C and D changed substantially. The correction is to compare and report every category, including rare categories, and to show counts and variation across completed datasets.
Check Your Understanding
Answer each question using category-specific proportions and the stated denominators.
- For three nominal categories, the observed counts are 30, 15, and 5 among 50 observed values. Imputed counts are 6, 3, and 1 among 10 missing values. Calculate both distributions and their total variation.
- Observed proportions for categories A, B, and C are 0.70, 0.20, and 0.10. Imputed proportions are 0.70, 0.10, and 0.20. What does a report of the proportion in A alone miss?
- Why does a marginal difference between observed and imputed proportions not automatically show that an imputation model is invalid under MAR?
- Across three completed datasets, the proportion imputed in a rare category is 0.02, 0.03, and 0.07. Give two features that should accompany a summary of its mean.
Solutions. (1) The observed proportions are \(30/50=0.600\), \(15/50=0.300\), and \(5/50=0.100\); the imputed proportions are \(6/10=0.600\), \(3/10=0.300\), and \(1/10=0.100\). Each difference is zero, so total variation is 0. (2) A is unchanged, but B falls by 0.10 and C rises by 0.10; comparing A alone misses the redistribution between B and C. (3) Under MAR, missingness can depend on observed variables, so observed and missing records may have different predictor compositions and therefore different marginal category distributions. (4) Show the range or individual completed-dataset proportions, and show the category counts or sample size. The mean is \((0.02+0.03+0.07)/3=0.040\), but it alone hides the spread and the rarity of the category.