Why Order Matters in an Ordinal Comparison
Ordinal variables have categories with a meaningful order, such as none, mild, moderate, and severe. That order carries information that a nominal-category display does not use: a shift from mild to moderate is in a different direction from a shift from severe to moderate. At the same time, ordinal labels do not generally imply equal distances between categories. Coding none, mild, moderate, and severe as 0, 1, 2, and 3 does not by itself make a numerical mean an adequate summary.
A useful diagnostic therefore examines both the proportion in each category and the cumulative proportion at each ordered threshold. The category proportions show where the mass lies; cumulative proportions show how much of the distribution is at or below a specified level. Together, they retain the category order without assuming equal spacing.
For an observed-versus-imputed diagnostic, the observed group consists of records whose value was observed, and the imputed group consists of the originally missing records assigned values in a completed dataset. The category definitions and their order must be identical in both groups. If higher categories mean greater severity, then a lower cumulative proportion at a threshold means a greater share of values lies above that threshold.
These groups are not automatically comparable samples from the same marginal distribution. Their records may differ in predictors associated with the ordinal outcome, and the missingness process may select which values are observed. A discrepancy is a prompt to investigate the imputation model and context, not, by itself, proof of model failure.
Category and Cumulative Summaries
For every category, calculate its count divided by the group size. Then add the proportions in order to obtain the cumulative profile. The last cumulative proportion is always 1, so comparisons are usually most informative at the first \(K-1\) thresholds. A side-by-side bar plot of category proportions should display categories in their substantive order, not alphabetically or according to arbitrary numeric codes. A cumulative plot should likewise connect thresholds in order.
The two summaries answer related but distinct visual questions. Category proportions identify which specific categories are relatively common or uncommon. Cumulative proportions reveal whether the distribution tends toward lower or higher categories at meaningful cut points. A profile of cumulative differences that stays on one side of zero suggests a consistent directional shift; a profile that changes sign suggests that the distributions differ in more than one direction.
Worked Example: Calculating Both Profiles
Suppose a hypothetical symptom-severity variable has four ordered categories: none, mild, moderate, and severe. Among 100 records with observed values, the counts are 40, 30, 20, and 10. In one hypothetical imputed dataset, 20 missing records have imputed counts of 3, 5, 7, and 5 in the same categories. Calculate and compare the profiles.
For the observed records, the category proportions are the counts divided by 100:
Adding from the lowest category upward gives the observed cumulative proportions:
For the 20 imputed values, the category proportions are:
Their cumulative proportions are \((0.15,\ 0.40,\ 0.75,\ 1)\). At the threshold moderate or below, for example, the cumulative proportions are 0.90 for observed records and 0.75 for imputed records. Equivalently, 10% of observed values and 25% of imputed values are severe. The imputed distribution in this draw is shifted toward higher severity. This is a descriptive contrast between two groups, not a test that the imputation model is wrong.
Summarizing Multiple Imputations
With \(m\) imputed datasets, calculate the imputed category proportions separately in each dataset. Each dataset contains the same originally missing records but may assign them different categories because imputation is stochastic. Summarize the resulting proportions across imputations, for example by reporting their mean and range. Do not concatenate all imputed records across datasets and treat them as independent observations: that would count the same missing records repeatedly and hide between-imputation variation.
The observed category proportions are calculated once from records with observed values. For each imputation \(i\), let \(p_{ik}\) be the proportion in category \(c_k\) among the imputed records, and let \(F_{ij}\) be its cumulative proportion through \(c_j\). The mean imputed profile is:
Because cumulative proportions are sums of category proportions, the mean cumulative profile is also the cumulative profile of the mean category proportions. The range across imputations provides a simple indication of how much the diagnostic itself varies across completed datasets; it is not a confidence interval for a population parameter.
Worked Example: Comparing a Mean Profile Across Four Imputations
In a hypothetical dataset, the observed counts in the four severity categories are 18, 24, 12, and 6, for 60 observed records. There are 20 records to impute. Across four hypothetical imputations, their category counts are respectively (4, 8, 6, 2), (3, 7, 7, 3), (5, 7, 5, 3), and (4, 6, 7, 3). Calculate the observed profile and the mean imputed profile.
The observed category proportions are \((18/60,\ 24/60,\ 12/60,\ 6/60)=(0.30,\ 0.40,\ 0.20,\ 0.10)\), with cumulative proportions \((0.30,\ 0.70,\ 0.90,\ 1)\).
Dividing each imputation’s counts by 20 gives category proportions \((0.20,0.40,0.30,0.10)\), \((0.15,0.35,0.35,0.15)\), \((0.25,0.35,0.25,0.15)\), and \((0.20,0.30,0.35,0.15)\). Their mean category proportions are:
For example, the mean proportion in the moderate category is \((0.30+0.35+0.25+0.35)/4=0.3125\). The components sum to 1. The four cumulative profiles are \((0.20,0.60,0.90,1)\), \((0.15,0.50,0.85,1)\), \((0.25,0.60,0.85,1)\), and \((0.20,0.50,0.85,1)\). Their mean is \((0.20,0.55,0.8625,1)\).
At the threshold mild or below, the mean imputed cumulative proportion is 0.55, compared with 0.70 among observed records. At moderate or below, it is 0.8625, compared with 0.90. The first difference is larger, indicating that the principal contrast in these summaries is at the lower threshold. The four imputation-specific values at the mild-or-below threshold range from 0.50 to 0.60, so the mean should not be mistaken for an identical result in every imputation.
What Cumulative Differences Reveal
Cumulative profiles are not merely another way to display the same bar chart. They express the distribution at ordered cut points, and their differences can be used to locate category-level changes. Let \(F_j^{(I)}\) and \(F_j^{(O)}\) be cumulative proportions for the imputed and observed groups, and define \(D_j=F_j^{(I)}-F_j^{(O)}\). Set \(D_0=0\). Since a category proportion is the increase in the cumulative profile from the previous threshold, the difference between the two groups’ proportions in category \(c_j\) is \(D_j-D_{j-1}\).
This identity explains why a cumulative plot can clarify which adjacent categories account for a difference. It also supplies a useful interpretation rule: if \(D_j\leq 0\) at every threshold, the imputed distribution has at least as much mass above every threshold as the observed distribution. With higher categories representing greater severity, that is a consistent shift toward higher categories in this marginal comparison. If the signs change, there is no single direction of shift across all cut points.
Worked Example: Locating a Difference Between Adjacent Categories
Consider a hypothetical comparison with observed category proportions \((0.25,0.35,0.25,0.15)\) and imputed proportions \((0.20,0.40,0.25,0.15)\). The observed cumulative profile is \((0.25,0.60,0.85,1)\); the imputed profile is \((0.20,0.60,0.85,1)\). Calculate the cumulative differences and identify the category changes.
Subtracting observed from imputed cumulative proportions gives \(D_1=-0.05\), \(D_2=0\), \(D_3=0\), and \(D_4=0\). With \(D_0=0\), the category-proportion differences are successive changes in \(D\):
Thus, the imputed distribution has 5 percentage points less mass in none and 5 percentage points more in mild, while the proportions in moderate and severe are unchanged. The cumulative profiles are equal from mild or below onward because the change moves mass only from the first category to the adjacent second category. A bar plot shows the specific categories; the cumulative comparison shows that the share at or below the first threshold is lower.
Assumptions, Diagnostics, and Alternatives
Calculating these proportions requires no distributional model. Interpreting them usefully does require care. The categories must have a defensible order, be defined consistently for observed and imputed records, and cover all possible recorded outcomes. If a category is absent in one group, retain it with a zero proportion rather than dropping it; otherwise the two profiles no longer refer to the same scale. Check that category labels have not been ordered alphabetically or by arbitrary stored codes.
Use the plots and tables as diagnostics alongside substantive knowledge and checks of the imputation model. Look for large differences at important thresholds, categories with unexpected zero or near-zero proportions, and variability across imputations. A difference can be expected if observed and missing records have different predictor profiles. Where appropriate, examine the comparison within relevant predictor groups before deciding that a marginal mismatch signals a problem. A good marginal match alone is also insufficient: it cannot establish that conditional relationships or associations are well represented.
The main alternatives answer different questions. Category bars are direct and useful for identifying individual categories, but can make an overall ordinal shift less apparent. Cumulative plots use order and make threshold contrasts visible, but may conceal which neighboring categories account for a change unless the category proportions are also inspected. A nominal-category comparison ignores order; a numerical mean imposes a spacing interpretation that ordinal categories may not support. These displays are descriptive, not formal tests of equality and not substitutes for model-based inference.
A common mistake is to read a difference between observed and imputed marginal proportions as direct evidence that the imputation is biased. That conclusion overlooks that the two groups contain different records, and observed values can be associated with predictors that also affect missingness. Conversely, a close marginal match does not prove that the conditional imputation model is adequate. Interpret the comparison as a targeted diagnostic, consider variation across imputations, and investigate discrepancies in context.
List every category in its substantive order and use that same order for every group and imputation.
Divide category counts by the number of observed records or imputed records in the group being summarized.
Add proportions from the lowest category upward, and compare the first \(K-1\) thresholds.
Summarize imputation-specific profiles, then investigate whether differences are plausible given predictors and the missingness process.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- For three ordered categories low, medium, and high, the proportions are 0.20, 0.50, and 0.30. What are the cumulative proportions?
- An imputed profile has a lower cumulative proportion than the observed profile at every threshold below the highest category. If higher categories indicate greater severity, what is the appropriate descriptive interpretation?
- Why should category proportions be calculated separately in each imputed dataset rather than by concatenating all imputed records?
- Observed proportions are \((0.30,0.40,0.30)\) and imputed proportions are \((0.20,0.50,0.30)\), in ascending order. Find both cumulative profiles and explain the difference.
- Does close agreement between observed and imputed marginal profiles prove that the imputation model is adequate? Explain.
Solutions. (1) Add in order: \(F_1=0.20\), \(F_2=0.20+0.50=0.70\), and \(F_3=1\), so the profile is \((0.20,0.70,1)\). (2) At each threshold, a lower cumulative proportion means a smaller share is at or below that category and a larger share is above it. The imputed distribution is descriptively shifted toward greater severity at all those cut points; this does not by itself establish model failure. (3) The same originally missing records appear in every completed dataset. Concatenation would count those records repeatedly as if independent and obscure variation across imputations. (4) The observed cumulative profile is \((0.30,0.70,1)\); the imputed profile is \((0.20,0.70,1)\). The change is 0.10 less mass in low and 0.10 more in medium, with high unchanged; cumulative proportions agree from medium or below onward. (5) No. Marginal agreement is one diagnostic only; it does not establish that conditional relationships are represented adequately, and observed and missing records may differ in their predictors.