Checking Distribution Fit Across Important Subgroups

Why Check Fit Within Important Groups?

A pooled observed-imputed plot can look reassuring while concealing a mismatch in a group that matters for the analysis. For example, imputed values might be systematically too high in one age group and too low in another. If those shifts offset each other, the overall distribution may show little difference. A subgroup check asks whether the observed and imputed distributions tell a similar descriptive story within each substantively important group.

The groups should be chosen because they matter for the scientific question, clinical interpretation, or analysis—not simply because many possible partitions are available. Examples include treatment arm, age category, study site, or a clinically meaningful severity group. The earlier material on stratifying observed-imputed comparisons by predictors introduces stratified displays; the emphasis here is on assessing whether fit is adequate across important groups and interpreting what pooled summaries can conceal.

This is a descriptive diagnostic, not a direct test of whether the imputation model is correct. The observed values are not necessarily representative of the missing values, even within a group. Thus, a difference may reflect selection into missingness, an imputation-model problem, or both. Conversely, similar plots do not establish that the imputed values are valid.

How Pooling Can Hide a Subgroup Pattern

A distribution in the full observed set is a mixture of the group-specific observed distributions. The corresponding imputed distribution is a mixture of the group-specific imputed distributions. The mixture weights need not be the same: the proportion of observed cases in each group can differ from the proportion of missing cases in each group.

Here \(F_{\mathrm{obs},g}(y)\) and \(F_{\mathrm{imp},g}(y)\) are the within-group cumulative distributions, and the weights \(w_{\mathrm{obs},g}\) and \(w_{\mathrm{imp},g}\) are the group proportions in the observed and missing sets, respectively. This identity explains two distinct ways a pooled plot can mislead: within-group differences can offset one another, and different group compositions can affect the pooled comparison.

When comparing groups, keep the scale and plotting choices consistent. For a continuous variable, side-by-side boxplots or density displays can reveal shifts, spread changes, tail differences, or shape differences. For categorical variables, side-by-side proportion displays show which categories differ. Earlier material covers these display types and variable-specific features; the key here is to read them within each important group before summarizing across groups.

Read the Group-Specific Patterns Before the Overall Pattern

A practical check is to inspect each prespecified group separately and then consider the pooled display. In each group, ask whether the distributions are similar in location, spread, shape, and relevant features such as bounds or tail behavior. A difference should be described in context: for example, “the imputed values have a higher upper tail in the severe-disease group,” rather than “the imputation is biased,” unless a suitable basis for that stronger claim exists.

The group label must be available and defined consistently for observed and missing cases. If the group itself is missing, a display stratified by that group cannot be interpreted straightforwardly without addressing how the group was handled. Also ensure that groups are not so finely divided that observed values provide little information. A plot with only a few observations may look irregular simply because it is based on little data; it may be difficult to distinguish a real pattern from random variation.

Assessing a Clinically Important Tail Within Groups

Not every useful subgroup check needs to compare an entire distribution visually. If a particular feature matters scientifically, such as an unusually high value, inspect that feature within each group and show the relevant denominator. This can prevent a pooled display from obscuring a discrepancy concentrated in a smaller but important group. It is still a descriptive check: it does not establish that observed cases represent missing cases in that subgroup.

Assumptions, Diagnostics, and Limits

These plots are most interpretable when the selected groups have substantive meaning, group membership is defined consistently, and the same variable scale is used for observed and imputed values. For numerical comparisons such as proportions, report both the numerator and denominator so a percentage based on a small group is not mistaken for a stable pattern. When multiple imputations are available, do not treat the imputed values from one completed data set as the full representation of imputation variability. Examine whether the visual pattern is broadly similar across imputations rather than relying on a single realization.

Several warning signs merit closer review: a conspicuous within-group shift; opposite shifts across groups that disappear in the pooled plot; a group with implausible values or a different shape; or a plot whose conclusion changes substantially across imputations. These patterns are prompts for investigation, not automatic evidence of a specific failure. Review whether the imputation model allows relevant group relationships and whether group-specific information is adequate. Avoid adding many unplanned subgroup displays and then interpreting isolated differences as discoveries.

A pooled plot remains useful for describing the overall observed and imputed sets, but it cannot replace subgroup checks when the groups matter to the analysis. Conversely, subgroup plots do not replace other diagnostics: they do not assess associations between variables, prove that the missingness mechanism is handled adequately, or verify imputation accuracy for values that are unobserved. Their role is narrower and valuable: to reveal whether an overall comparison conceals a pattern in an important group.