Why a Marginal Difference Can Be Expected
When reviewing imputations, it is tempting to compare the distribution of observed values with the distribution of imputed values and expect them to look alike. That expectation is not generally justified. The observed values belong to cases whose variable was recorded; the imputed values belong to cases whose variable was missing. If those two groups have different predictor profiles, their marginal distributions can differ even when the imputation model correctly represents the outcome conditional on those predictors.
The key distinction is between a conditional distribution, such as the distribution of an outcome among cases with a particular predictor value, and a marginal distribution, which averages over the predictor values present in a group. Marginals depend both on the conditional outcome model and on the group’s predictor distribution.
A useful working condition is missing at random (MAR) for \(Y\) given \(X\): conditional on \(X\), whether \(Y\) is observed does not additionally depend on the value of \(Y\). In probability notation, this says \(R\) and \(Y\) are conditionally independent given \(X\). Under MAR, \(P(Y\mid X,R=1)=P(Y\mid X,R=0)\). This does not imply that \(P(Y\mid R=1)\) and \(P(Y\mid R=0)\) are equal.
How Predictor Composition Changes the Marginal
For discrete predictors, the relationship follows by averaging the conditional outcome distribution across predictor levels. For continuous predictors, the corresponding average is an integral. In either case, the predictor distribution used in the average is group-specific.
Under MAR and a correctly specified imputation model, the first factor can be the same for observed and missing cases at each \(x\). The second factor, \(P(X=x\mid R=r)\), can still differ. Therefore, their marginal distributions can differ. For continuous \(X\), replace the sum by an integral over \(x\).
This reasoning assumes the predictors needed for the comparison are available for both groups, the conditional model is adequate, and there is sufficient overlap in predictor values between observed and missing cases. In practice, model estimates and finite samples add uncertainty; a visible discrepancy is not automatically evidence of failure, and a visually close match is not proof of validity.
Worked Example: Different Binary-Outcome Marginals Under MAR
Suppose \(Y\) is a binary outcome and \(X=1\) marks a high-risk group. In a hypothetical data set, 20% of observed cases are high risk, while 80% of cases with \(Y\) missing are high risk. Assume the conditional outcome probabilities are \(P(Y=1\mid X=0)=0.10\) and \(P(Y=1\mid X=1)=0.50\), and that these probabilities apply in both groups under MAR.
The observed-case marginal prevalence is
The corresponding imputed prevalence among missing cases is
Thus, the expected outcome prevalence is 18% among observed cases and 42% among cases receiving imputations, despite identical conditional outcome probabilities in both groups. The 24-percentage-point difference follows from the different proportions at high risk, not from a discrepancy in the conditional model. These are hypothetical probabilities, not results from a real study.
Continuous Outcomes and Marginal Shape
The same principle applies to quantitative outcomes. A difference in the predictor distribution can shift the marginal mean, alter the variance, and change the shape of the distribution. A marginal comparison therefore mixes information about the conditional outcome model with information about who is observed and who is missing.
Worked Example: A Shift in a Continuous Outcome
Consider a hypothetical conditional model \(Y=10+2X+\varepsilon\), where \(Y\) is measured in points, \(X\) is measured in years, and \(\varepsilon\sim N(0,9)\) is independent of \(X\). Suppose observed cases have \(X\sim N(2,1)\), while cases with \(Y\) missing have \(X\sim N(6,1)\). Assume MAR and that the stated conditional model is correct.
For observed cases, the marginal mean is
For cases with \(Y\) missing, the imputed marginal mean is
The marginal variances also follow from the model. Since \(Y=10+2X+\varepsilon\) and \(X\) and \(\varepsilon\) are independent, each group’s variance is \(2^2\operatorname{Var}(X)+\operatorname{Var}(\varepsilon)\). Here this is \(4(1)+9=13\) points squared in both groups. Because both predictor distributions are normal, the two outcome marginals are normal with standard deviation \(\sqrt{13}=3.606\) points, but their means differ. The conditional relationship is identical; the marginal locations are not.
More complex conditional relationships can produce differences in marginal shape as well as location. For example, if \(Y\) is binary and its probability changes sharply across \(X\), two groups with different concentrations of \(X\) can have different marginal proportions or mixtures. The discrepancy can be expected even when the model captures the conditional relationship correctly.
Why Equal Marginals Are Not Enough
The reverse warning is just as important: matching observed and imputed marginals does not establish that the imputation model is adequate. Aggregation can hide errors that point in opposite directions across predictor levels.
Worked Example: A Marginal Match That Hides Conditional Errors
Suppose a hypothetical binary outcome has two equally common predictor strata among observed cases and among cases with \(Y\) missing. In the observed cases, the outcome prevalences are 0.10 for \(X=0\) and 0.50 for \(X=1\). Suppose the true conditional prevalences among the missing cases are also 0.10 and 0.50, but an imputation model instead predicts 0.40 for \(X=0\) and 0.20 for \(X=1\).
The observed marginal prevalence is
The imputed marginal prevalence is also
The marginals match exactly, but the imputation model overpredicts by 0.30 in the \(X=0\) stratum and underpredicts by 0.30 in the \(X=1\) stratum. This constructed example shows why a single overall distribution check can miss important conditional errors. In actual data, the true conditional prevalences for missing cases are unknown; the example illustrates a diagnostic limitation, not a way to observe those values directly.
Choosing a Useful Comparison
A marginal observed-imputed plot answers a descriptive question: how do the observed values and the values generated for missing cases differ overall? It does not answer whether the imputation model reproduces the outcome distribution at comparable predictor values. For that question, compare distributions within meaningful predictor strata, or standardize both conditional distributions to the same predictor distribution.
For example, choose a common set of weights \(w(x)\), with nonnegative weights that sum to 1. The standardized outcome distribution is the weighted combination of the conditional distributions, using those same weights for each group:
Using the same weights removes differences caused solely by the groups’ predictor composition. It does not guarantee a good model: a standardized comparison can still hide errors within strata, and the result depends on the chosen weights. Stratified comparisons retain more detail but can be unstable with sparse data. When predictor distributions have little overlap, standardization may also require extrapolating beyond the information supported by observed cases.
Interpret diagnostics in light of the model’s assumptions and purpose. Under MAR, the conditional distributions should agree in principle, but observed conditional estimates fluctuate and may be imprecise. Imputations also reflect model and parameter uncertainty; comparing one imputed data set as if it were the only possible set can give a misleadingly definite picture. Distributional comparisons are diagnostic evidence, not a direct test of MAR, which cannot generally be verified from the observed data alone.
Assumptions, Failure Modes, and Practical Interpretation
A marginal difference is most readily explained by predictor composition when three conditions are plausible: missingness is MAR given the included predictors, the conditional imputation model represents \(Y\mid X\) adequately, and there is enough overlap to estimate that relationship for the missing cases. If important predictors are omitted, the conditional relationship is misspecified, or missing cases occupy predictor regions with few observed outcomes, the imputed conditional distributions may be wrong as well. In those situations, attributing the entire discrepancy to composition would be unjustified.
A useful diagnostic sequence is:
Record what differs between observed values and imputations, such as location, spread, or category proportions, without treating equality as the default requirement.
Assess whether the missing cases differ from observed cases on predictors related to \(Y\). A difference in such predictors can explain a marginal shift.
Compare conditional distributions across relevant predictor levels and note sparse strata or extrapolation. A marginal match cannot replace these checks.
Consider sampling variation and variation across imputations, and state that the conclusion depends on the included predictors and the assumed missingness mechanism.
A common mistake is to argue, “The imputed distribution differs from the observed distribution, so the imputation model must be wrong.” This confuses marginal and conditional comparisons. The correction is to ask whether the difference is expected from the predictor distributions, and then inspect the conditional fit and overlap. The opposite mistake—“The marginals match, so the imputations are valid”—is also incorrect because conditional errors can cancel when averaged.
Marginal comparisons remain useful. A striking, unexplained discrepancy may flag coding errors, implausible predictions, or a need to investigate the model. But they are not a universal equality target. Conditional comparisons are more informative about model behavior at comparable predictor values; marginal comparisons describe the overall distributions produced for the two groups. Neither alone establishes that the missing values have been recovered correctly.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- Under MAR given \(X\), what can be equal across observed and missing cases even when their marginal distributions of \(Y\) differ?
- A binary outcome has conditional prevalences 0.20 for \(X=0\) and 0.60 for \(X=1\). If 75% of observed cases have \(X=0\), what is the observed marginal prevalence?
- In the previous question, suppose only 25% of missing cases have \(X=0\). What is the expected imputed marginal prevalence under the same conditional model?
- Why can matching overall observed and imputed prevalences fail to detect a problem?
Solutions. (1) The conditional distributions \(P(Y\mid X,R=1)\) and \(P(Y\mid X,R=0)\) can be equal; the groups may still have different distributions of \(X\). (2) The prevalence is \((0.75)(0.20)+(0.25)(0.60)=0.15+0.15=0.30\), or 30%. (3) The missing group has 75% with \(X=1\), so its expected prevalence is \((0.25)(0.20)+(0.75)(0.60)=0.05+0.45=0.50\), or 50%. (4) Errors in different predictor strata can offset in the aggregate, so an overall match does not establish conditional fit.