Why Marginal Checks Cannot Settle Compatibility
An imputation can reproduce the observed range, mean, or even the full marginal distribution of each variable and still distort the analysis. The reason is that a regression analysis depends not only on how each variable behaves separately, but also on how variables behave together. If an incomplete covariate is imputed without preserving its relationship with the outcome, a completed data set may have plausible-looking covariate values but an altered regression coefficient.
This distinction matters when reviewing MICE imputations. Marginal checks can reveal problems such as implausible values or a badly mismatched distribution. Passing those checks does not establish that the imputed values retain the outcome-conditional patterns needed by the planned analysis. The target is not merely a plausible distribution for each variable in isolation; it is an adequate representation of the relevant joint distribution.
The concern applies when the analysis estimates an association, such as a regression slope or an odds ratio. For such an analysis, the outcome–covariate relationship is part of the information being estimated. Imputation models therefore need to represent the relationships relevant to the analysis, not simply the separate distributions of the variables.
Same Marginal Distribution, Different Regression Slopes
The following hypothetical example uses two completed data sets. They have exactly the same observed values of the outcome \(Y\) and covariate \(X\) separately, but pair those values differently. This controlled comparison isolates why the marginal distribution of \(X\) cannot determine the regression relationship.
Worked Example: Identical Covariate Margins, Different Slopes
Suppose eight hypothetical records have four \(Y=0\) outcomes and four \(Y=1\) outcomes. In both data sets, the \(X\) values are \(1,1,3,3,5,5,7,7\). In data set A, the four \(Y=0\) records have \(X=1,1,3,3\), and the four \(Y=1\) records have \(X=5,5,7,7\). In data set B, each outcome group has one record at each of \(X=1,3,5,7\). Thus, the marginal distributions of both variables are exactly the same in A and B.
For the least-squares regression of \(Y\) on \(X\), the slope is \(S_{XY}/S_{XX}\), where \(S_{XY}=\sum_i (X_i-\bar{x})(Y_i-\bar{y})\) and \(S_{XX}=\sum_i (X_i-\bar{x})^2\). In either data set, \(\bar{x}=4\), \(\bar{y}=0.5\), and \(S_{XX}=2(9+1+1+9)=40\).
In data set A, the \(Y=0\) records contribute \(1.5+1.5+0.5+0.5=4\) to \(S_{XY}\). The \(Y=1\) records contribute \(0.5+0.5+1.5+1.5=4\). Therefore \(S_{XY}=8\), and the fitted slope is \(8/40=0.2000\) outcome units per unit of \(X\).
In data set B, each outcome group has \(X\) deviations from 4 equal to \(-3,-1,1,3\), whose sum is zero. The contribution to \(S_{XY}\) is zero within each group, so \(S_{XY}=0\) and the fitted slope is \(0/40=0\). Identical marginal distributions have produced different fitted slopes because the pairing of \(X\) with \(Y\) differs.
These are deliberately specified allocations, not a claim that random marginal draws will yield either allocation. If the values in a finite imputation are drawn at random, their group-specific counts and means will generally vary. The example establishes that marginal equality alone does not determine the association.
A Population-Level Justification
A short covariance identity makes the issue precise. Suppose \(Y\) is binary, with \(P(Y=1)=p\), and let \(\mu_1=E(X\mid Y=1)\) and \(\mu_0=E(X\mid Y=0)\). The covariance between \(X\) and \(Y\) is
The first equality is the definition of covariance. For the second, \(E(XY)=p\mu_1\), because \(XY=0\) when \(Y=0\); also \(E(X)=p\mu_1+(1-p)\mu_0\) and \(E(Y)=p\). Expanding and collecting terms gives the final expression. Thus the covariance depends on a difference between outcome-specific means. A marginal distribution for \(X\) can constrain its overall mean and variance, but does not say how those values are distributed across the outcome groups.
When \(Y\) is regressed linearly on \(X\), the population slope is \(\operatorname{Cov}(X,Y)/\operatorname{Var}(X)\), provided \(\operatorname{Var}(X)>0\). A procedure that preserves the marginal distribution of \(X\) while changing its conditional means across \(Y\) can therefore change the slope. For other analyses, different features of the joint distribution may matter, but the same principle holds.
Worked Example: Independent Marginal Draws and the Population Slope
Consider a hypothetical population in which \(P(Y=0)=P(Y=1)=0.5\). Suppose \(X\mid Y=0\) takes values 1 and 3 with equal probabilities, while \(X\mid Y=1\) takes values 5 and 7 with equal probabilities. The marginal distribution of \(X\) assigns probability \(0.25\) to each of 1, 3, 5, and 7.
The outcome-specific means are \(\mu_0=(1+3)/2=2\) and \(\mu_1=(5+7)/2=6\). The overall mean is \(E(X)=0.5(2)+0.5(6)=4\). Also, \(E(X^2)=(1+9+25+49)/4=21\), so \(\operatorname{Var}(X)=21-4^2=5\).
Using the identity above, \(\operatorname{Cov}(X,Y)=0.5(0.5)(6-2)=1\). The population least-squares slope of \(Y\) on \(X\) is therefore \(1/5=0.2000\) per unit of \(X\).
Now consider imputing from the marginal distribution of \(X\) independently of \(Y\). In the resulting idealized population, \(X\) and \(Y\) are independent: both outcome groups have the same conditional distribution for \(X\), so \(E(X\mid Y=0)=E(X\mid Y=1)=4\), and the population covariance and slope are zero. The marginal distribution remains the same, but the outcome-conditional pattern has changed.
This is a population-level result. A finite set of random draws from the marginal distribution will not generally have exactly the same empirical distribution in each outcome group, nor exactly zero sample covariance. Its group means and fitted slope vary across draws. The conclusion is that independent marginal draws yield zero covariance in the population under the stated setup—not that every finite imputation must produce a zero fitted slope.
The Same Problem for a Binary Outcome
For logistic regression, the relevant association can be expressed by an odds ratio. The next example uses tables with exactly the same row and column totals, so even exact matching of both variables’ marginal proportions cannot distinguish the two associations.
Worked Example: Identical Table Margins, Different Odds Ratios
Consider two hypothetical \(2\times2\) tables of a binary outcome \(Y\) and binary covariate \(X\). In each table there are 10 records with \(Y=1\), 10 with \(Y=0\), 10 with \(X=1\), and 10 with \(X=0\).
| Table | \(Y=1, X=1\) | \(Y=1, X=0\) | \(Y=0, X=1\) | \(Y=0, X=0\) |
|---|---|---|---|---|
| Associated | 8 | 2 | 2 | 8 |
| Independent | 5 | 5 | 5 | 5 |
For the associated table, the odds ratio for \(Y=1\) comparing \(X=1\) with \(X=0\) is \((8/2)/(2/8)=16\). The corresponding logistic regression coefficient for \(X\), with \(X\) coded 0 and 1 and no other predictors, is \(\log(16)\approx2.773\) log-odds units. In the independent table, the odds ratio is \((5/5)/(5/5)=1\), and the coefficient is \(\log(1)=0\) log-odds units. The margins match exactly, but the outcome–covariate associations are sharply different.
What Must Be Assumed and Checked
A compatibility assessment concerns the intended analysis and the imputation model together. The central requirement is that the imputation procedure adequately represents the conditional distributions needed to recover the analysis relationship, including relevant functional forms and predictors. The analysis and imputation specifications must not conflict in a way that systematically removes the relationship of interest.
This claim is not a guarantee that a finite collection of imputations will reproduce every sample statistic exactly. Imputed values vary because they are draws, and estimation also varies across completed data sets. The goal is to use a suitable model and retain appropriate uncertainty, not to force identical observed and imputed summaries. The usual missing-data assumptions and the adequacy of the specified conditional models also matter; marginal agreement cannot compensate for a misspecified conditional model.
Useful checks therefore look beyond one-variable summaries. Compare outcome–covariate relationships in records with observed and imputed covariate values, examine conditional summaries across meaningful outcome groups, and assess whether the fitted association changes substantially across completed data sets. Interpret these checks with their uncertainty: a difference can reflect random variation, while an apparently good match in a small sample is not proof that the model is correct. The earlier material on checking imputed analysis relationships gives context for such relationship-focused checks.
Name the outcome, covariate, and association measure that the analysis estimates. A marginal distribution check cannot be interpreted fully without knowing which relationship matters.
Check whether imputed covariate values retain plausible relationships with the observed outcome and other analysis variables, rather than looking only at pooled marginal summaries.
Review whether coefficients or other target summaries are stable and plausible across imputations. Variation is expected; systematic attenuation or a changed pattern warrants investigation.
Alternatives, Limitations, and a Common Pitfall
Marginal-only imputation is simple, but it is generally not an adequate substitute when the analysis relies on outcome–covariate association. Conditional imputation models can use the outcome and relevant predictors to represent those relationships, although merely adding variables does not ensure that the conditional model is correctly specified. Complete-case analysis avoids assigning imputed values but changes the analyzed records and may lose precision or introduce bias under some missingness mechanisms. Mean imputation may preserve a rough location summary, but it does not preserve the covariate’s variability or its joint relationship with the outcome. These approaches answer different problems; none can be selected solely because a marginal histogram looks plausible.
There are limits to diagnostics. The missing values’ true conditional distribution is not directly observed, so checks cannot certify it from the data alone. Sparse outcome groups, weak overlap, small samples, and a poor conditional model can all make relationships hard to assess. Moreover, agreement in one relationship does not establish compatibility for every term in a more complex analysis. The earlier material on limitations of pairwise conditional reasoning is relevant when several variables and relationships must be considered together.
- Common mistake: “The imputed covariate has the right mean and histogram, so the regression must be preserved.”
- Correction: A mean and histogram describe a marginal distribution. Regression depends on how the covariate is paired with the outcome, so assess the conditional or joint relationship required by the analysis as well.
- Another mistake: “Independent random draws from the marginal distribution make the sample slope exactly zero.”
- Correction: Independence implies zero population covariance under the stated setup. A finite sample has random group means and a random fitted slope, which need not be exactly zero.
Check Your Understanding
Try each question, then compare your reasoning with the short solutions below.
- Two completed data sets have the same values of \(X\) and the same values of \(Y\), but different pairings of those values. Can their regression slopes differ? Why?
- In the population example above, what is the outcome-conditional mean of \(X\) in each group after independent marginal imputation, and what is the resulting population slope?
- Two \(2\times2\) tables have identical row and column totals. Does this establish that their odds ratios are equal? Explain.
- Name one relationship-focused diagnostic and one limitation of that diagnostic.
Solutions. (1) Yes. The slope depends on the covariance between \(X\) and \(Y\) as well as the variance of \(X\); changing the pairings can change the covariance while leaving both marginals fixed. (2) Both conditional means are 4, so the population covariance and slope are zero under independence; a finite-sample slope need not be exactly zero. (3) No. The tables can have different cell allocations and therefore different odds ratios, as the worked example demonstrates. (4) One may compare outcome–covariate conditional summaries for observed and imputed values; such a check cannot reveal the true distribution of missing values and may be inconclusive in a small or sparse sample.