Why Incompatibility Matters After the Chain Appears Stable
A chained-equations algorithm repeatedly draws each variable with missing values from a specified conditional model. If the conditional models are compatible, they can all be conditional distributions of one joint distribution. When they are incompatible, there is no such joint distribution with those conditionals. The updates may nevertheless settle into a stable pattern. The central consequence is that stability alone does not establish that the imputations represent the joint model the analyst intended.
This tutorial focuses on what incompatibility can do: make the resulting joint distribution depend on update order, leave a theoretically incoherent set of conditional assumptions, and affect estimates calculated from completed data. Compatibility and congeniality are distinct ideas; the earlier material on Compatibility Versus Congeniality separates them. Here the concern is what can happen when compatibility fails.
With incompatible models, chained updates still define a sequence of random draws. Under suitable conditions, that sequence may have a stationary distribution: a distribution that remains unchanged after an update cycle. But the stationary distribution need not have all the specified models as its conditionals. In addition, the distribution of the paired values at the end of a cycle can depend on which variable was updated last.
A Binary Example: Stable Marginals, Different Joint Distributions
Consider two binary variables, \(X\) and \(Y\), in a hypothetical imputation problem. Suppose the specified conditional models are
and
These are valid conditional probabilities: each is strictly between 0 and 1. But validity of each conditional separately does not ensure compatibility. For a binary joint distribution with all four cell probabilities positive, the conditional odds ratios must agree. The odds ratio implied by the \(Y\mid X\) model is
The \(X\mid Y\) model instead implies
Since \(23/8\ne36\), there is no positive joint distribution with both specified conditionals. The discrepancy is not a numerical approximation: the two conditional models impose conflicting association structures.
Worked Example: Exact Convergence and Scan-Order Dependence
Use the binary models above. First update \(Y\mid X\), then update \(X\mid Y\). Let \(p_t=P(X=1)\) at the start of cycle \(t\). After the \(Y\mid X\) update, the marginal probability of \(Y=1\) is
Substitution: \(P(Y=1)=\frac{2}{5}(1-p_t)+\frac{23}{35}p_t=\frac{2}{5}+\frac{9}{35}p_t\). The following \(X\mid Y\) update gives \(p_{t+1}=\frac{1}{10}+\frac{7}{10}P(Y=1)\). Therefore,
A fixed point \(p^*\) satisfies \(p^*=0.38+0.18p^*\), so \(p^*=0.38/0.82=19/41\approx0.4634\). This is an exact fixed point when written as \(19/41\). Subtracting its fixed-point equation from the recurrence gives
Because \(|0.18|<1\), the distance from the fixed point contracts by a factor of 0.18 per cycle. For example, if \(p_0=0\), then \(p_1=0.38\), and \(p_2=0.38+0.18(0.38)=0.4484\); these move toward \(19/41\). This establishes convergence of this marginal recurrence, not compatibility.
Now compare the joint distributions at the ends of the two possible scan orders. If \(Y\) is updated and then \(X\), the terminal pair satisfies the specified \(X\mid Y\) model. At stationarity, \(P(Y=1)=149/287\). The joint cell probabilities, in the order \((X,Y)=(1,1),(1,0),(0,1),(0,0)\), are
Calculation: \(P(1,1)=(4/5)(149/287)=596/1435\); \(P(0,1)=(1/5)(149/287)=149/1435\); \(P(1,0)=(1/10)(138/287)=69/1435\); and \(P(0,0)=(9/10)(138/287)=621/1435\).
If \(X\) is updated and then \(Y\), the terminal pair instead satisfies the specified \(Y\mid X\) model. Its cells are \(437/1435,\;228/1435,\;308/1435,\;462/1435\), in the same order. For instance, \(P(1,1)=(19/41)(23/35)=437/1435\). The two joint distributions differ even though the stationary marginal probabilities of \(X=1\) and \(Y=1\) are \(19/41\) and \(149/287\) in both scans. Thus apparently stable, agreeing marginal summaries do not guarantee an order-invariant joint distribution.
What a Stable Chain Does—and Does Not—Show
A trace plot with no visible drift can be evidence that the monitored quantity has reached a stable regime under the implemented update scheme. It does not show that all the conditional models arise from one joint distribution, nor that the stationary distribution has the conditionals the analyst specified. A chain can converge to a stable distribution generated by the sequence of updates while still violating the intended conditional statements.
The same distinction applies to common numerical convergence summaries. Agreement across chains for selected summaries, a small estimated Monte Carlo error, and stable marginal distributions address aspects of simulation behavior. None is a compatibility test. The binary example makes this precise: the recurrence contracts exactly to its fixed point, but the odds-ratio contradiction remains.
A useful diagnostic review therefore asks both whether the chain behaves stably and whether the model specification has a coherent joint interpretation. Depending on the variable types and model forms, useful checks include:
- Compare results under alternative update orders or scan schemes, looking beyond marginal means to joint summaries such as cross-tabulations, correlations, or interaction patterns.
- Check that conditional relationships implied by different parts of the model are not contradictory. For simple binary pairs, the conditional odds ratios provide a direct check; other model families require their own compatibility conditions.
- Inspect trace plots, between-chain behavior, and distributional diagnostics for drift, poor mixing, extreme values, or implausible conditional relationships. These can reveal problems but cannot certify compatibility.
- Repeat relevant analyses across defensible model specifications. Material changes in completed-data estimates or uncertainty are a warning that conclusions may depend on modeling choices.
Order sensitivity is a warning rather than a universal numerical measure of incompatibility. Similar results across orders do not prove compatibility: some discrepancies may be small for the observed data, and the summaries examined may not reveal them. Conversely, noticeable differences can arise from Monte Carlo error or insufficient run length, so simulation stability must be assessed before interpreting an order comparison.
Incompatibility and Downstream Inference
A completed data set contains values for the variables jointly, so a joint relationship matters even when the imputation models are specified one conditional at a time. Incompatible updates can affect associations, tail behavior, or combinations of variables that are not adequately summarized by their marginal distributions. An analysis based on those completed data can consequently depend on the update scheme or on choices in the conditional models.
For multiple imputation, let \(Q_j\) be the estimate from imputed data set \(j\), and let \(U_j\) be its estimated variance. The usual pooled estimate is \(\bar Q=m^{-1}\sum_{j=1}^{m}Q_j\), and the within-imputation variance is \(\bar U=m^{-1}\sum_{j=1}^{m}U_j\). Pooling also uses the between-imputation variance \(B=(m-1)^{-1}\sum_{j=1}^{m}(Q_j-\bar Q)^2\). These calculations summarize estimates and imputation variability under the procedure used; they do not repair incompatible conditional models or establish that the imputations represent a coherent intended joint distribution.
Worked Example: A Pooled Estimate Does Not Diagnose Compatibility
Suppose two hypothetical scan orders are applied to the same illustrative incomplete-data setting. Under order A, five completed-data estimates of a mean outcome, measured in points, are \(10,11,9,10,10\). Under order B, they are \(12,13,11,12,12\). For simplicity, suppose each completed-data analysis reports variance \(4\) points squared. These values are hypothetical and are used only to show the calculation.
For order A, \(\bar Q_A=(10+11+9+10+10)/5=10\) points. The deviations from 10 are \(0,1,-1,0,0\), so \(B_A=(0^2+1^2+(-1)^2+0^2+0^2)/(5-1)=0.5\) points squared. Also \(\bar U_A=4\) points squared, giving the usual total variance \(T_A=\bar U_A+(1+1/m)B_A=4+(1+1/5)(0.5)=4.6\) points squared and \(SE_A=\sqrt{4.6}=2.145\) points.
For order B, the estimates have mean \(\bar Q_B=12\) points and the same deviations, so \(B_B=0.5\) points squared, \(T_B=4.6\) points squared, and \(SE_B=2.145\) points. The pooled point estimates differ by \(12-10=2\) points despite equal within- and between-imputation variance summaries. This calculation is not a formal test of scan-order differences. It shows why an apparently routine pooled result cannot answer whether the imputation models are compatible: the scan-specific model behavior must be assessed separately.
Alternatives, Limitations, and Common Mistakes
A direct alternative is to specify a joint model for the incomplete variables and derive its conditional distributions from that joint model. By construction, those conditionals are compatible, provided the joint model is well-defined. This can make the target of the imputation process explicit, although the joint model still requires defensible distributional assumptions and may be difficult to specify for mixed variable types or complex data structures. Another option is to revise the conditional model forms and parameters so that they can arise from a common joint model. Neither alternative removes the need to assess fit and adequacy.
In some applications, a particular analysis may be relatively insensitive to the incompatibility present, or the incompatible specifications may be close enough that practical consequences are small. That possibility is not a general guarantee. Sensitivity depends on the missingness pattern, the variable distributions, the analysis, and which features of the joint distribution matter for the estimand. Compatibility also does not ensure that the model is substantively appropriate; it addresses coherence among the conditional models, not every source of model error.
- Wrong reasoning: “The chains have stable trace plots, so the imputation models must be compatible.” Correction: stability concerns the behavior of the update process; compatibility asks whether all specified conditionals can come from one joint distribution.
- Wrong reasoning: “The imputed marginal distributions agree across scan orders, so the imputations are equivalent.” Correction: joint distributions can differ while the marginals agree, as the binary example demonstrates.
- Wrong reasoning: “Pooling includes between-imputation variance, so it accounts for incompatibility.” Correction: pooling summarizes variation across the imputations generated by a chosen procedure. It does not add uncertainty for every modeling choice or make incompatible conditionals jointly coherent.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- In the binary example, why does the disagreement between conditional odds ratios establish incompatibility?
- For the recurrence \(p_{t+1}=0.38+0.18p_t\), calculate \(p_1\) and \(p_2\) when \(p_0=0\), and state the exact fixed point.
- Can convergence of a chain establish that all specified conditional models are compatible? Explain.
- What does a difference in pooled estimates across scan orders suggest, and what does it not prove on its own?
- Name one alternative to using incompatible conditional models and give one limitation of that alternative.
Solutions. (1) Any positive \(2\times2\) joint distribution has one odds ratio, which must be implied by both sets of conditionals. Here the \(Y\mid X\) model implies \(23/8\), whereas the \(X\mid Y\) model implies \(36\); because they differ, no joint distribution can have both as its conditionals. (2) \(p_1=0.38+0.18(0)=0.38\), and \(p_2=0.38+0.18(0.38)=0.4484\). Solving \(p^*=0.38+0.18p^*\) gives \(p^*=0.38/0.82=19/41\), exactly. (3) No. Convergence shows that the implemented updates may have reached a stable regime; it does not establish that their conditional models share a joint distribution. (4) It suggests that results may depend on the update scheme and warrants checks of simulation stability and model specification. By itself, it does not prove incompatibility, because Monte Carlo error or inadequate run length could also contribute. (5) One alternative is to define a joint model and derive its conditionals. This ensures compatibility by construction, but the joint model still needs appropriate assumptions and may be difficult to specify adequately.)