Why Check the Variable Order?
In fully conditional specification (FCS), each incomplete variable is updated from a model conditional on the other variables. A complete sweep updates every variable once. When there are several incomplete variables, a sweep must choose an order: for example, update \(X_1\), then \(X_2\), then \(X_3\), or update them in the reverse order. The order is part of the algorithm, not a change to the data or to the stated conditional models.
Comparing orders is a computational sensitivity check. It asks whether conclusions from the chain appear to depend on the sequence in which the same specified conditional models are applied. Differences can arise from ordinary Monte Carlo error, different transient behavior, or from features of the FCS specification and its implementation. In particular, when the conditional models are incompatible, there may be no single joint distribution that all the updates preserve. Order sensitivity is then possible even if the chains appear stable.
What Order Can Change
Let \(Z^{(t)}\) denote the current set of imputed values at the end of sweep \(t\). Updating variable \(j\) replaces its missing entries with draws from its specified conditional model given the current values of the other variables. A later update in the same sweep uses the values already updated earlier in that sweep. Thus, changing the order changes which values are available at each update.
There is an important distinction between the target distribution and the path taken by the chain. If the conditionals are compatible—that is, they are full conditionals of one joint distribution—and each update is an exact conditional draw, every individual update preserves that joint distribution. Applying several such updates in sequence preserves it as well. However, the transition from one sweep to the next can differ by order. The resulting chains can therefore have different autocorrelation, mixing, and finite-run summaries even when they share the same stationary target.
The justification is direct. If the current complete state has distribution \(\pi\), replacing one component by a draw from its conditional distribution given the remaining components leaves the joint distribution equal to \(\pi\). Applying that argument once for each variable shows that the distribution remains \(\pi\) at the end of the sweep. The argument concerns invariance, not how quickly a chain approaches that distribution or how well a finite run explores it.
When the conditionals do not come from any common joint distribution, this invariance argument does not apply. A stable chain can still have useful empirical summaries, but those summaries need not represent draws from a unique joint distribution implied by all the conditionals. Different update orders may then produce different long-run behavior, not just different short-run autocorrelation.
Worked Example: Compatible Conditionals and Different Chain Summaries
Consider hypothetical standardized variables \(X\) and \(Y\) with a bivariate normal joint distribution, mean zero for each variable, variance 1 for each, and correlation \(0.7\). The full conditionals are \(X\mid Y\sim N(0.7Y,0.51)\) and \(Y\mid X\sim N(0.7X,0.51)\), since \(1-0.7^2=0.51\). Both update orders therefore use conditionals from the same joint distribution.
Suppose two sufficiently long, separately simulated chains give the following hypothetical summaries for the mean of imputed \(X\): order \(X\) then \(Y\), estimate \(0.03\) with \(SE=0.04\); order \(Y\) then \(X\), estimate \(0.01\) with \(SE=0.05\). The estimates differ by
If the runs use independent random streams, an approximate standard error for this difference is
The difference is about \(0.02/0.0640=0.313\) standard errors, small relative to its Monte Carlo uncertainty. Both estimates are also close to the target mean of zero. This comparison does not establish that the chains behave identically. For example, the two orders could still have different autocorrelation or different finite-run efficiency. It does illustrate why a small difference between chain means should not, by itself, be interpreted as evidence that the underlying target changed.
Order Sensitivity When Conditionals Are Incompatible
A useful way to see the possible consequences of incompatibility is to work with two binary variables. For compatible binary full conditionals, the conditional odds ratios implied in the two directions must agree. If they do not, no joint distribution can have both conditionals as its full conditionals.
Worked Example: An Incompatible Pair Gives Different Joint Summaries
Consider hypothetical binary variables \(X\) and \(Y\), with the following fixed conditional probabilities: \(P(X=1\mid Y=0)=0.2\), \(P(X=1\mid Y=1)=0.8\), \(P(Y=1\mid X=0)=0.2\), and \(P(Y=1\mid X=1)=0.6\). The conditional odds ratio from the \(X\) model is
The corresponding conditional odds ratio from the \(Y\) model is
Since \(16\ne 6\), these conditionals are incompatible. Now update \(X\) and then \(Y\). At the end of a sweep, let \(q=P(X=1)\). The last update gives \(P(Y=1)=0.2(1-q)+0.6q=0.2+0.4q\). The next \(X\) update has probability \(0.2\) when the old \(Y\) is zero and \(0.8\) when it is one. Stationarity therefore requires
Under this order, \(P(X=1,Y=1)=0.6q=0.2526\). For the reverse order, \(Y\) is updated first and then \(X\). Let \(r=P(Y=1)\) at the end of a sweep. The last update gives \(P(X=1)=0.2+0.6r\), so stationarity requires
The final \(X\) update uses \(P(X=1\mid Y=1)=0.8\), giving \(P(X=1,Y=1)=0.8r=0.2947\). Thus the two orders produce different joint cell probabilities: \(0.2526\) versus \(0.2947\). In this example the marginal probabilities happen to agree across orders, but the joint association does not. This calculation shows why checking only marginal means or proportions can miss order sensitivity in joint summaries.
How to Run and Interpret the Check
Choose summaries that matter for the analysis: for example, the mean or proportion of imputed values, a joint summary involving multiple variables, or a downstream estimate computed from each completed data set. Use the same conditional-model specifications and the same data in each run. Keep other choices—such as the run length and burn-in rule—comparable, and use independent random streams if you intend to compare Monte Carlo errors using the simple difference formula. If the runs are deliberately paired, the covariance between their estimates must also be considered.
For two independently simulated estimates \(\hat{\theta}_A\) and \(\hat{\theta}_B\), with Monte Carlo standard errors \(SE_A\) and \(SE_B\), a useful descriptive measure is
The standardized difference \(D\) helps assess whether an observed discrepancy is large relative to simulation noise. It is a diagnostic, not a formal hypothesis test of compatibility or a rule that proves the specification is acceptable. Interpret summaries in context and with appropriate units; for example, a difference in an imputed measurement mean is in the measurement’s units, while a difference in a proportion is on the probability scale.
Worked Example: Distinguishing a Discrepancy From Monte Carlo Noise
In a hypothetical data set, 24 of 120 records have a missing value of \(X\). The same FCS models are run with two fixed orders, and each run uses the same burn-in and number of retained iterations. Suppose the mean of the 24 imputed values is \(12.4\) units with \(SE_A=0.18\) under order A, and \(12.9\) units with \(SE_B=0.20\) under order B. With independent random streams,
The estimated difference is about \(1.86\) Monte Carlo standard errors in magnitude. That is worth investigating, but it is not by itself proof of an algorithmic failure: it may reflect finite-run variation, and the calculation addresses only this one summary. Compare trace plots and other relevant summaries, and check whether the discrepancy persists in longer or additional runs. Also examine joint summaries, because agreement in this imputed mean would not establish agreement in dependence between variables.
Diagnostics, Alternatives, and Limitations
A meaningful ordering comparison depends on having runs that are themselves adequate for comparison. Review trace plots, autocorrelation, effective sample size, Monte Carlo error, and between-chain behavior using the earlier material on those diagnostics. In particular, a short or poorly exploring run can produce apparent order sensitivity simply because its summaries are imprecise. Conversely, a stable-looking summary does not prove that the conditional models are compatible.
Separate the ordering check from other comparisons. Repeating a run from different initial values probes sensitivity to initialization; repeating independent chains with the same order probes between-chain variation. An alternative fixed order probes dependence on the sweep sequence. Changing several of these choices at once makes the source of a discrepancy harder to identify. Even when all the displayed summaries agree, the check is limited to the summaries, variables, run lengths, and orders examined; it cannot establish compatibility or guarantee that the chain has explored every relevant part of its state space.
A practical warning is to avoid treating order sensitivity as a pass/fail test. A difference that is small relative to Monte Carlo error is not proof of no sensitivity, while a difference that is large relative to estimated Monte Carlo error is a reason to investigate, not a diagnosis of its cause. Review the specified models and the run behavior before drawing conclusions about the substantive analysis.
- Mistake: “The means under two orders are close, so the FCS conditionals must be compatible.” Correction: Marginal summaries can agree even when joint behavior differs; order comparisons do not establish compatibility.
- Mistake: “Any difference between orders proves the model is wrong.” Correction: Finite-run Monte Carlo variation and different transition behavior can produce differences. Compare them with their uncertainty and inspect other diagnostics.
- Mistake: “Changing order and initialization together tells me which caused the change.” Correction: Vary one feature at a time when the goal is to isolate sensitivity.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- In one sentence, define a fixed variable-update order in FCS.
- Why can two orders yield different autocorrelation even when the conditionals are compatible?
- Two independent runs give estimates 4.2 and 4.0, with Monte Carlo standard errors 0.10 and 0.10. Calculate the standardized difference \(D\).
- Why does agreement between orders in the marginal mean of one variable not establish agreement in the joint behavior of two variables?
- Name one other check that should not be confused with an ordering sensitivity check.
Solutions. (1) It is the specified sequence in which the incomplete variables are updated during each FCS sweep. (2) The sequence changes the chain’s transition dynamics and dependence between successive sweeps, although each update still preserves the common target. (3) \(SE_{\mathrm{diff}}=\sqrt{0.10^2+0.10^2}=\sqrt{0.02}=0.1414\), so \(D=(4.2-4.0)/0.1414=1.41\). This is a descriptive comparison, not a formal test. (4) Marginal summaries do not determine the joint distribution; dependence or joint cell probabilities can differ while marginal means agree. (5) For example, rerunning from different initial values checks sensitivity to initialization, while independent chains with the same order assess between-chain variation.