A Stable Chain With Incompatible Conditionals

When a Chain Settles but Its Conditionals Conflict

An FCS algorithm repeatedly updates variables from their specified conditional models. If those models are compatible, they can be understood as conditionals of a common joint distribution. But the updates still define a Markov chain whether or not such a joint distribution exists. The chain may settle into a stationary distribution even when the models conflict. Its stable behavior is then a property of the update mechanism, not evidence that the specified conditionals describe one coherent joint model.

A small binary example makes this distinction exact. It lets us check incompatibility directly, derive the stationary distribution of a particular update scheme, and compare the chain’s actual stationary conditionals with the ones supplied to it. The example is deliberately simple; the same conceptual issue can arise with more variables and richer conditional models.

A Compatibility Check for Two Binary Variables

Let \(X\) and \(Y\) each take values 0 or 1. Write \(a=P(X=1\mid Y=0)\), \(b=P(X=1\mid Y=1)\), \(c=P(Y=1\mid X=0)\), and \(d=P(Y=1\mid X=1)\). Assume all four probabilities are strictly between 0 and 1, so all conditional odds are defined and a joint distribution giving positive probability to all four cells is possible.

If the two conditionals come from the same positive joint distribution, each implies the same cross-product ratio of the four joint cell probabilities. The conditional odds of \(X=1\) at \(Y=1\), divided by those at \(Y=0\), must equal the conditional odds of \(Y=1\) at \(X=1\), divided by those at \(X=0\).

The equality is necessary because both expressions reduce to the same ratio of joint cells. For example, the odds of \(X=1\) given \(Y=1\) are \(P(X=1,Y=1)/P(X=0,Y=1)\); dividing by the corresponding odds given \(Y=0\) gives the displayed joint-cell ratio. The other side reduces to that same ratio. Thus, if the two conditional odds ratios disagree, no positive joint distribution can have both specified conditionals.

Deriving a Stationary Distribution for One Update Scheme

Use a fixed systematic scan: from the current pair \((X,Y)\), first draw a new \(X\) from the specified \(X\mid Y\) model, then draw a new \(Y\) from the specified \(Y\mid X\) model using that newly drawn \(X\). Denote the updated values by \(X'\) and \(Y'\). The conditionals in the example are all strictly between 0 and 1, so every possible new pair has positive probability from every current pair. For this finite-state chain, that positivity gives a unique stationary distribution and convergence to it from any starting state.

To find the stationary distribution, let \(q=P(Y=1)\) at the start of a sweep under stationarity, and let \(p=P(X'=1)\) after the first update. The specified \(X\)-conditional gives \(p=0.20(1-q)+0.80q\). The second update gives \(P(Y'=1)=0.20(1-p)+0.50p\). Stationarity requires \(P(Y'=1)=q\), so these equations can be solved together.

Substituting the first equation into the second gives \(q=0.20+0.30(0.20+0.60q)=0.26+0.18q\). Therefore \(0.82q=0.26\), so \(q=13/41\approx0.3171\). Substitution then gives \(p=0.20+0.60(13/41)=16/41\approx0.3902\).

The joint cell probabilities at the end of a sweep follow from the two draws. The first draw gives \(P(X'=1)=16/41\); conditional on \(X'\), the second draw uses the specified \(Y\mid X\) model. Thus \(P(X'=1,Y'=1)=(16/41)(0.50)=8/41\), and the other cells follow in the same way.

These probabilities sum to \(41/41=1\). Their \(Y'=1\) marginal is \((8+5)/41=13/41=q\), and their \(X'=1\) marginal is \((8+8)/41=16/41=p\). They therefore reproduce the values used in the stationary calculation. In fact, the joint distribution is stationary for the full update: at the start of the next sweep, the \(Y\) marginal is again \(13/41\), which gives the same \(X\) marginal after the first draw and the same joint probabilities after the second.

What Stability Can and Cannot Tell You

The assumptions behind the example matter. The update order is fixed, each update draws from the stated model using the current conditioning value, and every conditional probability is strictly between 0 and 1. These conditions make every full-sweep transition possible and ensure a unique stationary distribution for this finite chain. Other update schemes or models with zero-probability transitions require their own analysis; the example does not claim that every FCS chain must converge.

In practice, summaries may look stable across later iterations: for example, the proportion with \(X=1\) or \(Y=1\) may fluctuate around a nearly constant level. Such behavior is relevant to the chain’s sampling behavior, but it is not a compatibility check. A chain can settle under its transition mechanism even when the specified conditionals disagree about what a common joint distribution would require.

A useful assessment keeps two questions separate: do the chain’s summaries appear stable under the chosen update process, and do the conditional models satisfy the conditions needed to share a joint distribution? The first question concerns the chain; the second concerns the specification. Useful checks include algebraic compatibility conditions where available, inspection of the conditional models’ implications, and comparisons between the observed stationary relationships and the updates prescribed at each step. A trace that looks flat is not a substitute for those checks.

Alternatives, Limitations, and a Common Pitfall

When compatible conditionals are available, a Gibbs sampler updates from the full conditionals of a joint distribution. Its stationary target is that joint distribution, subject to the usual conditions for the chain. FCS instead specifies a collection of conditional models and cycles through updates; if the models are incompatible, the induced chain may still have a stationary distribution, but that distribution is determined by the transition kernel and need not have the specified models as its own full conditionals. Specifying a joint model directly is another approach: it makes the joint structure explicit, but requires choosing and fitting that joint model.

The binary calculation is unusually transparent. In larger systems, compatibility conditions can be harder to check, and a stationary distribution may be difficult to derive exactly. Also, finite-run summaries only approximate the behavior of a chain and can be affected by dependence and insufficient exploration. This example isolates compatibility from those computational complications; it does not establish that a particular practical FCS analysis has converged or that its imputations are suitable for an intended analysis.

• Common mistake: “The trace and marginal summaries have levelled off, so the conditionals must be compatible.”
• Correction: Stability describes behavior of the transition chain. Compatibility requires a separate check that one joint distribution can generate all the specified conditionals.
• Another mistake: “The stationary distribution reproduces the last update’s conditional, so it reproduces every update conditional.”
• Correction: In the example, the last-updated \(Y\mid X\) conditionals are reproduced, but the \(X\mid Y\) conditionals are not. Update order and the full transition mechanism matter.