Why Multimodality Matters in FCS
A chain of chained equations can appear orderly for hundreds of iterations and still represent only one part of the relevant completed-data state space. This can happen when the states visited by the chain form distinct regions, with very few transitions between them. A chain that starts in one region may spend a long time there, giving stable-looking summaries that do not reveal what it has failed to explore.
Here, a chain state means the current collection of imputed values for the missing entries after an FCS sweep. Depending on the implementation, the state may also include quantities used in the update, such as model parameters. The updates form a stochastic process: each new state is generated using the current state and the specified conditional models. The central issue in this tutorial is not ordinary gradual movement within a region, but the possibility that movement between regions is rare.
The definition concerns regions of the completed-data state space, not just multiple peaks in a single variable’s histogram. Two regions may correspond to different patterns across several missing values—for example, one set of imputations may assign several records to a low-value pattern while another assigns them to a high-value pattern. FCS updates variables in sequence, so reaching the second pattern may require a coordinated series of low-probability conditional draws. Intermediate states can be unlikely even when both regions are plausible.
A key qualification is that FCS conditionals need not define a compatible joint distribution. If a coherent stationary target exists, multimodality can refer to distinct regions of that target. If compatibility is absent, it is safer to describe the observed transition behavior—regions that the chain visits and rarely leaves—without claiming that they are modes of a well-defined joint distribution. In either case, the finite-run diagnostic problem is real: a chain’s observed history may not represent all relevant behavior.
A Two-Region Chain: Why Stability Can Mislead
A simple two-state Markov chain isolates the mechanism. Suppose the chain has states \(A\) and \(B\). On each iteration it switches from its current state to the other with probability \(\epsilon\), and otherwise stays where it is. Assume \(0<\epsilon<1\), and assume the transition rule is the same at every iteration. This chain is irreducible and has stationary probabilities \(P(A)=P(B)=0.5\). Those facts describe its long-run behavior; they do not guarantee that a finite run visits both states.
If the chain starts in \(A\), then remaining in \(A\) for \(T\) successive transitions has probability \((1-\epsilon)^T\). The waiting time to the next switch has a geometric distribution with mean \(1/\epsilon\) transitions. Thus, a small transition probability creates long runs in one state even though the chain has a well-defined stationary distribution that gives positive probability to both states.
Worked Example: A Stable Run That Misses Half the States
Consider the two-state chain above with \(\epsilon=0.002\), started at \(A\). Suppose it is run for 500 transitions. The probability of no switch is
So there is about a 36.8% probability that the entire run stays at \(A\). The expected waiting time for a switch is
A trace of the state indicator would be constant throughout such a run. Its lack of visible movement would not show that the chain has explored its stationary behavior; it would show only that no switch occurred in this run. The stationary probability of \(B\) is 0.5, yet the observed run could contain no visits to \(B\) at all.
The example is deliberately simple, but its lesson transfers to a high-dimensional imputation problem. A long stable stretch within one region is not evidence that all relevant regions have been explored. Stability describes what happened in the observed run; adequate exploration requires that the chain move through the regions that matter.
When Regions Have Different Long-Run Weights
Not all regions need have equal stationary probability. A chain may spend most of its long-run time in one region and still get trapped for a long time in a less common one. This is particularly important for diagnosis: seeing a rarely visited region for a short time does not establish its long-run weight, and failing to visit it does not establish that its weight is zero.
For a two-state chain with probability \(a\) of moving from \(A\) to \(B\), and probability \(b\) of moving from \(B\) to \(A\), the stationary probabilities satisfy the balance equation \(\pi(A)a=\pi(B)b\), together with \(\pi(A)+\pi(B)=1\). Solving these equations gives the stationary weights, provided both transition probabilities are positive.
Worked Example: A Chain Trapped in a Less Common Region
Suppose \(P(A\to B)=0.001\) and \(P(B\to A)=0.009\). The stationary probability of \(A\) is
Now start the chain in \(B\). The probability it remains in \(B\) for 200 transitions is
Even though \(B\) has stationary probability 0.1, there is about a 16.4% chance that this run makes no transition out of \(B\) during those 200 transitions. If the chain stays there, its observed occupancy is 100% in \(B\), far from the long-run occupancy of 10%. The discrepancy is not a contradiction: the stationary probability describes the long-run distribution, while the observed proportion comes from a finite, potentially trapped run.
This calculation also shows why the phrase “the chain has converged” should not be inferred from a steady-looking summary alone. Long residence in one region can be compatible with a very different long-run distribution when exits are rare. The chain’s start and its opportunities to move between regions both matter.
What to Look for in an FCS Run
In practice, a completed-data state is usually too large to inspect directly. Diagnostics therefore examine summaries of the imputations and compare chains, while recognizing that a summary can conceal changes in other coordinates. For this topic, the most useful evidence is whether chains initialized in different plausible regions reach and move among the same regions.
Use the structure of the missing-data problem and the conditional models to consider whether different combinations of imputed values could persist. Look for patterns across variables, not only separate marginal values.
Where feasible, create starting completions that represent distinct plausible patterns. Deliberately varied starts are more informative about trapping than several chains that all begin in nearly the same region.
Inspect trace plots of relevant summaries and record whether each chain visits the contrasting patterns. Agreement within one chain is not enough if chains started elsewhere remain in different regions.
If a chain does not change region, that is evidence of limited observed exploration, not proof that the other region has zero probability. If chains switch, the number and timing of switches still matter.
Worked Example: Contrasting Starts Reveal Trapping
Suppose an analyst runs two FCS chains for 1,000 sweeps on hypothetical data with missing values in two strongly associated variables. The analyst defines a simple diagnostic summary \(S\): the number of records whose two imputed values jointly follow a high-value pattern. A low-pattern starting completion has \(S=2\), and a high-pattern starting completion has \(S=18\). The two chains remain near their respective starting patterns throughout the run, with no observed transition between them.
The within-chain summaries may look steady: the first chain fluctuates around \(S=2\), and the second around \(S=18\). But the between-chain difference is \(18-2=16\) records, and neither chain has shown that it can move between these patterns. The evidence therefore supports a concern about region-specific trapping. It does not tell us whether the long-run probability of either region is larger, nor does it prove that the chains have different stationary distributions.
A useful next check is to examine whether the contrast is tied to a particular conditional update or to an implausible feature of the starting values. The analyst should also verify that both patterns are consistent with the observed data and the specified conditional models. If they are, the failure to move between them remains a serious limitation on conclusions drawn from either chain alone.
Initialization is a diagnostic probe, not a cure. If every chain starts in the same region, agreement among their summaries can simply reflect a shared inability to leave it. Conversely, chains that start apart and later meet provide more reassuring evidence than chains that start together, but finite agreement still cannot prove that every important region has been explored.
Distinguishing Trapping from Other Behaviors
Multimodality and trapping are related to, but not identical with, slow mixing. Slow mixing can occur within one broad region when successive states change only gradually. Trapping emphasizes rare movement between separated regions. A chain may experience both: it can move slowly within a region and switch infrequently between regions. Autocorrelation and effective sample size help describe dependence in a run, but they cannot recover a region that the run never visited.
A particularly important failure mode is treating a long run as an automatic remedy. In the two-state example, the probability of no switch declines as \(T\) increases, but when \(\epsilon\) is very small the required run can be much longer than is computationally practical. Extending a chain that has not switched may still leave it in the same region. A lack of transitions is a reason to question the adequacy of the run, not a reason to assume that more iterations will necessarily solve the problem.
| Approach | What it can reveal | Important limitation |
|---|---|---|
| One long chain | Whether the observed chain moves within its visited region and whether transitions occur during the run | Cannot show that unvisited regions do not exist; a trapped chain may look stable |
| Several chains with similar starts | Whether runs behave similarly from comparable initial states | Shared initialization can leave all chains in the same region |
| Several chains with contrasting starts | Whether distinct plausible starting regions lead to similar behavior or remain separated | Finite runs can still miss regions or give too few transitions to assess their weights |
| Review of conditional models and update behavior | Whether model specification or update structure may make transitions implausible | Model review alone does not establish that a revised chain explores adequately |
When trapping is suspected, practical responses include using additional chains from deliberately different starts, inspecting the variables jointly, and reviewing whether the conditional models and update scheme permit movement between the observed patterns. If one specific transition appears to be the bottleneck, examine the relevant conditional updates rather than relying only on a larger iteration count. Changes to the model or update strategy require statistical justification; forcing chains to agree by changing starts or discarding a region is not a valid solution.
Common Mistakes and Limits
- “The trace is steady, so the chain has converged.” A chain can be steady because it is trapped. Steadiness within one region does not establish exploration of other regions.
- “All chains agree, so trapping is ruled out.” If all chains started in the same region, their agreement may be shared failure to leave it. Contrasting initial states make this comparison more informative.
- “One region was never visited, so it has probability zero.” A finite run cannot establish this. Rare transitions can leave a region unseen even when it has positive long-run probability.
- “A large number of iterations guarantees a representative sample.” Iteration count alone does not measure movement between regions. The relevant question is whether the run has had adequate opportunities to move and has actually done so.
- “Two observed clusters must be modes of a joint target.” That interpretation requires a coherent target distribution. For FCS with incompatible conditionals, report the observed separated behavior without asserting a joint distribution that has not been established.
These diagnostics are necessarily incomplete. Observed transitions offer evidence about the regions a chain can reach during its run; they do not prove that no other regions exist. Likewise, several chains that appear to mix cannot guarantee that a very low-probability region was found. The appropriate conclusion is calibrated: describe the evidence, identify the unresolved uncertainty, and avoid treating finite-run stability as proof of global exploration.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- A symmetric two-state chain switches states with probability 0.01 per transition. What is the probability that a chain starting in \(A\) stays there for 100 transitions?
- Why can several chains with similar initial completions give weak evidence against trapping?
- In an FCS run, two chains started in distinct plausible patterns and remain separated. What can and cannot be concluded?
- How does trapping between regions differ from slow movement within one region?
Solutions. (1) The probability is \((1-0.01)^{100}=0.99^{100}\approx0.3660\), or about 36.6%. (2) The chains may all begin in the same region and share the same difficulty leaving it; agreement then reflects their common starting region as well as any features of the update process. (3) The evidence indicates that these finite runs have not demonstrated movement between the patterns, so conclusions based on either chain alone may be region-specific. It does not establish that either pattern has zero probability or determine their long-run weights. (4) Slow movement within one region concerns gradual dependence among nearby states; trapping concerns rare movement between separated regions. Both can occur together.