Reading an Unstable FCS Run
An unstable fully conditional specification (FCS) run can show several different warning signs: summaries of the imputed values keep moving, successive iterations are highly dependent, or results change when the variable update order changes. These patterns are related, but they do not call for the same response. Extending a chain may help with a transient that has not died away; it is unlikely to resolve a chain trapped in one region or a persistent difference induced by the update schedule.
Troubleshooting therefore starts by describing the problem, not by reflexively increasing the number of iterations. Compare traces across independent chains, examine autocorrelation, and make controlled changes to initialization or update order. Each change is a diagnostic perturbation: it helps locate the source of instability, but does not by itself prove that the imputation procedure is valid.
For a trace to be interpretable, define the summary before inspecting the result. For example, calculate the mean or proportion among the cells that were missing and imputed for a particular variable, using the same cells at every iteration. Alternatively, track a downstream estimate calculated from each completed data set. Do not silently change the cells or statistic being summarized between iterations.
The diagnostic comparisons assume that runs use the same observed data, missingness pattern, conditional models, and summary definition, except for the feature deliberately changed. Random-number variation means that two runs need not match exactly. The aim is to detect differences that are persistent or large relative to ordinary chain-to-chain variation, not to demand identical traces. Convergence diagnostics concern the behavior of the computational procedure; they do not establish that the imputation models are appropriate or that missingness assumptions are satisfied.
Diagnose Before Changing the Run
Inspect more than a single final summary. Plot traces by iteration and chain, compare early and late windows, and examine autocorrelation for the summaries of substantive interest. A trace can look flat over a short interval while still exploring poorly, and an average across chains can hide chains that occupy different regions. Earlier material on diagnosing drift across FCS iterations and flat trace plots explains these patterns in more detail; here the emphasis is on choosing the next troubleshooting action.
A useful first distinction is whether the difficulty appears to be an initial transient, slow exploration, or sensitivity to the update schedule. These can coexist. For instance, a chain may drift from its starting values and then mix slowly around a region that differs across schedules. Compare independent chains and, when appropriate, runs started from different plausible initial completions. The earlier material on sensitivity to FCS initialization provides the rationale for this comparison.
Worked Example: A Summary That Is Still Drifting
In a hypothetical FCS run, the mean of the imputed values for a continuous variable is recorded at four checkpoints. The average across chains is 12.1, 13.0, 13.8, and 14.4 units at iterations 50, 100, 150, and 200, respectively. These values are summaries for diagnosis, not observed measurements from a real study.
To quantify the direction and scale of movement, fit a simple descriptive linear trend to these four checkpoints. Let \(t\) be the iteration and \(q_t\) the recorded mean. The least-squares slope is
Here \(\bar{t}=125\) and \(\bar{q}=(12.1+13.0+13.8+14.4)/4=13.325\). The centered iteration values are \(-75,-25,25,75\), and the centered summaries are \(-1.225,-0.325,0.475,1.075\). Thus
The fitted trend is about 0.0154 units per iteration across these checkpoints. This calculation is a descriptive diagnostic, not a formal convergence test. The continued rise, together with traces from individual chains, supports checking whether starts were unusually influential and whether the later parts of the chains remain in motion. If different plausible initializations show similar late movement, extending the run is reasonable; if chains settle at distinctly different levels, simply extending each chain may not be enough.
Autocorrelation is a separate issue from drift. A stable-looking trace can still contain little effective information if adjacent iterations are very similar. For a scalar trace with lag-\(k\) autocorrelation \(\rho_k\), an approximate effective sample size for \(m\) retained iterations is
This expression follows from the variance of an average. If the trace has variance \(\sigma_q^2\), then the variance of its average is \(\sigma_q^2/m\) only when iterations are independent. With autocorrelation, the variance is approximately \(\sigma_q^2/m\) multiplied by \(1+2\sum_{k=1}^{\infty}\rho_k\). Equating that variance to \(\sigma_q^2/m_{\mathrm{eff}}\) gives the displayed approximation. It is an approximation for an adequately behaving stationary trace; it is not meaningful evidence of adequate exploration if the trace is still drifting or trapped.
Worked Example: High Autocorrelation Despite a Long Trace
Suppose 100 retained iterations of a summary have lag-one autocorrelation about 0.85, and for a rough calculation the autocorrelations decline geometrically, so \(\rho_k=0.85^k\). Then
Under this approximation, 100 recorded iterations carry information comparable to only about 8 independent draws for this summary. The approximation is not a general guarantee: the full autocorrelation pattern matters, and nonstationarity makes the calculation unreliable. The practical response is to investigate slow mixing and possible chain trapping, not to treat 100 as an adequate effective sample size merely because it is a large iteration count. Thinning alone discards draws and does not remove the underlying dependence.
Use Order Sensitivity as a Diagnostic
In FCS, a schedule specifies the sequence in which variables are updated. Changing that sequence can change finite-run behavior, especially when conditional updates feed back strongly on one another. Order sensitivity is assessed by holding the data and conditional models fixed and comparing runs under different defensible schedules. Randomized ordering can serve as a robustness check, but a changed result is a signal to investigate, not evidence that one order is automatically correct.
The earlier material on feedback loops in chained updates, randomized ordering as a robustness check, and designing an FCS update schedule develops these ideas. In troubleshooting, first ask whether the order comparison is fair: use comparable run lengths, multiple chains where feasible, the same summaries, and initializations that do not systematically favor one schedule. Then assess whether the differences exceed the variation seen among chains under a single schedule.
Worked Example: A Result Changes With the Update Order
In a hypothetical analysis, four chains under schedule A give a late-run average of a target summary of 18.4 units, with chain summaries between 18.0 and 18.8. Four comparable chains under schedule B give 16.9 units, with summaries between 16.6 and 17.2. The observed separation is 18.4 minus 16.9, or 1.5 units; the ranges do not overlap.
This contrast is large relative to the within-schedule spread, so it should not be dismissed as ordinary iteration-to-iteration noise. The next check is whether the separation persists when initializations are varied and both schedules are run longer. If the schedule-specific clusters remain separated, the run is order-sensitive in a practically important way. Consult the compatibility audit and inspect the feedback structure; also verify that no implementation detail besides update order changed.
A targeted schedule change can then be tested while preserving the conditional models: for example, compare a carefully chosen alternative ordering or a randomized-order robustness run, with replication across chains. If the results remain schedule-dependent, selecting whichever order gives a preferred answer is not a defensible remedy. Further model or algorithm investigation is needed.
Match the Intervention to the Evidence
There is no single repair for every unstable run. Choose the smallest change that addresses the observed pattern, and rerun diagnostics afterward. A useful distinction is between changes that improve observation of the process and changes that alter how the process is run. Increasing chain length gives more opportunity to see late behavior; changing initialization tests dependence on starting states; changing the schedule probes sensitivity to update order. None substitutes for checking whether the conditional models themselves are suitable.
| Observed pattern | Targeted next step | What the step can and cannot show |
|---|---|---|
| Early movement that appears to settle similarly across chains | Use a justified burn-in and retain enough later iterations for summaries | Can reduce the influence of an initial transient; cannot establish model validity |
| Continued drift in late iterations | Extend the chain and reassess traces and chain-to-chain behavior | Can reveal whether movement persists; more iterations do not guarantee resolution |
| Strong autocorrelation without clear drift | Investigate slow mixing, dependence, or trapping; use longer runs if exploration is progressing | May increase effective information if the chain continues to explore; thinning does not cure poor mixing |
| Dependence on plausible starting completions | Run multiple chains from dispersed, defensible initializations | Can reveal initialization sensitivity; identical-looking starts can conceal it |
| Material, persistent differences across update orders | Repeat a controlled schedule comparison and revisit the conditional-model and feedback structure | Can identify schedule sensitivity; does not justify choosing the schedule with the preferred result |
A longer chain is the main alternative to restarting or changing the schedule, but it is useful only when the additional iterations contribute exploration. Independent chains provide a different kind of evidence: they help show whether apparent stability is shared across starting points. A schedule comparison probes order dependence directly. Altering conditional models may ultimately be necessary, but that is a modeling decision rather than a generic convergence fix. The earlier material on conditional-model compatibility is relevant if persistent schedule sensitivity suggests that the chained conditionals do not define a coherent joint distribution.
Common Mistakes and Limits
- Extending every run automatically. More iterations can help with an unfinished transient, but a chain that remains trapped may simply produce a longer trapped trace. Check whether chains explore consistently before relying on added length.
- Calling a flat trace proof of convergence. A trace can be flat because it has not moved. Compare chains, inspect autocorrelation, and consider whether the summary captures the features that matter.
- Using a single final value to diagnose order effects. Final iterations are noisy and depend on random draws. Compare replicated late-run summaries and their within-schedule variation.
- Treating a diagnostic as a validity test. Stable traces do not establish that the conditional models are appropriate, that the missingness assumptions hold, or that inferential uncertainty is properly represented.
The evidence is necessarily empirical. FCS may not have a simple stationary joint target, especially when conditional models are incompatible; even a well-behaved-looking run cannot turn that limitation into a guarantee. Troubleshooting improves the basis for judging computational stability, while model specification and missing-data assumptions require separate assessment.
Check Your Understanding
Answer each question, then compare with the solutions below.
- A summary rises from 5.0 to 5.2 units over many late iterations, and independent chains show similar movement. What is the most direct next step, and what should be checked afterward?
- A trace has 80 retained iterations and an estimated effective sample size of about 10. Why is it misleading to describe the trace as 80 independent draws?
- Two update schedules produce different final values, but each has only one chain. What is missing from the comparison?
- Why is selecting the update order that produces the most scientifically appealing estimate not a sound troubleshooting rule?
Solutions. (1) Extend the run because late drift is still present, then examine the extended traces, autocorrelation, and agreement across chains; added length helps only if the chains continue to explore. (2) Autocorrelation means neighboring iterations are redundant, so the uncertainty of the average is larger than it would be for 80 independent draws; the effective size summarizes that loss of information. (3) Replication is missing: use comparable multiple chains and assess variation within each schedule, ideally controlling initialization and other run settings. (4) The order comparison is diagnostic, not a method for choosing a preferred answer. Persistent differences require investigation of schedule sensitivity and the conditional-model structure, rather than outcome-based selection.