Why a Stopping Rule Needs More Than a Flat Trace
A practical stopping rule for fully conditional specification (FCS) should answer a specific question: is there enough evidence that additional iterations are unlikely to change the summaries that matter for this imputation task by a practically important amount? A trace that looks flat is useful evidence, but it is not enough on its own. Chains can appear stable while exploring only one region, and successive iterations can be so correlated that many iterations contain little new information.
The earlier material on trace plots, independent chains, effective sample size, Monte Carlo error, burn-in, and drift provides the diagnostic pieces. Here they are assembled into a decision rule. There is no universal number of FCS iterations that guarantees an adequate run. The defensible choice depends on the quantities being monitored, their scale, and the precision needed.
Before running the chains, identify the summaries most relevant to the intended imputations and analysis. Include summaries that could reveal problematic behavior: for example, means and variances for continuous variables, category proportions for categorical variables, and important analysis estimates or contrasts. A diagnostic summary is not automatically the final inferential result; it is a way to monitor the stochastic FCS process.
What a Practical Rule Should Require
A defensible decision uses multiple independent chains initialized in different plausible states and run with independent random-number streams. After a chosen burn-in, inspect each chain and compare chains on the same summaries. Agreement between chains is important because a single chain can look stable without showing that its explored region is representative of the relevant distribution.
Assess stability over more than one recent window. Compare each chain’s window summaries, the average across chains, and the range among chain summaries. Set tolerances before examining the final diagnostics where possible. Tolerances should be in meaningful units: a shift of 0.1 mg/L might matter for one summary, while a shift of 0.001 in a probability might be negligible for another. A tolerance should reflect practical consequences, not merely what the observed run happens to achieve.
Then check autocorrelation-aware Monte Carlo error. If \(Q_t\) is a chain summary at iteration \(t\), its autocorrelation means the number of iterations is not the number of independent observations. Under a stationary chain with summable autocorrelations, define the integrated autocorrelation time as \(\tau=1+2\sum_{k=1}^{\infty}\rho_k\), where \(\rho_k\) is the lag-\(k\) autocorrelation. For \(N\) retained iterations, the approximate effective sample size is \(N_{\mathrm{eff}}=N/\tau\). If the stationary standard deviation of \(Q_t\) is \(\sigma_Q\), then the Monte Carlo standard error of its chain average is approximately \(\sigma_Q/\sqrt{N_{\mathrm{eff}}}\).
Why Autocorrelation Changes the Precision
The relationship between autocorrelation and Monte Carlo error follows from the variance of an average. For a stationary chain with variance \(\sigma_Q^2\), the variance of the average of \(N\) consecutive values is
When \(N\) is large and the autocorrelations decay sufficiently quickly, the bracketed quantity is approximately \(\tau\). Thus \(\operatorname{Var}(\bar{Q})\approx \sigma_Q^2\tau/N=\sigma_Q^2/N_{\mathrm{eff}}\). This derivation explains why a long run can still have substantial Monte Carlo error if successive iterations are highly correlated. In practice, autocorrelation and Monte Carlo error are estimated from finite chains, so the estimates themselves can be uncertain, especially for slow-mixing or nonstationary chains.
For \(C\) independent chains with the same retained length \(N\), let \(\bar{Q}_c\) be the average in chain \(c\), and estimate its chain-average variance by \(v_c\), accounting for that chain’s autocorrelation. The average across chains has estimated Monte Carlo variance \(\sum_{c=1}^{C}v_c/C^2\). This calculation uses independence between chains; it does not remove bias from inadequate burn-in or make a poorly explored chain reliable.
Worked Example: Checking Stability Across Recent Windows
In a hypothetical FCS run, three independent chains monitor the mean of an imputed laboratory measurement, in mg/L. The analyst prespecifies that the largest difference between any two chain-window means should be no more than 0.10 mg/L, and that the change in the across-chain average between the latest windows should be no more than 0.05 mg/L.
| Retained-iteration window | Chain 1 mean (mg/L) | Chain 2 mean (mg/L) | Chain 3 mean (mg/L) |
|---|---|---|---|
| 401–600 | 12.03 | 12.07 | 12.02 |
| 601–800 | 12.08 | 12.05 | 12.09 |
| 801–1000 | 12.10 | 12.11 | 12.06 |
For the latest window, the across-chain average is \((12.10+12.11+12.06)/3=12.09\) mg/L. For the preceding window it is \((12.08+12.05+12.09)/3=12.073\) mg/L, so the change is \(12.09-12.073=0.017\) mg/L, below 0.05 mg/L. The largest difference among the latest chain means is \(12.11-12.06=0.05\) mg/L, below 0.10 mg/L. These particular stability checks pass. They do not, by themselves, justify stopping: the analyst must also examine other prespecified summaries, chain behavior, and Monte Carlo error.
Worked Example: Autocorrelation, Effective Sample Size, and Monte Carlo Error
Suppose a hypothetical retained chain has \(N=1{,}000\) iterations for a summary measured in mg/L. Assume, for illustration, that its autocorrelation follows an AR(1) pattern with lag-one autocorrelation \(\rho_1=0.60\), so \(\rho_k=0.60^k\). Suppose the estimated standard deviation of the summary across iterations is 0.080 mg/L.
For this AR(1) pattern, the sum of the autocorrelations is \(\sum_{k=1}^{\infty}\rho_k=\rho_1/(1-\rho_1)=0.60/0.40=1.5\). Therefore \(\tau=1+2(1.5)=4.0\), giving \(N_{\mathrm{eff}}=1{,}000/4.0=250\). The estimated Monte Carlo standard error is
If the prespecified absolute Monte Carlo-error tolerance is 0.010 mg/L, this diagnostic passes because \(0.00506<0.010\) mg/L. The 1,000 iterations have delivered precision more like 250 independent draws for this summary, not 1,000. Actual FCS autocorrelations need not follow an AR(1) pattern; this assumption is used only to make the calculation transparent.
A Combined Operational Rule
A practical rule can be written as a sequence of gates. Each gate is applied to every prespecified summary, not just the one with the most reassuring trace. The required number of chains, retained iterations, windows, and tolerances should be chosen for the application; none has a universal value.
Choose chain summaries that matter for the imputation task. Specify meaningful bounds for recent-window shifts, between-chain differences, and Monte Carlo error.
Use distinct initial states and independent random-number streams. Exclude the burn-in period selected from diagnostic evidence before calculating retained-run summaries.
Require chain summaries and their across-chain averages to stay within the prespecified practical tolerances in more than one recent window. Look for systematic drift, not just a small difference between two averages.
Compare chain summaries and traces. A chain trapped in a distinct region, or chains that have not explored the same behavior, is a reason to continue or investigate rather than declare success.
Estimate effective sample size and Monte Carlo error for each important summary. Require the error to be smaller than the prespecified absolute or relative tolerance.
Run additional iterations, then repeat the checks. Stop extending only when all gates pass and the diagnostics show no unresolved warning signs.
This is an operational decision, not a theorem that convergence has occurred. In particular, passing the rule supports the claim that the monitored summaries appear stable and have adequate estimated simulation precision under the explored behavior. It does not prove that the FCS procedure has reached or correctly represents a single target distribution.
Worked Example: Applying the Rule to Four Chains
Consider four independent hypothetical chains monitoring the proportion of imputed values in a binary category. Each chain has 800 retained iterations after burn-in. Recent-window checks, repeated over two consecutive windows, show no systematic drift, and all summaries meet the prespecified stability tolerances. The chain averages are 0.402, 0.397, 0.405, and 0.396; each chain has estimated standard deviation 0.080 and an estimated autocorrelation time of 3.0.
The average across chain means is \((0.402+0.397+0.405+0.396)/4=0.400\). The largest difference between chain means is \(0.405-0.396=0.009\). For each chain, the variance of its average is approximately
Because the four chains are independent, the Monte Carlo variance of their average is \(4(0.0000240)/4^2=0.00000600\), so its Monte Carlo standard error is \(\sqrt{0.00000600}=0.00245\). Each chain has estimated effective sample size \(800/3.0\approx267\), or about 1,067 across the four independent chains for this average.
If the prespecified between-chain difference tolerance is 0.020, and the Monte Carlo-error tolerance is 0.005, these checks pass: \(0.009<0.020\) and \(0.00245<0.005\). Together with stable recent windows and no visible evidence of trapping or drift, this would support stopping the current chains under the stated rule. The conclusion is limited to these diagnostics and summaries; other important summaries must pass too.
Alternatives, Failure Modes, and Limits
| Approach | What it contributes | Main limitation |
|---|---|---|
| Stop after a fixed iteration count | Simple, reproducible computational budget | A fixed count does not show that a particular run mixed or achieved adequate precision. |
| Stop when a trace looks flat | Quick visual screening for large trends or shifts | Visual stability can conceal high autocorrelation, chain disagreement, or failure to explore other regions. |
| Use multiple chains, stability checks, and Monte Carlo error | Combines evidence about drift, agreement, and simulation precision | Depends on the summaries monitored and on the chains having explored relevant behavior; it cannot prove correctness of the FCS models. |
Common warning signs include a summary that changes across recent windows, one chain consistently separated from the others, estimated Monte Carlo error above its tolerance, or very low effective sample size. Autocorrelation estimates may be unreliable when the chain is short or dependence decays slowly. A flat trace is especially weak evidence when the earlier material on flat traces that conceal poor exploration applies: apparently stable summaries can coexist with poor exploration of the chain’s possible states.
A rule can also fail by monitoring too few quantities. If only an overall mean is checked, important distributional features such as tail behavior or category proportions may still be unstable. Conversely, demanding very small Monte Carlo error for every conceivable diagnostic can make the computational burden excessive without improving the analysis in a meaningful way. Choose the monitored summaries and tolerances to reflect the scientific use of the imputations, and report them transparently.
Finally, the number of FCS iterations and the number of completed data sets for multiple imputation are different choices. Iterations describe how far a chain is run; the number of completed data sets describes how many imputations are retained for subsequent analysis. Passing a chain-stopping rule does not, by itself, determine how many completed data sets are needed.
Check Your Understanding
Use the definitions and decision logic above. Solutions follow the questions.
- A chain summary has 1,200 retained iterations and estimated autocorrelation time 6. What is its approximate effective sample size?
- Why is a stable across-chain average not enough if one chain remains separated from the others?
- A summary has estimated standard deviation 0.12 units and effective sample size 400. Calculate its approximate Monte Carlo standard error.
- State one reason why passing a practical stopping rule does not prove that the FCS procedure is valid.
- Explain the difference between the number of FCS iterations and the number of completed data sets.
Solutions. (1) \(N_{\mathrm{eff}}=1{,}200/6=200\). (2) The average can conceal a chain that has not explored the same behavior as the others; chain separation is unresolved evidence against stopping. (3) \(SE_{\mathrm{MC}}\approx0.12/\sqrt{400}=0.12/20=0.006\) units. (4) The rule only assesses selected summaries and estimated simulation behavior; it cannot establish that conditional models are appropriate or that the algorithm explores all relevant regions. (5) Iterations are updates within the FCS chain; the number of completed data sets is how many imputed versions are retained for multiple-imputation analysis.