Two Different Uses of Repetition
FCS repeatedly updates missing values using conditional models. Because the updates involve random draws, repeating the algorithm can produce different completed data sets from the same incomplete data. That repetition serves two distinct purposes: running independent chains to inspect algorithm behavior, and retaining multiple completed data sets to carry missing-data uncertainty into an analysis. The two counts need not be the same.
A chain is one run of the iterative FCS procedure, with its own initial imputed values and random-number stream. At each iteration, a completed data set is available: the observed values remain fixed, while imputed values can change. A retained completed data set is one of the outputs selected for analysis after the run has progressed sufficiently for the intended use. Multiple completed data sets are not automatically multiple independent chains.
For a meaningful comparison, chains should use the same observed data, conditional models, conditioning sets, update rules, variable order, and other substantive settings. They should have separate random-number streams. Initial imputations may be deliberately varied so that chains begin in meaningfully different states. This design asks whether runs starting differently show similar behavior as they proceed; it does not create independent samples of the study participants.
Using Chains to Inspect Behavior
A practical diagnostic is to track one or more summaries \(Z_{ct}\) across iterations for each chain. Examples include the mean or standard deviation of an imputed continuous variable, the proportion assigned a category, or a regression coefficient from a chosen analysis. Plotting the summaries against iteration makes persistent chain separation, substantial drift, or unusual instability easier to notice. Compare like with like: summaries should be defined in the same way in every chain.
Chains that begin apart and later show similar ranges and patterns provide some reassurance that the observed behavior is not dominated by the chosen starting imputations. Persistent differences are a warning to investigate. They could reflect slow movement, an inadequate run, sensitivity to initialization, or problems with the conditional models or data. Similar-looking traces are not proof that the algorithm has reached a desired long-run regime or that the imputations are scientifically appropriate. Diagnostics reveal problems more readily than they certify their absence.
Worked Example: Comparing Three Diagnostic Chains
Suppose a hypothetical data set has missing values in a continuous variable, and three FCS chains start from different initial imputations. At five successive iterations, the mean of the imputed values (in units of the variable) is as follows.
| Iteration | Chain 1 | Chain 2 | Chain 3 |
|---|---|---|---|
| 1 | 4.1 | 7.2 | 2.9 |
| 2 | 4.8 | 5.8 | 4.2 |
| 3 | 5.0 | 5.2 | 4.8 |
| 4 | 5.1 | 5.0 | 5.0 |
| 5 | 4.9 | 5.1 | 4.9 |
The starting means differ substantially: 4.1, 7.2, and 2.9. By iteration 5 they are 4.9, 5.1, and 4.9. For the last three iterations, the chain-specific averages are:
These summaries are close, and the visibly large initial differences have diminished. That is a useful diagnostic observation, not a formal demonstration of convergence: five iterations are very few, and the means alone may conceal other behavior. Other summaries and longer traces could show a problem that this summary misses.
Retaining Multiple Completed Data Sets
The inferential role of multiple imputation is different. Each retained completed data set is analyzed using the same analysis specification and estimand. The resulting estimates and their uncertainty are then combined using an appropriate pooling method; the earlier material on Rubin's Rules and Pooling covers that step. The purpose of \(M\) is to represent uncertainty due to missing values in the analysis, not to replicate a convergence diagnostic.
One chain can supply several retained completed data sets, and several chains can collectively supply the retained data sets. Thus \(C=3\) does not imply \(M=3\), and \(M=20\) does not imply that 20 independent chains were run. If consecutive iterations from one chain are retained, their imputed values may be serially dependent. Saving more data sets does not necessarily provide the same amount of information as saving the same number of less-dependent draws.
A useful selection plan specifies the chain count, the iterations to be treated as an initial diagnostic period rather than retained outputs, and which later completed data sets will be saved. The appropriate number of retained imputations depends on the analysis and the desired Monte Carlo precision; the number of chains should be chosen for the diagnostic task. There is no universal rule that these numbers must match.
Worked Example: Chain Count Is Not Imputation Count
A hypothetical analysis runs two FCS chains. After inspecting a sufficiently long run for problematic behavior, the analyst plans to retain three completed data sets from each chain, for a total of \(M=6\). Suppose the selected iterations are 51, 71, and 91 in both chains. The outputs are \(D_{1,51}\), \(D_{1,71}\), \(D_{1,91}\), \(D_{2,51}\), \(D_{2,71}\), and \(D_{2,91}\).
There are \(C=2\) independently initialized runs and \(M=6\) retained completed data sets. Each retained output is a full data set with the observed values fixed and missing entries imputed. The counts describe different things: two chains were run, while six completed data sets will enter the later analysis.
For illustration, if the analysis estimate from the six data sets is 12.1, 12.8, 11.9, 13.2, 12.4, and 12.6 units, their simple average is:
This arithmetic average is the combined point estimate in this illustration; a complete inferential analysis must also combine uncertainty appropriately. The fact that the six estimates came from two chains does not change \(M\) to two. Conversely, the average alone is not a diagnostic that the chains behaved well.
Why Dependence Matters
A chain can produce a completed data set at every iteration, but successive outputs may resemble one another because each update begins from the current imputed state. This is serial dependence. It matters when assessing how much Monte Carlo information a finite set of outputs contains. For a scalar summary with variance \(\sigma_Z^2\), the variance of the average of \(M\) retained summaries is:
If the \(M\) summaries are independent and have the same variance, every covariance term is zero and this reduces to \(\sigma_Z^2/M\). Positive serial covariances make the variance larger than that independent-draw expression. The formula explains why simply saving many adjacent iterations does not guarantee a proportionate gain in Monte Carlo information. It does not prescribe a universal spacing or number of imputations; those choices depend on the model, the analysis, and the observed behavior.
Worked Example: The Effect of Positive Serial Covariance
Consider four retained scalar summaries, each with variance \(\sigma_Z^2=1\). For a simple hypothetical calculation, suppose every pair has covariance 0.2. There are \(\binom{4}{2}=6\) distinct pairs. The variance of their average is:
If the four summaries were independent with the same variance, the variance would instead be \(1/4=0.250\). Under the stated positive-covariance assumption, the average therefore has greater Monte Carlo variance than the independent-draw calculation suggests. This example illustrates the effect of dependence; in practice, the covariances are not usually all equal, and they are not known just from the number of saved data sets.
A Practical Workflow and Its Limits
The following workflow keeps diagnostic replication and inferential retention conceptually separate.
Use the same observed data, conditional-model choices, update settings, and variable order across runs so that chain behavior can be compared.
Use distinct initial imputations and independent random-number streams. Keep the other modeling choices consistent across chains.
Compare traces across iterations and chains. Look for persistent separation, drift, abrupt changes, or implausible values; examine more than one summary where appropriate.
After the diagnostic review, specify which completed data sets will be analyzed and consider whether dependence between saved outputs could materially reduce Monte Carlo information.
Apply the same analysis to each completed data set and combine the resulting estimates and uncertainty using the appropriate pooling method.
The main alternative to multiple diagnostic chains is to inspect one long chain. A long run can reveal within-chain drift and variation, but it cannot show whether a different initialization would lead to persistently different behavior. Multiple chains make that comparison possible, at the cost of additional computation. Neither approach removes the need to inspect appropriate summaries, check the imputed values for plausibility, and consider whether the conditional models are suitable for the analysis.
Chain comparisons also have limits. A few similar traces can miss slow or subtle problems; different summaries can tell different stories; and apparent agreement is not evidence that the missing-data assumptions or substantive models are correct. The chain diagnostic addresses the behavior of the algorithm under the chosen specification, not every source of uncertainty in the study. In particular, chain-to-chain variation should not be substituted for the between-imputation component in a pooled analysis: those quantities play different roles.
Common Mistake
A common mistake is to say, “I saved 20 completed data sets, so I ran 20 independent chains and have verified convergence.” This confuses retained outputs with independent runs. Twenty data sets may all be adjacent outputs from one chain, and a large \(M\) does not establish that the chain behaved acceptably. The correction is to report \(C\) and \(M\) separately, describe how the chains were initialized and inspected, and state which outputs were retained. A second mistake is to treat agreement across chains as proof that the imputations are valid. Agreement is a useful diagnostic observation, but it cannot establish that the models or assumptions are correct.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- An analyst runs four independently initialized FCS chains and retains five completed data sets from each. What are \(C\) and \(M\)?
- Why might two chains with persistent differences in a trace summary require further investigation?
- Does agreement among several chain traces prove that the imputation models and assumptions are correct? Explain.
- For three retained summaries with variance 2, suppose each of the three pairs has covariance 0.3. Calculate the variance of their average and compare it with the independent case.
- Give one reason that saving many consecutive outputs from a single chain may yield less Monte Carlo information than treating those outputs as independent.
Solutions. (1) \(C=4\) chains and \(M=4\times5=20\) retained completed data sets. (2) Persistent differences may indicate sensitivity to initialization, slow movement, or another issue with the run or specification; the traces should be investigated rather than assumed to be interchangeable. (3) No. Agreement is a diagnostic observation about the runs, not proof that the model or assumptions are correct. (4) There are three pairs, so \(\operatorname{Var}(\bar{Z})=[3(2)+2(3)(0.3)]/3^2=(6+1.8)/9=0.867\) (to three significant figures). Under independence the variance would be \(2/3=0.667\); the positive covariances increase it. (5) Consecutive outputs may be positively serially dependent, so the covariance terms increase the variance of their average relative to the independent-draw calculation.