Autocorrelation Between FCS Iterations

Why Dependence Between Iterations Matters

An FCS algorithm produces a sequence of completed-data states, not a collection of independent draws. After one sweep through the variables, the next sweep begins from the current state. If the conditional updates retain information from one sweep to the next, summaries of successive states can be similar. A chain may then move through its possible values slowly: recording many iterations does not necessarily mean the algorithm has explored many distinct regions of its distribution.

Autocorrelation quantifies this dependence for a chosen numerical summary. It is a property of that summary’s sequence across iterations, rather than a single property that describes every feature of the full imputed data. For example, the mean of an imputed continuous variable may have low autocorrelation even while a category proportion has high autocorrelation.

The summary \(Q_t\) must be defined consistently at every iteration. It could be the mean of an imputed variable, the proportion assigned to a category, or another scalar quantity computed from the current completed-data state. An FCS iteration ordinarily means one full sweep through the variables being imputed. If summaries are instead recorded after each individual conditional update, the time index has a different meaning, so the lag must be interpreted accordingly.

The population autocorrelation definition assumes a stationary sequence with finite, nonzero variance: its mean and variance do not change with \(t\), and the covariance between two terms depends on their separation \(k\), not on their absolute positions. These conditions make a single \(\rho_k\) meaningful. They are not guaranteed merely because a sequence was produced by an algorithm. If the chain is still changing systematically, an autocorrelation calculated over the whole sequence can mix dependence with that change.

Estimating Autocorrelation From a Sequence

For an observed finite sequence \(Q_1,\ldots,Q_T\), first calculate its sample mean \(\bar{Q}=T^{-1}\sum_{t=1}^T Q_t\). A commonly used sample autocorrelation at lag \(k\), for \(1\leq k<T\), is

The numerator pairs values \(k\) iterations apart, while the denominator measures the total squared variation around the sequence mean. This convention uses the same denominator for all lags. Other finite-sample estimators exist; when comparing reported estimates, check which convention was used. For a short sequence, the estimate can be quite variable, even if the underlying chain is stationary.

Reading the Sign and Persistence

A positive \(\rho_k\) means that values \(k\) iterations apart tend to deviate from their mean in the same direction. A negative value means that they tend to deviate in opposite directions. An autocorrelation near zero indicates little linear association at that particular lag; it does not rule out all dependence or dependence at other lags.

Persistence matters when positive dependence extends across several lags. If a summary is high at one iteration and remains likely to be high in nearby iterations, repeated records contain less distinct movement through the summary’s range than an equally long sequence of independent values would. This is why a large iteration count by itself does not guarantee thorough exploration. Autocorrelation describes the dependence relevant to that summary; it does not directly establish whether the chain has reached a stationary distribution.

A useful justification for why dependence can persist across multiple lags comes from a simple stationary autoregressive model. This model is an illustration, not an assumption that every FCS summary follows an autoregression. Suppose \(Q_t-\mu=\phi(Q_{t-1}-\mu)+\varepsilon_t\), where the innovations \(\varepsilon_t\) have mean zero, variance \(\sigma_\varepsilon^2\), are independent across time, and are independent of past values. If \(|\phi|<1\), the process has stationary variance. For \(k\geq1\), the innovation at time \(t+k\) is uncorrelated with \(Q_t\), so \(\operatorname{Cov}(Q_t,Q_{t+k})=\phi\operatorname{Cov}(Q_t,Q_{t+k-1})\). Repeated application gives \(\gamma_k=\phi^k\gamma_0\), and hence \(\rho_k=\phi^k\). The condition \(|\phi|<1\) also ensures that this dependence decays as the lag grows.

Assumptions, Diagnostics, and Limits

Interpret estimated autocorrelation with the conditions behind its definition in mind. The sequence should represent the same summary, recorded at the same point in each sweep. A stationary interpretation requires a stable mean and variance over the analyzed segment. A summary with no variation has an undefined autocorrelation because the denominator is zero. With a short or irregular sequence, estimates can be noisy, and sample values at individual lags should not be treated as exact population correlations.

Several checks help prevent misleading conclusions:

• Check that the summary is meaningful. Choose a quantity tied to the feature of the imputed data or analysis that matters. Autocorrelation in one summary does not automatically describe a different variable or statistic.
• Check the time index. Be explicit about whether a lag counts full FCS sweeps or individual conditional updates. A lag of one has no interpretation without this definition.
• Consider whether the sequence is stable. A clear shift in level or spread can make an overall autocorrelation hard to interpret as stationary dependence. Autocorrelation is not a test that the chain has converged.
• Look beyond lag one. A small lag-one estimate does not ensure small dependence at every higher lag, and a negative lag-one estimate can accompany alternating patterns. Dependence is a sequence across lags, not just one number.
• Remember sampling variability. Autocorrelation estimates from finite runs can fluctuate. A single estimate near zero is not proof of independence, and a large estimate should be interpreted alongside the sequence and the other summaries being examined.

Autocorrelation complements, rather than replaces, other ways of examining FCS output. It gives a numerical description of linear dependence for a selected summary at specified lags. Assessing whether the marginal distribution of that summary looks stable addresses a different question; neither assessment alone establishes compatibility of the conditional models. A sequence can have low autocorrelation while still failing to represent the intended distribution, and a high autocorrelation does not by itself show that the conditional models are incompatible.

A common mistake is to say, “There are hundreds of iterations, so the chain must have explored well.” That reasoning counts recorded states but ignores their dependence. If successive values are strongly persistent, many records can repeatedly reflect nearly the same part of the summary’s range. The correction is to interpret the sequence’s dependence together with whether its behavior is stable and with the substantive summaries of interest. Autocorrelation identifies a limitation on how much new movement is seen from one iteration to the next; it is not, by itself, a verdict about the entire FCS analysis.

Autocorrelation also has limits as a diagnostic. It concerns linear association in a chosen scalar sequence, so nonlinear dependence may go undetected. Different summaries can have very different dependence patterns. And even a convincing estimate of low autocorrelation over observed lags cannot prove that unobserved regions have been explored. The conclusion should therefore remain specific: state the summary, the lag, the estimated dependence, and the conditions under which its interpretation is reasonable.