Diagnosing Drift Across FCS Iterations

Why Examine Drift Across Iterations?

An FCS run generates a sequence of completed data sets, not a collection of independent draws. A trace plot shows how a chosen summary changes from iteration to iteration, but short stretches can look unusual simply because neighboring iterations are dependent and the chain fluctuates. Conversely, a gradual shift may be hard to see when the trace is noisy. Rolling summaries help by reducing the visual influence of individual iterations and making sustained directional movement easier to assess.

The diagnostic question is not whether the trace is perfectly flat. It is whether its behavior is consistent with a stable range of fluctuation, or instead shows a persistent shift in its location or other relevant feature. A trace that appears stable is not proof that the FCS procedure has converged, and a drift diagnostic does not assess whether the imputation models or missingness assumptions are appropriate.

The summary \(S_t\) should be chosen to reveal changes relevant to the imputation. Examples include the mean of the imputed values for a variable, the proportion of imputed binary values equal to one, or a coefficient from an analysis model fitted to the completed data. The same summary must be calculated in the same way at every iteration. A change in definition, scale, or included cases can create an apparent shift that is not chain drift.

For interpretation, assume the FCS update schedule, conditional models, and data being imputed are held fixed while the chain runs. If the chain has reached stable behavior, summaries should fluctuate around a stable distribution, although adjacent values may be correlated. This expectation does not guarantee that the chain has a stationary distribution, that it has reached one, or that a compatible joint distribution exists. Drift checks are empirical diagnostics, not proofs of convergence.

Using Rolling Summaries and Window Comparisons

A rolling mean smooths a trace: each new value averages the current iteration with the preceding \(w-1\) values. A rising or falling sequence of rolling means can make sustained movement more apparent than the raw trace. Non-overlapping windows offer a simpler comparison: divide a trace into consecutive segments of equal length and calculate the mean of \(S_t\) in each. Rolling windows give a more continuous display, while non-overlapping windows avoid the strong overlap between adjacent rolling means.

Window length matters. A very short window leaves much of the short-term noise in place; a very long one may hide a shift by averaging together values from before and after it. It is useful to examine more than one reasonable window length and compare rolling summaries with the underlying trace. The choice should reflect the number of iterations available and the dependence between iterations, rather than being chosen because it produces a reassuring-looking plot.

A numerical comparison between early and late windows can describe the size and direction of movement. It is not automatically a significance test. In particular, treating the iterations as independent observations will generally misrepresent the variability of a window mean when the chain is autocorrelated. The earlier material on autocorrelation between FCS iterations is relevant to this interpretation.

Why Serial Dependence Affects Window Comparisons

The variability of a window mean depends on both the variation in individual summaries and their serial dependence. A short derivation makes clear why the usual independent-observation calculation is not generally suitable. Suppose, for this justification, that the trace is stationary after burn-in, with mean \(\mu_S\) and autocovariance \(\gamma_h=\operatorname{Cov}(S_t,S_{t+h})\), where \(\gamma_0\) is the variance. For a window mean \(\bar{S}_w\), expansion of the variance of a sum gives $$ \operatorname{Var}(\bar{S}_w) =\operatorname{Var}\left(\frac{1}{w}\sum_{i=1}^{w}S_i\right) =\frac{1}{w^2}\left[w\gamma_0+2\sum_{h=1}^{w-1}(w-h)\gamma_h\right]. $$

There are \(w\) variance terms, each equal to \(\gamma_0\). For each lag \(h\), there are \(w-h\) pairs in each time order, giving the factor \(2(w-h)\gamma_h\). If all nonzero-lag autocovariances were zero, this would reduce to \(\gamma_0/w\). Positive autocorrelation usually makes a window mean more variable than that independent-observation expression suggests.

For two window means, the variance of their difference is $$ \operatorname{Var}(\bar{S}_B-\bar{S}_A) =\operatorname{Var}(\bar{S}_B)+\operatorname{Var}(\bar{S}_A) -2\operatorname{Cov}(\bar{S}_A,\bar{S}_B). $$ If the windows are close together, their covariance may matter too. In practice, estimating this variability reliably can be difficult, particularly when the chain itself may be drifting. Therefore, a window difference is best treated as a descriptive diagnostic unless an appropriate inferential procedure and its assumptions have been specified.

Distinguishing Oscillation from Drift

A large trace range alone is not a drift diagnosis. A chain may fluctuate sharply around a stable level, or oscillate in a recurring pattern, without showing a sustained change in its location. Conversely, a modest but steady movement can matter even if every value lies within a seemingly plausible overall range. Inspect the order of values, their rolling or window means, and whether the apparent movement persists across the chain.

Combining Diagnostics and Recognizing Failure Modes

Rolling summaries are most informative when used alongside the trace and other diagnostics already introduced for FCS chains. Examine the same scalar summary across the full post-burn-in trace, compare early, middle, and late windows, and check whether independent chains show similar behavior. A trend present in one chain but not another can indicate that the chains have not reached comparable behavior; apparent agreement between chains is reassuring only in a limited sense and is not proof of convergence.

The following checks help prevent overinterpretation:

• Check the raw trace as well as its smoothing. A rolling mean can hide abrupt changes, repeated cycles, or changes in variability.
• Vary the window length. A conclusion that changes completely with a small change in window size is not robust evidence about drift.
• Keep summaries comparable. Use the same cells, scale, and calculation at each iteration. Otherwise, movement may reflect a changed summary rather than chain behavior.
• Consider dependence. Adjacent rolling means share most of their values, and FCS iterations are serially dependent; do not treat either as independent evidence.
• Inspect more than one relevant summary. Stability in one mean does not imply stability in other features of the imputed data or analysis results.

A common mistake is to say, “The late window mean is higher than the early window mean, so drift is statistically significant.” A difference in means describes observed movement; without a valid uncertainty calculation that accounts for serial dependence and the diagnostic context, it does not establish statistical significance. Another mistake is to treat a flat rolling mean as proof that the chain has explored the relevant distribution. A mean can remain stable while other features or modes remain unexplored, as emphasized in the earlier material on flat trace plots that conceal poor exploration.

Drift diagnostics matter because summaries calculated from an unstable chain may depend on where in the run they are obtained. But no plot or window comparison can certify that an FCS procedure is valid. These checks do not establish that the conditional models are appropriate, that the missingness mechanism has been handled correctly, or that the imputation and analysis models answer the intended question. They also do not supply a universal numerical threshold for declaring stability. Interpret them as evidence to assess together, not as a mechanical pass/fail test.