Slow Mixing Under Strong Conditional Dependence

Why Strong Conditional Dependence Matters

In fully conditional specification (FCS), variables with missing values are updated in sequence from models conditional on the current values of the other variables. When two variables are strongly conditionally dependent, changing one can prompt a similar change in the other. The next update may then pull the first variable back toward where it started. A chain can be valid and stable yet make only small moves from one sweep to the next.

This tutorial isolates that mechanism in a two-variable Gaussian example. Its exact calculations explain a common pattern in chain summaries: a trace that wanders slowly and a high lag-one autocorrelation. The example is deliberately simpler than a typical MICE analysis, where there may be many incomplete variables, estimated model parameters, and conditionals that do not correspond to one joint distribution. The result is an illustration of slow mixing, not a universal formula for MICE chains.

Definition: A chain mixes slowly when successive states remain strongly dependent over many iterations, so the chain explores its stationary distribution gradually. A chain’s stationary distribution is a distribution that is preserved by its update rule. Slow mixing reduces the amount of effectively independent information in a fixed number of iterations; it does not, by itself, mean that the chain is invalid or has failed to converge.

A Two-Variable Gaussian FCS Chain

Suppose the two quantities being updated, \(X\) and \(Y\), have a compatible bivariate standard normal distribution with correlation \(\rho\), where \(-1<\rho<1\). “Standard” means that both marginal means are 0 and both marginal variances are 1. Assume the model is correctly specified, the conditional draws are exact, and \(\rho\) is fixed and known. For this illustration, there are no additional predictors or changing model parameters.

The conditional distributions implied by this joint model are

X\mid Y=y \sim N(\rho y,\,1-\rho^2), \qquad Y\mid X=x \sim N(\rho x,\,1-\rho^2).

A full sweep first draws \(X_t\) conditional on the previous value \(Y_{t-1}\), then draws \(Y_t\) conditional on the newly updated \(X_t\). Here \(t\) indexes completed sweeps. Write the two draws as

X_t=\rho Y_{t-1}+\epsilon_{X,t}, \qquad Y_t=\rho X_t+\epsilon_{Y,t},

where each conditional-draw error has mean 0 and variance \(1-\rho^2\), and the errors used in different draws are independent. Because the second conditional update uses the new \(X_t\), rather than \(X_{t-1}\), the order of updates matters for the transition calculations.

Worked Example: Following Two Sweeps

Let \(\rho=0.90\), and suppose the chain starts with \(Y_0=1.00\). The conditional standard deviation for either update is

\sqrt{1-\rho^2}=\sqrt{1-0.90^2}=\sqrt{0.19}\approx 0.4359.

For the first sweep, suppose the standardized random draw for \(X_1\) is \(0.50\), and the draw for \(Y_1\) is \(-0.40\). These are hypothetical random draws, not observed study data. Substituting them into the conditional updates gives

X_1=0.90(1.00)+0.50(0.4359)\approx 1.118, \qquad Y_1=0.90(1.118)-0.40(0.4359)\approx 0.8318.

For a second sweep, suppose the standardized draws are \(0.20\) for \(X_2\) and \(0.10\) for \(Y_2\). Then

X_2=0.90(0.8318)+0.20(0.4359)\approx 0.8358, \qquad Y_2=0.90(0.8358)+0.10(0.4359)\approx 0.7958.

The values change at both updates, so this is not a chain stuck at a single state. But they remain near their previous values. The example is only one realization: different random draws could produce larger moves. The transition calculation below shows why small movement is typical when \(|\rho|\) is large.

Deriving the Sweep-to-Sweep Dependence

Substitute the update for \(Y_{t-1}\) into the update for \(X_t\). Since \(Y_{t-1}\) was generated from \(X_{t-1}\), this gives

X_t=\rho\bigl(\rho X_{t-1}+\epsilon_{Y,t-1}\bigr)+\epsilon_{X,t} =\rho^2 X_{t-1}+\rho\epsilon_{Y,t-1}+\epsilon_{X,t}.

Thus the sequence of completed-sweep \(X\) values has the form of a first-order autoregression: its previous value is multiplied by \(\rho^2\), with a new random error added. The combined error has variance

\rho^2(1-\rho^2)+(1-\rho^2)=1-\rho^4.

It is independent of \(X_{t-1}\), under the independent conditional-draw assumptions above. This variance also preserves the standard normal marginal variance: if \(X_{t-1}\) has variance 1, then \(X_t\) has variance \(\rho^4+(1-\rho^4)=1\). The same calculation applies to the completed-sweep \(Y\) sequence. Under stationarity, the lag-\(k\) autocorrelation for either same-variable sequence is therefore \(\rho^{2k}\).

Key result: In this compatible, fixed-parameter Gaussian example, the lag-one autocorrelation between completed-sweep values of the same variable is \(\rho^2\). Strong conditional dependence, meaning \(|\rho|\) near 1, makes this autocorrelation close to 1 and causes slow mixing. The square means the same-variable chain has positive lag-one autocorrelation even when \(\rho\) is negative.

This result concerns autocorrelation in the sampled chain, not a claim that a high correlation in every fitted MICE data set will produce exactly this value. Real FCS updates can have different conditional models, multiple variables, parameter draws, and observed values that alter the dependence structure. The derivation identifies a mechanism to look for, rather than replacing chain diagnostics.

Worked Example: Effective Sample Size at \(\rho=0.95\)

For \(\rho=0.95\), the approximate lag-one autocorrelation in the Gaussian example is

\phi=\rho^2=0.95^2=0.9025.

For a stationary first-order autoregressive sequence with lag-\(k\) autocorrelation \(\phi^k\), the integrated autocorrelation time is the sum of the autocorrelation contributions across lags:

\tau=1+2\sum_{k=1}^{\infty}\phi^k =1+\frac{2\phi}{1-\phi} =\frac{1+\phi}{1-\phi}.

The infinite geometric series is valid because \(|\phi|<1\). Substituting the value above,

\tau=\frac{1+0.9025}{1-0.9025} =\frac{1.9025}{0.0975} \approx 19.51.

For a long run of \(N=1{,}000\) completed sweeps, the approximate effective sample size for estimating a chain mean is

ESS\approx\frac{N}{\tau} =\frac{1{,}000}{19.51} \approx 51.25.

So, for this particular mean-estimation task and under the model assumptions, 1,000 correlated sweep values carry roughly the same precision as about 51 independent draws. This is an approximation, not a count of independent observations actually present in the chain. It also does not imply that a run of 1,000 sweeps is sufficient for every analysis.

What Slow Mixing Looks Like in Chain Summaries

A trace plot of one variable may show long stretches at relatively high or low values, with gradual movement between them. The lag-one autocorrelation is high, and autocorrelation may remain substantial over many lags. These features agree with the Gaussian calculation: if \(\rho^2\) is near 1, then \((\rho^2)^k\) decays slowly. A chain summary that reports only the total number of iterations can consequently overstate how much information the run contains.

For example, under the same idealized model, \(\rho=0.50\) gives lag-one autocorrelation \(0.50^2=0.25\), while \(\rho=0.95\) gives \(0.9025\). The first chain’s correlations decay much faster. This comparison concerns the same-variable chain across completed sweeps; it does not say that every variable in a multivariable FCS system will share one autocorrelation value.

Worked Example: Comparing Two Dependence Levels

Suppose two idealized chains each have \(N=2{,}000\) completed sweeps. For \(\rho=0.50\), the sweep-to-sweep autocorrelation is \(\phi=0.25\), so

\tau=\frac{1+0.25}{1-0.25} =\frac{1.25}{0.75} \approx 1.667, \qquad ESS\approx\frac{2{,}000}{1.667}\approx 1{,}200.

For \(\rho=0.95\), the preceding calculation gives \(\tau\approx19.51\), and therefore

ESS\approx\frac{2{,}000}{19.51}\approx 102.5.

Under these assumptions, the weaker-dependence chain has far more effective information about its mean for the same number of sweeps. Both ESS values are approximations based on the specified autoregressive model; neither is a guarantee that the chain has reached stationarity.

Diagnostics, Alternatives, and Limitations

A useful assessment combines trace plots, autocorrelation summaries, and effective sample sizes for relevant variables and summaries. Look for long runs of similar values and slowly decaying autocorrelation, but do not diagnose slow mixing from one trace feature alone. Compare independent chains as well: chains can each move slowly, or can appear stable while exploring different regions. A flat-looking trace is not proof of good exploration.

Several responses are available, but they address different issues. Running more sweeps can increase effective sample size if the chain is exploring the intended distribution, though it may require many additional iterations when autocorrelation is high. Thinning by retaining only every \(k\)th state may reduce the correlation among retained values, but discards intervening draws and does not create information or repair a chain that has failed to explore. Independent chains help assess sensitivity to starting states and reveal disagreement; they do not automatically make each chain mix faster. Randomized update ordering can be a robustness check, but it does not remove strong dependence by itself.

These strategies are not substitutes for checking that the updates and conditional models are appropriate. The exact Gaussian calculation assumes compatible conditionals, fixed known parameters, exact draws, and a simple two-variable system. In practical MICE, estimated parameters, other predictors, incomplete-data patterns, and the chosen update schedule can change chain behavior. In particular, an estimated lag-one autocorrelation need not equal the square of an observed or fitted correlation. Use that formula to understand the mechanism in the stated example, not as a diagnostic threshold for all FCS runs.

Common pitfall: “The trace plot is flat, so the chain must have mixed well.” A flat trace can be produced by a chain that moves only within a narrow range or remains highly autocorrelated. Check autocorrelation, effective sample size, and agreement across independent chains; a stable-looking mean alone does not establish adequate exploration.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

In the Gaussian example, what is the lag-one autocorrelation between completed-sweep values of \(X\) when \(\rho=-0.80\)?
For \(\rho=0.90\), calculate the integrated autocorrelation time and approximate ESS for \(N=500\) sweeps.
Why can a chain show changing imputed values and still mix slowly?
Does thinning a highly autocorrelated chain increase the information in its original run? Explain.

Solutions. (1) The same-variable lag-one autocorrelation is \(\rho^2=(-0.80)^2=0.64\), which is positive despite the negative cross-variable correlation. (2) Here \(\phi=0.90^2=0.81\), so \(\tau=(1+0.81)/(1-0.81)=1.81/0.19\approx9.526\), and \(ESS\approx500/9.526\approx52.49\). These are model-based approximations. (3) Each conditional draw can change a value, but strong dependence can keep successive states close and correlated, so the chain explores its distribution slowly. (4) No. Thinning retains fewer states and may lower the correlation among those retained, but it does not add information to the original run or correct inadequate exploration.