Diagnosing Random-Effect Chain Mixing

Why Mixing Matters in SAEM

In SAEM, latent random effects are simulated from their conditional distribution given the observations and current population parameters. A Metropolis–Hastings chain can target that distribution without producing independent draws: successive states may remain strongly related. If the chain explores the conditional distribution slowly, the sufficient statistics supplied to the stochastic approximation can reflect only a narrow part of the random-effect space. The resulting updates may be unusually noisy, slow to settle, or distorted by a chain that has not reached its target distribution.

The key question is not simply whether proposals are accepted. It is whether the chain moves around the relevant conditional distribution well enough for the quantities used in the SAEM update to be represented. Acceptance rate, autocorrelation, and effective sample size describe different aspects of this question, so they should be considered together.

What the Diagnostics Measure

For a Metropolis–Hastings transition, a proposal \(b^\ast\) from the current state \(b\) is accepted with probability

where \(q(b^\ast\mid b)\) is the proposal density. If a proposal is rejected, the next chain state remains at \(b\). The observed acceptance rate is the number of accepted proposals divided by the number of proposals attempted. It indicates how often proposals move the chain, but it does not say whether those moves travel far enough to explore the target distribution.

For a scalar quantity \(G_m=g(b^{(m)})\), the lag-\(k\) autocorrelation describes how strongly \(G_m\) is linearly related to \(G_{m+k}\). Large positive autocorrelation over many lags means successive values tend to be similar. This is common when proposals are small or when the target distribution has difficult geometry, but the diagnostic should be assessed for the quantities that matter to the SAEM update, not only for a convenient random-effect coordinate.

The ESS is quantity-specific. A chain may mix adequately for a random-effect mean but poorly for its square, a covariance-related statistic, or a nonlinear function entering the complete-data sufficient statistics. Therefore, inspect traces and autocorrelations for relevant components of the sufficient-statistic vector as well as for the random effects themselves.

Why Autocorrelation Reduces Effective Information

The ESS formula follows from the variance of a chain average. Suppose \(G_1,\ldots,G_M\) come from a stationary chain, each with variance \(\sigma_G^2\), and lag-\(k\) correlation \(\rho_k\). The Monte Carlo average is \(\bar{G}=M^{-1}\sum_{m=1}^M G_m\). Expanding its variance gives

When \(M\) is sufficiently large and the autocorrelations are summable, the bracketed term is approximately \(\tau\). Thus the variance is approximately \(\sigma_G^2\tau/M\), the variance that \(M/\tau\) independent draws would have. This gives the approximation \(ESS\approx M/\tau\). The derivation assumes a chain sampling a fixed target distribution and concerns Monte Carlo precision; it does not make a poorly targeted or nonstationary chain valid.

Consequences for SAEM Sufficient-Statistic Updates

Let \(S(b)\) denote the complete-data sufficient-statistic vector evaluated at a simulated latent random effect. A basic stochastic-approximation update has the form

where \(s_k\) is the running statistic at iteration \(k\) and \(\gamma_k\) is the current gain. The simulated value \(S(b^{(k)})\) is meant to provide information about the conditional expectation of the statistic. If the chain has barely moved, several successive values can be nearly repetitions. The update then receives less new information than a sequence of independent draws of the same length would provide.

Serial dependence does not, by itself, imply that a correctly stationary chain targets the wrong conditional distribution. Under appropriate ergodicity and moment conditions, a Markov chain can estimate expectations consistently, although strong dependence reduces efficiency. A different problem occurs if the chain is initialized far from its target and has not adequately explored it: then the values supplied to the update can be systematically unrepresentative. In SAEM, the target may also change as \(\theta\) is updated, so diagnostics must be interpreted in the context of the current simulation and parameter iteration rather than as if all draws came from one fixed target.

Practical Diagnosis and Response

A useful assessment combines complementary checks. Examine trace plots for random-effect components and relevant functions such as squares, products, or model-specific sufficient statistics. A trace that stays in a narrow band for long stretches, jumps rarely, or shifts its overall level over time raises concern. Autocorrelation plots show how quickly dependence decays. ESS summarizes that dependence for a selected quantity, but should not be reported as if it were a universal property of the entire multivariate chain.

When feasible, compare multiple chains initialized in different plausible regions of the conditional random-effect distribution. Agreement in explored ranges and summaries is reassuring, whereas persistent separation suggests that the chains may not be exploring the same target region. This comparison is a diagnostic, not proof of convergence. Also check that the Metropolis–Hastings acceptance calculation uses the intended conditional target and the correct proposal-density ratio: a chain can appear stable while targeting the wrong distribution if the transition is implemented incorrectly.

If diagnostics indicate poor mixing, first identify which quantities are affected and whether the problem is widespread or limited to particular random effects or subjects. Then review proposal scale and geometry, consider a more suitable proposal, and run longer chains where justified. A proposal that is too narrow can yield frequent, tiny moves; one that is too broad can yield frequent rejections. Neither response should be automatic: revise the proposal and reassess the chain behavior. Increasing the number of iterations alone may be inefficient when the chain is moving extremely slowly.

The principal alternatives and complements are not interchangeable. Acceptance rate is inexpensive to calculate but only describes proposal outcomes. Autocorrelation identifies the persistence of dependence at different lags. ESS translates that dependence into an approximate precision scale, but depends on the chosen statistic and autocorrelation estimation. Longer chains can increase information if mixing is adequate, while improved proposals aim to increase exploration per iteration. Independent conditional draws, when available, avoid this particular serial-dependence problem, but may be impractical for nonlinear mixed-effects conditional distributions.

Common Pitfall and Limitations

Wrong reasoning: “The acceptance rate is high, so the random-effect chain mixes well and the SAEM sufficient statistics are reliable.” This confuses the frequency of accepted proposals with the distance and coverage of chain exploration. A chain can accept many tiny moves and remain highly autocorrelated. Correction: inspect traces, autocorrelation, and ESS for the statistics actually used in the update, and check for signs of transient behavior or implementation errors.

No single diagnostic proves that a chain has converged or that every part of a high-dimensional conditional distribution has been explored. Autocorrelation and ESS estimates are themselves uncertain for short chains, and scalar summaries can miss dependence in other directions. These measures assess simulation quality; they do not replace checks of the model, the target distribution, or the broader SAEM fit.