Monte Carlo Error in FCS Chain Summaries

Why Monte Carlo Error Matters

An FCS run produces a finite sequence of completed-data states. A statistic calculated at each iteration—such as the mean of an imputed variable or a regression coefficient—will usually fluctuate. Some fluctuation reflects the variation represented by the FCS procedure; some reflects the fact that we have simulated only finitely many iterations. Monte Carlo error quantifies the latter: how imprecisely the finite chain estimates its long-run summary because of simulation noise.

This is a numerical-accuracy question, not a question about whether the missing values are known. Even if the chain is sampling appropriately, its estimated average can change slightly when the random simulation is repeated. Conversely, a small Monte Carlo error does not establish that the conditional models are appropriate, that the chain has explored its target adequately, or that the resulting imputations represent missing-data uncertainty well.

The phrase “same target” is important. The interpretation assumes that the chain is sampling from a stable target distribution and that the summary has a finite variance. In practice, an MCSE calculation also relies on the retained sequence being representative of that target, and on the dependence in the chain being estimated adequately. These conditions cannot be guaranteed by the MCSE formula itself.

Estimating the Error of a Chain Average

Iterations of an FCS chain are generally dependent. If adjacent summaries are positively autocorrelated, a sequence of \(n\) iterations contains less information about its long-run average than \(n\) independent draws would. Let \(\gamma_k\) denote the lag-\(k\) autocovariance of a stationary sequence \(S_t\), with \(\gamma_0=\operatorname{Var}(S_t)\). The variance of the chain average is

This expression follows by expanding the variance of the sum. There are \(n\) variance terms, each equal to \(\gamma_0\), and for each lag \(k\), there are \(n-k\) pairs of observations separated by that lag. The covariance terms occur twice, once for each order of a pair. Thus

which is the stated formula after factoring out \(1/n\). For a sufficiently long stationary chain with summable autocovariances, the finite-sample weights approach 1 at each fixed lag. The resulting long-run variance is \(\gamma_0+2\sum_{k=1}^{\infty}\gamma_k\).

In practice, the autocovariances must be estimated from the finite chain. A common approach estimates the long-run variance directly, using a suitable truncation or smoothing rule for the autocovariance sequence. Another uses the effective sample size from FCS iterations. In the usual mean-summary setting, if \(s_S\) is the sample standard deviation of the \(S_t\) values and \(ESS\) is the effective sample size, then the corresponding MCSE estimate is \(s_S/\sqrt{ESS}\). This uses the effective sample size to account for autocorrelation; it is not generally correct to substitute the raw iteration count for \(ESS\).

These formulas are for the error of an average. They do not automatically give the MCSE of every possible chain summary. For example, an estimated quantile is not a chain average of the original summary values. Its MCSE requires a method appropriate to quantiles, such as a quantile-specific variance estimate or a carefully designed batch method. The target statistic and the MCSE method must match.

Several Independent Chains

When independent chains are available and each targets the same long-run distribution, their chain averages can be combined. If \(K\) chains have estimates \(\bar{S}_j\) and estimated MCSEs \(e_j\), the equal-weight average is \(\bar{S}_{\text{all}}=K^{-1}\sum_{j=1}^K\bar{S}_j\). If the chain-average errors are independent, its estimated MCSE is \(\sqrt{\sum_{j=1}^K e_j^2}/K\). This calculation describes simulation precision only. It does not establish that the chains reached the same target or explored it adequately; noticeable disagreement among chain averages deserves investigation rather than being hidden by pooling.

Autocorrelation and the Size of Simulation Error

An equivalent way to understand the adjustment for dependence is through the integrated autocorrelation time, \(\tau=1+2\sum_{k=1}^{\infty}\rho_k\), where \(\rho_k\) is the lag-\(k\) autocorrelation of \(S_t\), when the sum is well-defined. For a long chain, \(ESS\) is approximately \(n/\tau\). Positive autocorrelation makes \(\tau\) larger than 1 and the effective sample size smaller than \(n\). The uncertainty in a chain average is therefore larger than the independent-draw calculation would suggest.

Monte Carlo Error Is Not Missing-Data Uncertainty

In multiple imputation, variation among completed-data estimates can reflect uncertainty about the unobserved values. Monte Carlo error is different: it describes how much finite simulation perturbs an estimate of a chain’s long-run summary. A useful way to keep the concepts separate is to ask two questions:

• Missing-data uncertainty: How much would the inferential estimate vary because different plausible values for the missing data are represented?
• Monte Carlo error: How much would the estimated summary vary because this particular FCS simulation used a finite, dependent sequence?

A chain can have substantial variation in imputed values and still estimate its long-run average precisely if it has enough effective information. A chain can also have little apparent variation while estimating the average imprecisely—for example, if summaries move slowly and are highly autocorrelated. These quantities answer different questions and should not be substituted for one another.

Diagnostics, Alternatives, and Limitations

An MCSE is only as trustworthy as the chain and the dependence estimate from which it is calculated. Check that the summary being monitored is the one whose average you intend to report, that it has no obvious unresolved drift, and that the autocorrelation estimate is based on enough information to be stable. Comparing independent chains can reveal discrepancies that a single-chain MCSE does not show. A small computed MCSE cannot repair a chain that has not adequately represented its target.

Different estimators of Monte Carlo error have different practical strengths. The \(s_S/\sqrt{ESS}\) calculation is compact and interpretable when an appropriate effective sample size is available for a chain average. Direct long-run-variance estimators use the observed autocovariance structure but can be sensitive to how that structure is truncated or smoothed. Batch-means methods divide a chain into batches and estimate variance from the batch summaries; they can be useful when the chain is long enough for the batches to capture dependence, but poor choices of batch size can make the estimate unstable. None of these methods removes the need to inspect the underlying chain summaries.

The MCSE is an estimate, not a guarantee of accuracy. In short chains, autocorrelation estimates can be noisy; slow movement or other unrepresentative exploration can make the estimated effective sample size misleading. The usual standard-error interpretation is also most direct for averages under conditions that support a central limit approximation. For nonlinear summaries, use a method designed for that summary rather than applying the mean formula mechanically.