Why the Number of Iterations Can Mislead
An FCS run can retain many completed-data states, yet consecutive iterations may be similar because each chained update depends on the current state. If a scalar summary changes only gradually, 1,000 retained iterations do not generally provide as much information about its long-run mean as 1,000 independent draws would. Effective sample size (ESS) expresses this loss of precision on an independent-draw scale.
ESS is a property of a particular scalar sequence, not a single universal property of an entire FCS run. The summary might be a completed-data mean, a regression coefficient calculated at each sweep, or an indicator for whether a chosen condition holds. Different summaries from the same run can have different autocorrelation patterns and therefore different ESS values.
From Autocorrelation to Effective Sample Size
For a stationary sequence \(Y_t\) with finite variance, let \(\gamma_k\) be its autocovariance at lag \(k\), and let \(\rho_k=\gamma_k/\gamma_0\) be its autocorrelation. Thus, \(\rho_0=1\), while \(\rho_k\) describes the linear dependence between values \(k\) sweeps apart. If the sequence is positively autocorrelated over many lags, nearby iterations tend to carry overlapping information.
The ESS approximation relies on a stable, stationary sequence and conditions strong enough for the variance of its average to have a long-run limit. In particular, the autocorrelations must be summable for the usual integrated-autocorrelation-time expression to be finite. In practice, the retained segment should not show substantial transient drift, and estimated autocorrelations at long lags should not be treated as precise when few data points inform them.
This finite-\(N\) expression shows why the nominal iteration count alone is insufficient: the variance depends on the correlations as well as on \(N\). When \(N\) is large and the autocorrelations decay sufficiently quickly, the bracketed quantity is approximated by the integrated autocorrelation time, \(\tau\). The corresponding ESS is approximately \(N/\tau\).
The factor of 2 appears because dependence at lag \(k\) contributes to covariance on both sides of an observation in the variance of the average. When the cumulative autocorrelation is positive, \(\tau>1\), so \(N_{\mathrm{eff}}<N\). ESS is an equivalent-precision measure, not a literal count of distinct states: iterations can be different while still being strongly dependent.
Worked Calculations
Worked Example: An AR(1)-Like FCS Summary
Suppose a hypothetical FCS run retains \(N=1{,}000\) values of a completed-data summary. Assume, for illustration, that this scalar sequence is stationary and has the autocorrelation pattern of an AR(1) process with lag-one autocorrelation \(\rho_1=0.60\). For an AR(1) process, \(\rho_k=(0.60)^k\), so the correlation assumptions are explicit and decay geometrically.
The integrated autocorrelation time is
Therefore,
Although the run retained 1,000 iterations, the average of this summary has approximately the same variance as an average from 250 independent draws with the same marginal variance. This does not say that exactly 750 iterations were duplicates; it quantifies the impact of dependence on the average.
Worked Example: Using a Truncated Lag-Correlation Summary
Consider a hypothetical stable chain with \(N=800\) retained values of a scalar summary. Suppose a reliable autocorrelation calculation reports \(\rho_1=0.45\), \(\rho_2=0.24\), \(\rho_3=0.12\), and \(\rho_4=0.05\), and the calculation's prespecified truncation rule stops after lag 4 because subsequent paired lag estimates no longer support a positive cumulative contribution. The reported estimate uses the first four lags; the truncation rule is important because very distant sample autocorrelations can be dominated by noise.
Using that truncated sum gives
The approximate ESS is
For this summary, the 800 retained values therefore have roughly the same average-estimation precision as 294 independent values. This is an approximation: it depends on the chain being sufficiently stationary and on the truncated autocorrelation estimate being adequate.
Worked Example: Two Summaries, Two Effective Sample Sizes
A hypothetical FCS run retains \(N=1{,}000\) sweeps. The analyst tracks both a continuous completed-data coefficient and an indicator equal to 1 when a specified completed-data condition holds and 0 otherwise. Suppose the coefficient sequence is adequately described by an AR(1) autocorrelation pattern with \(\rho_1=0.70\), while the indicator sequence has an AR(1) pattern with \(\rho_1=0.20\). These are assumptions about the two scalar sequences, not claims that the full FCS state follows an AR(1) model.
For the coefficient, the geometric autocorrelation sum gives
For the indicator, the corresponding calculation is
The same run thus has markedly different ESS values for the two summaries. Reporting only one number for the entire run would hide this difference. In particular, an ESS for a coefficient does not automatically describe the precision of a proportion or another derived quantity.
Assumptions, Diagnostics, and Interpretation
A useful ESS estimate requires more than a formula. First, the segment being summarized must behave approximately as a stationary sequence: its location and dependence structure should not be changing substantially over the retained sweeps. Second, the autocorrelation estimate must capture the important dependence at short and moderate lags without accumulating unreliable noise at very long lags. Third, the scalar summary must have a finite variance and a sufficiently well-behaved dependence structure for the long-run variance approximation to apply.
Trace plots and comparisons of independent FCS chains can help reveal conspicuous changes in level or disagreement between chains. They do not, by themselves, establish stationarity or guarantee a reliable ESS. Inspect autocorrelation plots for the specific summary, note whether correlations decline with lag, and check whether the estimated ESS is sensitive to reasonable choices in the truncation or estimation method. An estimate that changes greatly under small changes in the lag window is weak evidence about effective information.
The ESS calculation concerns the sampling behavior of the observed chain summary. It does not establish that the conditional models are compatible, that an appropriate joint distribution exists, or that the chain has reached the distribution relevant to the scientific analysis. As the earlier material on compatibility and convergence emphasizes, stability and convergence behavior are distinct from compatibility. A stable chain can still have incompatible conditional specifications; its ESS does not repair that issue.
How ESS Differs From Other Summaries
| Quantity or approach | What it describes | What it does not establish |
|---|---|---|
| Retained iteration count, \(N\) | How many values of a summary were retained | How much dependence those values contain |
| Effective sample size | Approximate independent-draw equivalent for the variance of a particular chain average | Correctness of the FCS models or a literal count of unique states |
| Independent-chain comparison | Whether separate runs show conspicuous differences in their explored summaries | The within-chain autocorrelation of every summary |
| Trace or autocorrelation plot | Visual evidence about movement, drift, and dependence over iterations or lags | A guaranteed numerical estimate of effective information |
Independent draws are the main conceptual benchmark: if all retained values were independent, every \(\rho_k\) for \(k>0\) would be zero, \(\tau=1\), and \(N_{\mathrm{eff}}\approx N\). For positively correlated chains, ESS is usually smaller than \(N\). Negative autocorrelation can make the theoretical ESS exceed \(N\), indicating lower average variance than for \(N\) independent draws with the same marginal variance. In finite samples, estimates in that setting can be especially sensitive to the estimation method; do not interpret an ESS as a physical count of states.
ESS is also not a substitute for the analysis of uncertainty in a multiple-imputation procedure. It characterizes the dependence-related information in a chain summary under the stated approximation. The number of FCS sweeps, the number of completed data sets ultimately used, and the uncertainty of a scientific estimate are related but distinct quantities.
Common Mistakes and Limitations
- Using \(N\) as though iterations were independent. If adjacent summaries are positively correlated, the variance of their average is larger than the independent-draw variance for the same \(N\). Use autocorrelation information rather than the retained count alone.
- Assigning one ESS to every summary. The coefficient and indicator example shows that dependence can vary substantially by quantity. Calculate or inspect ESS for the specific summaries that matter.
- Summing noisy correlations indefinitely. Sample autocorrelations at large lags are uncertain. A practical estimator needs a defensible truncation or smoothing rule; an unexamined long sum can give an unstable estimate.
- Treating a large ESS as proof of a good imputation model. ESS addresses dependence-related precision under a chain approximation. It cannot verify compatibility, model adequacy, or scientific validity.
- Calling ESS the number of unique imputed data sets. ESS is defined by variance equivalence for a scalar average, not by counting distinct completed data sets or states.
The main limitation is that ESS is estimated from a finite chain. If the chain is drifting, trapped in a region, or too short to estimate its correlation decay, a numerical ESS can look precise while being misleading. When these warning signs arise, report the diagnostic uncertainty plainly rather than presenting the estimate as definitive.
Check Your Understanding
Use the definitions and approximations above. For each numerical question, treat the stated autocorrelation pattern as valid for the calculation.
- A retained scalar sequence has \(N=600\) and an AR(1) autocorrelation pattern with \(\rho_1=0.50\). Calculate its approximate ESS.
- A chain summary has \(N=900\), and a justified truncated autocorrelation sum is \(\sum_k \rho_k=0.75\). Calculate \(\hat{\tau}\) and the approximate ESS.
- Why can two summaries from the same FCS run have different ESS values?
- Does an ESS larger than \(N\) necessarily indicate an arithmetic error? Explain.
- A summary's trace shows persistent upward drift. Should its calculated ESS be treated as a reliable estimate of effective information? Why or why not?
Solutions. (1) For the AR(1) pattern, \(\tau=1+2[0.50/(1-0.50)]=1+2(1)=3\). Thus \(N_{\mathrm{eff}}\approx600/3=200\). (2) \(\hat{\tau}=1+2(0.75)=2.5\), so \(\widehat{N}_{\mathrm{eff}}=900/2.5=360\). (3) ESS depends on the autocorrelations of the particular scalar sequence; summaries can respond differently to the same sequence of FCS updates. (4) No. Negative autocorrelation can reduce the variance of an average relative to \(N\) independent draws, so the theoretical variance-equivalent ESS can exceed \(N\). The estimate still needs careful interpretation. (5) No. Persistent drift is evidence against approximate stationarity, an assumption underlying the usual ESS interpretation. A numerical result may be reported as a calculation, but it should not be treated as a trustworthy measure of effective information without resolving the nonstationarity concern.