Effective Sample Size for Variance-Component Summaries

Why the Number of Draws Is Not the Amount of Information

A Bayesian GLMM may produce thousands of retained draws for a variance component, yet nearby draws can be similar because the chain moves slowly through the posterior. The raw draw count then overstates how much independent information the simulation contains. Effective sample size (ESS) expresses this loss in information relative to independent draws, and its value depends on which feature of the posterior is being summarized.

This distinction matters for variance components. Their posterior distributions can be right-skewed, concentrated near zero, or have relatively sparse tails. A chain may represent the central portion reasonably well while moving slowly through tail regions. Bulk and tail ESS summarize these aspects separately; neither is a verdict on the validity of the model or a substitute for examining chain behavior.

The interpretation below assumes retained draws come from chains that have reached the posterior’s stationary distribution, that chains target the same posterior, and that the autocorrelation relevant to the summary is adequately estimated. For the standard long-run ESS interpretation, the autocorrelations must have a finite sum (or the corresponding long-run variance must be finite). If these conditions are doubtful, an ESS number does not repair the problem. The earlier material on diagnosing MCMC mixing discusses the broader role of chain diagnostics.

Autocorrelation and Effective Sample Size

Definition: For a stationary sequence of $M$ retained draws and a scalar quantity $g$ computed from each draw, let $\rho_k$ be the lag-$k$ autocorrelation of $g$ across the chain. Its integrated autocorrelation time is $$ \tau_g = 1 + 2\sum_{k=1}^{\infty}\rho_k, $$ and its effective sample size is $$ \operatorname{ESS}_g = \frac{M}{\tau_g}. $$ This is the approximate number of independent draws that would provide comparable information about the chosen summary of $g$.

For a variance component $\theta$, choosing $g(\theta)=\theta$ makes the ESS relevant to its posterior mean. Other summaries can require other functions of the draws. For example, a tail probability depends on whether each draw falls beyond a threshold, so its relevant autocorrelation is the autocorrelation of that indicator—not necessarily the autocorrelation of $\theta$ itself.

The formula gives an asymptotic comparison, not a literal count of independent observations hidden inside the chain. Positive autocorrelation makes $\tau_g$ greater than one and reduces ESS. Negative autocorrelation can make ESS exceed $M$: this is possible because the chain’s dependence can make an average more precise than the average of the same number of independent draws. It does not mean the sampler generated extra physical observations.

Why Autocorrelation Reduces Information

The connection follows from the variance of an average. Let $g_t$ denote the value of $g$ at retained iteration $t$, with stationary variance $\sigma_g^2$. Then $$ \operatorname{Var}\left(\frac{1}{M}\sum_{t=1}^{M}g_t\right) = \frac{\sigma_g^2}{M} \left[ 1+2\sum_{k=1}^{M-1}\left(1-\frac{k}{M}\right)\rho_k \right]. $$ The terms $(1-k/M)$ count how many pairs of draws are separated by lag $k$, relative to the total number of draws.

When the autocorrelations decay sufficiently quickly, the bracket approaches $\tau_g$ as $M$ grows. An independent sample of size $M$ has variance $\sigma_g^2/M$. Equating its variance to the chain-average variance gives the approximate independent sample size $M/\tau_g$. Thus ESS is not an arbitrary rescaling: it compares the simulation’s long-run information with the information from independent posterior draws.

Key result: If a chain has positive serial dependence for the summary of interest, its effective sample size is generally below its retained draw count. The reduction depends on the whole autocorrelation pattern, not just the lag-one autocorrelation.

Worked Examples

Worked Example: A Slowly Moving Variance-Component Chain

Suppose a hypothetical stationary chain retains $M=2{,}000$ draws of a random-intercept variance component $\theta$. For illustration, assume the autocorrelation at lag $k$ is $\rho_k=0.6^k$. This geometric pattern has positive, declining autocorrelation and a finite sum, so the long-run ESS formula applies.

The autocorrelation sum is $$ \sum_{k=1}^{\infty}\rho_k = \sum_{k=1}^{\infty}0.6^k = \frac{0.6}{1-0.6} = 1.5. $$ Therefore, $$ \tau_\theta=1+2(1.5)=4, \qquad \operatorname{ESS}_\theta=\frac{2{,}000}{4}=500. $$

Although the chain contains 2,000 retained values, its information about the posterior mean of $\theta$ is approximately that of 500 independent draws under this autocorrelation model. This is an approximation for that summary, not a claim that the chain literally contains 500 independent values.

Worked Example: Comparing Two Variance Components

Consider hypothetical retained draws for two variance components, $\theta_1$ and $\theta_2$, from the same set of $M=4{,}000$ stationary iterations. Suppose, for a simple illustration, their autocorrelations follow geometric patterns with lag-$k$ correlations $0.2^k$ and $0.8^k$, respectively. These assumptions give finite autocorrelation sums.

For $\theta_1$, $$ \tau_{\theta_1} = 1+2\left(\frac{0.2}{1-0.2}\right) = 1+2(0.25) = 1.5, \qquad \operatorname{ESS}_{\theta_1} = \frac{4{,}000}{1.5} \approx 2{,}667. $$ For $\theta_2$, $$ \tau_{\theta_2} = 1+2\left(\frac{0.8}{1-0.8}\right) = 1+2(4) = 9, \qquad \operatorname{ESS}_{\theta_2} = \frac{4{,}000}{9} \approx 444.4. $$

The same raw number of iterations can therefore provide very different amounts of information for different variance components. Here the draws for $\theta_2$ are much more persistent, so the chain provides less effective information about summaries tied to its values. In actual analyses, autocorrelation usually does not follow an exact geometric pattern; these calculations illustrate the definition rather than replace an estimator based on the observed chains.

Worked Example: Tail ESS Can Be Smaller Than Bulk ESS

Suppose three hypothetical stationary chains yield $M=3{,}000$ retained draws in total for a positive variance component $\theta$. Assume for illustration that the relevant central, rank-based sequence has geometric autocorrelation $0.1^k$. Also suppose the indicator for being below the posterior fifth percentile, $I(\theta\le q_{0.05})$, has autocorrelation $0.7^k$, while the indicator for being above the ninety-fifth percentile has autocorrelation $0.5^k$. These patterns are assumed stationary and summable.

The central sequence has $$ \tau_{\text{bulk}} = 1+2\left(\frac{0.1}{1-0.1}\right) \approx 1.222, \qquad \operatorname{ESS}_{\text{bulk}} \approx \frac{3{,}000}{1.222} \approx 2{,}455. $$ For the lower-tail indicator, $$ \tau_{\text{lower}} = 1+2\left(\frac{0.7}{1-0.7}\right) \approx 5.667, \qquad \operatorname{ESS}_{\text{lower}} \approx \frac{3{,}000}{5.667} \approx 529.4. $$ For the upper-tail indicator, $$ \tau_{\text{upper}} = 1+2\left(\frac{0.5}{1-0.5}\right)=3, \qquad \operatorname{ESS}_{\text{upper}}=1{,}000. $$

A common tail-ESS summary takes the smaller of the lower- and upper-tail ESS values, which here is about 529.4. The central sequence has substantially higher ESS, but the chain moves more persistently in the lower tail. This warns that tail-related summaries may be less well resolved than central summaries, even though the total draw count is 3,000.

Bulk ESS and Tail ESS in Practice

ESS is summary-specific, and modern software commonly reports bulk and tail versions based on multiple chains. A common convention is to split chains into segments, rank-normalize the draws, and estimate an ESS from the resulting sequences. Bulk ESS describes information in the central distribution and is useful for assessing central summaries. Tail ESS is commonly based on the ESS of indicators for falling below a low posterior quantile and above a high posterior quantile; the smaller of those two values is reported. Many implementations use the 5th and 95th percentiles. Details can vary across software, so a report should identify the implementation when that distinction matters.

Rank-based diagnostics are particularly useful when variance-component draws are skewed or have different scales across chains. But a bulk ESS is not an exact ESS for every possible statistic. If the target is an unusual transformation or a specific tail probability, inspect the behavior relevant to that target rather than treating one reported bulk number as universal. Tail ESS is a useful general check of tail exploration, not a guarantee that every interval endpoint or tail probability is precise.

For multiple chains, the reported ESS typically uses the total number of retained draws across chains while accounting for within-chain autocorrelation and differences among chains. Pooling draws does not make badly separated chains trustworthy. Chain agreement and stationarity remain necessary for a meaningful posterior simulation.

Reading ESS Alongside Other Diagnostics

Quantity or check	What it addresses	What it does not establish by itself
Raw retained draw count	How many values were stored	How many effectively independent values they represent
Bulk ESS	Information in the central part of the posterior	That the tails or every transformed summary are well explored
Tail ESS	Information in lower and upper tail behavior	That all tail probabilities or interval endpoints are accurate
Chain agreement diagnostic	Whether chains appear to target a common distribution	That within-chain autocorrelation is negligible
Trace plots and chain histories	Visible trends, sticking, and between-chain separation	A precise numerical measure of information

A useful assessment combines these checks. Low ESS indicates that autocorrelation has reduced information for the relevant sequence. A large ESS does not prove convergence: chains can all be trapped in the same region and still look locally efficient. Conversely, a small ESS is not evidence that the posterior itself is narrow or broad; it describes the simulation’s efficiency, not the posterior’s substantive uncertainty.

Common Mistakes and Limitations

Equating retained draws with independent draws. The wrong reasoning is “I saved 10,000 iterations, so I have 10,000 independent posterior draws.” The correction is to inspect ESS, because serial dependence can make the effective information much smaller.
Using lag-one autocorrelation alone. The wrong reasoning is “The first-lag correlation is modest, so dependence is negligible.” ESS depends on the sum of autocorrelations across lags; a long tail of moderate correlations can still reduce information substantially.
Treating bulk ESS as a tail guarantee. A chain can explore the center efficiently yet move slowly through rare low or high values. Compare bulk and tail ESS, especially for positive variance components with skewed posterior shapes.
Interpreting ESS as convergence proof. ESS is conditional on the chains representing the intended stationary posterior. It cannot detect every failure to reach that posterior, and it should be read alongside chain behavior and agreement diagnostics.
Assuming every summary has the same ESS. The relevant autocorrelation changes with the function of the draws. ESS for a component’s mean, a tail indicator, and another nonlinear summary need not match.

Thinning—saving only every $r$th draw—can reduce storage and may lower the autocorrelation among saved values, but it also discards draws. It does not generally create more information than retaining the full chain. The more relevant comparison is the information in the retained sequence, summarized by an appropriately estimated ESS, together with evidence that the chains explore the target distribution.

Key takeaway: Read bulk and tail ESS as estimates of how much information the chains contain for different parts of a variance component’s posterior. Autocorrelation reduces that information; raw draw count alone does not measure it.

Check Your Understanding

Answer each question, then compare with the solutions below.

A stationary chain has $M=1{,}200$ draws and autocorrelations $\rho_k=0.5^k$. Calculate its ESS.
Two variance components each have 2,400 retained draws. One has estimated integrated autocorrelation time 2 and the other 6. Calculate both ESS values and state which has less effective information.
Why can a chain have a high bulk ESS but a low tail ESS?
Does a high ESS alone demonstrate that the chains have converged to the intended posterior? Explain.

Solutions. (1) The autocorrelation sum is $0.5/(1-0.5)=1$, so $\tau=1+2(1)=3$, and $\operatorname{ESS}=1{,}200/3=400$. (2) The ESS values are $2{,}400/2=1{,}200$ and $2{,}400/6=400$. The second component has less effective information because its larger integrated autocorrelation time reflects more persistent dependence. (3) Central draws may move relatively freely while extreme draws are visited infrequently or persist for long stretches; tail indicators then have greater autocorrelation than central rank-based values. (4) No. ESS measures simulation efficiency for a particular sequence under the assumed posterior behavior. Chains can have high within-chain efficiency while failing to explore the full target distribution, so chain agreement and other diagnostics are also needed.