Why Correlated Simulation Output Needs a Different Standard Error
Many simulation calculations produce a sequence of outputs rather than independent observations. For example, a simulated system may be observed at successive time steps, or a Markov simulation may produce one output at each iteration. Nearby outputs can be similar because the system retains memory. In that setting, the usual independent-observation standard error \(s/\sqrt{n}\) can understate or overstate the uncertainty in the overall average.
Batch means provide a practical way to estimate uncertainty while accounting for serial dependence. The sequence is divided into consecutive blocks, or batches. The average within each batch becomes one observation for estimating the variability of the overall average. The method does not make the original outputs independent; it relies on batches being long enough that dependence between batch averages is small.
If \(n\) is not divisible by \(b\), a common simple convention is to use \(m=\lfloor n/b\rfloor\) complete batches and omit the remaining outputs from the batch calculation. State that choice, since the omitted observations do not contribute to the estimate. For a clean comparison of batch sizes, it is often preferable to choose sizes that divide the available run length.
Why the Batch-Means Formula Works
Suppose the sequence is stationary, has finite variance, and has sufficiently weak dependence for its autocovariances to be summable. Write \(\gamma_k=\operatorname{Cov}(X_t,X_{t+k})\). The long-run variance is
Under these conditions, for a sufficiently long run, the variance of the sample average is approximately \(\sigma_L^2/n\). This is the quantity whose square root is the standard error of interest. Positive serial dependence typically makes \(\sigma_L^2\) larger than the individual-output variance \(\gamma_0\), which explains why treating correlated outputs as independent can be misleading.
The link to batch means follows from grouping the sum into batches. Since \(n=mb\), the overall average is exactly the average of the batch means:
When batches are long enough that their means have little remaining serial dependence, the sample variance \(s_B^2\) estimates the variance of a batch mean. The variance of the average of \(m\) approximately independent batch means is then \(s_B^2/m\). Taking its square root gives \(s_B/\sqrt{m}\). Also, a batch mean averages \(b\) consecutive outputs, so its variance is approximately \(\sigma_L^2/b\); consequently \(b s_B^2\) estimates \(\sigma_L^2\). These approximations require both a sufficiently long run and suitable batch sizes; they are not exact guarantees for a short, strongly dependent sequence.
Use outputs from a part of the simulation that can reasonably be treated as stationary. If an initial transient is present, decide on a warm-up period before calculating batch means.
Divide the retained sequence into \(m\) complete batches of \(b\) outputs each. Record whether any remainder is omitted.
Average the outputs separately within each batch, then calculate their sample variance \(s_B^2\).
Calculate \(s_B/\sqrt{m}\). Repeat with other reasonable batch sizes; substantial changes suggest the run or the batch design may not yet support a stable estimate.
Worked Calculations
Worked Example: Standard Error From Four Batches
A hypothetical simulation produces the stationary output sequence \(2,4,3,5,7,6,4,6,5,7,9,8\). Treat these as correlated observations with finite variance; no parametric distribution is assumed. Use batch size \(b=3\) and estimate the standard error of the overall average.
Method. Calculate the four consecutive batch means, then use their sample variance. This method accounts for within-batch dependence through the observed variation between batch means.
The batch means are
Their average is \((3+6+5+8)/4=5.5\), which is also the average of all 12 outputs. The sum of squared deviations from 5.5 is
There are \(m-1=3\) degrees of freedom, so \(s_B^2=13/3=4.3333\). The estimated variance of the overall average is \(s_B^2/m=4.3333/4=1.0833\). Therefore the estimated standard error is \(\sqrt{1.0833}=1.0408\), rounded to four decimal places. As a check, the estimated long-run variance is \(b s_B^2=3(4.3333)=13\), and \(\sqrt{13/12}=1.0408\) gives the same result.
Common mistake avoided. The standard deviation of the batch means is \(\sqrt{4.3333}=2.0817\), not the standard error of the overall average. It must be divided by \(\sqrt{m}=2\). With only four batches, this estimate is also quite uncertain and should not be treated as conclusive evidence of stable precision.
Worked Example: Comparing Two Batch Sizes
A hypothetical stationary simulation gives the 16 correlated outputs \(1,3,2,4,5,7,6,8,3,5,4,6,7,9,8,10\). Compare the estimated standard error using batch sizes \(b=2\) and \(b=4\). Assume finite variance and no parametric distribution. The purpose is to assess batch-size stability, not to select a size based on a predetermined direction of the answer.
Batch size 2. The eight batch means are \(2,3,6,7,4,5,8,9\). Their average is \(44/8=5.5\). The squared deviations from 5.5 sum to
Thus \(s_B^2=42/7=6\), and the estimated standard error is \(\sqrt{6/8}=\sqrt{0.75}=0.8660\), rounded to four decimal places. The corresponding long-run variance estimate is \(2(6)=12\).
Batch size 4. The four batch means are \(2.5,6.5,4.5,8.5\). Their average is \(5.5\), and the squared deviations sum to \(9+1+1+9=20\). Hence \(s_B^2=20/3=6.6667\). The estimated standard error is \(\sqrt{6.6667/4}=\sqrt{1.6667}=1.2910\), rounded to four decimal places. The corresponding long-run variance estimate is \(4(6.6667)=26.6667\).
Check and interpretation. Both calculations use the same overall average, 5.5, but produce materially different standard errors: 0.8660 and 1.2910. This is a warning that the estimate is not stable across these batch sizes. The batch-size-2 calculation has more batches, but that alone does not make its standard error more trustworthy: adjacent short batches may still be dependent. The appropriate conclusion is to investigate the run and, if possible, collect a longer sequence and compare a range of reasonable sizes.
Common mistake avoided. Choosing the smaller standard error simply because it is more attractive would ignore the instability. Batch-size comparison is a diagnostic, not a rule to pick the lowest estimate.
Worked Example: Comparing Batch Means With an Independent-Output Calculation
A hypothetical stationary simulation yields five batches of four outputs each: \(8,10,8,10;\ 9,11,9,11;\ 10,12,10,12;\ 9,11,9,11;\ 9,11,9,11\). Assume finite variance, but do not assume the individual outputs are independent. Estimate the standard error by batch means, then compare it with the usual calculation that would apply if all 20 outputs were independent and identically distributed.
Batch-means calculation. The batch means are \(9,10,11,10,10\), and their average is 10. The sum of squared deviations from 10 is \(1+0+1+0+0=2\). Thus \(s_B^2=2/(5-1)=0.5\), and the batch-means standard error is
Independent-output comparison. The overall average is also 10. The sum of squared deviations of the 20 individual outputs from 10 is 28: the first batch contributes 8, the second 4, the third 8, and each of the last two contributes 4. The individual sample variance is therefore \(28/19=1.4737\). If independence were justified, the standard error would be \(\sqrt{(28/19)/20}=\sqrt{0.0736842}=0.2714\), rounded to four decimal places.
The two estimates differ because they use different information about dependence. The independent-output value is not justified merely because it is easy to calculate; it requires independent observations. The batch-means estimate uses variation among the five batch averages, though five batches still provide limited information. This comparison is a diagnostic illustration, not proof that either estimate is accurate for every correlated process.
Common mistake avoided. Do not use the individual sample variance divided by the number of outputs unless the independent-observation assumption is appropriate. For correlated output, that calculation may miss the effect of serial dependence.
Choosing Batch Size and Checking Stability
Batch size involves a trade-off. If \(b\) is too small, adjacent batch means may remain correlated, so the calculation treats them as more independent than they are. If \(b\) is too large, the number of batches \(m\) becomes small, making \(s_B^2\) itself imprecise. Increasing \(b\) while keeping the run length fixed always reduces the number of batches; it does not automatically improve the estimate.
A practical assessment compares the estimated long-run variance \(b s_B^2\), or equivalently the standard error, over several batch sizes. A reasonably stable range is reassuring, while large changes indicate that the run, the batch sizes, or both deserve further attention. There is no universal batch size that works for every process. Asymptotically, consistency is obtained under suitable dependence conditions when both the batch size and number of batches grow, with the batch size growing more slowly than the total run length. This describes a principle, not a guarantee for a particular finite simulation.
Also check for an initial transient: if early outputs reflect initialization rather than the long-run process, including them can distort the estimated average and its standard error. Choose any warm-up exclusion based on the simulation design and apply it before forming batches. Do not remove observations selectively just to obtain a preferred estimate. When a simulation has too few batches to assess stability, the appropriate response is to report that limitation or extend the run, rather than imply unjustified precision.
Check Your Understanding
For each problem, assume the stated output sequence is the retained simulation run. Use complete nonoverlapping batches and round standard errors to four decimal places.
- A run has batch means \(4,6,5,9\), each based on \(b=5\) outputs. Calculate \(s_B^2\), the estimated standard error of the overall average, and the estimated long-run variance. (Answer: \(s_B^2=4.6667\), standard error \(=1.0801\), long-run variance \(=23.3333\))
- A run has 30 retained outputs and a proposed batch size of 8. How many complete batches are used, how many outputs are omitted, and how many batch means enter the variance calculation? (Answer: 3 complete batches, 6 outputs omitted, and 3 batch means)
- For batch size 3, four batch means are \(7,8,10,11\). Find the overall average, the sample variance of the batch means, and the estimated standard error. (Answer: average \(=9\), \(s_B^2=10/3=3.3333\), standard error \(=\sqrt{(10/3)/4}=0.9129\))
- Two batch sizes give estimated standard errors 0.42 and 0.91 from the same run, while the smaller batch size leaves few outputs per batch. What should be concluded, and what is one sensible next step? (Answer: the estimate is not stable across the tested sizes; extend the run and reassess across a range of batch sizes rather than selecting 0.42 automatically)
- A colleague uses the individual-output sample standard deviation divided by the square root of the number of outputs for a serially correlated sequence. State the condition required for that calculation and the batch-means alternative. (Answer: the usual calculation requires independent, identically distributed outputs; for correlated output, estimate the standard error as \(s_B/\sqrt{m}\) using sufficiently long batches and assess batch-size stability)
This treatment of correlated simulation output is aligned with the SOA Exam ASTAM (Advanced Short-Term Actuarial Mathematics) syllabus.