Block Bootstrap for Dependent Observations

Why Resampling Must Respect Order

An ordinary bootstrap draws individual observations independently with replacement. That is appropriate when the observations are independent and identically distributed, but it can be misleading for an ordered sequence whose neighboring values tend to be related. For example, a hypothetical insurer’s daily claim count may be elevated for several consecutive days after a particular event. Drawing days independently breaks those runs and can understate the uncertainty of a statistic that depends on the sequence.

The block bootstrap resamples consecutive groups of observations. It keeps the order within each selected group, so short stretches of the observed dependence can recur in a bootstrap sample. It does not reproduce every dependence pattern exactly: joins between separately sampled blocks, the choice of block length, and the finite sample all matter.

The usual goal is to approximate the sampling distribution of a statistic, such as the sample mean, when observations are dependent. A block-bootstrap replicate is one resampled sequence of length \(n\); the statistic is recalculated on that sequence. Repeating this produces bootstrap replicates. The purpose is not to make the resampled values independent, but to preserve some of their local ordering.

Moving and Circular Blocks

For the moving-block bootstrap, the possible starting positions are \(1,\ldots,n-\ell+1\). Each block has the form \((X_s,\ldots,X_{s+\ell-1})\). Draw starting positions independently and uniformly from these possibilities, using replacement. Concatenate the selected blocks and truncate the result to length \(n\). The final block may therefore contribute fewer than \(\ell\) observations.

For the circular-block bootstrap, define indices cyclically: after \(n\) comes \(1\). A block starting at \(s\) is \((X_s,X_{s+1},\ldots)\), with indices wrapped around as needed. There are \(n\) possible starting positions. This removes the special restriction that a moving block cannot start near the end of the observed sequence, but it introduces a potential seam: \(X_n\) followed by \(X_1\) was not an observed linear-time transition.

Counting Transitions in a Finite Sample

A transition is a pair of consecutive values in the resampled sequence. Within a moving block, each transition is an original linear adjacency. A join between blocks is not guaranteed to be one. In a circular block, a within-block transition that crosses from \(X_n\) to \(X_1\) is also not an original linear adjacency.

For a moving-block replicate with \(K=\lceil n/\ell\rceil\) selected blocks, write the block lengths after truncation as \(b_1,\ldots,b_K\), where each \(b_j\) is at most \(\ell\) and \(\sum_{j=1}^K b_j=n\). The number of within-block transitions is \(\sum_{j=1}^K(b_j-1)=n-K\). The resampled sequence has \(n-1\) adjacent pairs in total, so the fraction that are within-block transitions is \((n-K)/(n-1)\). This counts within-block positions; it does not imply that block joins retain dependence.

For example, if \(n=4\) and \(\ell=2\), then \(K=2\). There are \(4-2=2\) within-block transitions among the \(4-1=3\) adjacent pairs, giving a fraction of \(2/3\), not \(1/2\). In the circular-block bootstrap, the same count \(n-K\) gives the number of within-block positions, but some may be seam transitions rather than observed linear adjacencies.

Why Blocks Can Preserve Information About Dependence

For a stationary sequence with mean \(\mu\), let \(\gamma_h=\operatorname{Cov}(X_t,X_{t+h})\) be the autocovariance at lag \(h\). The variance of the sample mean \(\bar X=n^{-1}\sum_{t=1}^n X_t\) is

The first equality follows by expanding the variance of a sum: the \(n\) individual variances contribute \(n\gamma_0\), and each lag-\(h\) covariance occurs in \(n-h\) pairs in each order. This explains why dependence matters. The variance of the mean is affected by covariances at nonzero lags, not just by the marginal variance \(\gamma_0\).

An independent resampling of individual values discards their observed consecutive pairings. Block resampling retains consecutive pairings inside selected blocks, allowing the bootstrap replicates to reflect some short-lag dependence. A block of length \(\ell\) cannot retain an observed lag-\(h\) pair when \(h\geq\ell\), and joins between blocks may add transitions that were not observed together. Thus blocks provide an approximation, not an exact reconstruction of the sequence’s dependence structure.

Choosing a Block Length

The block length \(\ell\) controls a bias-variance tradeoff in the bootstrap approximation. If \(\ell\) is too short, important local patterns may be broken frequently. At \(\ell=1\), every block contains a single observation, so the procedure reduces to the ordinary individual-observation bootstrap and preserves no within-block transitions. If \(\ell\) is longer, more observed local structure can remain intact, but fewer blocks contribute to each resample. That can make the bootstrap replicates less varied and can leave fewer distinct blocks available.

In standard large-sample arguments for block bootstrap methods, conditions typically require the block length to increase with \(n\) while remaining small relative to \(n\): \(\ell\to\infty\) and \(\ell/n\to0\). These conditions express competing needs: blocks must eventually be long enough to retain increasingly extended local dependence, yet many blocks must still fit into a sample. Formal validity also requires assumptions on the sequence’s dependence and moment behavior; the block-length conditions alone do not establish validity for every process or statistic.

For a finite data set, the analyst must balance the observed dependence scale against the number of blocks and the purpose of the statistic. A sequence with persistent dependence may need longer blocks than one whose dependence fades quickly. Sensitivity checks across plausible block lengths can reveal whether an uncertainty estimate depends heavily on this choice. Circular blocks reduce the restriction on starting positions, but their seam transitions can be questionable when the beginning and end of the observed sequence are not naturally connected.

• Too short: the resamples may break dependence at scales relevant to the statistic.
• Too long: fewer sampled blocks can make resampling variability harder to represent.
• Boundary awareness: moving-block joins are new joins; circular blocks can also contain a seam transition.
• Interpretation: block bootstrap uncertainty is an approximation conditional on the observed sequence and the assumptions supporting resampling.