Block and Sieve Bootstraps for Dependent Vectors

Why Dependence Changes Resampling

For independent observations, resampling individual data vectors can reproduce the sampling variation of many statistics. In a time series, however, a vector at one time may be informative about nearby vectors. Resampling single time points independently breaks that temporal pattern, even if it retains the association among coordinates measured at the same time. Block and sieve bootstraps address this problem in different ways: blocks retain observed runs of consecutive vectors, while a sieve approximates the dependence with a fitted time-series model.

Let \(X_1,\ldots,X_n\) be observations from a strictly stationary sequence in \(\mathbb{R}^d\), with mean \(\mu=E[X_t]\). A central example is inference for \(\mu\), whose uncertainty depends not just on the contemporaneous covariance but on covariances across time. Write \(\Gamma_h=\operatorname{Cov}(X_t,X_{t+h})\). When the covariance series is absolutely summable, the long-run covariance is \(\Gamma=\sum_{h=-\infty}^{\infty}\Gamma_h\). It governs the limiting covariance of the scaled sample mean.

Moving and Stationary Blocks

In the moving-block bootstrap, choose a block length \(b\). The overlapping observed blocks are \((X_1,\ldots,X_b), (X_2,\ldots,X_{b+1}),\ldots,(X_{n-b+1},\ldots,X_n)\). Sample blocks independently and uniformly from this collection, concatenate them, and retain the first \(n\) vectors. Each sampled block is a whole vector sequence: its within-time coordinates and its observed temporal order remain intact. The join between two sampled blocks does not generally continue the original series.

The stationary bootstrap also joins observed runs, but uses random block lengths. To generate each next vector, continue from the next observed time point, wrapping from \(n\) to \(1\), with probability \(1-q\); with probability \(q\), restart at a uniformly chosen observed index. Block lengths are geometric with mean \(b=1/q\). This randomization gives the resampled series a stationary construction on the circular sample, while its blocks still come from the observed data.

What the Mean’s Uncertainty Depends On

The long-run covariance is the relevant scale for the sample mean, not merely \(\Gamma_0\). The following result makes that distinction precise. Assume strict stationarity, \(E\|X_t\|^2<\infty\), and \(\sum_{h=-\infty}^{\infty}\|\Gamma_h\|<\infty\), where \(\|\cdot\|\) is any matrix norm.

Proof. Expand the covariance of the sum: \[ n\,\operatorname{Cov}(\bar X_n) =\frac{1}{n}\sum_{s=1}^n\sum_{t=1}^n\operatorname{Cov}(X_s,X_t). \] For a lag \(h=t-s\), there are \(n-|h|\) pairs \((s,t)\) when \(|h|<n\), and stationarity gives \(\operatorname{Cov}(X_s,X_t)=\Gamma_h\). Grouping the terms by lag yields the displayed finite sum. For each fixed \(h\), the weight \(1-|h|/n\), when \(|h|<n\), tends to 1. Its absolute value is at most 1, so each summand is bounded in norm by \(\|\Gamma_h\|\). Absolute summability permits dominated convergence for this series, giving the stated limit. \(\square\)

This result is about the true sampling variance. Bootstrap methods aim to reproduce it conditionally on the observed series. For block methods, a useful finite-sample calculation shows why the block length controls which lags contribute.

Proof. There are \(m=n/b\) independently sampled blocks. Let \(B\) denote one sampled block sum after subtracting \(\bar X_n\) from each of its vectors. The bootstrap mean is \(\bar X_n\) plus the sum of the \(m\) centered block sums divided by \(n\). The block sums are conditionally independent and have conditional mean zero, so its conditional covariance is \(m\,\operatorname{Cov}^*(B)/n^2\). A block contains \(b\) terms with lag zero and \(b-h\) ordered pairs at lag \(h\) in each direction, for \(1\leq h<b\). Averaging over the \(n\) circular starting positions gives \[ \operatorname{Cov}^*(B)=b\widehat\Gamma_0+ \sum_{h=1}^{b-1}(b-h)(\widehat\Gamma_h+\widehat\Gamma_h^T). \] Substitute \(m=n/b\) into \(m\,\operatorname{Cov}^*(B)/n^2\), then divide each coefficient by \(b\). This gives the claimed expression. \(\square\)

The factors \(1-h/b\) taper the contribution of longer lags, and lags at least \(b\) do not contribute within a sampled block. This is a finite-sample identity for the circular construction; it is not by itself a general proof of bootstrap consistency.

Dependence Conditions and the Sieve Alternative

For block-bootstrap inference about a mean, a common sufficient framework is a fixed vector dimension \(d\), strict stationarity, \(E\|X_t\|^{2+\delta}<\infty\) for some \(\delta>0\), and strong mixing with \(\sum_{k=1}^{\infty}\alpha(k)^{\delta/(2+\delta)}<\infty\). Here \(\alpha(k)\) is the supremum of \(|P(A\cap B)-P(A)P(B)|\) over \(A\) determined by observations at times at most 0 and \(B\) determined by observations at times at least \(k\). These assumptions impose weak enough dependence for a central limit theorem and control the distant-lag contribution. For the usual block-bootstrap approximation to the distribution of the scaled mean, choose \(b=b_n\) with \(b_n\to\infty\) and \(b_n/n\to0\). The first requirement lets retained blocks include increasingly long dependence; the second leaves increasingly many blocks to resample. Variants and stronger targets may require additional conditions.

The sieve bootstrap is model-based rather than based on a block-length approximation. A common version fits a vector autoregression of order \(p\), resamples centered residual vectors as whole vectors, and recursively generates a bootstrap series. For example, it can be motivated by a causal linear process \(X_t-\mu=\sum_{j=0}^{\infty}A_j\varepsilon_{t-j}\), where the \(\varepsilon_t\) are independent and identically distributed mean-zero vectors. Under suitable invertibility, moment, and approximation conditions, increasing-order autoregressions can approximate the process. A typical sieve analysis requires the fitted order to grow, while remaining small relative to the sample size; the precise rate and regularity conditions depend on the model and inferential target. The essential requirement is that the fitted autoregressive dynamics and resampled innovations reproduce the dependence relevant to the statistic. A sieve is not automatically valid for every stationary process.

Choosing a Method and a Block Length

Block length is a bias–variability choice. If blocks are too short, important dependence at longer lags is cut off, and uncertainty can be underestimated. If blocks are too long, relatively few blocks are effectively available; resampling then varies less across replicates and estimates can be unstable. A practical analysis examines whether the target uncertainty estimate changes materially over a reasonable range of block lengths. The range should be justified by the time scale of dependence and the sample size, not chosen solely because one value gives a preferred interval.

Moving blocks use a fixed length and overlapping observed runs. The stationary bootstrap randomizes run lengths, which can reduce sensitivity to fixed block boundaries, but it still requires a defensible mean block length. A sieve can be useful when a credible autoregressive approximation captures the dependence, especially when dependence extends beyond a short local window. It introduces a different risk: a misspecified or poorly estimated model can give misleading uncertainty.