Constructing Bands with Self-Similarity and Bias Bounds

Why Resolution Selection Needs a Bias Bound

A confidence band for a regression function must account for two different errors. At a coarse resolution, the estimate is stable but can omit substantial fine-scale structure. At a fine resolution, that approximation bias is smaller, but the estimate is more affected by noise. Choosing a resolution from the data can balance these errors, but a band centered at the selected estimate needs an allowance for both.

Self-similarity supplies a way to estimate the omitted structure: it requires the unresolved tail to be controlled by the size of the coefficient block just beyond the fitted resolution. That block is observable, up to noise. The construction below adds a noise allowance to its observed size, combines this data-dependent bias bound with a simultaneous stochastic bound, and selects the resolution with the smallest total radius.

Model, Estimator, and Self-Similarity

Consider a regression function represented in bounded basis functions. For clarity, group the basis functions by resolution level \(j\). At level \(j\), write \(e_j(x)=\sum_{k=1}^{K_j}\theta_{jk}\psi_{jk}(x)\), where \(K_j\) is the number of coefficients and \(\theta_{jk}\) is the coefficient of \(\psi_{jk}\). Suppose \(f(x)=\sum_{j=0}^{\infty}e_j(x)\), with the series converging uniformly. Define \(D_j=\sup_x\{\sum_{k=1}^{K_j}\psi_{jk}(x)^2\}^{1/2}\) and \(A_j=D_j\|\theta_j\|_2\), where \(\theta_j=(\theta_{j1},\ldots,\theta_{jK_j})\). Cauchy–Schwarz gives \(\|e_j\|_\infty\leq A_j\).

The coefficient observations are assumed to satisfy the Gaussian sequence model \(Y_{jk}=\theta_{jk}+\sigma n^{-1/2}Z_{jk}\), where the \(Z_{jk}\) are independent \(N(0,1)\) variables and \(\sigma\) is known. We observe coefficients through level \(J_{\max}+1\). For a candidate resolution \(J\in\{J_{\min},\ldots,J_{\max}\}\), estimate the function by keeping levels through \(J\): \(\hat f_J(x)=\sum_{j=0}^{J}\sum_{k=1}^{K_j}Y_{jk}\psi_{jk}(x)\).

By uniform convergence and the triangle inequality, the truncation bias is bounded by \(\|f-\sum_{j=0}^{J}e_j\|_\infty\leq\sum_{j>J}\|e_j\|_\infty\leq\sum_{j>J}A_j\). Self-similarity therefore yields the deterministic bound \(\|f-\sum_{j=0}^{J}e_j\|_\infty\leq C_{\mathrm{ss}}D_{J+1}\|\theta_{J+1}\|_2\). The unknown norm in this expression will be replaced by an upper bound derived from the observed next-level block.

Simultaneous Bounds Across Resolutions

A data-driven choice can depend on noise at every candidate resolution. It is therefore not enough to establish a probability bound separately at each fixed \(J\) and then act as if that bound automatically applies to the selected \(J\). Instead, use one event that controls all observed coefficients simultaneously.

Let \(M=\sum_{j=0}^{J_{\max}+1}K_j\), the total number of observed coefficients, and for \(0<\alpha<1\) set \(t=\sqrt{2\log(2M/\alpha)}\). For each candidate \(J\), define \[ V_J=\frac{\sigma t}{\sqrt n}\sum_{j=0}^{J}D_j\sqrt{K_j}. \] This is a stochastic fluctuation bound for the truncated estimator. Define the observed next-block allowance and total radius by \[ \widehat B_J=C_{\mathrm{ss}}D_{J+1} \left(\|Y_{J+1}\|_2+\frac{\sigma t}{\sqrt n}\sqrt{K_{J+1}}\right), \qquad R_J=V_J+\widehat B_J. \] Choose \(\hat J\) to minimize \(R_J\) over the candidate set, breaking ties by choosing the smallest resolution. The resulting band is \[ \mathcal C(x)= [\hat f_{\hat J}(x)-R_{\hat J},\ \hat f_{\hat J}(x)+R_{\hat J}], \quad x\text{ in the domain}. \]

Proof. For each observed coefficient, \(P(|Z_{jk}|>t)\leq 2\exp(-t^2/2)\). A union bound over the \(M\) coefficients gives \[ P\left(\max_{j,k}|Z_{jk}|>t\right) \leq 2M\exp(-t^2/2)=\alpha. \] Let \(E\) be the complementary event, so \(P(E)\geq1-\alpha\). On \(E\), for every level \(j\), \(\|Y_j-\theta_j\|_2\leq(\sigma t/\sqrt n)\sqrt{K_j}\).

For every \(x\), Cauchy–Schwarz gives \[ |\hat f_J(x)-\sum_{j=0}^{J}e_j(x)| \leq\sum_{j=0}^{J} \left|\sum_{k=1}^{K_j}(Y_{jk}-\theta_{jk})\psi_{jk}(x)\right| \leq\frac{\sigma t}{\sqrt n}\sum_{j=0}^{J}D_j\sqrt{K_j} =V_J. \] The same event also gives \(\|\theta_{J+1}\|_2\leq\|Y_{J+1}\|_2+(\sigma t/\sqrt n)\sqrt{K_{J+1}}\). Combining this inequality with self-similarity bounds the truncation bias by \(\widehat B_J\). The triangle inequality now implies, simultaneously for every candidate \(J\), \[ \|f-\hat f_J\|_\infty \leq V_J+\widehat B_J=R_J. \] In particular, this inequality holds for the data-dependent choice \(J=\hat J\), because it holds for all candidates on the same event. Thus \(f(x)\in\mathcal C(x)\) for every \(x\) on \(E\), and the probability of this event is at least \(1-\alpha\). \(\square\)

Worked Examples

Why the Construction Matters—and Where It Can Fail

The construction makes explicit how adaptation and coverage fit together. The resolution is selected by comparing total radii, not by treating an estimated best-fitting resolution as if it were fixed in advance. The simultaneous event protects every candidate at once, and the self-similarity assumption converts the observed next block into a bound on the unseen tail. If one wants to claim coverage over a function class, the required constants and self-similarity condition must hold uniformly over that class.

The price of this guarantee is that the radius can be conservative. A union bound over all observed coefficients and a sum of levelwise fluctuations are convenient, transparent bounds, but they need not be sharp. More importantly, the data cannot certify the self-similarity condition by themselves: a small observed block can coexist with a larger later tail if the assumed condition fails. In that case the reported bias allowance need not cover the actual approximation bias.

A common mistake is to select \(\hat J\) by minimizing an estimate of prediction error and then attach only the stochastic radius \(V_{\hat J}\). This omits the truncation bias, which is generally nonzero and can be largest at the coarsest resolutions. The correction is to include a valid bias allowance at every candidate resolution before selection, and to use a fluctuation event that is simultaneous over those candidates. In this construction, self-similarity and the next-block noise correction provide precisely that allowance.