Integrated-Risk Adaptation Across Hölder Classes

Why Integrated Risk Changes the Adaptation Question

An estimator can be accurate at most locations and still have a large error at a few points. Integrated squared error measures the average squared discrepancy across the whole domain, rather than the largest pointwise discrepancy. For this loss, both smoothing and wavelet methods can adapt to unknown smoothness: they choose a resolution or scale that balances approximation error against noise.

We use the periodic Gaussian white-noise model on \([0,1]\), \(dY(x)=f(x)\,dx+n^{-1/2}dW(x)\), where \(W\) is standard Brownian motion and \(n\) indexes the noise level. The loss and risk are

Periodic boundary conditions avoid edge corrections; they are a modeling convention here, not a claim that every regression problem is periodic. For \(s>0\), the periodic Hölder class \(\mathcal H^s(L)\) consists of functions whose derivatives through order \(r=\lfloor s\rfloor\) are bounded by \(L\), with the order-\(r\) derivative Hölder continuous of exponent \(s-r\) when \(s\) is not an integer. At integer \(s\), one may use the standard Lipschitz-derivative convention. Constants below may depend on the class radius and on fixed kernel or wavelet choices.

Bandwidth Smoothing: Approximation Versus Variance

Let \(K\) be a bounded, compactly supported kernel with integral one and vanishing moments through order \(m-1\), where \(m>s_1\). Write \(K_h\) for its periodic rescaling and define \(\hat f_h(x)=\int_0^1 K_h(x-u)\,dY(u)\). For \(f\in\mathcal H^s(L)\), \(s\le s_1\), the kernel approximation error satisfies \(\|E_f\hat f_h-f\|_2\le C h^s\). The stochastic term has integrated variance \(n^{-1}\|K_h\|_2^2=n^{-1}h^{-1}\|K\|_2^2\), for bandwidths small enough that the periodic rescaling has the usual norm.

To verify the bound, write \(\hat f_h-f=(E_f\hat f_h-f)+(\hat f_h-E_f\hat f_h)\). The second term is a centered Gaussian random function, so the cross term has expectation zero. Its expected squared \(L^2\) norm is the integrated pointwise variance, \(n^{-1}\int_0^1\int_0^1K_h(x-u)^2\,du\,dx=n^{-1}\|K_h\|_2^2\). Adding this to the squared bias proves the result.

If \(s\) were known, balancing the two terms gives \(h^{2s}=(nh)^{-1}\), hence \(h=n^{-1/(2s+1)}\) and risk of order \(n^{-2s/(2s+1)}\). A data-driven choice can use Lepski’s method: compute estimates on a geometric grid of bandwidths, compare a candidate with all finer-bandwidth estimates, and reject candidates whose differences exceed a noise-calibrated threshold. For a finite geometric grid and Gaussian white noise, the integrated-risk version of this comparison gives, under the kernel conditions above,

The threshold is proportional to \(\sqrt{\log n/(ng)}\) when comparing against a finer scale \(g\). The logarithm allows simultaneous control over the grid comparisons; the bound is an oracle inequality up to a constant. Taking a grid point within a constant factor of \((\log n/n)^{1/(2s+1)}\) yields risk at most \(C(\log n/n)^{2s/(2s+1)}\). The rule itself does not use \(s\).

Wavelets: Coefficient Decay and Thresholding

Consider an orthonormal periodic wavelet basis with regularity greater than \(s_1\). Write \(\theta_{jk}=\langle f,\psi_{jk}\rangle\), where level \(j\) has \(2^j\) coefficients. In the white-noise model, the observed coefficient is \(Y_{jk}=\theta_{jk}+n^{-1/2}Z_{jk}\), with independent standard normal \(Z_{jk}\). Hölder smoothness implies the level-energy bound

A universal hard-threshold estimator retains \(Y_{jk}\) when \(|Y_{jk}|>\lambda_n\), sets it to zero otherwise, and uses \(\lambda_n=\sqrt{2\log n/n}\). It also keeps the scaling coefficients and uses levels \(0\le j\le J_n\), with \(2^{J_n}\) of order \(n\). The final resolution is sufficient for the rates below; the smoothness range is bounded away from zero.

Here is the rate derivation. At a coefficient with mean \(\theta\), hard thresholding at \(\lambda_n\) has mean squared error bounded by \(C\min(\theta^2,\lambda_n^2)+C n^{-1}n^{-c}\), for a fixed positive \(c\) when the threshold constant is chosen sufficiently large. The first term covers the choice between suppressing a small signal and retaining a large one; the second bounds Gaussian tail errors. Summing over at most order \(n\) coefficients makes the tail contribution negligible compared with the displayed rate. At level \(j\), summing the first term over \(k\) is bounded by \(C\min\{2^{-2js},\,2^j\log n/n\}\): one bound comes from coefficient energy and the other from counting coefficients and multiplying by \(\lambda_n^2\).

The crossover level \(j_*\) solves \(2^{-2j_*s}=2^{j_*}\log n/n\), or \(2^{j_*}\asymp(n/\log n)^{1/(2s+1)}\). Below this level, sum the variance bounds \(2^j\log n/n\); the geometric sum is bounded by a constant times its final term. Above it, sum the approximation bounds \(2^{-2js}\), also a geometric series. Both sums are of order \((\log n/n)^{2s/(2s+1)}\). The unresolved wavelet tail obeys the same approximation bound. This establishes the stated adaptive rate without knowing \(s\).

How the Two Adaptive Strategies Compare

Both methods express the same statistical trade-off in different coordinates. A bandwidth determines a single global smoothing scale: coarse smoothing has more approximation bias but less variance. Wavelet thresholding makes a coefficient-by-coefficient decision, suppressing small details while retaining large ones. The rates derived above are the same in their dependence on smoothness, although their constants and behavior for spatially irregular functions can differ.

The logarithm in these particular adaptive bounds is a price of the stated selection rules: simultaneous bandwidth comparisons use a multiple-comparison threshold, and universal wavelet thresholding uses a threshold that controls many coefficients at once. It is not the minimax benchmark for integrated squared error itself. More refined procedures can improve logarithmic losses under suitable conditions. In contrast, the supremum-norm loss involves control at every point and has its own logarithmic considerations; the loss and its risk bound must always be stated before comparing rates.

Common Mistakes

• Confusing the loss with pointwise error. Integrated risk is an expectation of an integral over the domain. A pointwise risk bound or a supremum-norm bound is not automatically an integrated-risk result.
• Balancing only variance. Making \(h\) very small reduces smoothing bias but increases variance as \(1/(nh)\). The optimal scale balances both terms.
• Calling a logarithmic rate exactly minimax. The rate \((\log n/n)^{2s/(2s+1)}\) has the minimax exponent but includes a logarithmic loss relative to \(n^{-2s/(2s+1)}\).
• Assuming adaptation means estimating smoothness explicitly. Lepski selection and thresholding can adapt through comparisons or coefficient decisions without returning a separate estimate of \(s\).