Adaptive Minimax Rates and Their Costs

Two Losses, Two Benchmarks

A smoothing method must choose how much local detail to retain. If the regression function is rough, a relatively small smoothing bandwidth can be useful; if it is smooth, more smoothing reduces noise without introducing much bias. When smoothness is unknown, one estimator must work across several possibilities. The cost of doing so depends both on the available information and on the loss used to judge the estimate.

We make this comparison in the periodic Gaussian white-noise model on \([0,1]\):

where \(W\) is standard Brownian motion and \(n\) indexes the noise level. Let \(\mathcal H^s(L)\) be a periodic Hölder ball of radius \(L\), with smoothness \(s\) in a fixed interval \(0<s_0\le s\le s_1<\infty\). For clarity, take the interval to avoid integer smoothness values; the integer cases can be defined using the usual Hölder-Zygmund convention. These classes are nested: if \(s_2>s_1\), then, for a common radius convention, \(\mathcal H^{s_2}(L)\subseteq\mathcal H^{s_1}(L)\).

The losses are not interchangeable. Integrated squared error averages error across the domain and is quadratic in its size. Supremum loss records the largest error anywhere and is not squared here. Consequently, their rate exponents differ, and the logarithmic factors associated with the supremum norm need careful interpretation.

Integrated Squared Error: The Smoothness-Specific Rate

For a kernel estimator with bandwidth \(h\), a kernel of sufficient order gives a bias of size \(O(h^s)\) over \(\mathcal H^s(L)\). In the white-noise model, integrated variance is of order \(1/(nh)\). Thus the leading squared-error terms have orders \(h^{2s}\) and \(1/(nh)\). The following calculation makes the balance precise for a compactly supported kernel.

Here is the derivation. Define \(K_h^{\mathrm{per}}(x)=\sum_{j\in\mathbb Z}h^{-1}K((x+j)/h)\), and let \(\hat f_h(x)=\int_0^1K_h^{\mathrm{per}}(x-t)\,dY(t)\). If the support of \(K\) has diameter \(D\) and \(Dh<1\), distinct translated copies do not overlap except possibly at endpoints of measure zero. Changing variables across the translated intervals gives

The noise contribution to \(\hat f_h(x)\) is a centered Gaussian variable with variance \(n^{-1}\int_0^1\{K_h^{\mathrm{per}}(x-t)\}^2dt\). Translation invariance makes this variance \(n^{-1}h^{-1}\int K^2\). Integrating over \(x\) gives the same order for integrated variance. The Hölder bias bound gives \(\|\mathbb E_f\hat f_h-f\|_2^2\le C h^{2s}\). Therefore

To balance the terms, solve \(h^{2s}=(nh)^{-1}\), or \(h^{2s+1}=n^{-1}\). This gives \(h=n^{-1/(2s+1)}\), up to a constant. Substitution yields \(h^{2s}=n^{-2s/(2s+1)}\), and \((nh)^{-1}=n^{-2s/(2s+1)}\) as well. This proves the asserted upper-rate calculation. Standard bump-function lower bounds show that this order is minimax over the Hölder ball, so it is not just a limitation of the kernel estimator.

What Changes Under Supremum Loss?

Under supremum loss, it is not enough to control error at one fixed location. Roughly \(1/h\) locations can carry distinct local fluctuations at bandwidth \(h\). The largest of that many noise fluctuations is typically larger than one fluctuation by a factor of order \(\sqrt{\log(1/h)}\). The elementary Gaussian calculation below exhibits the source of this factor.

For \(t>0\), the Gaussian tail bound and the union bound give \(P(\max_j|Z_j|>t)\le \min\{1,2M e^{-t^2/2}\}\). Put \(t_0=\sqrt{2\log(2M)}\). The identity \(E U=\int_0^\infty P(U>t)\,dt\) for a nonnegative random variable \(U\), followed by the Gaussian tail integral bound \(\int_a^\infty e^{-t^2/2}dt\le e^{-a^2/2}/a\), gives

This proves the bound. A kernel noise process is not a collection of independent normal variables, so the calculation is not itself a proof of a kernel estimator's supremum risk. Rather, it explains the logarithmic scale: maximal inequalities for suitably regular Gaussian processes give the corresponding control over locations. Combining the noise scale \(\sqrt{\log(1/h)/(nh)}\) with a bias of order \(h^s\) leads to the familiar balance.

The replacement of \(\log(1/h)\) by \(\log n\) is valid at this bandwidth up to constant factors: \(\log(1/h)\) is of order \(\log n\) for fixed \(s>0\). Standard lower bounds show that the resulting supremum-norm order is minimax over a Hölder ball even when \(s\) is known. Thus its logarithm is already present in the smoothness-specific benchmark.

Adaptation Across Nested Classes

An estimator is adaptive over a range of smoothness values if it does not know which \(\mathcal H^s(L)\) contains \(f\), yet attains the appropriate rate for each such \(s\). In the Gaussian model above, standard adaptive wavelet minimax results give the following comparison, for fixed \(0<s_0<s_1\) and suitable wavelet regularity:

The integrated-risk conclusion is that adaptation to unknown smoothness need not cost a logarithmic factor: one appropriately constructed wavelet estimator attains the smoothness-specific order across the range. The supremum-risk conclusion must be read relative to its own column. Its adaptive rate contains a logarithm, but the known-smoothness minimax rate contains the same logarithm. In this standard setting there is no additional logarithmic factor merely because \(s\) is unknown.

This distinction is important whenever rates are compared. If the reference is the pointwise error scale \(n^{-s/(2s+1)}\), the supremum rate is slower by a logarithmic factor. That comparison alone does not establish an adaptation penalty: it compares two different loss criteria. For simultaneous control over a continuum of locations, the maximum noise fluctuation must be controlled, even if smoothness is known. The extra factor reflects that simultaneous requirement.

Worked Comparisons

Common Pitfall and Practical Interpretation

Wrong reasoning: “The adaptive supremum rate has a logarithm, so adaptation necessarily caused a logarithmic loss.” This compares the adaptive rate to an inappropriate benchmark. The correction is to first write the known-smoothness minimax rate for the same loss. For supremum loss in this model, the logarithm is already part of that rate. For integrated squared error, adaptation over the stated Hölder range does not require a further logarithmic factor.

The conclusion is specific to estimation risk in this standard model. Different demands can change the answer. For example, requiring an uncertainty band to cover the entire function with a prescribed probability is not the same problem as estimating it under expected loss; honesty and width impose additional constraints. Likewise, assumptions about the design, noise, boundaries, or function classes matter when transferring a rate statement. Always specify the loss, the function class, and whether the benchmark assumes smoothness is known.