Why Adaptation Can Fail for Confidence Bands
An estimator can adapt its smoothing to the unknown regularity of a function and achieve a favorable estimation error over several smoothness classes. A confidence band has a harder task: it must cover the whole unknown function with a prescribed probability, uniformly over the functions under consideration. If rough functions can be statistically difficult to distinguish from a smooth function, a band cannot always be both honest over the rough class and as narrow as the smooth-class benchmark.
The obstruction is not that confidence bands are inherently impossible, nor that every pair of nested classes creates a contradiction. It is that, under a particular indistinguishability condition, a band that is narrow at a smooth function cannot also cover nearby-in-distribution rough alternatives. The condition makes the source of the conflict precise without asserting that it holds for every model or every choice of smoothness classes.
Confidence Bands, Honesty, and Width
Consider observations \(Y\) from a statistical model indexed by an unknown regression function \(f\) on \([0,1]\). Write \(P_f\) for the distribution of the observations when the function is \(f\). A confidence band is specified by random lower and upper functions \(L_n(x)\) and \(U_n(x)\), with \(L_n(x)\le U_n(x)\). It covers \(f\) if it contains the function at every point, not merely at a selected point.
The global width measures the largest vertical thickness of the band. Thus, if one band contains two functions that differ by at least \(\rho\) somewhere, its global width must be at least \(\rho\). The smooth-class width target considered below is a uniform expected-width bound: for a sequence \(r_n\) and constant \(C\), \(\sup_{f\in\mathcal F_{\mathrm{smooth}}}E_f[W_n]\le Cr_n\). This is stronger than requiring a small width only at a single favorable function.
A common setting uses nested Hölder classes. For \(0<\beta\le1\), one version is \(\mathcal H(\beta,L)=\{f:\sup_x|f(x)|\le L,\ |f(x)-f(y)|\le L|x-y|^\beta\text{ for all }x,y\in[0,1]\}\). When \(0<\beta_1<\beta_2\le1\), \(\mathcal H(\beta_2,L)\subseteq\mathcal H(\beta_1,L)\), since \(|x-y|^{\beta_2}\le |x-y|^{\beta_1}\) on this domain. Call the larger, less regular class \(\mathcal F_{\mathrm{rough}}\), and the nested class \(\mathcal F_{\mathrm{smooth}}\).
Worked Example: A Function in the Rough Class but Not the Smooth Class
Take \(\beta_1=1/2\), \(\beta_2=1\), and \(f(x)=\sqrt{x}\) on \([0,1]\). The function is bounded by 1. For \(0\le y\le x\le1\),
because \(x+y-2\sqrt{xy}\le x-y\) is equivalent to \(y\le\sqrt{xy}\), which holds when \(x\ge y\). Therefore \(|f(x)-f(y)|\le |x-y|^{1/2}\), so \(f\in\mathcal H(1/2,1)\). But \(f\) is not Lipschitz: with \(y=0\) and \(x=1/100\), the ratio \(|f(x)-f(y)|/|x-y|=0.1/0.01=10\), and this ratio grows without bound as \(x\) decreases to zero. Thus \(f\notin\mathcal H(1,1)\). Nesting permits rough functions that do not belong to the smoother class; the impossibility argument will require something more, namely alternatives that are hard to distinguish statistically from a smooth function.
An Indistinguishability Criterion
Fix a smooth function \(f_0\in\mathcal F_{\mathrm{smooth}}\). For each sample size \(n\), let \(\mathcal A_n\subseteq\mathcal F_{\mathrm{rough}}\) be a set of rough alternatives, all separated from \(f_0\) in supremum norm: \(\inf_{g\in\mathcal A_n}\|g-f_0\|_\infty\ge\rho_n\), where \(\|h\|_\infty=\sup_{x\in[0,1]}|h(x)|\). Suppose a probability distribution \(\pi_n\) is placed on these alternatives. The induced mixture distribution of the data is \(Q_n=\int P_g\,d\pi_n(g)\). The relevant condition is that this mixture is close to the data distribution under \(f_0\), even though each alternative is separated from \(f_0\).
Here \(\operatorname{TV}(Q,P)=\sup_A|Q(A)-P(A)|\), the supremum being over measurable data events \(A\). The theorem uses only the mixture's closeness to the smooth model; it does not require each individual alternative distribution to be close to \(P_{f_0}\).
Proof. Let \(B_n\) be the event that the band intersects the alternative set \(\mathcal A_n\). Under data generated by any \(g\in\mathcal A_n\), honesty gives \(P_g(g\in C_n)\ge1-\alpha-o(1)\), uniformly over the rough class. Since \(g\in\mathcal A_n\) implies that \(C_n\) intersects \(\mathcal A_n\), averaging this coverage bound over \(\pi_n\) gives \(Q_n(B_n)\ge1-\alpha-o(1)\). By the definition of total variation distance, \(|Q_n(B_n)-P_{f_0}(B_n)|\le\operatorname{TV}(Q_n,P_{f_0})\). Consequently, \(P_{f_0}(B_n)\ge1-\alpha-o(1)-\operatorname{TV}(Q_n,P_{f_0})\).
Honesty also gives \(P_{f_0}(f_0\in C_n)\ge1-\alpha-o(1)\), because \(f_0\) belongs to the rough class. The probability that both the band contains \(f_0\) and intersects \(\mathcal A_n\) is therefore at least \(1-2\alpha-o(1)-\operatorname{TV}(Q_n,P_{f_0})\), by the union bound applied to the two complementary events. On this intersection, the band contains \(f_0\) and some \(g\in\mathcal A_n\). Since \(\|g-f_0\|_\infty\ge\rho_n\), there is a point where their values differ by at least \(\rho_n\), so \(W_n\ge\rho_n\).
On the other hand, Markov's inequality and the smooth-class width bound give \(P_{f_0}(W_n\ge\rho_n)\le E_{f_0}[W_n]/\rho_n\le Cr_n/\rho_n\to0\). The probability of the intersection just obtained is at most this vanishing probability, but its lower bound tends to \(1-2\alpha>0\). This contradiction proves the claim. \(\square\)
Worked Example: Checking the Contradiction Numerically
Suppose \(\alpha=0.05\), the total variation distance is at most \(0.02\), and the expected width at \(f_0\) is at most \(0.01\rho_n\). Ignoring vanishing asymptotic errors, the probability under \(P_{f_0}\) that the band both covers \(f_0\) and intersects the alternatives must be at least
But the event forces \(W_n\ge\rho_n\). Markov's inequality bounds its probability by \(P_{f_0}(W_n\ge\rho_n)\le E_{f_0}[W_n]/\rho_n\le0.01\). The lower bound \(0.88\) and upper bound \(0.01\) cannot both be true. This calculation illustrates the contradiction; in an asymptotic application, the total variation distance and the ratio \(r_n/\rho_n\) must tend to zero.
How the Rate Comparison Enters
For orientation, in standard regular nonparametric regression settings, the usual benchmark order for uniform estimation or band width over a Hölder class of smoothness \(\beta\) is often \(r_\beta=(\log n/n)^{\beta/(2\beta+1)}\), subject to model and design conditions. Greater smoothness gives a narrower benchmark. This rate comparison alone does not prove impossibility: the theorem also needs rough alternatives separated by \(\rho_n\) whose mixture distribution is close to that at a smooth function, with \(r_n/\rho_n\to0\).
Worked Example: Comparing Two Benchmark Widths
Take roughness \(\beta_1=1/2\) and smoothness \(\beta_2=1\). Their benchmark orders are \(r_{\mathrm{rough}}=(\log n/n)^{1/4}\) and \(r_{\mathrm{smooth}}=(\log n/n)^{1/3}\). Their ratio is
At \(n=10^{12}\), \(\log n\) is about \(27.63\), so this ratio is approximately \((10^{12}/27.63)^{1/12}\), or \(7.6\). Thus the smooth benchmark is substantially narrower at this sample size, and the gap grows asymptotically. This comparison explains what an adaptation goal might demand, but it does not establish that a mixture of rough alternatives is indistinguishable from a smooth function. That separate condition is essential to the theorem.
Interpretation and Scope
The theorem identifies a precise conflict. Uniform honesty over the rough class requires coverage even for alternatives that the data cannot reliably distinguish, in aggregate, from a smooth function. At the smooth function, honest coverage also requires the band to include that function. If a narrow band were to do both, it would have to contain two functions separated by at least \(\rho_n\), forcing its width to be large with nonvanishing probability. A smooth-class expected width that is negligible relative to \(\rho_n\) rules that out.
This is an impossibility under stated conditions, not a universal prohibition on adaptive uncertainty quantification. If the mixture is not close to the smooth model, if the alternatives are not sufficiently separated, or if the target width is not smaller than that separation, this argument gives no contradiction. Additional restrictions on the function classes or the model can also change the problem. The central point is that favorable estimation adaptation does not by itself guarantee honest adaptation for bands covering an entire function.
- Mistake: “The smooth class is nested inside the rough class, so an honest band can simply adapt its width to whichever function generated the data.” Nesting alone does not make the smoothness identifiable from the data.
- Correction: Check whether rough alternatives can be separated in function distance while remaining difficult to distinguish statistically from a smooth function. When they can, the theorem shows why a uniformly honest band may have to be wider than the smooth-class target.
Check Your Understanding
Use the theorem's conditions and the distinction between rate comparison and indistinguishability to answer these questions.
- Why does a band containing both \(f_0\) and an alternative \(g\) with \(\|g-f_0\|_\infty\ge\rho_n\) have width at least \(\rho_n\)?
- In the theorem, why is closeness of the mixture \(Q_n\) to \(P_{f_0}\) used instead of requiring every \(P_g\) to be close to \(P_{f_0}\)?
- What role does the condition \(\alpha<1/2\) play in the proof?
- Why does comparing rough- and smooth-class benchmark rates not, by itself, prove that honest adaptive bands are impossible?
- If \(r_n/\rho_n\) does not tend to zero, which step in the contradiction may fail?