Adaptation for Linear Functionals Versus Entire Functions

Why the Target of Inference Matters

An estimator can be accurate in one sense and inadequate in another. Estimating one prescribed feature of a regression function asks for uncertainty about a single number; a simultaneous confidence band asks for one statement that covers the function at every location. The distinction matters for adaptation: the first target may be estimable without knowing the function’s smoothness, while the second must control errors across many locations and can face substantially greater testing difficulty.

The comparison is not a rule that every scalar target permits adaptation or every band is impossible. The decisive features include how the target weights the unknown function, how the estimator’s bias depends on smoothness, and whether functions with different target values can be distinguished from the data.

Linear Functionals in a Gaussian Sequence Model

Use the Gaussian sequence model \(Y_j=\theta_j+n^{-1/2}\xi_j\), \(j=1,2,\ldots\), where the \(\xi_j\) are independent \(N(0,1)\) random variables and \(\theta=(\theta_j)\) is an unknown coefficient sequence. For a fixed, known coefficient sequence \(a=(a_j)\), the target is the linear functional \(T_a(\theta)=\sum_{j=1}^{\infty}a_j\theta_j\), whenever this series is well-defined. A familiar example is a function value expressed in a basis: the coefficients \(a_j\) encode how each basis coefficient contributes at the specified location.

A particularly favorable case occurs when \(a\) is square-summable, meaning \(\sum_j a_j^2<\infty\). If \(\theta\) is also square-summable, then \(T_a(\theta)\) is the inner product of two square-summable sequences and is well-defined. The data noise in this functional has finite variance, regardless of the unknown smoothness of \(\theta\).

When Smoothness Controls the Bias

Not every useful functional has square-summable weights. To see how this changes the problem, consider the smoothness class \(\Theta_s(R)=\{\theta:\sum_{j=1}^{\infty}j^{2s}\theta_j^2\le R^2\}\), where \(s>0\) and \(R>0\). For any sequence \(a\) with \(\sum_j a_j^2j^{-2s}<\infty\), the functional is well-defined on this class: Cauchy–Schwarz gives \(\sum_j|a_j\theta_j|\le R(\sum_j a_j^2j^{-2s})^{1/2}<\infty\).

A natural estimator retains only the first \(m\) coordinates: \(\widehat T_m=\sum_{j=1}^{m}a_jY_j\). Its error has two parts. The retained coordinates contribute random noise, while omitted coordinates contribute deterministic truncation bias. The sign of that bias is worth keeping straight: \(\widehat T_m-T_a(\theta)\) has mean \(-\sum_{j>m}a_j\theta_j\), not the positive omitted tail.

To prove the bias bound, write the omitted tail as \(\sum_{j>m}a_j\theta_j=\sum_{j>m}(a_jj^{-s})(j^s\theta_j)\). Cauchy–Schwarz bounds its absolute value by \((\sum_{j>m}a_j^2j^{-2s})^{1/2}(\sum_{j>m}j^{2s}\theta_j^2)^{1/2}\), and the second factor is at most \(R\). The random part of the error is \(n^{-1/2}\sum_{j=1}^m a_j\xi_j\), which is normal with mean zero and variance \(V_m\). Thus, with probability \(1-\alpha\), its absolute value is at most \(z_{1-\alpha/2}\sqrt{V_m}\). On that event, the error is bounded in absolute value by the sum of this noise bound and \(B_m\); this is exactly the stated coverage claim. \(\square\)

The interval is honest over the specified class because the bias allowance holds for every member of that class. But its width depends on \(s\), through the tail bound, and choosing an effective \(m\) may require knowing or learning about smoothness. The balance between \(B_m\) and \(\sqrt{V_m}\) is target-specific: it depends on \(a\), not just on the function class.

Testing Difficulty Limits Interval Width

Smoothness-dependent bias is only one part of the adaptation problem. A second part is whether the data can distinguish parameter values that give different values of the target. If two parameter sequences are statistically difficult to tell apart but their functional values are far apart, an honest interval cannot be short with high probability at both sequences. This is the same basic testing geometry that appears in adaptive-band impossibility arguments, applied here to a specified scalar target.

For the proof, define the event \(A=\{T_a(\theta_0)\in C\}\). Coverage under \(\theta_0\) implies \(\mathbb P_{\theta_0}(A)\ge1-\alpha\). On the event \(A\cap\{|C|<\Delta\}\), the interval cannot also contain \(T_a(\theta_1)\), since the two target values are separated by \(\Delta\). Coverage under \(\theta_1\) therefore implies \(\mathbb P_{\theta_1}(A\cap\{|C|<\Delta\})\le\alpha\). In particular, \(\mathbb P_{\theta_1}(A)\le\alpha+\mathbb P_{\theta_1}(|C|\ge\Delta)\). By the definition of total variation, \(\mathbb P_{\theta_1}(A)\ge\mathbb P_{\theta_0}(A)-\operatorname{TV}(\mathbb P_{\theta_0},\mathbb P_{\theta_1})\). Combining these inequalities and rearranging proves the result. \(\square\)

For finitely supported \(\theta_1-\theta_0=\delta\), the Kullback–Leibler divergence from \(\mathbb P_{\theta_0}\) to \(\mathbb P_{\theta_1}\) is \(n\|\delta\|_2^2/2\). Indeed, the log density ratio is a sum of Gaussian coordinate log density ratios; under \(\theta_0\), each coordinate contributes \(n\delta_j^2/2\) in expectation. Pinsker’s inequality then gives \(\operatorname{TV}\le\sqrt{n\|\delta\|_2^2/4}\). Thus, if \(n\|\delta\|_2^2\) is small while \(|\sum_j a_j\delta_j|\) is appreciable, the two target values are hard to distinguish and the interval must have substantial length with non-negligible probability. Which perturbations create this conflict depends on the functional weights and the parameter classes under consideration.

Why a Whole-Function Band Is a Different Target

A fixed functional asks about one prescribed weighted sum. Even when its weights are not square-summable, a suitable smoothness condition may control its omitted tail, and inference can be designed around that particular bias–variance trade-off. A simultaneous band instead requires coverage at every location. Errors at separate locations need not be independent, but controlling the probability of any failure is inherently a joint problem; pointwise intervals alone do not establish simultaneous coverage.

Common Mistakes and Practical Implications

• Assuming every scalar functional has root-\(n\) uncertainty. That conclusion follows for square-summable weights in the Gaussian sequence model, not for arbitrary weights. When the weights are not square-summable, truncation bias and smoothness can determine the rate.
• Treating a pointwise interval as a band. A collection of intervals with 95% coverage at each fixed location need not have 95% probability of covering the function everywhere.
• Ignoring the sign of the truncation error. For \(\widehat T_m-T_a(\theta)\), the omitted signal enters with a minus sign. The absolute bias bound uses its magnitude, but the signed mean is still negative the omitted tail.
• Claiming exact coverage in a degenerate case. If \(a=0\), the zero-width interval covers with probability one. Exact \(1-\alpha\) coverage from a normal quantile formula applies to a nonzero square-summable \(a\).

The central lesson is to specify the target before discussing adaptation. For a regular fixed functional, finite noise variance can yield an exact interval that needs no smoothness selection. For an irregular functional, smoothness controls the bias, and alternatives that are hard to test can force wider intervals. For an entire function, simultaneous coverage adds a joint error-control requirement. These are distinct inferential tasks, with distinct limits; restrictions on the function class may be important when seeking adaptive bands.