Adaptive Wavelet Rates Across Besov Smoothness

Why Thresholding Can Adapt to Smoothness

A fixed-resolution wavelet estimator must choose its finest retained scale in advance. That choice depends on smoothness: a rough function needs resolution to represent its detail, while a smooth function benefits from suppressing noisy fine-scale coefficients. Thresholding offers a different strategy. It keeps coefficients whose observed magnitudes are large relative to the noise level and discards the rest, without requiring the smoothness index to be supplied to the estimator.

The risk calculation has two parts. At retained scales, thresholding limits the cost of many small, noisy coefficients; at scales beyond the useful resolution, the Besov condition bounds the signal energy left out. Balancing those terms gives an upper rate. For the universal threshold considered here, the bound contains a logarithmic factor, so it is important to state the rate precisely rather than describe it as the log-free minimax rate.

Model, Threshold, and Besov Condition

Work with an orthonormal wavelet expansion on a one-dimensional compact interval, with boundary-corrected wavelets where needed. Write \(\theta_{jk}\) for the true coefficient at resolution \(j\) and location \(k\), and suppose its estimate satisfies the Gaussian sequence model below. At each resolution there are at most \(C2^j\) coefficients, for a constant \(C\) independent of \(j\) and \(n\).

For the risk proof, Gaussianity and the stated marginal variance are enough; independence among the coefficient errors is not required. In a regression application, this model is an assumption on the coefficient estimates (or a justified approximation to them). The noise scale \(\sigma\) is taken as known for this result.

The restriction \(p\ge2\) is useful because it turns the Besov condition directly into an \(L^2\) bound at each scale. Indeed, for a vector with at most \(C2^j\) entries, \(\|a\|_2\le C'2^{j(1/2-1/p)}\|a\|_p\). Since each scale term in the Besov condition is at most \(M\), this gives \(\|\theta_j\|_2\le C'M2^{-js}\). This is the coefficient decay that controls the omitted fine-scale signal.

Take \(J_n\) such that \(2^{J_n}\) is of order \(n\), and use a threshold \(\lambda_n=\sqrt{32\sigma^2\log n/n}\) for \(n\ge3\). The particular constant is chosen generously to make the aggregate Gaussian tail contribution negligible. Other sufficiently large constants give the same rate.

A Threshold Risk Bound and the Adaptive Rate

The scalar calculation behind the result is that thresholding a noisy coefficient costs, up to constants, the smaller of its squared signal size and the squared threshold. For a Gaussian error, the rare events where noise carries a small coefficient past the threshold contribute an additional exponentially small term.

To verify the bound, split into three ranges. If \(|\theta|\le\lambda/2\), a nonzero output requires \(|\varepsilon|>\lambda/2\). On this event the squared error is at most \(2\theta^2+2\varepsilon^2\); Gaussian tail bounds for both \(P(|\varepsilon|>\lambda/2)\) and \(E[\varepsilon^2\mathbf 1\{|\varepsilon|>\lambda/2\}]\) give the stated exponential remainder, with the \(\theta^2\) contribution bounded by \(C\theta^2\). If \(\lambda/2<|\theta|\le2\lambda\), then \(\min(\theta^2,\lambda^2)\ge\lambda^2/4\), and the risk is at most \(2\theta^2+2E[\varepsilon^2]\le 8\lambda^2+2v\), which is bounded by a constant times that minimum. Finally, if \(|\theta|>2\lambda\), failure to retain the coefficient implies \(|\varepsilon|\ge|\theta|-\lambda>|\theta|/2\); Gaussian tails bound the squared error on that event, while on retention the error is \(\varepsilon\). The resulting risk is at most \(C\lambda^2\). These cases prove the inequality.

Proof. Orthonormality makes integrated squared error the sum of squared coefficient errors, including the coarse-scale part. The coarse-scale part has a fixed number of coefficients and contributes \(O(1/n)\). At any wavelet coefficient, apply the lemma with \(v=\sigma^2/n\). Since \(\lambda_n^2/(16v)=2\log n\), the exponential remainder is \(Cv n^{-2}\). There are \(O(n)\) coefficients through \(J_n\), so their total remainder is \(O(nv n^{-2})=O(n^{-2})\).

For a scale \(j\), summing the main term in the lemma over locations gives at most \(C\min(\|\theta_j\|_2^2,2^j\lambda_n^2)\): the first bound sums the squared coefficients, and the second uses the number of coefficients. Let \(j_s\) be an integer within a fixed distance of \(\log_2\{(n/\log n)^{1/(2s+1)}\}\). For \(j\le j_s\), use the threshold part and sum the geometric series: \(\sum_{j\le j_s}2^j\lambda_n^2\le C\lambda_n^2 2^{j_s}\). For \(j>j_s\), use the Besov coefficient decay proved above: \(\sum_{j>j_s}\|\theta_j\|_2^2\le C M^2 2^{-2sj_s}\). The choice of \(j_s\) balances these terms because \(2^{j_s}\) is of order \((n/\log n)^{1/(2s+1)}\). In particular, \[ \lambda_n^2 2^{j_s} \asymp \frac{\log n}{n}\left(\frac{n}{\log n}\right)^{1/(2s+1)} = \left(\frac{\log n}{n}\right)^{2s/(2s+1)}, \] and \(2^{-2sj_s}\) has the same order. The omitted scales above \(J_n\) have energy at most \(CM^2 2^{-2sJ_n}=O(n^{-2s})\), which is no larger than the displayed rate for sufficiently large \(n\). Combining the scale bounds, the coarse-scale risk, and the negligible Gaussian remainder proves the theorem. \(\square\)

What the Rate Does—and Does Not—Say

The useful feature is that one threshold rule gives an upper bound at multiple smoothness levels without inserting \(s\) into the estimator. The rate is slower than \(n^{-2s/(2s+1)}\) by a logarithmic factor: this proof uses a universal threshold large enough to control the aggregate noise contribution across many coefficients. Its conclusion is an adaptive upper bound, not a claim that this logarithmic factor is inevitable for every possible estimator.

The integrability condition matters. When \(p\ge2\), the coefficient norm inequality directly yields the \(L^2\) tail bound used in the proof. For \(p<2\), that inequality does not give the same scale-wise conclusion, and sparse coefficient patterns require a different risk analysis. Nor does the theorem automatically cover arbitrary regression designs or non-Gaussian errors: those settings must first justify an appropriate coefficient-noise bound.

Common Mistake

• Dropping the logarithm when stating the result. Balancing \(2^j\lambda_n^2\) against \(2^{-2js}\) gives \((\log n/n)^{2s/(2s+1)}\), because \(\lambda_n^2\) is proportional to \(\log n/n\). Replacing it by \(n^{-2s/(2s+1)}\) does not follow from this threshold argument.
• Assuming the result covers every Besov integrability index. The proof uses \(p\ge2\) to convert the Besov condition into an \(L^2\) tail bound. For \(p<2\), that step needs a different argument.
• Calling the bound an exact risk. The theorem is an inequality with constants depending on the class and model. A rate factor or a threshold value alone does not determine the estimator's exact risk.