Spatial Adaptation Through Wavelet Thresholding

Why Spatial Adaptation Matters

A signal may be nearly flat over most of its domain but change abruptly in a few small regions. A single global smoothing level faces a trade-off: preserving a sharp change may leave excess noise throughout smooth stretches, while suppressing noise everywhere may blur the change. Wavelets offer a way to make this decision locally. Their coefficients are indexed both by scale and by location, so a thresholding rule can retain prominent coefficients near a discontinuity while discarding weak coefficients elsewhere.

The key is not that wavelets automatically identify every feature perfectly. It is that localization makes the coefficient pattern informative: smooth stretches tend to produce small coefficients, while supports intersecting a jump can produce comparatively large coefficients. We make this connection precise in a finite Gaussian sequence model and then relate it to integrated squared-error risk.

Wavelet Coefficients and Localized Structure

At a finer scale \(j\), wavelets have narrower supports and there are more possible locations \(k\). If the wavelets have compact support and \(r\) vanishing moments, then a wavelet coefficient on an interval where \(f\) is \(s\)-times smooth is small when \(r>s\). To see the source of this property, expand \(f\) locally as a polynomial \(p\) of degree at most \(r-1\), plus a remainder. The vanishing moments give \(\int p(x)\psi_{jk}(x)\,dx=0\); only the remainder contributes. For \(f\) with \(s\) derivatives and a suitable Taylor remainder bound, this yields \(|\theta_{jk}|\le C2^{-j(s+1/2)}\). The constant depends on the smoothness bound and wavelet normalization.

A discontinuity has a different signature. At a given scale, only a bounded number of compactly supported wavelets have supports crossing a particular jump. For a bounded jump, the integral over each such support is at most a constant times its width \(2^{-j}\), multiplied by the wavelet amplitude \(2^{j/2}\). Thus its coefficient is at most \(C2^{-j/2}\). The jump can therefore generate appreciable coefficients at locations near itself, rather than throughout the domain. The exact size also depends on where the jump falls within each support.

Thresholding Each Location

Suppose a finite collection of wavelet coefficients is observed with independent Gaussian noise:

Here \(\theta_{jk}\) is the true coefficient, \(\tau^2\) is the noise variance, and the index set \(I\) contains the coefficients being estimated. For a threshold \(\lambda>0\), hard thresholding retains an observed coefficient only when its magnitude is strictly greater than \(\lambda\).

The strict inequality is part of the rule: if \(|Y_{jk}|=\lambda\), the coefficient is discarded. Thresholding is spatially adaptive in the limited but useful sense that this same rule acts separately on every scale-location coefficient. It does not use the coefficient's location directly; localization makes its observed magnitude responsive to local signal structure.

A Coefficientwise Risk Bound

The following bound formalizes the compromise made by thresholding. For a coefficient with true value \(\theta\), the benchmark \(\min(\theta^2,\lambda^2)\) is small when the coefficient is small (discarding it incurs little squared bias) and is \(\lambda^2\) when it is large (retaining it incurs noise variance on the threshold scale).

The remainder is a Gaussian tail term. If \(a=\lambda/(2\tau)\), direct integration of the normal density gives

where \(\phi\) and \(\Phi\) are the standard normal density and distribution function. In particular, this term decreases rapidly as \(\lambda/\tau\) grows.

Proof. First consider \(|\theta|\le\lambda/2\). If the observation is discarded, the squared error is \(\theta^2\). If it is retained, then \(|\varepsilon|=|Y-\theta|>\lambda-|\theta|\ge\lambda/2\), and the squared error is \(\varepsilon^2\). Consequently,

In this case \(\min(\theta^2,\lambda^2)=\theta^2\), so the stated bound follows.

Next suppose \(\lambda/2<|\theta|\le2\lambda\). The squared error is at most \(\theta^2\) when the observation is discarded and equals \(\varepsilon^2\) when it is retained. Therefore its expectation is at most \(\theta^2+E[\varepsilon^2]\le4\lambda^2+\tau^2\le5\lambda^2\). Also \(\min(\theta^2,\lambda^2)\ge\lambda^2/4\), so this risk is at most \(20\min(\theta^2,\lambda^2)\).

Finally suppose \(|\theta|>2\lambda\). On retention, the squared error is \(\varepsilon^2\). Discarding implies \(|\varepsilon|\ge|\theta|-\lambda>|\theta|/2\). The standard Gaussian tail inequality \(P(|\varepsilon|\ge t)\le2\exp(-t^2/(2\tau^2))\) then gives

The penultimate inequality follows by maximizing \(x e^{-x/8}\) over \(x\ge0\), whose maximum is \(8/e\), with \(x=\theta^2/\tau^2\). Here \(\min(\theta^2,\lambda^2)=\lambda^2\), so the desired bound holds. Adding the per-coefficient inequalities proves the total bound. \(\square\)

From Coefficient Risk to Integrated Risk

Orthonormality translates coefficient errors into integrated squared error, but only for the coefficients represented in the reconstruction. If \(\theta_{jk}=\langle f,\psi_{jk}\rangle\), and \(\hat f_I\) includes a finite set \(I\), Parseval's identity gives the exact decomposition

when the wavelet system is a complete orthonormal basis and the indicated sums include all detail and scaling coefficients. The second sum is omitted signal energy; it does not vanish merely because the represented coefficients are estimated without error. Taking expectations and applying the theorem bounds the first sum, while the second remains the truncation or approximation error.

How the Bound Expresses Spatial Adaptation

For a piecewise-smooth function, the coefficientwise sum in the risk bound reflects the signal's uneven structure. In smooth stretches, wavelet coefficients decay with scale; many eventually fall below the threshold, and their contribution to the benchmark is their squared size. Near a jump, a small number of coefficients per scale can remain relatively large; their benchmark contribution is at most \(\lambda^2\) each. The threshold therefore does not have to choose one global smoothing strength that treats every location identically.

This is an oracle-style description of risk, not a guarantee that the threshold is optimal for every signal or that its uncertainty statements are automatically valid. The threshold level still matters, and the bound includes a noise-tail remainder. Choosing a threshold to control many noisy coefficients often introduces a logarithmic factor through the number of coefficients considered. The useful point here is the spatial structure: local coefficients allow the estimator to respond to a jump without requiring the same behavior in smooth regions.

Common Mistake

• Counting only the errors of represented coefficients. Orthonormality equates the represented component's squared error with the sum of represented coefficient errors. For full integrated error, omitted coefficients also contribute their squared energy.
• Changing the threshold rule at equality. With the strict rule \(|Y|>\lambda\), an observation exactly at \(\lambda\) is discarded. A proof must respect that convention; it cannot silently treat equality as retention.
• Assuming a jump makes every coefficient large. Wavelet localization means only supports intersecting the jump are directly affected. At a particular scale, the coefficient also depends on the jump's position within the support.

For the strict rule, the risk bound above remains valid even at equality: the proof uses separate signal-size cases and the actual discard/retain definition. A stronger good-noise inequality should not be asserted at a boundary unless that boundary case has been checked under the stated convention.