Block Thresholding for Wavelet Estimation

Why Pool Wavelet Coefficients?

Wavelet coefficients at a fixed resolution are often arranged in a natural order, and neighboring coefficients can describe nearby features of a signal. If a feature is present across a region, several coefficients in that region may carry information together, even when no single coefficient is striking on its own. A coordinatewise rule makes a separate keep-or-remove decision for each coefficient; a block rule uses the combined evidence from a group.

Pooling is not automatically better. A block can combine several weak but coherent signals, while it can also dilute the evidence from one isolated large coefficient. The choice of block size and threshold therefore matters. The Gaussian sequence model makes the basic trade-off precise without requiring a full regression derivation.

Definition: In a block of $L$ coefficients, observe $Y_i=\theta_i+\sigma Z_i$, $i=1,\ldots,L$, where $\theta_i$ is the unknown coefficient, $\sigma>0$ is known, and the $Z_i$ are independent $N(0,1)$ variables. Write $Y=(Y_1,\ldots,Y_L)^\mathsf{T}$, $\theta=(\theta_1,\ldots,\theta_L)^\mathsf{T}$, and $\|Y\|^2=\sum_{i=1}^L Y_i^2$. A block thresholding rule uses the block’s observed coefficients jointly to estimate $\theta$.

For an orthonormal wavelet transform with Gaussian white noise, the idealized coefficient model has this form: transformed noise coefficients are independent Gaussian variables with a common variance. In practice, blocks are often formed from neighboring coefficients at the same resolution. One may handle a short final block by allowing its length to be smaller, or by choosing block sizes that divide the number of coefficients at that resolution.

Two Block Rules

The simplest rule compares the total observed energy in a block with a threshold. For a tuning constant $\lambda>0$, the threshold is $\lambda L\sigma^2$. A block with energy below threshold is set entirely to zero; one above threshold is retained unchanged.

Definition: The block hard-thresholding estimator is $$ \hat\theta^{\,H}=Y\,\mathbf{1}\{\|Y\|^2>\lambda L\sigma^2\}. $$ Here $\mathbf{1}\{A\}$ equals one when event $A$ holds and zero otherwise. The rule retains or removes the entire block.

A related rule both tests the block and shrinks it. It uses the same energy comparison, but, when the block passes the threshold, multiplies every coordinate by the same factor. This preserves the direction of $Y$ within the block while reducing its magnitude.

Definition: The positive-part block-shrinkage estimator is $$ \hat\theta^{\,S}=\left(1-\frac{\lambda L\sigma^2}{\|Y\|^2}\right)_+Y, $$ where $a_+=\max(a,0)$. At $Y=0$, define the estimator to be zero. Equivalently, it is zero when $\|Y\|^2\leq\lambda L\sigma^2$, and otherwise it is $Y$ multiplied by $1-\lambda L\sigma^2/\|Y\|^2$.

Both rules use the same block statistic, $\|Y\|^2$, but their actions above threshold differ. Hard thresholding retains the observed vector as is. Positive-part shrinkage retains its direction and reduces its length. The latter is continuous at the threshold boundary, while hard thresholding has a jump there.

Worked Example: A Block Decision From Observed Energy

Take a block of length $L=2$, noise standard deviation $\sigma=1$, and $\lambda=2$. The energy threshold is $\lambda L\sigma^2=4$. Suppose the observed block is $Y=(1.5,1.2)^\mathsf{T}$. Its energy is $$ \|Y\|^2=1.5^2+1.2^2=2.25+1.44=3.69. $$

Because $3.69<4$, block hard thresholding gives $\hat\theta^{\,H}=(0,0)^\mathsf{T}$, and positive-part block shrinkage gives the same result.

Now suppose instead that $Y=(1.8,1.5)^\mathsf{T}$. Its energy is $1.8^2+1.5^2=3.24+2.25=5.49$, which exceeds 4. Hard thresholding returns $(1.8,1.5)^\mathsf{T}$. Positive-part shrinkage multiplies this vector by $$ 1-\frac{4}{5.49}=\frac{1.49}{5.49}\approx 0.2714, $$ so its estimate is approximately $(0.4885,0.4071)^\mathsf{T}$. Both coordinates are treated as one block decision, rather than tested separately.

The Oracle Comparison Behind Pooling

An oracle that knows the true coefficient vector can compare two simple actions: set the whole block to zero, or retain the noisy vector $Y$. This comparison is not an implementable data-driven rule, because it depends on $\theta$. It does, however, show why the total signal energy is the relevant scale for the block decision.

Proof. If the estimate is zero, its squared error is $\|0-\theta\|^2=\|\theta\|^2$, with no randomness. If the estimate is $Y$, then $Y-\theta=\sigma Z$, so $$ E\|Y-\theta\|^2 =E\left[\sigma^2\sum_{i=1}^L Z_i^2\right] =\sigma^2\sum_{i=1}^L E[Z_i^2] =L\sigma^2. $$ Comparing these two risks gives the stated decision. The data-driven threshold uses observed energy as evidence about the unknown signal energy; it is not the oracle decision itself. $\square$

Worked Example: The Oracle Trade-Off

Let $L=4$ and $\sigma=1$. If $\theta=(1,1,1,1)^\mathsf{T}$, setting the block to zero has risk $\|\theta\|^2=4$, while retaining $Y$ has risk $L\sigma^2=4$. The oracle is indifferent between these two actions.

If instead $\theta=(1.5,1.5,1.5,1.5)^\mathsf{T}$, then $\|\theta\|^2=4(1.5^2)=9$, while retaining $Y$ still has risk 4. The oracle prefers retaining the block. In contrast, if $\theta=(0.5,0.5,0.5,0.5)^\mathsf{T}$, the zero estimate has risk $4(0.5^2)=1$, less than the risk 4 of retaining noisy observations. These comparisons explain the signal-versus-noise trade-off; they do not tell us which action to take from the data without uncertainty.

Risk Estimate for Positive-Part Block Shrinkage

For a smooth-enough estimator in the Gaussian sequence model, Stein’s identity gives an unbiased estimate of its squared-error risk. In this setting, the identity applies to an estimator $\delta(Y)$ that is weakly differentiable, has integrable components and derivatives under the Gaussian distribution, and grows slowly enough for the relevant expectations to exist. The block shrinkage rule above satisfies these conditions: it is zero inside the positive-radius threshold and continuous across the boundary, with bounded derivatives on the outside region.

Result: Let $R(\theta,\delta)=E_\theta\|\delta(Y)-\theta\|^2$, and suppose $\delta$ satisfies the regularity conditions for Stein’s identity. Then $$ R(\theta,\delta) =E_\theta\left[\|\delta(Y)-Y\|^2+2\sigma^2\operatorname{div}\delta(Y)-L\sigma^2\right], $$ where $\operatorname{div}\delta(Y)=\sum_{i=1}^L \partial\delta_i(Y)/\partial Y_i$ wherever the derivatives exist.

Derivation. Expand the squared error using $Y$: $$ \|\delta(Y)-\theta\|^2 =\|\delta(Y)-Y\|^2+\|Y-\theta\|^2 +2\{\delta(Y)-Y\}^\mathsf{T}(Y-\theta). $$ The expectation of $\|Y-\theta\|^2$ is $L\sigma^2$. Stein’s identity gives $E_\theta[(Y-\theta)^\mathsf{T}g(Y)]=\sigma^2 E_\theta[\operatorname{div}g(Y)]$ for $g(Y)=\delta(Y)-Y$. Since $\operatorname{div}g=\operatorname{div}\delta-L$, taking expectations in the expansion yields the result. $\square$

For positive-part block shrinkage, write $r^2=\|Y\|^2$ and $t=\lambda L\sigma^2$. When $r^2>t$, the $i$th coordinate is $\delta_i(Y)=(1-t/r^2)Y_i$. Differentiating each coordinate with respect to its matching observation gives $$ \frac{\partial\delta_i}{\partial Y_i} =1-\frac{t}{r^2}+\frac{2tY_i^2}{r^4}. $$ Summing over $i$ gives $\operatorname{div}\delta=L-t(L-2)/r^2$. When $r^2<t$, the estimator and its divergence are zero. Thus an unbiased risk estimate is $$ \|\delta(Y)-Y\|^2+2\sigma^2\operatorname{div}\delta(Y)-L\sigma^2, $$ with the appropriate inside or outside expression. For $r^2>t$, the first term is $t^2/r^2$. At the boundary the derivative need not be specified for this expectation: the boundary has probability zero under a nondegenerate Gaussian distribution. This calculation makes the shrinkage rule’s risk assessable, but an unbiased estimate can still fluctuate substantially for a single observed block.

Worked Example: False Block Retention Under the Null

Consider block hard thresholding with $L=2$, $\sigma=1$, and $\lambda=2$, so the rule retains $Y$ when $Q=\|Y\|^2>4$. Suppose the true block is zero. Then $Q=Z_1^2+Z_2^2$ has a chi-squared distribution with 2 degrees of freedom and density $f_Q(q)=\frac12 e^{-q/2}$ for $q>0$.

The probability that noise alone activates the block is $$ P(Q>4)=\int_4^\infty \frac12 e^{-q/2}\,dq=e^{-2}\approx 0.1353. $$ When the block is activated, its squared estimation error is $Q$, and otherwise the error is zero. Its risk at the zero signal is therefore $$ E[Q\mathbf{1}\{Q>4\}] =\int_4^\infty \frac{q}{2}e^{-q/2}\,dq =6e^{-2}\approx 0.8120. $$ For the integral, the substitution $u=q/2$ gives $2\int_2^\infty ue^{-u}\,du=2(3e^{-2})=6e^{-2}$. The risk is positive despite the true block being zero: the threshold reduces false retention but does not eliminate it.

Pooling Versus Coordinatewise Thresholding

A coordinatewise threshold makes a separate decision for each $Y_i$, often retaining a coefficient when its magnitude exceeds a chosen cutoff. A block rule instead asks whether the combined energy is large enough. Consequently, several moderate observations can jointly pass a block threshold when no individual observation passes a coordinate threshold. The reverse can also happen: one large observation can be diluted by a block rule if its total block energy is below the chosen cutoff.

Worked Example: Shared Evidence and an Isolated Coefficient

Let $L=4$, $\sigma=1$, and $\lambda=2$, so the block threshold is $\lambda L\sigma^2=8$. Compare this with coordinatewise hard thresholding at $\sqrt{2\log 4}\approx1.665$.

For $Y=(1.6,1.5,1.4,1.3)^\mathsf{T}$, the block energy is $$ 1.6^2+1.5^2+1.4^2+1.3^2 =2.56+2.25+1.96+1.69=8.46. $$ Block hard thresholding retains all four observations. Each coordinate is below $1.665$, so coordinatewise hard thresholding removes all four. This is the pooling advantage when moderate evidence is spread across the block.

For $Y=(2.2,0.4,0.3,0.2)^\mathsf{T}$, the energy is $4.84+0.16+0.09+0.04=5.13$, below 8, so block hard thresholding removes the block. Coordinatewise thresholding retains 2.2 and removes the other three observations. This illustrates the cost of pooling when evidence is concentrated in one coefficient. Neither decision is universally preferable; the comparison depends on how signal energy is distributed within blocks.

Practical Choices and Common Pitfalls

The block length determines how much neighboring information is pooled. Short blocks limit the chance that unrelated coefficients are grouped together, but offer less aggregation. Long blocks can gather diffuse evidence, but a strong signal in only part of the block may be diluted. In wavelet estimation, blocks are commonly formed within a resolution level rather than across levels, since coefficients at different scales play different roles.

Threshold calibration also matters. The threshold $\lambda L\sigma^2$ scales with both the noise variance and the number of coefficients. If $\sigma$ is underestimated, blocks will be retained too often; if it is overestimated, real signal blocks will be discarded. The Gaussian calculations here assume known $\sigma$. When noise is estimated, its estimation uncertainty can affect the block decisions as well.

Mistake: treating an energy threshold as a coordinate threshold. The condition is $\sum_iY_i^2>\lambda L\sigma^2$, not $|Y_i|>\lambda L\sigma^2$. The units and scaling differ.
Mistake: assuming every coefficient in a selected block is signal. Selection says the observed block energy passed a rule; it does not establish that each coordinate is nonzero or that every retained observation is accurate.
Mistake: applying a smooth-estimator risk formula blindly to hard thresholding. The block hard-thresholding rule jumps at its boundary. The Stein calculation above was derived for a regular, continuous shrinkage rule, not by ignoring that discontinuity.
Mistake: claiming pooling always beats coordinatewise thresholding. Pooling helps when neighboring coefficients share signal evidence, but can suppress a localized coefficient or combine signal with mostly noise.

Key takeaway: Block thresholding replaces many separate coefficient decisions with a decision based on aggregate energy. Its potential gain comes from pooling weak neighboring evidence; its main risk is that the chosen blocks do not match how signal energy is distributed.

Check Your Understanding

Use the definitions and calculations in this tutorial to answer the following.

For a block with $L=3$, $\sigma=2$, and $\lambda=1.5$, calculate the block hard-threshold energy cutoff.
Explain why the oracle comparison between zero and $Y$ depends on $\|\theta\|^2$ and $L\sigma^2$.
For positive-part block shrinkage, what happens to the estimate when $\|Y\|^2\leq\lambda L\sigma^2$?
Give one situation in which pooling can retain a block that coordinatewise thresholding removes, and one situation in which pooling can discard a coefficient that coordinatewise thresholding retains.
Why does the Stein risk derivation for the continuous block-shrinkage rule not automatically provide an unbiased risk estimate for block hard thresholding?