Risk Bounds for Hard and Soft Thresholding

Why Separate the Sources of Thresholding Error?

Thresholding estimates many coefficients by setting small noisy observations to zero. This can remove noise when a true coefficient is zero, but it can also discard a real signal. If a coefficient survives thresholding, its estimate may still be inaccurate because of the noise; soft thresholding adds shrinkage even when the signal is retained. A useful risk calculation therefore distinguishes noise-driven error at zero coefficients from shrinkage and missed-signal error at nonzero coefficients.

We work coordinate by coordinate in the Gaussian sequence model. A vector estimate’s total squared-error risk is the sum of its coordinatewise risks, by additivity of expectation; independence between coordinates is not required for that identity. The calculations below help identify which part of the risk is controlled by thresholding and which part is paid for by shrinking or missing signals.

Hard Thresholding: Exact Risk and Its Two Components

Hard thresholding either keeps \(Y\) unchanged or replaces it by zero. Let \(A=\{|Y|>\lambda\}\) be the selection event. On \(A\), the estimation error is \(Y-\theta=\sigma Z\); on \(A^c\), it is \(-\theta\). Thus the exact risk decomposition is

The first term is the squared signal lost when the coefficient is thresholded to zero. The second is the noise error on the event that the coefficient is retained. At a zero coefficient the first term vanishes, and the remaining risk is exactly the cost of false inclusion.

To obtain this formula, use symmetry and the standard normal tail-moment identity \(\int_t^\infty z^2\phi(z)\,dz=t\phi(t)+\bar\Phi(t)\). This identity follows by integration by parts, since \(\phi'(z)=-z\phi(z)\).

Hard-Thresholding Bounds for Nonzero Signals

For a nonzero coefficient, the exact decomposition immediately gives useful bounds. If \(|\theta|>\lambda\), thresholding can miss the signal only when the noise moves \(Y\) toward or across the threshold. By symmetry, the probability of this event is at most \(\Phi((\lambda-|\theta|)/\sigma)\). Also, the selected-noise contribution is at most \(E[\sigma^2Z^2]=\sigma^2\). Therefore

For \(\theta>0\), the event \(|Y|\leq\lambda\) implies \(Z\leq(\lambda-\theta)/\sigma\), so its probability is at most that normal lower-tail probability. For \(\theta<0\), apply the same argument to \(-Y\). This proves the claimed miss-probability bound. When \(|\theta|\leq\lambda\), the general decomposition still applies, but the signal is not separated from the threshold by a positive margin; a simple bound is \(R_H(\theta)\leq\theta^2+\sigma^2\).

Soft Thresholding: Shrinkage and a Uniform Risk Bound

Soft thresholding also sets observations inside \([-\lambda,\lambda]\) to zero, but outside that interval it moves the observation \(\lambda\) units toward zero. Define \(A=\{|Y|>\lambda\}\). On \(A^c\), the squared error is \(\theta^2\). On \(A\), the error is \(\sigma Z-\lambda\operatorname{sgn}(Y)\). This gives the exact decomposition

For \(\theta=0\), symmetry and integration of \((z-\lambda/\sigma)^2\) over the upper tail give the exact expression below. For nonzero \(\theta\), the formula separates the squared error from rejecting the coefficient and the error—including shrinkage—when it is retained.

Proof. Write \(\delta(y)=\eta_S(y;\lambda)\). The function \(\delta\) is absolutely continuous, has at most linear growth, and has derivative \(\delta'(y)=\mathbf{1}\{|y|>\lambda\}\) except at the two threshold points, which have probability zero under a Gaussian distribution. Expanding the squared error and applying integration by parts to the Gaussian density gives

For the second equality, Gaussian integration by parts states \(E[(Y-\theta)g(Y)]=\sigma^2E[g'(Y)]\) for absolutely continuous \(g\) of suitable integrability. Apply it to \(g(y)=\delta(y)-y\), whose derivative is bounded almost everywhere. Since \(\delta(y)-y\) is the amount removed from \(y\), its magnitude is at most \(\lambda\). Also \(E[\delta'(Y)]\leq1\). Consequently,

All expectations in this calculation are finite because \(\delta\) has at most linear growth and \(Y\) is Gaussian. This proves the bound. \(\square\)

Comparing the Rules on a Strong Signal

For a signal well above the threshold, hard thresholding has no systematic shrinkage when it selects the coefficient; its main extra risk is the rare event that noise pushes the observation below threshold. Soft thresholding avoids a discontinuous jump at the threshold, but on a strong positive signal its estimate is approximately \(Y-\lambda\), so it incurs a shrinkage error of approximately \(\lambda^2\) in addition to noise variance. The following example shows both effects through the exact event decomposition.

Common Pitfall and Practical Interpretation

A common mistake is to count only the probability that a coefficient is selected and call that the risk at a zero coefficient. Risk is expected squared error, not selection probability. At \(\theta=0\), hard thresholding’s risk is \(E[Y^2\mathbf{1}\{|Y|>\lambda\}]\), which weights a false inclusion by its squared magnitude. Soft thresholding instead weights the selected observation by \((|Y|-\lambda)^2\). Confusing these quantities can badly misstate the cost of thresholding.

For nonzero coefficients, it is equally important not to treat the thresholded estimate as if selection were certain. The exact decompositions include a missed-signal term \(\theta^2P(|Y|\leq\lambda)\); omitting it is particularly serious when \(|\theta|\) is near or below \(\lambda\). Conversely, a signal far above threshold has a small miss probability, and the main difference between the rules is then hard thresholding’s retained noise versus soft thresholding’s shrinkage. These coordinatewise bounds can be summed across a sequence to study total squared-error risk.