SURE Threshold Selection for Wavelet Coefficients

Why Select a Threshold by SURE?

A wavelet transform can represent a signal with many coefficients close to zero and a smaller number carrying substantial structure. Soft thresholding shrinks small observed coefficients to zero and reduces the magnitude of larger ones. The threshold controls that trade-off: too small a threshold leaves noise in the estimate, while too large a threshold removes signal along with noise.

The universal threshold gives a useful noise-based benchmark, but a particular data set may benefit from another choice. Stein’s unbiased risk estimate, or SURE, provides a data-dependent criterion for comparing thresholds under Gaussian noise. Its interpretation requires care: SURE estimates expected squared error for each fixed threshold; it is not the realized error, and minimizing it over thresholds does not make the selected estimate’s risk estimate exactly unbiased.

Gaussian Coefficients and Soft Thresholding

Consider \(m\) observed coefficients in the Gaussian sequence model \(Y_i=\theta_i+\varepsilon_i\), where the noise variables are independent and \(\varepsilon_i\sim N(0,\sigma^2)\). In this tutorial, \(\sigma^2\) is assumed known. The target is the coefficient vector \(\theta=(\theta_1,\ldots,\theta_m)\), and squared-error loss is the sum of squared coefficient errors.

If an orthonormal wavelet transform is applied to a signal with Gaussian white noise, the transformed noise coefficients remain Gaussian with the same variance and are independent. Thus, for a specified set of coefficients subject to thresholding, the sequence model applies. By orthonormality, squared error in those coefficients equals squared error for the corresponding reconstructed component. This risk statement concerns the coefficients being estimated; any omitted or unthresholded components require their own treatment.

Soft thresholding has two useful identities. First, \(Y_i-\eta_t(Y_i)\) has magnitude \(\min(|Y_i|,t)\). Second, away from the two points \(y=\pm t\), the derivative of \(\eta_t(y)\) is \(1\) when \(|y|>t\), and \(0\) when \(|y|<t\). The thresholded estimate is continuous and piecewise linear, so these derivatives suffice in the expectation calculations below; under a Gaussian model, the probability of observing exactly either boundary is zero.

Deriving Stein’s Unbiased Risk Estimate

The key identity is Stein’s formula. If \(Y\sim N(\theta,\sigma^2)\) and \(g\) is absolutely continuous with suitable integrability, then \(E_\theta[(Y-\theta)g(Y)]=\sigma^2 E_\theta[g'(Y)]\). For the function used here, \(g(y)=\eta_t(y)-y\), these conditions hold: \(g\) is bounded and piecewise linear, its derivative exists except at finitely many points, and its derivative is bounded. To see the identity in this case, write the normal density as \(f_\theta(y)\); it satisfies \((y-\theta)f_\theta(y)=-\sigma^2 f_\theta'(y)\). Integration by parts on the intervals separated by the breakpoints gives \(E[(Y-\theta)g(Y)]=\sigma^2 E[g'(Y)]\). The boundary terms at infinity vanish because \(g\) is bounded and the normal density tends to zero, and terms at interior breakpoints cancel because \(g\) is continuous.

To prove the claim, first take any coordinatewise estimator \(\delta_i(Y_i)\) for which Stein’s identity applies, and set \(g_i(Y_i)=\delta_i(Y_i)-Y_i\). Expand its total squared error:

Taking expectations, the first sum has expectation \(m\sigma^2\). Applying Stein’s identity to each cross-product gives \(E[(Y_i-\theta_i)g_i(Y_i)]=\sigma^2 E[g_i'(Y_i)]\). Since \(g_i'(Y_i)=\delta_i'(Y_i)-1\) almost surely, the risk becomes

For soft thresholding, \((\eta_t(Y_i)-Y_i)^2=\min(Y_i^2,t^2)\), and its derivative is \(\mathbf{1}\{|Y_i|>t\}\) almost surely. Substitution gives the displayed SURE formula. Its expectation equals \(R(\theta,t)\) for each fixed \(t\), proving unbiasedness. \(\square\)

Using SURE to Choose a Threshold

In practice, one specifies a candidate set \(\mathcal{T}\), often a finite grid of nonnegative thresholds, evaluates SURE at each candidate, and chooses a minimizer. For a finite set, a deterministic tie-breaking rule makes the selected threshold well-defined. In the wavelet setting, the coefficients and the variance \(\sigma^2\) used in the formula must match the coefficient set and noise scale being analyzed. If the variance is replaced by an estimate, the displayed expression is no longer guaranteed to be exactly unbiased under the stated theorem.

Estimated Risk, Realized Error, and Selection

The distinction between risk and error is fundamental. The realized squared error is \(\sum_i(\eta_t(Y_i)-\theta_i)^2\); it depends on the unknown target coefficients and the particular noise realization. The risk \(R(\theta,t)\) is the expectation of that loss over repeated data sets with the same \(\theta\) and fixed \(t\). SURE is a random quantity calculated from the data whose expectation equals this risk for a fixed threshold.

There is a further caution when choosing the threshold from the same observations. Let \(S(t)\) denote SURE and let \(\mathcal{T}=\{t_1,\ldots,t_K\}\) be a finite candidate set. Since \(E[S(t_j)]=R(\theta,t_j)\) for each fixed candidate,

The inequality follows because \(\min_j S(t_j)\leq S(t_k)\) for every \(k\); take expectations and then choose \(k\) with the smallest expected value. Thus the minimum SURE can be optimistic as an estimate of the best candidate’s risk. More importantly, the fixed-threshold unbiasedness proof does not establish unbiasedness for the selected estimator \(\eta_{\hat t(Y)}(Y)\): the choice \(\hat t(Y)\) itself depends on the observations, so the fixed-function application of Stein’s identity used above is not the argument for that data-dependent rule. SURE remains a useful selection criterion, but its selected minimum should not be reported as an exactly unbiased estimate of the selected procedure’s risk.

Common Mistakes

• Confusing total and average risk. The formula estimates total squared error over \(m\) coefficients. Divide by \(m\) only when the desired quantity is mean squared error per coefficient.
• Dropping the divergence term. The indicator contribution \(2\sigma^2\sum_i\mathbf{1}\{|Y_i|>t\}\) is essential; it accounts for the local sensitivity of the estimate to the observations.
• Calling SURE the observed loss. The true coefficients are unknown in ordinary use, so the realized squared error cannot usually be calculated. SURE is a data-based estimate of expected loss.
• Assuming unbiasedness survives minimization. Unbiasedness holds for each fixed threshold. Choosing the smallest observed SURE introduces data-dependent selection, and the selected minimum is generally optimistic.