Hard and Soft Thresholding of Covariance Entries

Why Threshold Covariance Entries?

When the number of variables is large relative to the sample size, a sample covariance matrix can contain many noisy off-diagonal estimates. If the underlying covariance structure is sparse, many population covariances are zero or small enough to be treated as negligible. Entrywise thresholding makes that sparsity explicit: it removes some off-diagonal estimates and, depending on the rule, shrinks others toward zero.

Let \(X_1,\ldots,X_n\) be independent and identically distributed random vectors in \(\mathbb{R}^p\), with finite second moments and covariance matrix \(\Sigma=(\sigma_{jk})\). Write \(\bar X=n^{-1}\sum_{i=1}^n X_i\), and use the sample covariance matrix

The rules below act on each off-diagonal entry \(S_{jk}\), \(j\ne k\). They leave the diagonal entries unchanged. This convention preserves the estimated marginal variances and, when the same rule is used on both sides of the diagonal, preserves symmetry.

At \(|x|=\lambda\), hard thresholding sets the entry to zero under the strict inequality in its definition. Soft thresholding also sets it to zero. For \(|x|>\lambda\), hard thresholding retains the observed value exactly, whereas soft thresholding reduces its magnitude by \(\lambda\).

Two Scalar Optimization Derivations

Both rules can be derived by treating one observed entry \(s\) as a noisy estimate of a target value \(z\). The squared-error term \(\tfrac12(z-s)^2\) favors staying close to the observation. Different penalties produce the two thresholding rules.

Hard thresholding derivation. If \(z=0\), the objective is \(s^2/2\). Among all nonzero \(z\), the squared-error term is minimized at \(z=s\), giving objective \(\lambda^2/2\) when \(s\ne0\). Thus the best nonzero choice is preferable exactly when \(s^2>\lambda^2\), or \(|s|>\lambda\). When \(|s|=\lambda\), zero and \(s\) tie, and the stated convention selects zero. If \(s=0\), zero is also the minimizer. These cases give \(H_\lambda(s)\).

Soft thresholding derivation. For \(z>0\), differentiating the objective gives \(z-s+\lambda\); a stationary point in that region is \(z=s-\lambda\), which is valid when \(s>\lambda\). For \(z<0\), the derivative is \(z-s-\lambda\); the stationary point \(z=s+\lambda\) is valid when \(s<-\lambda\). If \(-\lambda\le s\le\lambda\), the subgradient at \(z=0\) contains zero: it is \(-s+\lambda[-1,1]\). The objective is strictly convex because of its quadratic term, so these candidates give its unique minimizer. Together they are exactly \(T_\lambda(s)\). \(\square\)

Bias and Variance Under Gaussian Entry Noise

To compare repeated-sample behavior, consider a scalar model for one off-diagonal entry: \(Y=\theta+\varepsilon\), where \(\theta\) is the population covariance entry and \(\varepsilon\sim N(0,\tau^2)\). This is a marginal noise model for an entry; it does not assert that all sample covariance entries are mutually independent. Write \(\phi\) and \(\Phi\) for the standard normal density and distribution function, and define \(a=(\lambda-\theta)/\tau\), \(b=(-\lambda-\theta)/\tau\), assuming \(\tau>0\).

To verify these expressions, note that \(H_\lambda(Y)=Y\) on \(Y>\lambda\) and on \(Y<-\lambda\), and is zero otherwise. With \(Z=(Y-\theta)/\tau\), the two retained regions are \(Z>a\) and \(Z<b\). The standard normal tail identities \(E[Z\mathbf{1}\{Z>a\}]=\phi(a)\) and \(E[Z^2\mathbf{1}\{Z>a\}]=1-\Phi(a)+a\phi(a)\), with corresponding lower-tail identities, follow by integrating the normal density and using \(\phi'(z)=-z\phi(z)\). Applying them to \(Y=\theta+\tau Z\) gives the displayed first and second moments. Subtracting the square of the mean from the second moment gives the variance. \(\square\)

For soft thresholding, the mean can be written in the same tail notation:

Indeed, on \(Y>\lambda\) the estimate is \(Y-\lambda\), and on \(Y<-\lambda\) it is \(Y+\lambda\); elsewhere it is zero. The final term is the expected signed subtraction on those two tails. Its variance is \(E[T_\lambda(Y)^2]-m_S^2\), where

Small and Large Entries: What Changes?

At a truly zero entry, symmetry gives zero expected estimate for both rules, so both have zero bias at that exact parameter value. They do not have zero variance: repeated noise can push an observed entry beyond the threshold. Soft thresholding has a useful pointwise comparison at this null value. For every observation \(y\), \(|T_\lambda(y)|\le |H_\lambda(y)|\), so its squared estimate is no larger than the hard-thresholded squared estimate. Both estimates have mean zero when \(\theta=0\); therefore this pointwise inequality also proves that soft thresholding has no greater variance than hard thresholding at \(\theta=0\).

For small but nonzero \(\theta\), the estimates need not remain unbiased. Thresholding can set a real covariance to zero, creating bias toward zero; when a small entry is retained, soft thresholding additionally pulls it toward zero. The reduced variability can be useful when the sample noise is substantial, but the cost is potential bias. There is no general ordering of hard and soft variances at every nonzero \(\theta\): the exact comparison depends on \(\theta,\tau,\lambda\).

For a large positive signal relative to the noise and threshold, hard thresholding is almost always in its retain region and then equals \(Y\). Its bias approaches zero and its variance approaches \(\tau^2\). Soft thresholding is also almost always retained, but then equals \(Y-\lambda\); its bias approaches \(-\lambda\) and its variance approaches \(\tau^2\). For a large negative signal, the corresponding soft-threshold bias approaches \(+\lambda\). Thus hard thresholding avoids shrinkage bias on retained large entries, while soft thresholding continues to shrink them.

Practical Interpretation and a Common Pitfall

The choice between the rules is a choice about what to do with retained entries. Hard thresholding gives an exact zero pattern while leaving retained estimates unchanged. Soft thresholding gives the same zeroing behavior below the threshold, but applies continuous shrinkage to every retained entry. Near zero, either rule can remove genuine weak covariance; at large magnitudes, soft thresholding can retain a persistent bias. These trade-offs explain why the threshold is part of the estimator, not merely a cosmetic display setting.

• Pitfall: calling the retained soft-thresholded entry unbiased because it is nonzero. For example, \(0.58\) becomes \(0.38\) at threshold \(0.20\). Conditional on being retained, its value has been shifted toward zero. Moreover, conditioning on retention changes the distribution of the observation, so the conditional bias is not simply the unconditional Gaussian bias formula.
• Correction: distinguish the rule's output for one observed entry from its expectation across repeated samples. Use \(E[\hat\sigma_{jk}]-\sigma_{jk}\) for bias and \(\operatorname{Var}(\hat\sigma_{jk})\) for variance; neither is determined by a single thresholded matrix.

Finally, symmetry is preserved by applying the same threshold to \(S_{jk}\) and \(S_{kj}\), but positive semidefiniteness is not guaranteed. An entrywise thresholded matrix therefore need not itself be a valid covariance matrix. Thresholding is an entrywise sparsification operation, not a general guarantee of every matrix constraint.