Why Describe an Inverse by a Filter?
An inverse problem asks us to recover an unknown \(x\) from data \(y\) related by a forward operator \(A\). If \(A\) has very small singular values, directly undoing its action can amplify small data components without bound. A spectral-filter estimator describes a controlled alternative: in each singular direction, it applies a specified multiplier. The filter says how strongly the estimator inverts that direction; the residual says how much of the corresponding solution component remains unrecovered.
Here \(A:\mathcal H\to\mathcal G\) is a compact linear operator between Hilbert spaces. Its positive singular values are \(\sigma_j\), with corresponding right and left singular vectors \(v_j\in\mathcal H\) and \(u_j\in\mathcal G\), so \(Av_j=\sigma_j u_j\) and \(A^*u_j=\sigma_jv_j\). Write \(t_j=\sigma_j^2\). The singular vectors are orthonormal within their respective spaces. Data components orthogonal to all the \(u_j\) cannot be inverted by \(A\).
The factor \(\sigma_j q_\lambda(\sigma_j^2)\) is the data-to-solution gain in direction \(u_j\). Equivalently, the formula applies \(q_\lambda\) to \(A^*A\) and then applies \(A^*\), when that composition is interpreted through the singular expansion. The residual has a direct interpretation for exact data: if \(y=Ax\), the estimated coefficient in direction \(v_j\) is \(t_jq_\lambda(t_j)\langle x,v_j\rangle\), so the remaining fraction of that coefficient is \(r_\lambda(t_j)\).
When Is the Estimator Well Defined?
A finite multiplier at every individual singular value does not by itself guarantee convergence for every data vector. If the singular values approach zero, the gains may grow without bound. The relevant condition is boundedness of the gains over the positive spectrum.
Proof. Suppose first that \(C_\lambda<\infty\). For any \(y\in\mathcal G\), orthonormality of the \(v_j\) gives $$ \|\hat{x}_\lambda(y)\|_{\mathcal H}^2 =\sum_j \sigma_j^2|q_\lambda(\sigma_j^2)|^2|\langle y,u_j\rangle_{\mathcal G}|^2 \le C_\lambda^2\sum_j|\langle y,u_j\rangle_{\mathcal G}|^2 \le C_\lambda^2\|y\|_{\mathcal G}^2. $$ Thus the series converges in \(\mathcal H\) and defines a bounded linear map with norm at most \(C_\lambda\). Conversely, if the estimator is a bounded map with norm \(M\), take \(y=u_j\). Its image is \(\sigma_jq_\lambda(\sigma_j^2)v_j\), so \(\sigma_j|q_\lambda(\sigma_j^2)|\le M\) for every \(j\). Hence \(C_\lambda\le M\). These two inequalities establish the claimed operator norm and the equivalence. \(\square\)
This condition is for each fixed regularization level. It does not say that the operator norms stay bounded as \(\lambda\) changes. That distinction matters: a family may define a valid estimator for every positive \(\lambda\), while its gains become larger as \(\lambda\) decreases.
Worked Example: Checking a Filter on a Finite Spectrum
Let \(A\) have singular values \(2,1,0.1\), and use \(q_\lambda(t)=1/(t+\lambda)\) with \(\lambda=1/4\). Suppose the data have left-singular coordinates \(\langle y,u_j\rangle=(3,-2,5)\). Calculate the gains, estimate, and residual values.
The gain at singular value \(\sigma\) is \(\sigma/(\sigma^2+1/4)\). In the three directions, the gains are \(2/(4.25)=8/17\), \(1/1.25=4/5\), and \(0.1/0.26=5/13\). Thus the largest gain is \(4/5\); by the theorem, the estimator is bounded with operator norm \(4/5\).
Multiplying each gain by its data coordinate gives $$ \hat{x}_\lambda(y)=\frac{24}{17}v_1-\frac{8}{5}v_2+\frac{25}{13}v_3. $$ The residual is \(r_\lambda(t)=(1/4)/(t+1/4)\). Its values at \(t=4,1,0.01\) are, respectively, \(1/17,1/5,25/26\). The small singular direction is therefore much less completely inverted than the large ones.
Residuals and Reconstruction of Exact Solutions
Boundedness makes an estimator well defined, but it does not alone show that the estimator recovers a solution as regularization changes. A useful additional condition is that residuals remain uniformly bounded and vanish at each fixed positive spectral value. The next result makes this statement precise for exact data.
Proof. Write \(x^\dagger=\sum_j x_jv_j\), where \(\sum_j|x_j|^2<\infty\). Since \(y=Ax^\dagger\), its coordinate in direction \(u_j\) is \(\sigma_jx_j\). Therefore the estimated coefficient is \(\sigma_jq_\lambda(\sigma_j^2)\sigma_jx_j=(1-r_\lambda(\sigma_j^2))x_j\). Consequently, $$ \|\hat{x}_\lambda(y)-x^\dagger\|_{\mathcal H}^2 =\sum_j|r_\lambda(\sigma_j^2)|^2|x_j|^2. $$ To show this tends to zero, fix any \(\epsilon>0\). Choose \(J\) so that \(\sum_{j>J}|x_j|^2<\epsilon\). The tail of the displayed sum is at most \(M^2\epsilon\). For the finite sum over \(j\le J\), each residual tends to zero, so that finite sum tends to zero. Since \(\epsilon\) is arbitrary, the full squared norm tends to zero. \(\square\)
The restriction on \(x^\dagger\) excludes a component in the null space of \(A\): such a component has no effect on \(y\), so no estimator based only on \(y\) can recover it. The conclusion concerns exact data and norm convergence; it does not assert convergence for arbitrary noisy data.
Worked Example: A Filter with Bounded Gains and Vanishing Residuals
For \(t>0\), define \(q_\lambda(t)=1/[\sqrt{t}(\sqrt{t}+\sqrt{\lambda})]\). Check its gain and residual, then evaluate them for a finite spectrum with singular values \(1,1/4,1/16\) and \(\lambda=1/256\).
At singular value \(\sigma=\sqrt t\), the gain and residual are $$ \sigma q_\lambda(\sigma^2)=\frac{1}{\sigma+\sqrt{\lambda}}, \qquad r_\lambda(\sigma^2)=1-\frac{\sigma}{\sigma+\sqrt{\lambda}} =\frac{\sqrt{\lambda}}{\sigma+\sqrt{\lambda}}. $$ For every fixed \(\lambda>0\), the gain is at most \(1/\sqrt{\lambda}\), so the estimator is bounded. For the three specified singular values, \(\sqrt{\lambda}=1/16\), giving gains \(16/17,16/5,8\) and residuals \(1/17,1/5,1/2\).
As \(\lambda\) decreases, each residual at a fixed positive singular value tends to zero, and the residuals are bounded by 1. Thus the exact-solution convergence result applies. The gains can nevertheless grow as \(\lambda\) decreases: the theorem guarantees a well-defined estimator at each level, not a uniform bound over all levels.
What Fails Without the Gain Bound?
The gain condition is not a technical convenience. Without it, a data vector with finite norm can produce a formal series whose solution coefficients are not square-summable. The following example shows why defining a finite filter value at every positive spectral value is insufficient.
Worked Example: Finite Filter Values but an Undefined Estimator
Let \(\mathcal H=\mathcal G=\ell^2\), and let \(A\) be diagonal with singular values \(\sigma_j=1/j\), \(j=1,2,\ldots\). Use the unregularized inverse filter \(q(t)=1/t\) at every positive spectral value, and choose data with coordinates \(\langle y,u_j\rangle=1/j\).
The data are valid because \(\|y\|^2=\sum_{j=1}^{\infty}1/j^2<\infty\). Each filter value \(q(\sigma_j^2)=j^2\) is finite, but the gain is \(\sigma_jq(\sigma_j^2)=(1/j)j^2=j\), which is unbounded. The proposed solution coefficient is \(j(1/j)=1\) for every \(j\), so the squared norm of the formal solution would be \(\sum_{j=1}^{\infty}1=\infty\). It is not an element of \(\ell^2\); the estimator is not defined for this data vector.
A common mistake is to check only that \(q_\lambda(t)\) is finite at each eigenvalue and conclude that the estimator is well defined. The correction is to check the data-to-solution gains \(\sqrt{t_j}|q_\lambda(t_j)|\) uniformly over the spectrum. Another mistake is to infer uniform stability across regularization levels from boundedness at each fixed level. These are different claims: the former concerns one estimator, while the latter requires a single bound that works for the whole family.
Check Your Understanding
Use the definitions and conditions in this tutorial to answer the following.
- For a filter \(q_\lambda\), write its residual function and interpret the residual at a positive spectral value.
- State the necessary and sufficient condition for the estimator to be bounded on all of \(\mathcal G\).
- Why does a finite value of \(q_\lambda(t_j)\) at every \(j\) fail to ensure that the estimator is well defined?
- What two residual conditions support exact-solution convergence in the theorem, and why must the solution lie in the closed span of the right singular vectors?
- Does boundedness for every fixed \(\lambda>0\) imply a uniform bound on the estimator norms as \(\lambda\) decreases? Explain.