Stability Under Forward-Operator Error

Why Forward-Operator Error Matters

In an inverse problem, data $y$ are related to an unknown $x$ through a forward operator $A$, often with $y$ approximately equal to $Ax$. The operator may itself be estimated, discretized, or measured imperfectly. Even when the data are held fixed, replacing $A$ by a nearby operator can change the recovered estimate. This tutorial quantifies that change for Tikhonov regularization and shows why the regularization parameter matters.

The central comparison is between two operators acting on the same spaces and the same observed data. The resulting bound is an upper bound, not a promise that the change will be small for every problem: it also depends on the operator sizes and on the value of the regularization parameter.

Setup and a Useful Operator Bound

Definition: Let $\mathcal H$ and $\mathcal G$ be Hilbert spaces, let $A:\mathcal H\to\mathcal G$ be a bounded linear operator, let $y\in\mathcal G$, and let $\lambda>0$. The Tikhonov estimate is $$ x_\lambda(A;y)=(A^*A+\lambda I_{\mathcal H})^{-1}A^*y. $$ Here $A^*$ is the adjoint and $I_{\mathcal H}$ is the identity on $\mathcal H$. Equivalently, $x_\lambda(A;y)$ uniquely minimizes $\|Ax-y\|^2+\lambda\|x\|^2$ over $x\in\mathcal H$.

Because $A^*A$ is bounded, self-adjoint, and nonnegative, $A^*A+\lambda I_{\mathcal H}$ has a bounded inverse, with operator norm at most $1/\lambda$. The positive penalty $\lambda\|x\|^2$ makes the estimate well-defined even if $A$ has a nontrivial null space or does not have a bounded inverse.

Proposition: For every bounded $A:\mathcal H\to\mathcal G$ and every $\lambda>0$, $$ \left\|(A^*A+\lambda I_{\mathcal H})^{-1}A^*\right\|_{\mathrm{op}} \leq \frac{1}{2\sqrt{\lambda}}. $$ Consequently, $\|x_\lambda(A;y)\|\leq \|y\|/(2\sqrt{\lambda})$.

Proof. Put $T=A^*A$. The square of the operator norm on the left is the norm of the product of the operator with its adjoint: $$ \left\|(T+\lambda I_{\mathcal H})^{-1}A^*\right\|_{\mathrm{op}}^2 = \left\|(T+\lambda I_{\mathcal H})^{-1}T(T+\lambda I_{\mathcal H})^{-1}\right\|_{\mathrm{op}}. $$ By the spectral theorem for bounded self-adjoint operators, the last norm is bounded by the supremum of $t/(t+\lambda)^2$ over $t\geq 0$. Since $(t+\lambda)^2\geq 4t\lambda$, this supremum is at most $1/(4\lambda)$. Taking square roots proves the assertion, and applying the operator to $y$ gives the estimate bound. $\square$

This bound explains one role of regularization: as $\lambda$ decreases, the largest possible norm of the regularized inverse increases. It does not by itself compare estimates formed from two different operators; for that, the normal equations give a direct perturbation identity.

Worked Example: A Scalar Tikhonov Estimate

Take $\mathcal H=\mathcal G=\mathbb R$, $A x=x$, $y=2$, and $\lambda=0.5$. The estimate is the scalar $$ x_\lambda(A;y)=\frac{1\cdot 2}{1^2+0.5}=\frac{2}{1.5}=\frac{4}{3}\approx 1.3333. $$

The operator norm of the regularized inverse is $1/(1+0.5)=2/3$ in this example. The general proposition gives the less specific upper bound $1/(2\sqrt{0.5})\approx0.7071$, which is indeed larger than $2/3$. The proposition is uniform over all bounded operators; it need not be an equality for each particular operator.

A Perturbation Bound

Theorem: Let $A,B:\mathcal H\to\mathcal G$ be bounded linear operators between the same Hilbert spaces, let $y\in\mathcal G$, and fix $\lambda>0$. Define $\delta=\|A-B\|_{\mathrm{op}}$. Then $$ \|x_\lambda(A;y)-x_\lambda(B;y)\| \leq \frac{\delta}{\lambda} \left(\|y\|+(\|A\|_{\mathrm{op}}+\|B\|_{\mathrm{op}})\|x_\lambda(B;y)\|\right). $$ In particular, $$ \|x_\lambda(A;y)-x_\lambda(B;y)\| \leq \frac{\delta\|y\|}{\lambda} \left(1+\frac{\|A\|_{\mathrm{op}}+\|B\|_{\mathrm{op}}}{2\sqrt{\lambda}}\right). $$

Proof. Write $H_A=A^*A+\lambda I_{\mathcal H}$ and $H_B=B^*B+\lambda I_{\mathcal H}$. The definitions of the estimates give the normal equations $H_Ax_\lambda(A;y)=A^*y$ and $H_Bx_\lambda(B;y)=B^*y$. Subtracting after expressing both sides with $H_A$ yields $$ H_A\bigl(x_\lambda(A;y)-x_\lambda(B;y)\bigr) =(A^*-B^*)y+(B^*B-A^*A)x_\lambda(B;y). $$ The inverse of $H_A$ has operator norm at most $1/\lambda$, because $A^*A$ is nonnegative. Also, $$ A^*A-B^*B=A^*(A-B)+(A^*-B^*)B, $$ so $\|A^*A-B^*B\|_{\mathrm{op}}\leq(\|A\|_{\mathrm{op}}+\|B\|_{\mathrm{op}})\delta$. Applying these bounds to the identity proves the first inequality. The proposition gives $\|x_\lambda(B;y)\|\leq\|y\|/(2\sqrt{\lambda})$; substitution proves the second. $\square$

For fixed $\lambda$, bounded $\|A\|_{\mathrm{op}}$ and $\|B\|_{\mathrm{op}}$, and fixed $y$, the bound is proportional to $\delta$. Thus the estimate changes continuously, at a quantified rate, under small operator perturbations. The displayed dependence on $\lambda$ also warns that the guarantee can deteriorate as regularization weakens. The bound is deliberately general and may be conservative for a particular operator and data vector.

Worked Example: Applying the Bound to Nearby Scalar Operators

Let $A x=x$, $B x=1.1x$, $y=2$, and $\lambda=0.5$. The operator error is $\delta=|1-1.1|=0.1$. Direct calculation gives $$ x_\lambda(A;y)=\frac{2}{1+0.5}=\frac{4}{3}\approx1.33333, \qquad x_\lambda(B;y)=\frac{1.1(2)}{1.1^2+0.5} =\frac{2.2}{1.71}\approx1.28655. $$ Thus the actual change is approximately $1.33333-1.28655=0.04678$.

The theorem's first bound is $$ \frac{0.1}{0.5}\left(2+(1+1.1)(1.28655)\right) \approx 0.2(4.702)=0.9404. $$ This is much larger than the actual change, but it correctly bounds it. The example illustrates why a norm bound is a robust guarantee rather than an exact prediction.

Weakly Observed Directions and Regularization

The operator size alone does not describe how a perturbation affects each component of an estimate. Directions that $A$ observes weakly can be especially sensitive: small changes in the forward map may be large relative to the information in that direction. In finite dimensions, singular values make this behavior visible. Tikhonov regularization replaces unregularized division by a singular value $s$ with the factor $s/(s^2+\lambda)$; when $s$ is small, $\lambda$ prevents that factor from becoming arbitrarily large.

Worked Example: A Perturbation in a Weak Direction

Consider $\mathcal H=\mathcal G=\mathbb R^2$, with $$ A=\begin{pmatrix}1&0\\0&0.2\end{pmatrix}, \qquad B=\begin{pmatrix}1&0\\0&0.3\end{pmatrix}, \qquad y=\begin{pmatrix}1\\2\end{pmatrix}, \qquad \lambda=0.1. $$ For a diagonal operator with entries $a_1,a_2$, the estimate has coordinates $a_i y_i/(a_i^2+\lambda)$. Hence $$ x_\lambda(A;y) =\begin{pmatrix}1/1.1\\0.4/0.14\end{pmatrix} \approx\begin{pmatrix}0.90909\\2.85714\end{pmatrix}, \qquad x_\lambda(B;y) =\begin{pmatrix}1/1.1\\0.6/0.19\end{pmatrix} \approx\begin{pmatrix}0.90909\\3.15789\end{pmatrix}. $$ The operator perturbation norm is $0.1$, and the estimate changes only in the second coordinate, by approximately $3.15789-2.85714=0.30075$. The data's second coordinate is acted on through the relatively small diagonal entries $0.2$ and $0.3$, so the change is substantial compared with the first coordinate, which is unchanged.

This calculation is not a claim that every weak direction has a large perturbation response. It shows why assessing only the overall error $\|A-B\|_{\mathrm{op}}$ can conceal which directions are most affected.

Worked Example: Regularization Controls a Scalar Perturbation

Let $A=0$, $B x=0.001x$, $y=1$, and compare three positive regularization values. For $A=0$, the estimate is zero. For $B$, it is $0.001/(0.001^2+\lambda)$. The exact changes are $$ \begin{array}{c|c|c} \lambda & |x_\lambda(B;1)-x_\lambda(A;1)| & \delta/\lambda\\ 0.1 & 0.001/0.100001\approx0.0099999 & 0.01\\ 0.01 & 0.001/0.010001\approx0.0999900 & 0.1\\ 0.001 & 0.001/0.001001\approx0.9990010 & 1 \end{array} $$ Here $\delta=0.001$. The changes are close to $\delta/\lambda$ because $\delta^2$ is small relative to each listed $\lambda$. Lowering $\lambda$ by a factor of ten increases the observed change by approximately a factor of ten. This example verifies the sensitivity mechanism in a simple case; it does not assert that the general theorem's full bound is always attained.

Interpretation and a Common Pitfall

The theorem holds with the same $y$ on both sides, so it isolates forward-operator error. If the observed vector also changes, say from $y$ to $y+e$ while the operator $B$ is used, the additional change is bounded by $$ \|x_\lambda(B;y+e)-x_\lambda(B;y)\| \leq \frac{\|e\|}{2\sqrt{\lambda}}, $$ by linearity in the data and the proposition. One can therefore separate operator error from data error by adding this term to the operator-perturbation bound.

Common mistake: It is tempting to say that a small $\|A-B\|_{\mathrm{op}}$ guarantees a negligible change in the inverse estimate, or to reason that adding $\lambda I$ makes the estimate insensitive. The theorem instead gives a bound containing inverse powers of $\lambda$, and the scalar example shows that reducing $\lambda$ can amplify a fixed perturbation. The correction is to state the regularization level and the relevant operator and data norms whenever making a stability claim. For a fixed positive $\lambda$, small operator error does yield a small change under the theorem's assumptions; the guarantee is not uniform as $\lambda$ approaches zero.

The result is useful when the forward map is estimated from data or approximated numerically: it translates an operator error bound into an estimate error bound. Its role is distinct from the spectral effects of observation noise. Both kinds of error may matter in an inverse problem, but this comparison isolates the contribution of uncertainty in the forward operator.

Key takeaway: With $\lambda>0$, Tikhonov estimates are stable under bounded perturbations of the forward operator, with an explicit bound proportional to $\|A-B\|_{\mathrm{op}}$. The bound worsens as regularization is weakened, so the operator error and the value of $\lambda$ must be considered together.

Check Your Understanding

Work through these questions using the assumptions and definitions above.

In the perturbation theorem, which quantities must remain fixed when comparing $x_\lambda(A;y)$ and $x_\lambda(B;y)$?
Why does $\|A^*A-B^*B\|_{\mathrm{op}}\leq(\|A\|_{\mathrm{op}}+\|B\|_{\mathrm{op}})\|A-B\|_{\mathrm{op}}$ hold?
For $A=0$, $B x=\delta x$, and $y=1$, derive the exact change in the Tikhonov estimate as a function of $\delta$ and $\lambda$.
What does the bound $1/(2\sqrt{\lambda})$ control, and why does it increase when $\lambda$ decreases?
If the forward operator and the observed vector both change, how can the data-error contribution be bounded separately?