Landweber Iteration and Its Spectral Filter

Why the Step Size Shapes the Spectral Filter

Landweber iteration solves a linear inverse problem by repeatedly moving in the direction that reduces the squared residual. Its computational simplicity is useful, but the step size has a direct spectral effect: it determines whether components are steadily damped, oscillate, or grow. The number of iterations matters too. In particular, stopping after finitely many steps gives a specific spectral filter, rather than the exact inverse.

We derive that filter and establish the step-size range for which the error-propagation operator is nonexpansive. A key distinction is that stability does not mean every signal component is recovered monotonically. Within the stable range, some components can overshoot their exact coefficients.

Landweber Iteration and Its Spectral Multipliers

Let \(A:\mathcal{X}\to\mathcal{Y}\) be a bounded linear operator between Hilbert spaces, with adjoint \(A^*\). Given data \(y\in\mathcal{Y}\), Landweber iteration applies gradient descent to the least-squares objective \(\frac12\|Ax-y\|^2\):

Here \(\eta>0\) is the step size. Unless stated otherwise, take \(x_0=0\). The operator \(B=A^*A\) is bounded, self-adjoint and positive, and its spectrum is contained in \([0,M]\), where \(M=\|A\|^2\). We assume \(A\ne 0\), so \(M>0\).

To see how the data enter the iterate, substitute the update repeatedly. The resulting polynomial in \(B\) is

The value \(g_k(0)=k\eta\) is the continuous extension, obtained by taking the limit as \(\lambda\to0\). Also, \(A^*y\) has no component in the null space of \(B\), so this value does not create a null-space component in \(x_k\).

When \(y=Ax^\dagger\), the iteration can instead be written as \(x_k=[I-(I-\eta B)^k]x^\dagger\). Thus a coefficient of \(x^\dagger\) at a positive spectral value \(\lambda\) is multiplied by \(q_k(\lambda)\). In a singular system, where \(Av_i=s_i u_i\) and \(\lambda_i=s_i^2>0\), the corresponding data coefficient \(\langle y,u_i\rangle\) is multiplied by \(s_i g_k(\lambda_i)\) to form the coefficient of \(x_k\) along \(v_i\).

The Step-Size Condition for Nonexpansive Stability

The theorem concerns propagation of an initial error under the iteration, not a uniform bound on amplification of arbitrary data noise through all iteration counts. The proof uses the spectral theorem for the bounded self-adjoint operator \(B\): the norm of a continuous function of \(B\) is the supremum of its absolute value over the spectrum of \(B\).

Proof. The spectrum of \(B\) lies in \([0,M]\), and its maximum is \(M\). The spectrum of \(I-\eta B\) consists of values \(1-\eta\lambda\) for \(\lambda\) in the spectrum of \(B\). Consequently,

For \(\eta>0\), the scalar inequality \(|1-\eta\lambda|\le1\) is equivalent to \(0\le\eta\lambda\le2\). It holds for every spectral value precisely when \(\eta M\le2\), or \(0<\eta\le2/M\). This proves sufficiency. If \(\eta>2/M\), then \(2/\eta<M\). Since \(M\) is the maximum of the spectrum, there is a spectral value \(\lambda>2/\eta\); for that value \(|1-\eta\lambda|>1\), so the operator is not nonexpansive. This proves necessity. In the nonexpansive range, \(\|I-\eta B\|\le1\), hence submultiplicativity gives \(\|(I-\eta B)^k\|\le1\) for every integer \(k\ge0\). \(\square\)

For \(0<\eta<2/M\), each positive spectral value has \(|1-\eta\lambda|<1\), so its residual multiplier tends to zero as \(k\to\infty\). This pointwise decay does not imply a uniform contraction over the whole space: spectral values can approach zero, making \(1-\eta\lambda\) arbitrarily close to one. At the boundary \(\eta=2/M\), the multiplier at \(\lambda=M\) is \(-1\). The iterates remain nonexpansive, but a component at that spectral value does not decay.

When the Step Size Exceeds the Stable Range

The same scalar factors that give the filter also expose instability. If some spectral value satisfies \(\eta\lambda>2\), then \(1-\eta\lambda<-1\), so the residual multiplier grows in magnitude with each step. This is distinct from overshoot: overshoot can occur while the residual remains bounded, whereas an unstable residual grows without bound.

Interpreting the Recovery Multiplier

Within the nonexpansive range \(0<\eta\le2/M\), the residual multiplier satisfies \(-1\le r_k(\lambda)\le1\), and consequently \(0\le q_k(\lambda)\le2\). It need not be at most one. If \(0<\eta\le1/M\), then \(0\le1-\eta\lambda<1\) for positive \(\lambda\), so \(q_k(\lambda)\in(0,1]\). It is strictly less than one when \(\eta\lambda<1\); if \(\eta\lambda=1\), it equals one. Thus in the smaller step-size range, the multiplier behaves like a fraction of the exact coefficient, with this endpoint qualification.

For \(1/M<\eta\le2/M\), some spectral values can have \(1-\eta\lambda<0\). Their residuals change sign across iterations, and the signal-recovery multiplier can exceed one. Outside the nonexpansive range, it can also be negative. These facts make it important to name \(q_k\) a signal-recovery multiplier unless its range has been checked; “recovered fraction” is not a valid general description.

Common Mistake and Practical Significance

• Mistake: Calling \(q_k(\lambda)\) a fraction of the exact coefficient for every stable step size. Stability only guarantees \(|r_k(\lambda)|\le1\), which permits \(q_k(\lambda)>1\). For example, the stable choice \(\eta=0.3\), \(M=4\), \(\lambda=4\), \(k=3\) gives \(q_k(\lambda)=1.008\).
• Correction: Call \(q_k\) the signal-recovery multiplier. It lies in \((0,1]\) for positive \(\lambda\) when \(0<\eta\le1/M\); the endpoint value one is possible. For larger stable step sizes it may exceed one, and for step sizes outside the nonexpansive range it may be negative.
• Mistake: Treating nonexpansiveness as convergence. At \(\eta=2/M\), a component at \(\lambda=M\) can oscillate without decaying.
• Correction: Nonexpansiveness prevents growth of homogeneous error norms; convergence requires decay of the relevant spectral components. A finite iteration count also leaves low-eigenvalue components heavily damped.

The spectral form explains the role of early stopping: each eigencomponent is acted on by a different multiplier, and both the step size and the iteration count shape that action. The condition \(0<\eta\le2/\|A\|^2\) prevents homogeneous errors from expanding, while the more conservative \(0<\eta\le1/\|A\|^2\) keeps positive-eigenvalue signal multipliers within \((0,1]\). Neither statement alone says that data noise is uniformly controlled over arbitrarily many iterations; that requires analyzing the data-to-solution filter as well.