Spectral Coordinates for Statistical Inverse Problems

Why Spectral Coordinates Matter

An inverse problem starts with an unknown object \(x\), a forward operator \(T\) that maps it to an observable quantity, and data that may include noise. When \(T\) is compact, its singular values describe how strongly it transmits different directions. In singular-vector coordinates, the forward map becomes a collection of scalar equations. Those equations reveal which components of \(x\) leave a trace in the data, and how the noise enters each trace.

This coordinate representation is a way to describe the inverse problem, not yet a prescription for estimating \(x\). A crucial distinction is between a component that is not observable at all and one that is observable but transmitted weakly. The first lies in the null space of \(T\); the second corresponds to a small positive singular value.

Compact Operators and Their Singular Coordinates

Let \(\mathcal H\) and \(\mathcal G\) be real separable Hilbert spaces, with inner products and norms denoted by \(\langle\cdot,\cdot\rangle\) and \(\|\cdot\|\). A bounded linear operator \(T:\mathcal H\to\mathcal G\) is compact if it maps every bounded sequence in \(\mathcal H\) to a sequence in \(\mathcal G\) with a convergent subsequence. The adjoint \(T^*:\mathcal G\to\mathcal H\) is defined by \(\langle Tx,g\rangle=\langle x,T^*g\rangle\).

The compact, positive, self-adjoint operator \(T^*T\) has an orthonormal system of eigenvectors for its positive eigenvalues. Taking square roots of those eigenvalues gives the singular values of \(T\).

The singular system follows from the compact self-adjoint spectral theorem applied to \(T^*T\): its positive eigenvalues are \(s_j^2\), and corresponding unit eigenvectors are \(v_j\). Setting \(u_j=Tv_j/s_j\) gives orthonormal left singular vectors, since \(\langle Tv_i,Tv_j\rangle=\langle v_i,T^*Tv_j\rangle=s_j^2\langle v_i,v_j\rangle\). The null spaces account for directions not represented by positive singular values.

For \(x\in\mathcal H\), define its right singular coordinates \(x_j=\langle x,v_j\rangle\). The component of \(x\) in \(\ker(T)\) has no effect on \(Tx\), and

The series for \(Tx\) converges in \(\mathcal G\). In the output, any component orthogonal to all \(u_j\) is also zero for an exact signal \(Tx\); such components belong to \(\ker(T^*)\). Thus the left singular vectors span the part of the output space that the operator can reach, while the right singular vectors span the part of the input space it can observe.

Coordinate Form of the Noisy Inverse Problem

Consider the observation model \(Y=Tx+\varepsilon\), where \(x\in\mathcal H\) is fixed and \(\varepsilon\) is an output error. If \(Y\) and \(\varepsilon\) are \(\mathcal G\)-valued, define \(Y_j=\langle Y,u_j\rangle\) and \(\varepsilon_j=\langle\varepsilon,u_j\rangle\). Taking inner products with \(u_j\) gives

This identity is exact: it follows from \(\langle Tx,u_j\rangle=s_j\langle x,v_j\rangle\), which in turn follows from \(T^*u_j=s_jv_j\). The component of \(Y\) in \(\ker(T^*)\) is just the corresponding component of \(\varepsilon\), because the signal \(Tx\) has none there. It is not an additional observation of \(x\).

What the Coordinate Equations Identify

Proof. If \(Tx=T\tilde{x}\), linearity gives \(T(x-\tilde{x})=0\), so \(x-\tilde{x}\in\ker(T)\). Conversely, if \(x-\tilde{x}\in\ker(T)\), then \(T(x-\tilde{x})=0\), hence \(Tx=T\tilde{x}\). The decomposition of \(x\) into its null-space component and its component in \(\ker(T)^\perp\) therefore shows that exact data identify precisely the latter. Taking the inner product of \(y=Tx\) with \(u_j\) gives \(y_j=s_jx_j\); since every listed \(s_j\) is positive, \(x_j=y_j/s_j\).

For instability, suppose there are infinitely many singular values, so \(s_j\to0\). The input \(v_j\) has norm one, while its output \(Tv_j=s_ju_j\) has norm \(s_j\). Thus arbitrarily small output changes can correspond to unit changes in the input along sufficiently high-index singular directions. This establishes instability without asserting that those directions are unidentifiable: each remains identifiable from exact data. \(\square\)

If there are only finitely many positive singular values, the operator has finite rank and there is no sequence of positive singular values tending to zero. The null-space distinction still applies. In infinite rank, the range of a compact operator need not be closed; in that case some output sequences can be limits of attainable signals without themselves being attainable exact outputs. The coordinate equations describe attainable signals, and noisy data need not lie in the range.

How Noise Appears in Singular Coordinates

Suppose \(\varepsilon\) is a centered, square-integrable \(\mathcal G\)-valued random error, and let its covariance operator \(C:\mathcal G\to\mathcal G\) be defined by \(Ch=E[\langle\varepsilon,h\rangle\varepsilon]\). Then its singular-coordinate errors have covariance

In particular, the coordinate errors are uncorrelated when this expression is zero for distinct \(i,j\); the coordinate variances are \(\langle Cu_j,u_j\rangle\). The formulas describe covariance, not by themselves independence or a full distribution. A change to an orthonormal basis preserves a covariance operator proportional to the identity, but independence after that change requires an additional distributional condition.

There is an important infinite-dimensional qualification. A \(\mathcal G\)-valued, square-integrable error must have finite expected squared norm. By Parseval's identity and monotone convergence, the sum of its coordinate variances over any orthonormal basis is at most \(E\|\varepsilon\|^2<\infty\). Consequently, infinitely many independent coordinates all having the same positive variance cannot be the coordinates of such a \(\mathcal G\)-valued error. Idealized white noise with constant variance per coordinate is instead formulated as a generalized observation, or through finite-dimensional projections.

Common Pitfall and Practical Interpretation

• Confusing weak observation with no observation. A small positive \(s_j\) means the signal coordinate reaches the output with small amplitude; it does not mean that the coordinate is absent. A zero singular value corresponds to a null-space direction and no signal contribution at all.
• Assuming covariance implies independence. Diagonal covariance gives uncorrelated coordinates, not generally independent ones. The claim of independent coordinates in every orthonormal basis applies to isotropic Gaussian noise, not to every error with covariance \(\sigma^2 I\).
• Treating every coordinate sequence as a Hilbert-space observation. A sequence of independent errors with the same positive variance in infinitely many coordinates is not square-summable in expectation. An idealized white-noise model needs a generalized-observation interpretation.

The singular coordinates provide a diagnostic language for inverse problems. They separate the null-space question of identifiability from the small-singular-value question of sensitivity, while keeping the signal and noise contributions explicit. In a noisy problem, the equation \(Y_j=s_jx_j+\varepsilon_j\) makes clear why a coordinate with small \(s_j\) can be particularly vulnerable: the signal is attenuated in that coordinate, whereas the noise contribution remains part of the observation.