Mercer’s Theorem and Uniform Kernel Expansions

Why an Operator Spectrum Can Represent a Kernel

A kernel $k(x,y)$ assigns a similarity to each pair of inputs. Its integral operator compresses that two-variable object into a linear operator on functions, whose eigenvalues and eigenfunctions describe how the kernel acts under a chosen measure. Mercer’s theorem supplies a stronger conclusion than an operator expansion alone: under suitable conditions, those eigenfunctions reconstruct $k(x,y)$ at every pair of points, with uniform convergence.

The measure matters. An integral operator only sees the kernel on the measure’s support. If some points are invisible to the measure, its eigenfunctions need not determine the kernel there. Full support, together with continuity and compactness, is what lets the operator’s spectral description extend across the entire input space.

Definition: Let $X$ be a compact metric space. A real-valued kernel $k:X\times X\to\mathbb{R}$ is positive definite if it is symmetric and, for every integer $m\geq 1$, points $x_1,\ldots,x_m\in X$, and scalars $c_1,\ldots,c_m\in\mathbb{R}$, $$ \sum_{i=1}^m\sum_{j=1}^m c_i c_j k(x_i,x_j)\geq 0. $$ A finite Borel measure $\mu$ has full support on $X$ if every nonempty open subset of $X$ has positive measure.

In this tutorial, assume $X$ is nonempty and compact metric, $\mu$ is a finite nonzero Borel measure with full support, and $k$ is continuous and positive definite. The associated integral operator on $L^2(\mu)$ is $$ (Tf)(x)=\int_X k(x,y)f(y)\,d\mu(y). $$ Because $k$ is bounded and $\mu$ is finite, this integral is defined for $f\in L^2(\mu)$. The operator is compact and self-adjoint: its kernel is square-integrable and symmetric. It is also positive. For example, using the canonical RKHS feature sections $k_x=k(\,\cdot\,,x)$, continuity makes $x\mapsto k_x$ continuous in RKHS norm, and $$ \langle f,Tf\rangle_{L^2(\mu)} =\left\|\int_X f(y)k_y\,d\mu(y)\right\|_{\mathcal H}^2\geq 0. $$ The integral on the right exists as a Hilbert-space-valued integral because $f\in L^1(\mu)$ and the feature sections are bounded. Thus the spectral theorem for compact self-adjoint operators applies.

Mercer’s Theorem

Theorem (Mercer): Under the assumptions above, the positive eigenvalues of $T$, repeated according to multiplicity, can be listed as $\lambda_1,\lambda_2,\ldots>0$ (with a finite list allowed). There are corresponding $L^2(\mu)$-orthonormal eigenfunctions with continuous versions $\phi_j:X\to\mathbb{R}$, satisfying $T\phi_j=\lambda_j\phi_j$. For every $x,y\in X$, $$ k(x,y)=\sum_j \lambda_j\phi_j(x)\phi_j(y), $$ and the series converges absolutely and uniformly on $X\times X$. If there are no positive eigenvalues, the sum is zero.

The eigenfunctions here are normalized in $L^2(\mu)$, not at particular input points. An eigenfunction is initially an equivalence class of functions equal almost everywhere. The theorem specifies a continuous representative, so its values at every $x\in X$ are meaningful.

Worked Example: A Rank-One Kernel on the Unit Interval

Let $X=[0,1]$, let $\mu$ be uniform probability measure, and set $k(x,y)=(1+x)(1+y)$. This kernel is continuous and positive definite: its Gram quadratic form is $\left(\sum_i c_i(1+x_i)\right)^2$, which is nonnegative.

The operator is $Tf(x)=(1+x)\int_0^1(1+y)f(y)\,dy$. Its only possible nonzero eigenfunction is proportional to $1+x$. Its squared $L^2(\mu)$ norm is $$ \int_0^1(1+x)^2\,dx =\left[x+x^2+\frac{x^3}{3}\right]_0^1 =\frac{7}{3}. $$

Therefore $\phi_1(x)=\sqrt{3/7}(1+x)$ is normalized. Direct substitution gives $$ T\phi_1(x)=(1+x)\sqrt{\frac{3}{7}}\int_0^1(1+y)^2\,dy =\frac{7}{3}\phi_1(x). $$ Thus $\lambda_1=7/3$, and the expansion is exact: $\lambda_1\phi_1(x)\phi_1(y)=(1+x)(1+y)=k(x,y)$. No additional positive eigenvalues are possible because the range of $T$ consists only of scalar multiples of $1+x$.

Why the Eigenfunctions Are Continuous

For a positive eigenvalue, continuity follows directly from the integral equation. Begin with an $L^2(\mu)$-normalized eigenfunction $\phi_j$. Since $\mu$ is finite, Cauchy–Schwarz implies $\phi_j\in L^1(\mu)$. Define $$ \widetilde\phi_j(x)=\frac{1}{\lambda_j}\int_X k(x,y)\phi_j(y)\,d\mu(y). $$ For $x_n\to x$, continuity of $k$ gives $k(x_n,y)\to k(x,y)$ for every $y$. The integrands are bounded in absolute value by $\sup_{u,v\in X}|k(u,v)|\,|\phi_j(y)|$, which is integrable. Dominated convergence therefore gives $\widetilde\phi_j(x_n)\to\widetilde\phi_j(x)$. The eigenvalue equation says $\widetilde\phi_j=\phi_j$ almost everywhere, so this defines the required continuous version.

With these versions, the eigenvalue equation holds at every point, not merely almost everywhere. Both sides of $T\widetilde\phi_j=\lambda_j\widetilde\phi_j$ are continuous, and they agree almost everywhere. If two continuous functions differed at a point, they would differ on a nonempty open set; full support would make that set have positive measure, contradicting equality almost everywhere.

Worked Example: A Linear Kernel on a Symmetric Interval

Take $X=[-1,1]$, $\mu$ uniform probability measure, and $k(x,y)=1+xy$. For any $x_1,\ldots,x_m$ and $c_1,\ldots,c_m$, its Gram quadratic form is $\left(\sum_i c_i\right)^2+\left(\sum_i c_i x_i\right)^2$, so it is positive definite.

The functions $\phi_0(x)=1$ and $\phi_1(x)=\sqrt{3}x$ are orthonormal: their squared norms are $1$ and $3\int_{-1}^1x^2\,dx/2=1$, and their inner product is zero by symmetry. The operator is $Tf(x)=\int_{-1}^1(1+xy)f(y)\,dy/2$. Consequently $T\phi_0=1=\phi_0$, while $$ T\phi_1(x) =\frac{\sqrt{3}}{2}\int_{-1}^1 (y+xy^2)\,dy =\frac{\sqrt{3}}{2}\left(0+\frac{2x}{3}\right) =\frac{1}{3}\phi_1(x). $$

Thus the eigenvalues are $1$ and $1/3$, and the expansion is $1\cdot\phi_0(x)\phi_0(y)+(1/3)\phi_1(x)\phi_1(y)=1+xy$. The operator’s range is contained in the span of $1$ and $x$, so there are no other positive eigenvalues.

Deriving the Uniform Kernel Expansion

The key step is to connect the operator eigenfunctions to orthonormal directions in the RKHS. Let $\mathcal H$ be the RKHS of $k$, and let $J:\mathcal H\to L^2(\mu)$ be the inclusion map, $Jh=h$. It is bounded because $|h(x)|\leq \|h\|_{\mathcal H}\sqrt{k(x,x)}$, and $k(x,x)$ is bounded on compact $X$. Its adjoint satisfies $(J^*g)(x)=\int_X k(x,y)g(y)\,d\mu(y)$, so $JJ^*=T$.

For each positive eigenvalue define $v_j=J^*\phi_j/\sqrt{\lambda_j}\in\mathcal H$. These vectors are orthonormal, since $$ \langle v_i,v_j\rangle_{\mathcal H} =\frac{\langle\phi_i,JJ^*\phi_j\rangle_{L^2(\mu)}}{\sqrt{\lambda_i\lambda_j}} =\frac{\lambda_j\langle\phi_i,\phi_j\rangle_{L^2(\mu)}}{\sqrt{\lambda_i\lambda_j}} =\begin{cases}1,&i=j,\\0,&i\ne j.\end{cases} $$ The reproducing property and the eigenvalue equation also give $\langle k_x,v_j\rangle_{\mathcal H}=\sqrt{\lambda_j}\phi_j(x)$.

Key result: Under the Mercer assumptions, the vectors $v_j$ form an orthonormal basis of $\mathcal H$. Consequently, Parseval’s identity gives the pointwise kernel expansion, and compactness upgrades it to uniform convergence.

To verify the basis claim, first note that the closure of the span of $J^*\phi_j$ is the closure of the range of $J^*$. Indeed, the compact positive operator $T=JJ^*$ has its positive-eigenvalue eigenfunctions spanning $(\ker T)^\perp$, and $J^*$ vanishes on $\ker T$, since $\|J^*g\|_{\mathcal H}^2=\langle g,Tg\rangle_{L^2(\mu)}=0$ there. For any bounded operator, $\overline{\operatorname{range}(J^*)}=(\ker J)^\perp$. Finally, if $Jh=0$, then the continuous function $h$ is zero almost everywhere. Full support implies $h=0$ everywhere, hence $h=0$ in $\mathcal H$. So $\ker J=\{0\}$, and the $v_j$ are a basis.

Apply Parseval’s identity to $k_x,k_y\in\mathcal H$. For each $x,y\in X$, $$ k(x,y)=\langle k_y,k_x\rangle_{\mathcal H} =\sum_j\langle k_x,v_j\rangle_{\mathcal H}\langle k_y,v_j\rangle_{\mathcal H} =\sum_j\lambda_j\phi_j(x)\phi_j(y). $$ In particular, $\sum_j\lambda_j\phi_j(x)^2=k(x,x)$. For an infinite list, the partial sums on the diagonal are continuous, increase pointwise to the continuous function $k(x,x)$, and $X$ is compact. Dini’s theorem therefore implies that the diagonal tails $d_N(x)=k(x,x)-\sum_{j=1}^N\lambda_j\phi_j(x)^2$ converge uniformly to zero.

For every $x,y$, Cauchy–Schwarz bounds the absolute off-diagonal tail: $$ \sum_{j>N}\lambda_j|\phi_j(x)\phi_j(y)| \leq \sqrt{d_N(x)d_N(y)} \leq \sup_{z\in X}d_N(z). $$ The last bound tends to zero independently of $x$ and $y$. This proves both absolute and uniform convergence on $X\times X$. For a finite list, the expansion is already exact after its last term. $\square$

Worked Example: What Fails Without Full Support

Let $X=[0,1]$, take $\mu$ to be the point mass at $0$, and use the continuous positive definite kernel $k(x,y)=1+xy$. The operator is $Tf(x)=k(x,0)f(0)=f(0)$. As an operator on $L^2(\mu)$, it has one normalized eigenfunction $\phi_1=1$ and eigenvalue $1$. Its continuous representative is also the constant function $1$.

The eigenfunction series is therefore $1\cdot\phi_1(x)\phi_1(y)=1$. It agrees with $k(x,y)$ when $x=0$ or $y=0$, the only points observed by the measure, but it fails at $x=y=1$: the series gives $1$, whereas $k(1,1)=2$. The support of $\mu$ is only $\{0\}$, not all of $X$. Continuity cannot determine kernel values away from this support from the operator spectrum alone.

Common Pitfall and Practical Meaning

A common mistake is to assume that an eigen-expansion of $T$ automatically gives a pointwise expansion of $k$ everywhere. The operator acts on $L^2(\mu)$, where functions are identified up to equality almost everywhere; it does not directly record values on measure-zero regions. The correction is to check the theorem’s domain, continuity, and full-support hypotheses. Under those conditions, the RKHS argument shows the eigenvectors span all directions needed to reconstruct the kernel, while Dini’s theorem and the tail bound establish uniform convergence.

Uniform convergence is useful because it controls approximation error simultaneously for every pair of inputs. Truncating after $N$ terms yields a finite-dimensional approximation whose largest absolute error over $X\times X$ tends to zero. This conclusion concerns the kernel expansion itself; how RKHS functions are represented or normed in these coordinates is a separate question.

Check Your Understanding

Use the assumptions and proof structure above to answer the following.

Why does continuity of an eigenfunction’s integral representation require $\phi_j\in L^1(\mu)$, and why is that guaranteed here?
Where is full support used to show that the vectors $v_j$ span the RKHS?
What role does Dini’s theorem play in proving uniform convergence on $X\times X$?
For the kernel $1+xy$ on $[-1,1]$ with uniform probability measure, verify that the eigenfunction for eigenvalue $1/3$ has $L^2$ norm one.
In the point-mass example, why can the integral operator fail to recover $k(1,1)$?