Gaussian and Matérn Kernels as Function Spaces

Why the Kernel Spectrum Determines the Function Space

A kernel does more than measure similarity between inputs: it determines which functions have finite RKHS norm. For translation-invariant kernels on \(\mathbb{R}^d\), this relationship can be read from the kernel's Fourier transform. The Gaussian and Matérn kernels make a useful contrast. The Gaussian spectrum decays exponentially, imposing a strong penalty on high frequencies; the Matérn spectrum decays polynomially, producing a Sobolev space.

Throughout, take \(d\geq 1\), \(x,\omega\in\mathbb{R}^d\), and use the Fourier transform convention \(\widehat{f}(\omega)=\int_{\mathbb{R}^d}e^{-i\omega\cdot x}f(x)\,dx\), with inverse factor \((2\pi)^{-d}\). We consider real-valued functions, using the usual conjugate-symmetric Fourier transforms. The discussion concerns the RKHS on the whole space \(\mathbb{R}^d\).

Definition: A translation-invariant kernel has the form \(k(x,y)=\kappa(x-y)\). Suppose its spectral density \(s\) is positive and integrable, and \[ \kappa(t)=(2\pi)^{-d}\int_{\mathbb{R}^d}e^{i\omega\cdot t}s(\omega)\,d\omega. \] Its Fourier-weighted function space consists of functions \(f\) whose Fourier transforms satisfy \[ \|f\|_{\mathcal{H}}^2=(2\pi)^{-d}\int_{\mathbb{R}^d}\frac{|\widehat{f}(\omega)|^2}{s(\omega)}\,d\omega<\infty. \] For the kernels considered here, this space, with this norm, is the RKHS of \(k\).

The Fourier Characterization

The spectral density determines the cost of each frequency: a small value of \(s(\omega)\) makes that frequency expensive in the RKHS norm. We establish the characterization that will be used for both kernel families.

Theorem: Let \(s\) be positive, integrable, and even, and define \(\kappa\) by the displayed inverse Fourier formula. The functions with finite Fourier-weighted norm form the RKHS of \(k(x,y)=\kappa(x-y)\).

Proof. Consider the space of Fourier transforms with inner product \[ \langle \widehat f,\widehat g\rangle_{\mathcal H} =(2\pi)^{-d}\int_{\mathbb{R}^d} \frac{\widehat f(\omega)\overline{\widehat g(\omega)}}{s(\omega)}\,d\omega. \] It is a weighted \(L^2\) space and hence complete. For any member, Cauchy–Schwarz gives \[ \int_{\mathbb{R}^d}|\widehat f(\omega)|\,d\omega \leq \left(\int_{\mathbb{R}^d}\frac{|\widehat f(\omega)|^2}{s(\omega)}\,d\omega\right)^{1/2} \left(\int_{\mathbb{R}^d}s(\omega)\,d\omega\right)^{1/2}<\infty. \] Thus Fourier inversion defines a continuous function \(f\). For a fixed \(x\), the Fourier transform of the kernel section \(k(\,\cdot\,,x)\) is \(e^{-i\omega\cdot x}s(\omega)\). It has finite norm, since \[ \|k(\,\cdot\,,x)\|_{\mathcal H}^2 =(2\pi)^{-d}\int_{\mathbb{R}^d}s(\omega)\,d\omega =\kappa(0). \] Taking its inner product with \(f\) gives \[ \langle f,k(\,\cdot\,,x)\rangle_{\mathcal H} =(2\pi)^{-d}\int_{\mathbb{R}^d}e^{i\omega\cdot x}\widehat f(\omega)\,d\omega =f(x). \] So point evaluation is reproduced by the kernel section. The space is complete, contains all kernel sections, and has this reproducing property; it is therefore the RKHS of \(k\). \(\square\)

Gaussian Kernels: Exponential Frequency Penalties

Take the Gaussian kernel with variance parameter \(\sigma^2>0\) and length scale \(\ell>0\):

k(x,y)=\sigma^2\exp\left(-\frac{\|x-y\|^2}{2\ell^2}\right).

Its spectral density, under the convention above, is \(s(\omega)=\sigma^2(2\pi)^{d/2}\ell^d \exp(-\ell^2\|\omega\|^2/2)\). Consequently, its RKHS norm is

\|f\|_{\mathcal H}^2 =\frac{1}{(2\pi)^d\sigma^2(2\pi)^{d/2}\ell^d} \int_{\mathbb{R}^d}|\widehat f(\omega)|^2 \exp\left(\frac{\ell^2\|\omega\|^2}{2}\right)\,d\omega.

The exponential weight is the important feature. A function in this RKHS must have a Fourier transform that decays sufficiently rapidly to offset that weight. This requirement implies analyticity, not merely the existence of many derivatives.

Theorem: Every function in the Gaussian RKHS on \(\mathbb{R}^d\) has an extension to an entire function on \(\mathbb{C}^d\); in particular, its restriction to \(\mathbb{R}^d\) is real analytic.

Proof. For \(f\) in the Gaussian RKHS, define, for \(z\in\mathbb{C}^d\), \[ F(z)=(2\pi)^{-d}\int_{\mathbb{R}^d}e^{i\omega\cdot z}\widehat f(\omega)\,d\omega. \] Write \(z=u+iv\), with \(u,v\in\mathbb{R}^d\). On any bounded set of \(z\)'s, let \(\|v\|\leq M\). Cauchy–Schwarz bounds the absolute integral by a constant times \[ \left(\int_{\mathbb{R}^d} \exp\left(-\frac{\ell^2\|\omega\|^2}{2} +2M\|\omega\|\right)\,d\omega\right)^{1/2}, \] which is finite because the negative quadratic term dominates the linear term as \(\|\omega\|\) grows. For any derivative of order \(m\), differentiation of the integrand adds a factor bounded by \(\|\omega\|^m\); the corresponding Gaussian integral remains finite. The same bounds hold uniformly on bounded sets of \(z\), so differentiation under the integral is justified by dominated convergence. Thus \(F\) is holomorphic in every complex coordinate, and its restriction to \(\mathbb{R}^d\) equals \(f\) by Fourier inversion. \(\square\)

Worked Example: The Norm of a Gaussian Kernel Section

Let \(f(x)=k(x,a)\) for a fixed \(a\in\mathbb{R}^d\). Its Fourier transform is \(\widehat f(\omega)=e^{-i\omega\cdot a}s(\omega)\). Since \(|e^{-i\omega\cdot a}|=1\), the Fourier characterization gives

\|k(\,\cdot\,,a)\|_{\mathcal H}^2 =(2\pi)^{-d}\int_{\mathbb{R}^d} \frac{|e^{-i\omega\cdot a}s(\omega)|^2}{s(\omega)}\,d\omega =(2\pi)^{-d}\int_{\mathbb{R}^d}s(\omega)\,d\omega =\sigma^2.

The last equality follows from \(\kappa(0)=\sigma^2\), or directly by integrating the Gaussian spectral density. This also checks that the norm of a kernel section agrees with its squared evaluation norm.

Worked Example: Analyticity Does Not Guarantee Gaussian RKHS Membership

In one dimension, consider \(f(x)=e^{-a x^2}\), with \(a>0\). Its Fourier transform is \(\widehat f(\omega)=\sqrt{\pi/a}\,e^{-\omega^2/(4a)}\). Substitution into the Gaussian RKHS norm shows that, apart from a positive constant, the integral to check is

\int_{\mathbb{R}}\exp\left[ -\left(\frac{1}{2a}-\frac{\ell^2}{2}\right)\omega^2 \right]\,d\omega.

This integral is finite exactly when \(1/(2a)-\ell^2/2>0\), or \(a<1/\ell^2\). At equality its integrand is constant, and for \(a>1/\ell^2\) it grows exponentially, so the integral diverges in both cases. For example, \(a=1/(2\ell^2)\) gives finite norm, whereas \(a=2/\ell^2\) does not. Both functions are analytic. Analyticity is therefore necessary for Gaussian RKHS membership but is not sufficient.

Matérn Kernels: Sobolev Frequency Penalties

For smoothness parameter \(\nu>0\), range parameter \(\rho>0\), and variance parameter \(\sigma^2>0\), set \(\lambda=\sqrt{2\nu}/\rho\). The Matérn kernel can be specified by its spectral density:

s(\omega)= \sigma^2\frac{(4\pi)^{d/2}\Gamma(\nu+d/2)\lambda^{2\nu}}{\Gamma(\nu)} \left(\lambda^2+\|\omega\|^2\right)^{-(\nu+d/2)}.

This is the spectrum of the usual Matérn covariance \(\sigma^2\,2^{1-\nu}(\lambda r)^\nu K_\nu(\lambda r)/\Gamma(\nu)\), where \(r=\|x-y\|\) and \(K_\nu\) is the modified Bessel function of the second kind. Its decay is polynomial, with exponent determined by \(\nu+d/2\).

Definition: For \(t\geq 0\), the Bessel-potential Sobolev space \(H^t(\mathbb{R}^d)\) consists of functions with \[ \int_{\mathbb{R}^d}(1+\|\omega\|^2)^t|\widehat f(\omega)|^2\,d\omega<\infty. \] An equivalent norm may include the constant factor \((2\pi)^{-d}\).

Theorem: For fixed \(d,\nu,\rho,\sigma^2>0\), the Matérn RKHS on \(\mathbb{R}^d\) is \(H^{\nu+d/2}(\mathbb{R}^d)\) as a set. Its RKHS norm and the standard \(H^{\nu+d/2}\) norm are equivalent: each is bounded by a positive constant times the other.

Proof. The Fourier characterization and the displayed Matérn spectrum show that the RKHS norm squared is a positive constant times \[ \int_{\mathbb{R}^d} |\widehat f(\omega)|^2 (\lambda^2+\|\omega\|^2)^{\nu+d/2}\,d\omega. \] Write \(t=\nu+d/2\). Since \(\lambda>0\), there are constants \(c,C>0\) such that \[ c(1+\|\omega\|^2)\leq \lambda^2+\|\omega\|^2 \leq C(1+\|\omega\|^2) \quad\text{for all }\omega. \] For instance, one can take \(c=\min(\lambda^2,1)\) and \(C=\max(\lambda^2,1)\). Raising these positive quantities to \(t\) and multiplying by \(|\widehat f(\omega)|^2\) bounds the Matérn integrand above and below by constant multiples of the Sobolev integrand. Integration proves both equality of the function sets and equivalence of the norms. \(\square\)

Worked Example: Reading the Sobolev Order from a Matérn Kernel

Suppose \(d=2\) and \(\nu=3/2\). The Matérn spectrum has power \(\nu+d/2=3/2+1=5/2\). Therefore the RKHS on \(\mathbb{R}^2\) is \(H^{5/2}(\mathbb{R}^2)\), with an equivalent norm given by

\int_{\mathbb{R}^2}|\widehat f(\omega)|^2 (\lambda^2+\|\omega\|^2)^{5/2}\,d\omega.

Because \(\lambda>0\), replacing \(\lambda^2+\|\omega\|^2\) with \(1+\|\omega\|^2\) changes this quantity by at most fixed multiplicative constants. Thus membership is exactly the \(H^{5/2}\) condition, not a claim that the two displayed norms are numerically identical.

Comparing the Function Spaces

Every Gaussian RKHS function belongs to \(H^t(\mathbb{R}^d)\) for every finite \(t\geq0\). Indeed, its Fourier norm controls an integral weighted by \(\exp(\ell^2\|\omega\|^2/2)\), which dominates any fixed polynomial weight \((1+\|\omega\|^2)^t\). In contrast, a Matérn RKHS imposes one specific finite Sobolev order, \(t=\nu+d/2\).

The Gaussian RKHS is a strict subset of each such finite-order Sobolev space. To see why, choose a nonzero smooth function with compact support. Repeated integration by parts shows that its Fourier transform decays faster than any fixed inverse power, so it belongs to \(H^t\) for every finite \(t\). But the function vanishes on an open set without being identically zero, so it cannot be real analytic on connected \(\mathbb{R}^d\). The Gaussian analyticity theorem therefore excludes it from every Gaussian RKHS.

This comparison concerns function spaces, not a blanket ranking of kernels as better or worse. The Gaussian kernel enforces very strong global regularity. A Matérn kernel gives a tunable polynomial smoothness scale through \(\nu\), and its RKHS norm penalizes derivatives in the Sobolev sense encoded by its order.

Common Pitfalls

Confusing analyticity with membership. Every Gaussian RKHS function is analytic, but an analytic function can still have too much high-frequency content to have finite Gaussian RKHS norm, as the explicit Gaussian-function calculation showed.
Using the wrong Matérn order. The Sobolev order is \(\nu+d/2\), not \(\nu\) alone. The dimension contributes \(d/2\).
Calling equivalent norms identical. For a fixed positive range parameter, the Matérn norm is equivalent to a standard Sobolev norm. The constants depend on kernel parameters; the integrands need not be equal.
Mixing up kernel amplitude and smoothness. Changing \(\sigma^2\) rescales the RKHS norm, while \(\nu\) changes the polynomial spectral decay and thus the Sobolev order.

Key takeaway: On \(\mathbb{R}^d\), the Gaussian spectrum yields an exponentially weighted Fourier norm and analytic functions; the Matérn spectrum yields a polynomially weighted Fourier norm and the Sobolev space \(H^{\nu+d/2}\). These distinct frequency penalties explain their different function-space properties.

Check Your Understanding

Use the Fourier characterization and the stated parameter conventions to answer the following.

What feature of a Gaussian spectral density forces high frequencies to be strongly penalized in the RKHS norm?
For a Matérn kernel with \(d=3\) and \(\nu=2\), what is the Sobolev order of its RKHS?
Why does real analyticity alone not establish that a function belongs to a Gaussian RKHS?
For \(f(x)=e^{-a x^2}\) in one dimension, state the condition on \(a\) for membership in the Gaussian RKHS with length scale \(\ell\).
Why are the Matérn RKHS norm and the standard Sobolev norm equivalent when \(\lambda>0\)?