Evaluation Norms and Continuity in an RKHS

Why Evaluation Norms Matter

An RKHS is a Hilbert space of functions in which evaluating a function at an input is a continuous linear operation. The size of that operation depends on the input: some evaluation functionals can have a larger operator norm than others. The reproducing property gives an exact answer in terms of the kernel’s diagonal. It also provides a direct route from continuity of the kernel to continuity of every function in the space.

Throughout, let \(X\) be a set, let \(K:X\times X\to\mathbb{R}\) be a positive definite kernel, and let \(\mathcal{H}_K\) be its real RKHS. Write \(K_x=K(\cdot,x)\) for the kernel section at \(x\). The reproducing property is \(f(x)=\langle f,K_x\rangle_{\mathcal{H}_K}\) for all \(f\in\mathcal{H}_K\) and \(x\in X\). In particular, \(\langle K_x,K_y\rangle_{\mathcal{H}_K}=K(x,y)\).

The operator norm measures the largest absolute value that evaluation at \(x\) can produce among functions with RKHS norm at most one. The definition of an RKHS ensures that each \(E_x\) is bounded. The next result identifies its norm exactly, including the case where evaluation is identically zero.

The Exact Norm of Point Evaluation

Proof. By the reproducing property and the Cauchy–Schwarz inequality, for every \(f\in\mathcal{H}_K\),

Also, \(\|K_x\|_{\mathcal{H}_K}^2=\langle K_x,K_x\rangle_{\mathcal{H}_K}=K(x,x)\). Thus \(\|E_x\|_{\mathrm{op}}\leq\sqrt{K(x,x)}\). If \(K(x,x)>0\), take \(f=K_x/\sqrt{K(x,x)}\), which has RKHS norm one. Then

So the upper bound is attained. If \(K(x,x)=0\), then \(\|K_x\|_{\mathcal{H}_K}=0\), hence \(K_x\) is the zero element of the Hilbert space. The reproducing property then gives \(f(x)=\langle f,K_x\rangle_{\mathcal{H}_K}=0\) for every \(f\). In this case \(\|E_x\|_{\mathrm{op}}=0\) as required. \(\square\)

The diagonal value is nonnegative because it equals \(\|K_x\|_{\mathcal{H}_K}^2\). The theorem also supplies a pointwise bound for every RKHS function:

This inequality is sharp at each fixed \(x\) when \(K(x,x)>0\), as the normalized section used in the proof shows. It is a bound at one input; it does not by itself assert that the function is bounded over all of \(X\). A global bound follows if the diagonal is uniformly bounded: if \(\sup_{x\in X}K(x,x)\leq M<\infty\), then \(\sup_{x\in X}|f(x)|\leq\sqrt{M}\|f\|_{\mathcal{H}_K}\).

Kernel Continuity Gives Function Continuity

Now suppose that \(X\) is a topological space and that \(K\) is jointly continuous on \(X\times X\). The key quantity is the RKHS distance between two kernel sections. For \(x,y\in X\), the inner-product identities for sections give

This identity connects a change in input to a change in the corresponding section as a vector in the Hilbert space. It leads to a bound on the change of any function in the space.

Proof. Apply the reproducing property to the difference of the two function values, then apply Cauchy–Schwarz:

Fix \(x_0\in X\). The function \(G(x,y)=K(x,x)+K(y,y)-2K(x,y)\) is continuous on \(X\times X\), since \(K\) is jointly continuous and the diagonal maps \(x\mapsto(x,x)\) and \(y\mapsto(y,y)\) are continuous. Also \(G(x_0,x_0)=0\). Given \(\varepsilon>0\), if \(\|f\|_{\mathcal{H}_K}=0\), then \(f\) is identically zero by the reproducing property and is continuous. Otherwise, continuity of \(G\) at \((x_0,x_0)\) gives a neighborhood \(U\) of \(x_0\) such that \(G(x,x_0)<(\varepsilon/\|f\|_{\mathcal{H}_K})^2\) for \(x\in U\). The displayed bound with \(y=x_0\) then yields \(|f(x)-f(x_0)|<\varepsilon\). Thus \(f\) is continuous at \(x_0\). Since \(x_0\) was arbitrary, every \(f\in\mathcal{H}_K\) is continuous. \(\square\)

What the Continuity Condition Does—and Does Not—Say

Positive definiteness alone does not imply that RKHS functions are continuous. The kernel condition in the theorem is substantive, not a consequence of the reproducing property. To see this, take \(X=\mathbb{R}\) with its usual topology and define \(K(x,y)=1\) if \(x=y\), and \(K(x,y)=0\) otherwise. This kernel is positive definite: for any finite inputs \(x_1,\ldots,x_m\) and real coefficients \(c_1,\ldots,c_m\),

where the outer sum is over the distinct input values among \(x_1,\ldots,x_m\). But the kernel section \(K_0\) is the function equal to one at zero and zero elsewhere. It belongs to its RKHS by construction and is discontinuous at zero. Correspondingly, \(K\) is not jointly continuous at \((0,0)\).

Joint continuity is a sufficient condition for continuity of every RKHS function. The bound also describes how continuity is controlled: for a fixed \(f\), changes in its values are bounded by its RKHS norm multiplied by the distance between kernel sections. This does not claim that every continuous function on \(X\) belongs to the RKHS, nor that every such function has a finite RKHS norm.

Common Mistakes

• Using the diagonal value instead of its square root. The section has squared norm \(K(x,x)\), but the operator norm of evaluation is \(\|K_x\|_{\mathcal{H}_K}=\sqrt{K(x,x)}\).
• Assuming the upper bound is always attained by a normalized section. The expression \(K_x/\sqrt{K(x,x)}\) is defined only when \(K(x,x)>0\). If the diagonal is zero, the section and evaluation functional are both zero.
• Inferring continuity from positive definiteness. Positive definiteness ensures an RKHS construction, not continuity of its functions. The discontinuous equality kernel above is a counterexample.
• Claiming more than the bound establishes. The continuity result is pointwise. It does not assert a uniform continuity rate over a noncompact domain without additional conditions.