Isotropic Hölder Balls

Why Specify a Smoothness Class?

In nonparametric regression, a smoothness class describes which regression functions are plausible before any particular estimator is chosen. The smoothness order determines how rapidly the function can change locally; the radius limits its overall size. A precise definition matters because conventions at integer orders are not universal, and changing them can change which functions belong to the class.

Here the domain is the cube $Q=[-1,1]^d$, where $d$ is a positive integer. The class is isotropic: the same smoothness order is imposed in every coordinate direction, and distances are measured with the Euclidean norm. We use the convention that an integer order $r$ requires derivatives through order $r-1$ to be Lipschitz. It does not require derivatives of order $r$ to exist everywhere.

Definition of an Isotropic Hölder Ball

A multi-index is a vector $\nu=(\nu_1,\ldots,\nu_d)$ of nonnegative integers. Write $|\nu|=\sum_{j=1}^d \nu_j$, and let $D^\nu f$ denote the corresponding partial derivative; for example, $D^{(2,1)}f=\partial^3 f/\partial x_1^2\partial x_2$. Derivatives on the boundary of $Q$ are understood as continuous extensions from the interior.

Definition: Let $\beta>0$, set $k=\lceil\beta\rceil-1$, and set $\gamma=\beta-k$. Thus $0<\gamma\leq 1$. For a function with continuous derivatives through order $k$, define $$ \|f\|_{C^\beta(Q)} = \sum_{|\nu|\leq k}\|D^\nu f\|_\infty + \max_{|\nu|=k}[D^\nu f]_{\gamma,Q}, \qquad [g]_{\gamma,Q} = \sup_{\substack{x,y\in Q\\x\ne y}} \frac{|g(x)-g(y)|}{\|x-y\|_2^\gamma}. $$ The isotropic Hölder ball of radius $L>0$ is $$ \mathcal H^ \beta_d(L) = \{f:\|f\|_{C^\beta(Q)}\leq L\}. $$

The supremum norm is $\|g\|_\infty=\sup_{x\in Q}|g(x)|$. The seminorm $[g]_{\gamma,Q}$ measures how quickly $g$ can vary across the domain. The sum in the norm controls the size of every derivative through order $k$, while the final term controls the modulus of continuity of derivatives of the highest required order.

For noninteger $\beta$, $k=\lfloor\beta\rfloor$ and $0<\gamma<1$: derivatives of order $k$ must be $\gamma$-Hölder. For integer $\beta=r$, this convention gives $k=r-1$ and $\gamma=1$: derivatives of order $r-1$ must be Lipschitz. This explicit integer convention is important; do not silently replace it with a definition that requires continuous derivatives of order $r$.

Worked Examples of Membership

Worked Example: A Function in a Ball of Order One-Half

Let $d\geq1$, take $\beta=1/2$, and set $f(x)=x_1$ on $Q$. Here $k=0$ and $\gamma=1/2$, so the norm consists of the supremum of $|f|$ plus its $1/2$-Hölder seminorm. Since $|x_1|\leq1$, $\|f\|_\infty=1$. For $x,y\in Q$, $$ |f(x)-f(y)|=|x_1-y_1| \leq \|x-y\|_2 \leq (2\sqrt d)^{1/2}\|x-y\|_2^{1/2}. $$ The last inequality follows from $\|x-y\|_2\leq2\sqrt d$. Therefore $[f]_{1/2,Q}\leq(2\sqrt d)^{1/2}$, and $f\in\mathcal H^{1/2}_d(L)$ whenever $L\geq1+(2\sqrt d)^{1/2}$.

Worked Example: A Noninteger Order and a Derivative Modulus

Take $\beta=3/2$ and $f(x)=x_1^2$. Now $k=1$ and $\gamma=1/2$. The supremum of $|f|$ is 1; the only nonzero first derivative is $\partial f/\partial x_1=2x_1$, whose supremum is 2. For any $x,y\in Q$, $$ \left|\frac{\partial f}{\partial x_1}(x)-\frac{\partial f}{\partial x_1}(y)\right| =2|x_1-y_1| \leq 2\|x-y\|_2 \leq 2(2\sqrt d)^{1/2}\|x-y\|_2^{1/2}. $$ All other first-derivative seminorms are zero. Hence $\|f\|_{C^{3/2}(Q)}\leq3+2(2\sqrt d)^{1/2}$, so this function belongs to the ball for any radius at least that value. The fractional condition applies to the first derivatives, not merely to $f$.

Worked Example: Integer Order Under the Lipschitz Convention

Take $\beta=2$ and $f(x)=x_1|x_1|$. Its first partial derivative in coordinate 1 is $2|x_1|$, including the value zero at $x_1=0$; all other first partial derivatives are zero. The function is continuously differentiable, $\|f\|_\infty=1$, and $\|\partial f/\partial x_1\|_\infty=2$. Moreover, $$ |2|x_1|-2|y_1|| \leq 2|x_1-y_1| \leq 2\|x-y\|_2. $$ Thus the first derivative is Lipschitz with constant at most 2, and $\|f\|_{C^2(Q)}\leq5$. This function belongs to $\mathcal H^2_d(5)$ under the stated convention, even though its second derivative does not exist at points where $x_1=0$.

Taylor Approximation Follows from the Definition

A central use of the Hölder condition is to control the error when a function is approximated locally by a polynomial. For a multi-index $\nu$, write $\nu!=\prod_{j=1}^d\nu_j!$ and $h^\nu=\prod_{j=1}^d h_j^{\nu_j}$.

Theorem: If $f\in\mathcal H^\beta_d(L)$, then for any $x,y\in Q$, with $h=y-x$, the degree-$k$ Taylor polynomial $$ T_{x,k}(y)=\sum_{|\nu|\leq k}\frac{D^\nu f(x)}{\nu!}h^\nu $$ satisfies $$ |f(y)-T_{x,k}(y)|\leq C_{d,k,\gamma}L\|h\|_2^\beta, $$ where $C_{d,k,\gamma}$ is finite and depends only on $d,k,\gamma$.

Proof. If $k=0$, then $\beta=\gamma$, $T_{x,0}(y)=f(x)$, and the bound follows immediately from $[f]_{\gamma,Q}\leq L$. Now suppose $k\geq1$. The segment $x+th$, $0\leq t\leq1$, lies in $Q$, since $Q$ is convex. Define $g(t)=f(x+th)$. The chain rule gives $$ g^{(j)}(t)=\sum_{|\nu|=j}\frac{j!}{\nu!}h^\nu D^\nu f(x+th). $$ One-dimensional Taylor expansion with integral remainder, followed by subtracting the constant value $g^{(k)}(0)$ inside the integral, gives $$ g(1)-\sum_{j=0}^{k}\frac{g^{(j)}(0)}{j!} = \frac{1}{(k-1)!}\int_0^1(1-t)^{k-1}\bigl(g^{(k)}(t)-g^{(k)}(0)\bigr)\,dt. $$ For each multi-index of order $k$, the Hölder condition implies $$ |D^\nu f(x+th)-D^\nu f(x)|\leq L t^\gamma\|h\|_2^\gamma. $$ Also, $\sum_{|\nu|=k}(k!/\nu!)|h^\nu|=(\sum_{j=1}^d|h_j|)^k\leq d^{k/2}\|h\|_2^k$. Consequently, $$ |g^{(k)}(t)-g^{(k)}(0)| \leq Ld^{k/2}t^\gamma\|h\|_2^{k+\gamma}. $$ Substituting this bound into the integral proves the result, because $\int_0^1(1-t)^{k-1}t^\gamma\,dt$ is finite and $k+\gamma=\beta$. Finally, the chain-rule expression for $g^{(j)}(0)$ identifies the Taylor sum with $T_{x,k}(y)$. $\square$

The proof also explains why the integer convention works: when $\beta$ is an integer, the last derivatives used in the argument are Lipschitz, which supplies exactly the order-$\beta$ remainder bound. No derivative of order $\beta$ is needed for this conclusion.

At Smoothness at Least One, Functions Are Lipschitz

Result: If $f\in\mathcal H^\beta_d(L)$ for $\beta\geq1$, then $$ |f(y)-f(x)|\leq \sqrt d\,L\|y-x\|_2 \qquad (x,y\in Q). $$

Proof. If $\beta=1$, then $k=0,\gamma=1$, and the definition gives $[f]_{1,Q}\leq L$, which implies the stated bound. If $\beta>1$, then $k\geq1$. The norm bound implies $\|D^{e_j}f\|_\infty\leq L$ for each coordinate vector $e_j$. Along the segment from $x$ to $y$, the fundamental theorem of calculus gives $$ f(y)-f(x)=\int_0^1 \nabla f(x+t(y-x))^\mathsf{T}(y-x)\,dt. $$ At every point on the segment, $\|\nabla f\|_2\leq\sqrt d\,L$. The Cauchy–Schwarz inequality therefore bounds the absolute value of the integral by $\sqrt d\,L\|y-x\|_2$, as claimed. $\square$

This implication is useful when interpreting a smoothness assumption: at order one or higher, the class in particular prevents arbitrarily abrupt changes. At higher orders, the additional derivative and modulus conditions provide more refined local approximation control.

Common Mistake: Treating Integer Orders as Automatic Extra Derivatives

A common mistake is to read “order two” as requiring a continuous second derivative, or to assume that every source uses the same integer-order definition. Under the convention here, order two requires a continuous first derivative that is Lipschitz. The worked example $x_1|x_1|$ satisfies that condition but is not twice differentiable everywhere. Conversely, merely having a continuous first derivative is not enough: its first derivative must also have a finite Lipschitz seminorm.

The radius $L$ is another part of the specification, not a decorative constant. It bounds both the function and the relevant derivatives and modulus. Two classes with the same $\beta$ but different radii need not contain the same functions. In applications, the smoothness order describes the type of regularity, while the radius controls its scale.

Check Your Understanding

Use the stated integer convention and Euclidean-distance seminorm in your answers.

For $\beta=2.4$, what are $k$ and $\gamma$, and which derivatives receive the fractional Hölder condition?
For integer $\beta=3$, state the highest derivative order required to be Lipschitz.
Why does the line segment argument in the Taylor bound remain inside $Q$?
Does the order-two convention require $x_1|x_1|$ to have a second derivative everywhere? Explain.
For $\beta\geq1$, which part of the norm bound yields a bound on the gradient?

What You'll Learn