Two Different Kinds of Smoothness
In binary classification, the conditional class probability and the geometry of the decision boundary describe different features of a model. The conditional probability tells us how likely a label is at each covariate value; the boundary describes where the preferred class changes. These features can be related, but they are not interchangeable: a smooth probability function can have a geometrically nonsmooth boundary, and a geometrically smooth boundary can come from a rough probability function.
Let \(X\) take values in a domain \(\mathcal X\subseteq\mathbb R^d\), and let \(Y\in\{0,1\}\). Write \(\eta(x)=P(Y=1\mid X=x)\). A Bayes classifier assigns class 1 where \(\eta(x)\) exceeds \(1/2\), and class 0 where it is below \(1/2\). The level \(1/2\) is where the two classes have equal conditional probability.
One common way to describe function smoothness is the Hölder condition. For \(0<\beta\leq 1\), a function is Hölder-\(\beta\) on \(\mathcal X\) if there is a finite constant \(L\) such that \(|\eta(x)-\eta(y)|\leq L\|x-y\|^\beta\) for all \(x,y\in\mathcal X\). The case \(\beta=1\) is Lipschitz continuity. For larger smoothness orders, Hölder classes additionally require derivatives to exist and satisfy appropriate Hölder bounds.
Geometric smoothness describes the shape of the boundary itself. A boundary is a \(C^k\) hypersurface if, in a neighborhood of each of its points, it can be represented after a choice of coordinates as the graph of a \(k\)-times continuously differentiable function of \(d-1\) variables. This says nothing by itself about how quickly \(\eta(x)\) moves away from \(1/2\) on either side.
Smooth Probabilities Can Produce a Nonsmooth Boundary
Worked Example: A Smooth Function With a Crossing Boundary
Take \(\mathcal X=[-1,1]^2\), let \(X\) be uniform on this square, and define \(\eta(x_1,x_2)=1/2+(x_1^2-x_2^2)/8\). Given \(X=(x_1,x_2)\), let \(Y\) be Bernoulli with success probability \(\eta(x_1,x_2)\).
The probability is valid. Since \(x_1^2,x_2^2\in[0,1]\), their difference is in \([-1,1]\). Therefore \(3/8\leq\eta(x_1,x_2)\leq5/8\), so \(\eta\) is a valid conditional probability.
The function is smooth. It is a polynomial plus a constant, hence infinitely differentiable. In particular, it is Lipschitz on the compact square: its gradient is \((x_1/4,-x_2/4)\), whose Euclidean norm is at most \(\sqrt{2}/4\). The mean-value theorem then gives a finite Lipschitz bound.
The boundary is not a smooth curve at the origin. Solving \(\eta=1/2\) gives \(x_1^2=x_2^2\), or the union of the two diagonal lines \(x_2=x_1\) and \(x_2=-x_1\). They cross at the origin. In every sufficiently small neighborhood of the origin, removing the origin from this boundary leaves four branches. A one-dimensional smooth curve has only two local branches there, so this crossing is not a \(C^1\) curve. Thus smoothness of \(\eta\) alone does not guarantee a geometrically smooth boundary.
The issue is visible in the gradient. At the origin, \(\nabla\eta=(0,0)\), so the level set does not have a single locally preferred direction there. Smoothness controls how the function varies; it does not prevent its level sets from meeting or becoming singular.
A Smooth Boundary Can Come From a Rough Function
Worked Example: A Hölder Probability Profile With a Flat Boundary
Again use \(\mathcal X=[-1,1]^2\) and a uniform covariate distribution. Define \(\eta(x_1,x_2)=1/2+(1/4)\operatorname{sgn}(x_1)|x_1|^{1/2}\), with the added term set to zero at \(x_1=0\), and let \(Y\mid X\) be Bernoulli with this success probability.
The probability is valid. The signed square root lies in \([-1,1]\) on the domain, so \(\eta\) lies in \([1/4,3/4]\).
The function is Hölder-\(1/2\). For nonnegative \(a,b\), \(|\sqrt a-\sqrt b|\leq\sqrt{|a-b|}\). If \(u\) and \(v\) have opposite signs, then \[ |\operatorname{sgn}(u)\sqrt{|u|}-\operatorname{sgn}(v)\sqrt{|v|}| =\sqrt{|u|}+\sqrt{|v|} \leq\sqrt{2|u-v|}. \] For equal signs, the nonnegative inequality applies. Thus the signed square-root function is Hölder-\(1/2\), and scaling it by \(1/4\) preserves that property.
It is not Lipschitz. For \(h>0\), \[ \frac{|\eta(h,0)-\eta(0,0)|}{\|(h,0)-(0,0)\|} =\frac{\sqrt h/4}{h} =\frac{1}{4\sqrt h}, \] which diverges as \(h\) decreases to zero. No finite Lipschitz constant can satisfy the required inequality.
The boundary is a smooth line. The signed square root is positive for \(x_1>0\), negative for \(x_1<0\), and zero exactly when \(x_1=0\). Hence the decision boundary is the vertical line \(x_1=0\), a \(C^\infty\) hypersurface. Geometric smoothness does not imply Lipschitz, or differentiable, smoothness of the probability function.
This example also shows why the two notions should not be conflated in assumptions about a classification problem. Describing the line \(x_1=0\) specifies where the class preference changes, but it does not specify the behavior of \(\eta\) near that line. In this example the probability approaches \(1/2\) with a square-root profile, rather than a linear one.
When Function Smoothness Does Imply Boundary Smoothness
A useful sufficient condition combines smoothness of \(\eta\) with a nonzero gradient on the boundary. The gradient condition rules out the kind of singular crossing in the first example.
Justification. Since the gradient is nonzero, at least one partial derivative is nonzero; relabel coordinates so that \(\partial\eta(x_0)/\partial x_d\neq0\). The implicit function theorem says that, in a neighborhood of \(x_0\), the equation \(\eta(x)=1/2\) can be solved uniquely as \(x_d=g(x_1,\ldots,x_{d-1})\), where \(g\) is \(C^k\). This is exactly the local graph definition of a \(C^k\) hypersurface. The hypothesis also ensures that this level set is a boundary between the two classes locally: by continuity, \(\partial\eta/\partial x_d\) keeps its nonzero sign in a sufficiently small neighborhood, so moving in the \(x_d\) direction across the graph moves \(\eta\) to opposite sides of \(1/2\). Applying the argument at each level-set point gives the final claim. \(\square\)
The condition is sufficient, not necessary. A boundary can be smooth even where the gradient vanishes.
Worked Example: A Smooth Boundary With a Vanishing Gradient
On \([-1,1]^2\), take \(\eta(x_1,x_2)=1/2+x_1^3/4\), with \(X\) uniform and \(Y\mid X\) Bernoulli with this probability. Since \(|x_1^3|\leq1\), \(\eta\) lies between \(1/4\) and \(3/4\). It is a polynomial, so it is \(C^\infty\).
The equation \(\eta=1/2\) is \(x_1^3=0\), so the boundary is again the smooth line \(x_1=0\). But \(\partial\eta/\partial x_1=3x_1^2/4\), which is zero at every point of that line. The regular-level-set condition cannot be applied, even though the boundary is smooth. It is a convenient sufficient test, not a complete characterization of smooth boundaries.
What the Distinction Means in Practice
Function smoothness is a condition on probability values throughout the covariate domain. Boundary smoothness is a condition on the shape of one particular level set. They answer different questions, as the examples show:
| Assumption | What it controls | What it does not guarantee by itself |
|---|---|---|
| Hölder or differentiable smoothness of \(\eta\) | How conditional probabilities vary with \(x\) | A smooth, nonintersecting boundary |
| Geometric smoothness of the boundary | The local shape of the class-separating set | Smooth probability variation near that set |
| Smooth \(\eta\) and nonzero boundary gradient | Both probability regularity and a regular level set | Behavior of the covariate distribution or other noise properties |
In particular, boundary geometry alone does not determine how quickly \(\eta(x)\) moves away from \(1/2\) as \(x\) moves away from the boundary. Nor does smoothness of \(\eta\) rule out a boundary with a crossing or singular point. Those distinctions matter whenever assumptions are used to describe a classification problem or justify a rate calculation: verify which property the argument actually requires rather than substituting one for the other.
Common Mistake
Incorrect reasoning: “The regression function is smooth, so its decision boundary must be smooth.” The first worked example disproves this: a polynomial probability function has two level-set branches crossing at the origin.
Correction: Smoothness of the function controls its derivatives, but a smooth level set additionally requires suitable behavior of those derivatives at the level of interest. A nonzero gradient provides a standard sufficient condition. Conversely, a smooth geometric boundary does not force the probability function to be differentiable, as the signed square-root example shows.
Check Your Understanding
Use the definitions and examples above to check the distinction.
- For \(\eta(x_1,x_2)=1/2+(x_1^2-x_2^2)/8\), what is the boundary, and why is it not a smooth curve at the origin?
- Why is the signed square-root probability function Hölder-\(1/2\) but not Lipschitz?
- State the regular-level-set condition that makes a \(C^k\) regression function have a locally \(C^k\) boundary.
- Does a nonzero-gradient condition have to hold for every smooth boundary? Explain using the cubic example.
- What information about probabilities near the boundary is not supplied by knowing only its geometric shape?