Calibration Functions for Piecewise-Linear Convex Losses

Conditional Risk and the Calibration Question

A convex margin loss assigns a penalty $\phi(ya)$ to a score $a$ when the label is $y\in\{-1,+1\}$. To understand calibration, fix the covariates and let $\eta=P(Y=+1\mid X)$. The conditional risk describes the expected loss at a particular score. Its unconstrained minimum is the best surrogate risk at this covariate value; its minimum over scores predicting the wrong class measures the cost of making a classification error.

The difference between these minima is the conditional calibration gap. Its dependence on the conditional classification difficulty determines how much surrogate regret must accompany a wrong classification. For a piecewise-linear loss, both minima can be found from a finite set of candidate scores, and the resulting gap is piecewise linear in the class-probability imbalance.

Definition: Let $\phi:\mathbb R\to[0,\infty)$ be a finite convex piecewise-linear margin loss, and suppose $P(Y=+1\mid X)=\eta$. The conditional risk at score $a$ is $$ C_\eta(a)=\eta\phi(a)+(1-\eta)\phi(-a). $$ Write $C_\eta^*=\inf_{a\in\mathbb R}C_\eta(a)$. For $\eta\ne 1/2$, the wrong-sign conditional minimum is the infimum over scores on the side that predicts the less probable class, including zero as a boundary score. The calibration function at imbalance $t\in[0,1]$ is the wrong-sign conditional minimum minus $C_\eta^*$, where $t=|2\eta-1|$. Continuity of $C_\eta$ makes this infimum agree with the infimum over strictly wrong-sign scores when the decision rule assigns zero to the other class.

It is useful to orient the score so that positive values point toward the more probable class. Set $q=(1+t)/2$, the probability of that class, and let $u$ be the score after this orientation. The conditional risk becomes

R_t(u)=q\phi(u)+(1-q)\phi(-u),\qquad q=\frac{1+t}{2}.

In these coordinates, the wrong-sign region is $u\le 0$. The calibration function is therefore

\delta_\phi(t)=\inf_{u\le 0}R_t(u)-\inf_{u\in\mathbb R}R_t(u).

At $t=0$, the classes are equally probable and the classification regret is zero, so the calibration gap is defined as zero. The formula also gives zero: the constrained and unconstrained risks coincide at an optimal score of zero or at another score with equal conditional risk.

Finding Minima from Breakpoints and Slopes

Suppose the distinct breakpoints of $\phi$ are $b_1,\ldots,b_m$. Between consecutive breakpoints the loss is affine. Consequently, the conditional risk $R_t$ is affine except at scores in the finite set

K=\{b_1,\ldots,b_m,-b_1,\ldots,-b_m\}.

A risk breakpoint can come either from $\phi(u)$ changing slope at $u=b_j$, or from $\phi(-u)$ changing slope at $u=-b_j$. If the slope of $\phi$ on a segment containing $u$ is $s_1$, and its slope on a segment containing $-u$ is $s_2$, then the slope of the conditional risk on the corresponding interval is

$$ q s_1-(1-q)s_2. $$

Thus slopes determine whether risk falls or rises across each interval, while breakpoints delimit the intervals that must be checked.

Theorem: For a finite, nonnegative convex piecewise-linear loss with finitely many breakpoints, the unconstrained minimum of $R_t$ can be found by evaluating it at the points in $K\cup\{0\}$. The constrained minimum over $u\le 0$ can be found by evaluating it at the points in $(K\cap(-\infty,0])\cup\{0\}$. In particular, $$ \delta_\phi(t)= \min_{c\in(K\cap(-\infty,0])\cup\{0\}}R_t(c) - \min_{c\in K\cup\{0\}}R_t(c). $$

Proof. The function $R_t$ is convex and piecewise linear, with breakpoints contained in $K$. On any interval between consecutive breakpoints it is affine. An affine function on a closed bounded interval attains a minimum at an endpoint, unless it is constant, in which case both endpoints attain the same minimum. The same reasoning applies to the intervals cut out by $K$ on the constrained half-line. On an unbounded tail, an affine function bounded below either has its minimum at the finite endpoint or is constant on the tail; in the latter case its value at the endpoint is also minimal. Since $R_t\ge 0$, it is bounded below. Adding zero to the candidate sets also covers the case in which no breakpoint lies in the relevant region. Therefore the listed finite candidate evaluations suffice for both minima, proving the formula. $\square$

At any fixed candidate score $c$, its risk is affine in $t$:

R_t(c)=\frac{\phi(c)+\phi(-c)}{2} +t\frac{\phi(c)-\phi(-c)}{2}.

Each minimum in the theorem is thus the minimum of finitely many affine functions of $t$. The active candidate can change as $t$ changes, producing a piecewise-linear calibration function. Its intervals and slopes are determined by which score candidates minimize the constrained and unconstrained risks.

Worked Example: The Hinge Loss

Take $\phi(z)=\max(0,1-z)$. Its breakpoint is $1$, so the conditional risk changes slope at $u=1$ and $u=-1$. For $0\le u\le 1$, $$ R_t(u)=q(1-u)+(1-q)(1+u)=1-tu. $$ For $u\ge 1$, the first loss term is zero and $$ R_t(u)=(1-q)(1+u), $$ which is nondecreasing in $u$. The minimum over $u\ge0$ is attained at $u=1$, with value $2(1-q)=1-t$.

On the wrong side, for $-1\le u\le0$, $R_t(u)=1-tu\ge1$. For $u<-1$, the risk is convex and its slope is no larger than its slope immediately to the left of zero, which is $-t\le0$; moving toward zero therefore cannot increase the risk. Hence the constrained minimum is $R_t(0)=1$. Subtracting gives $$ \delta_\phi(t)=1-(1-t)=t. $$ A wrong-sign score therefore has conditional surrogate regret at least the conditional classification regret $t$.

Worked Example: A Shifted Hinge

Now let $\phi(z)=\max(0,2-z)$, whose breakpoint is $2$. For $0\le u\le2$, $$ R_t(u)=q(2-u)+(1-q)(2+u)=2-tu. $$ For $u\ge2$, $R_t(u)=(1-q)(2+u)$, which is nondecreasing. Thus the unrestricted minimum on the nonnegative side is attained at $u=2$, with value $4(1-q)=2(1-t)$. On $u\le0$, the risk is at least its value $2$ at zero: on $[-2,0]$ it equals $2-tu$, and convexity gives the same lower bound farther left. Therefore $$ \delta_\phi(t)=2-2(1-t)=2t. $$ The margin breakpoint changes the optimal score, while the slope of the calibration function reflects the loss’s scale.

Worked Example: A Loss with Two Active Breakpoints

Consider $$ \phi(z)=\max\{0,\,2-z,\,4-3z\}. $$ The active pieces are $4-3z$ for $z\le1$, $2-z$ for $1\le z\le2$, and $0$ for $z\ge2$. For $0\le u\le1$, direct substitution gives $R_t(u)=4-3tu$, which decreases to $u=1$ when $t>0$. For $1\le u\le2$, $$ R_t(u)=q(2-u)+(1-q)(4+3u), $$ whose slope is $3-4q=1-2t$. For $u\ge2$, the risk is $(1-q)(4+3u)$, with nonnegative slope $3(1-q)$. The unrestricted minimizer is therefore $u=1$ when $0\le t\le1/2$, and $u=2$ when $1/2\le t\le1$. The respective risk values are $4-3t$ and $10(1-q)=5(1-t)$.

For $u\le0$, the risk is minimized at zero. To see this, its slope immediately to the left of zero is $-3t$, and convexity makes the slope no greater farther to the left; the risk cannot be smaller there than at zero. Since $R_t(0)=4$, the calibration function is $$ \delta_\phi(t)= \begin{cases} 3t, & 0\le t\le 1/2,\\ 5t-1, & 1/2\le t\le1. \end{cases} $$ At $t=1/2$, both formulas give $3/2$. The change of slope occurs because the unrestricted minimizer switches from the breakpoint $1$ to the breakpoint $2$.

From Conditional Gaps to Classification Regret

For a fixed covariate value, the conditional classification regret of a wrong-sign prediction is $t=|2\eta-1|$: choosing the less probable class incurs error probability $\max(\eta,1-\eta)$, rather than the Bayes error $\min(\eta,1-\eta)$, and their difference is $t$.

Regret bound: At any covariate value with $t>0$, if a score predicts the less probable class, its conditional surrogate regret is at least $\delta_\phi(t)$. In symbols, for a wrong-sign score $u\le0$, $$ R_t(u)-\inf_{v\in\mathbb R}R_t(v)\ge \delta_\phi(t). $$

Proof. By definition, $\inf_{u\le0}R_t(u)$ is the smallest conditional risk among wrong-sign scores. Every such score satisfies $$ R_t(u)\ge \inf_{v\le0}R_t(v). $$ Subtracting the unrestricted minimum $\inf_{v\in\mathbb R}R_t(v)$ from both sides gives the stated inequality. If the decision rule makes an error only at a boundary tie, continuity lets the same bound be obtained by taking the limit from the wrong-sign side. $\square$

This is a pointwise conversion: the calibration function specifies the surrogate-regret cost that must accompany a classification error at each imbalance level. For a particular loss, its exact shape—not just convexity—determines how strong that conversion is. The hinge example gives a linear gap with slope one; the two-breakpoint example gives a different slope in each imbalance range.

Common Mistake: Using Only the Slope at Zero

A tempting shortcut is to inspect the conditional-risk slope at zero and assume it identifies the unrestricted minimizer for every $t$. That slope indicates the initial direction of improvement, but it cannot determine where the minimum lies after the risk crosses further breakpoints. In the two-breakpoint example, the risk initially decreases for every $t>0$, yet the minimizing score is $1$ for $t<1/2$ and $2$ for $t>1/2$. The correction is to calculate the risk slope on every interval, evaluate the candidate breakpoints, and compare the resulting risks. A change in the active candidate can create a new piece of the calibration function even when the initial direction stays the same.

Key takeaway: For a convex piecewise-linear margin loss, reflected loss breakpoints determine where conditional-risk slopes can change. Evaluating the risk at those candidate scores gives the exact constrained and unconstrained minima; their difference is the calibration function and yields a pointwise lower bound on surrogate regret whenever the prediction is wrong.

Check Your Understanding

Use the candidate-score method and show the intermediate risk calculations.

For the hinge loss, what are the breakpoints of the conditional risk in oriented-score coordinates?
For the shifted hinge $\phi(z)=\max(0,2-z)$, why does the unrestricted minimum occur at $u=2$ rather than at zero when $t>0$?
In the two-breakpoint example, what is the slope of the risk on $1\le u\le2$, and at what imbalance does its sign change?
Explain why the minimum of finitely many affine candidate risks is piecewise linear in $t$.
State the pointwise regret inequality that follows when a score predicts the less probable class.

Calibration Functions for Piecewise-Linear Convex Losses

What You'll Learn