Squared-Hinge Calibration and Conditional Minimizers

Conditional Risk and Its Minimizers

A margin loss assigns a cost to a score according to whether that score agrees with the observed class. For squared-hinge loss, the cost grows quadratically when the signed margin falls below one, and is zero once the margin reaches one. To assess calibration, we first need to know exactly which score minimizes the conditional surrogate risk at each posterior probability.

Definition: Let $Y\in\{-1,+1\}$, let $t\in\mathbb{R}$ be a score, and write $\eta=P(Y=+1\mid X=x)$ for the posterior probability at a fixed $x$. The squared-hinge margin loss and its conditional risk are $$ \phi(u)=(1-u)_+^2,\qquad C_\eta(t)=\eta\phi(t)+(1-\eta)\phi(-t), $$ where $a_+=\max(a,0)$. Thus, explicitly, $$ C_\eta(t)=\eta(1-t)_+^2+(1-\eta)(1+t)_+^2. $$

The first term is the loss when $Y=+1$, whose signed margin is $t$; the second is the loss when $Y=-1$, whose signed margin is $-t$. The posterior probability weights these two possible outcomes. We will minimize $C_\eta(t)$ over all real scores $t$, treating $\eta$ as fixed.

For $-1\le t\le 1$, both positive-part terms are active, so the risk is an ordinary quadratic: $$ C_\eta(t)=\eta(1-t)^2+(1-\eta)(1+t)^2 =t^2+2(1-2\eta)t+1. $$ Completing the square gives $$ C_\eta(t)=\bigl(t-(2\eta-1)\bigr)^2+4\eta(1-\eta). $$ The vertex $t=2\eta-1$ lies in $[-1,1]$ for every $\eta\in[0,1]$. To check that no score outside this interval improves the risk, observe that for $t\ge1$, $C_\eta(t)=(1-\eta)(1+t)^2$, which is nondecreasing, and for $t\le-1$, $C_\eta(t)=\eta(1-t)^2$, which is nonincreasing as $t$ increases. For $0<\eta<1$, the respective exterior minima occur at the endpoints $1$ and $-1$; the quadratic decreases to its vertex before $1$ and increases after its vertex, so neither exterior region improves on that vertex. At $\eta=0$ or $1$, the risk is constant at zero on one exterior ray, giving the endpoint cases below.

Theorem: For $0<\eta<1$, the squared-hinge conditional risk has the unique minimizer $t_\eta^*=2\eta-1$, and its minimum is $C_\eta(t_\eta^*)=4\eta(1-\eta)$. If $\eta=0$, every $t\le-1$ minimizes the risk, with minimum zero. If $\eta=1$, every $t\ge1$ minimizes the risk, also with minimum zero.

For an interior posterior, substituting $t_\eta^*=2\eta-1$ makes the two margins $2\eta-1$ and $1-2\eta$. Equivalently, the two hinge residuals are $1-t_\eta^*=2(1-\eta)$ and $1+t_\eta^*=2\eta$. Therefore $$ C_\eta(t_\eta^*)=\eta\,4(1-\eta)^2+(1-\eta)\,4\eta^2 =4\eta(1-\eta)\bigl((1-\eta)+\eta\bigr) =4\eta(1-\eta). $$ When $\eta=1$, the positive-class term is zero for every $t\ge1$ and the negative-class term has weight zero. When $\eta=0$, the corresponding zero-risk set is $t\le-1$. This explains why the minimizer is not unique at the two endpoints.

Worked Example: Posterior Probability 0.8

Suppose $\eta=0.8$. The interior minimizer is $t_\eta^*=2(0.8)-1=0.6$. Its risk is $$ C_{0.8}(0.6) =0.8(1-0.6)^2+0.2(1+0.6)^2 =0.8(0.16)+0.2(2.56) =0.128+0.512 =0.64. $$ The minimum-risk formula agrees: $4(0.8)(0.2)=0.64$. The minimizing score is positive, as expected when the positive class has posterior probability $0.8$.

Worked Example: Posterior Probability 0.25

Here $t_\eta^*=2(0.25)-1=-0.5$. Direct substitution gives $$ C_{0.25}(-0.5) =0.25(1.5)^2+0.75(0.5)^2 =0.25(2.25)+0.75(0.25) =0.5625+0.1875 =0.75. $$ The formula $4(0.25)(0.75)=0.75$ confirms the calculation. In this case the minimizing score is negative, matching the more probable negative class.

Worked Example: A Boundary Posterior

Let $\eta=1$, so $Y=+1$ with conditional probability one. Then $$ C_1(t)=(1-t)_+^2. $$ This is zero for every $t\ge1$, and strictly positive for $t<1$. Thus the full set of minimizers is $[1,\infty)$, not just the endpoint $t=1$. For example, $t=2$ gives $C_1(2)=0$, while $t=0.5$ gives $C_1(0.5)=0.25$.

The Derivative Criterion for Calibration

The minimizers already show the essential sign property: when $\eta>1/2$, $2\eta-1>0$; when $\eta<1/2$, it is negative; and at $\eta=1/2$, it is zero. The derivative criterion expresses this property for a broader class of convex margin losses, without requiring a direct calculation of every conditional minimizer.

Definition: For $\eta>1/2$, define the wrong-sign conditional surrogate gap by $$ \Delta_\eta=\inf_{t\le0}C_\eta(t)-\inf_{t\in\mathbb{R}}C_\eta(t). $$ For $\eta<1/2$, define it by $$ \Delta_\eta=\inf_{t\ge0}C_\eta(t)-\inf_{t\in\mathbb{R}}C_\eta(t). $$ A margin loss is classification calibrated if $\Delta_\eta>0$ for every $\eta\ne1/2$. These constraints count a zero score as wrong, making the definition conservative at the decision boundary.

For a differentiable convex margin loss $\phi$, the conditional risk is $C_\eta(t)=\eta\phi(t)+(1-\eta)\phi(-t)$, and differentiation gives $$ C_\eta'(0)=\eta\phi'(0)-(1-\eta)\phi'(0) =(2\eta-1)\phi'(0). $$ This identity leads to the derivative criterion.

Theorem: If $\phi:\mathbb{R}\to\mathbb{R}$ is differentiable and convex, then its binary margin loss is classification calibrated if and only if $\phi'(0)<0$.

Proof. First suppose $\phi'(0)<0$. For $\eta>1/2$, $C_\eta'(0)<0$. Convexity makes $C_\eta'$ nondecreasing, so $C_\eta'(t)\le C_\eta'(0)<0$ for all $t\le0$. Hence $C_\eta$ decreases as $t$ increases on that half-line, and its infimum over $t\le0$ is $C_\eta(0)$. Since $C_\eta'(0)<0$, the definition of derivative ensures that for some sufficiently small $h>0$, $C_\eta(h)<C_\eta(0)$. Thus the unrestricted infimum is strictly below the wrong-sign infimum, and $\Delta_\eta>0$. For $\eta<1/2$, $C_\eta'(0)>0$. The derivative is at least $C_\eta'(0)$ for $t\ge0$, so the infimum over that half-line is $C_\eta(0)$; a sufficiently small move to the left lowers the risk. Again, $\Delta_\eta>0$.

Conversely, suppose $\phi'(0)\ge0$. Take any $\eta>1/2$. Then $C_\eta'(0)\ge0$, and convexity implies $C_\eta(t)\ge C_\eta(0)$ for all $t\ge0$. The unrestricted infimum is therefore attained, or approached, on $t\le0$, since $0$ itself is in that region and no score on the positive half-line has lower risk than $C_\eta(0)$. Consequently the wrong-sign infimum equals the unrestricted infimum, so $\Delta_\eta=0$. The loss is not calibrated. This proves both directions. $\square$

Applying the Criterion to Squared-Hinge Loss

For squared-hinge loss, differentiation of $\phi(u)=(1-u)_+^2$ yields $$ \phi'(u)=-2(1-u)_+, \qquad \phi'(0)=-2. $$ The derivative exists even at $u=1$: the derivative from below is $-2(1-u)$, which tends to zero, matching the derivative from above. The loss is convex because its derivative is nondecreasing: it increases from negative values to zero on $u<1$, and remains zero on $u\ge1$. Since $\phi'(0)=-2<0$, the derivative criterion proves classification calibration.

For this loss, the conditional gap can also be calculated exactly. If $\eta>1/2$, then $C_\eta'(0)=-2(2\eta-1)<0$. By convexity, $C_\eta$ is decreasing on $t\le0$, so the wrong-sign infimum is $C_\eta(0)=1$. The unrestricted infimum is $4\eta(1-\eta)$. Therefore $$ \Delta_\eta=1-4\eta(1-\eta)=(2\eta-1)^2. $$ For $\eta<1/2$, symmetry gives the same expression. In either case, with $z=|2\eta-1|$, the exact gap is $z^2$. Since $z>0$ whenever $\eta\ne1/2$, this also verifies calibration directly.

Worked Example: Wrong-Sign Gap at Posterior 0.8

For $\eta=0.8$, a wrong-sign score satisfies $t\le0$. The conditional derivative at zero is $C_{0.8}'(0)=-2(2(0.8)-1)=-1.2$, so convexity implies that the smallest risk over $t\le0$ occurs at $t=0$. There, $C_{0.8}(0)=0.8+0.2=1$. The unrestricted minimum is $0.64$, calculated earlier. Thus $$ \Delta_{0.8}=1-0.64=0.36=(2(0.8)-1)^2. $$ The gap is strictly positive: restricting the score to the wrong sign necessarily increases surrogate risk.

Interpretation and a Common Pitfall

The minimizer $2\eta-1$ is a score aligned with the posterior imbalance, not the posterior probability itself. Its magnitude is the difference between the two class probabilities, while the minimum risk $4\eta(1-\eta)$ is smallest at the posterior endpoints and largest at $\eta=1/2$. These facts distinguish the surrogate score from a probability estimate; calibration here concerns recovery of the more probable class by the score's sign.

A common mistake is to check only that $\phi(u)\ge0$, or to observe that the loss is convex, and then conclude calibration. Neither fact alone establishes that conditional minimizers have the correct sign. The derivative at zero matters: for squared hinge it is strictly negative, and the conditional derivative is consequently negative at score zero when $\eta>1/2$, positive when $\eta<1/2$. Convexity then ensures the risk favors the corresponding sign. Another error is to use the endpoint minimizer formula $2\eta-1$ as if it were the unique minimizer at $\eta=0$ or $1$; the risk is flat over an entire ray at those endpoints.

Key takeaway: Squared-hinge conditional risk is minimized at $t=2\eta-1$ for interior posterior probabilities, with minimum $4\eta(1-\eta)$. Its derivative at zero is $-2$, so it satisfies the convex margin derivative criterion; the wrong-sign conditional surrogate gap is exactly $(2\eta-1)^2$.

Check Your Understanding

Use the conditional risk and derivative calculations to answer the following.

For $\eta=0.65$, find the minimizing score and minimum conditional risk.
Why is the minimizer set at $\eta=1$ a ray rather than a single score?
Calculate the squared-hinge wrong-sign conditional gap when $\eta=0.3$.
State the derivative criterion for a differentiable convex margin loss and identify the derivative value that verifies it for squared hinge.
Explain why convexity alone is not enough to conclude that a margin loss is calibrated.