The Exponential Loss Calibration Function

Conditional Risk and the Exponential Loss

A classification score is useful when its sign agrees with the more probable class. Calibration quantifies the conditional surrogate-risk penalty for a score that fails to do so. For exponential loss, that penalty can be calculated explicitly: first minimize the conditional risk over all scores, then minimize it subject to a wrong-sign constraint.

We use the loss $\phi(u)=2e^{-u}$ for a signed margin $u$. The factor of two is a normalization: it gives the calibration function in the convention used here. If instead the loss is written $e^{-u}$, all conditional risks and the calibration function below are divided by two.

Definition: Let $Y\in\{-1,+1\}$, let $\eta=P(Y=+1\mid X=x)$, and let $t\in\mathbb{R}$ be a score at $x$. The conditional risk of the exponential loss is $$ C_\eta(t)=E[\phi(Yt)\mid X=x] =2\left[\eta e^{-t}+(1-\eta)e^t\right]. $$ The conditional classification regret of a wrong-sign score is $q=|2\eta-1|$. For $0\leq\varepsilon\leq1$, define the calibration function by $$ \psi(\varepsilon)= \inf_{\substack{\eta\in[0,1],\,t\in\mathbb{R}\\ |2\eta-1|\geq\varepsilon,\;t(2\eta-1)\leq0}} \left[C_\eta(t)-\inf_{v\in\mathbb{R}}C_\eta(v)\right]. $$ The constraint includes a score of zero as a wrong-sign score.

The quantity $q$ is the difference between the probabilities of the two classes. For example, if $\eta=0.8$, then $q=|0.8-0.2|=0.6$. When the score chooses the less probable class, the conditional zero-one regret is exactly $q$; at a tie, $q=0$. The calibration function asks for the smallest conditional surrogate excess risk possible when this regret is at least $\varepsilon$.

It is convenient to orient the score toward the more probable class. Write $p=\max(\eta,1-\eta)=(1+q)/2$, and let $u$ be the score after orienting its sign so that $u\geq0$ points toward that class. The conditional risk then takes the form $$ C_q(u)=2\left[p e^{-u}+(1-p)e^u\right],\qquad p=\frac{1+q}{2}. $$ A score constrained to the wrong side has $u\leq0$. This reparameterization lets us do both minimizations using the same expression.

The Unrestricted Conditional-Risk Minimum

For $0<p<1$, differentiate the oriented risk: $$ C_q'(u)=2\left[-p e^{-u}+(1-p)e^u\right],\qquad C_q''(u)=2\left[p e^{-u}+(1-p)e^u\right]>0. $$ Strict convexity means that a stationary point is the unique global minimizer. Solving $C_q'(u)=0$ gives the minimizing score and risk.

Result: For $0<p<1$, the unrestricted conditional-risk minimizer and minimum are $$ u^*=\frac12\log\left(\frac{p}{1-p}\right), \qquad \inf_{u\in\mathbb{R}}C_q(u)=4\sqrt{p(1-p)}=2\sqrt{1-q^2}. $$ At $p=0$ or $p=1$, the same minimum value is an infimum, approached as the score tends without bound toward the certain class.

To verify the minimum, the stationary-point equation gives $p e^{-u^*}=(1-p)e^{u^*}$. Equivalently, $e^{2u^*}=p/(1-p)$. Substituting yields $p e^{-u^*}=\sqrt{p(1-p)}$ and $(1-p)e^{u^*}=\sqrt{p(1-p)}$. Their sum is $2\sqrt{p(1-p)}$, and the factor of two in the loss makes the risk $4\sqrt{p(1-p)}$. Finally, $p(1-p)=(1-q^2)/4$, giving $2\sqrt{1-q^2}$.

Worked Example: Minimizing Risk When the Positive Class Has Probability 0.8

Let $\eta=0.8$. The positive class is more probable, so $p=0.8$ and $q=|2(0.8)-1|=0.6$. The minimizing oriented score is $$ u^*=\frac12\log\left(\frac{0.8}{0.2}\right) =\frac12\log 4=\log 2. $$ At this score, $e^{-u^*}=1/2$ and $e^{u^*}=2$. Therefore $$ C_q(u^*)=2\left[0.8\left(\frac12\right)+0.2(2)\right] =2(0.4+0.4)=1.6. $$ The formula for the minimum gives the same answer: $2\sqrt{1-0.6^2}=2\sqrt{0.64}=1.6$.

The Minimum Under a Wrong-Sign Constraint

Now restrict to $u\leq0$. For $u\leq0$, $e^{-u}\geq1$ and $e^u\leq1$, so $$ C_q'(u)=2[-p e^{-u}+(1-p)e^u] \leq 2[-p+(1-p)]=-2q\leq0. $$ Thus the risk is nonincreasing as $u$ increases on this constrained interval. Its minimum is attained at the endpoint $u=0$, where $$ C_q(0)=2[p+(1-p)]=2. $$ This also covers $q=0$: then the derivative is zero at the endpoint and the minimum is still 2. For $q=1$, the expression is $C_q(u)=2e^{-u}$, which decreases over $u\leq0$, again giving its constrained minimum of 2 at zero.

Result: For every $q\in[0,1]$, the minimum conditional risk among scores with the wrong sign is $$ \inf_{u\leq0}C_q(u)=2. $$ Consequently, at a fixed class imbalance $q$, the smallest surrogate excess risk of a wrong-sign score is $2-2\sqrt{1-q^2}$.

Worked Example: The Cost of a Wrong-Sign Score at Class Imbalance 0.6

Continue with $\eta=0.8$, so the oriented risk is $2[0.8e^{-u}+0.2e^u]$ and $q=0.6$. A wrong-sign score has $u\leq0$. The constrained minimum occurs at $u=0$, where $$ C_q(0)=2[0.8+0.2]=2. $$ The unrestricted minimum is 1.6, as calculated above. The smallest excess risk among wrong-sign scores is therefore $2-1.6=0.4$. Equivalently, the fixed-$q$ formula gives $$ 2(1-\sqrt{1-0.6^2})=2(1-0.8)=0.4. $$ A score farther into the wrong-sign region cannot improve this value: the constrained risk is minimized at its boundary, zero.

Deriving the Calibration Function

At fixed $q$, the constrained-minus-unconstrained risk is $$ \Delta(q)=2-2\sqrt{1-q^2}=2(1-\sqrt{1-q^2}). $$ The definition of $\psi(\varepsilon)$ also allows any imbalance $q\geq\varepsilon$. We therefore need to check which such $q$ gives the smallest gap.

Theorem: For the loss $\phi(u)=2e^{-u}$, the calibration function is $$ \psi(\varepsilon)=2\left(1-\sqrt{1-\varepsilon^2}\right), \qquad 0\leq\varepsilon\leq1. $$

Proof. Fix $q\in[0,1]$. The unrestricted minimum is $2\sqrt{1-q^2}$, and the minimum under the wrong-sign constraint is 2. Hence every feasible pair with this imbalance has excess risk at least $\Delta(q)=2(1-\sqrt{1-q^2})$; equality is attained by the score $u=0$. For $0\leq q\leq1$, $1-q^2$ decreases as $q$ increases, so $\sqrt{1-q^2}$ decreases and $\Delta(q)$ increases. Among all $q\geq\varepsilon$, the smallest value is therefore $\Delta(\varepsilon)$. It is attainable by choosing an imbalance $q=\varepsilon$ and the constrained score $u=0$. Thus the infimum in the definition equals $2(1-\sqrt{1-\varepsilon^2})$, as claimed. $\square$

Worked Example: Evaluating the Calibration Function at 0.8

Suppose the required conditional classification regret is at least $\varepsilon=0.8$. The formula gives $$ \psi(0.8)=2\left(1-\sqrt{1-0.8^2}\right) =2(1-\sqrt{0.36})=2(1-0.6)=0.8. $$ This value is attained at imbalance $q=0.8$, for example when $\eta=0.9$, and score zero. At that class probability, the unrestricted risk is $2\sqrt{1-0.8^2}=1.2$, while the wrong-sign constrained minimum is 2; their difference is $0.8$.

Checks, Interpretation, and a Common Pitfall

The endpoints provide useful checks. If $\varepsilon=0$, then $\psi(0)=0$: at a class-probability tie, there is no conditional classification regret to penalize. If $\varepsilon=1$, then $\psi(1)=2$: at a certain class label, the unrestricted exponential risk has infimum zero, while the best wrong-sign score has risk 2. For $0<\varepsilon<1$, the function is positive and increasing. These properties match its role: a larger required classification regret cannot be achieved at a smaller surrogate-risk cost.

The conditional minimizer also explains how exponential loss responds to class imbalance. For $0<p<1$, its score is $u^*=\tfrac12\log(p/(1-p))$, positive when the positive class is more probable and negative when it is less probable. Its magnitude grows with imbalance. At a certain class probability, the infimum is approached only as the oriented score tends to $+\infty$; there is no finite minimizing score. That endpoint does not affect the constrained minimum or the calibration calculation.

Pitfall: dropping the loss normalization. With the unscaled loss $e^{-u}$, the conditional minima are $1$ under the wrong-sign constraint and $\sqrt{1-q^2}$ without it. Their difference is $1-\sqrt{1-q^2}$, not $2(1-\sqrt{1-q^2})$. The correction is to keep the loss convention fixed throughout: multiplying the loss by two multiplies both risks and their difference by two.
Pitfall: minimizing only over strictly wrong scores. The calibration constraint includes the boundary score $u=0$. If one instead imposes $u<0$, the infimum is still approached as $u$ tends to zero from below, but it need not be attained. Keeping the non-strict constraint makes the minimizing score explicit.
Pitfall: comparing the wrong-sign risk with zero. Calibration uses the excess over the unrestricted conditional minimum, not the constrained risk alone. The relevant quantity is $2-2\sqrt{1-q^2}$, not 2.

The calculation matters because a conditional-risk gap translates the quality of a surrogate score into a cost associated with an incorrect classification decision. In the exponential case, the explicit formula provides that translation without requiring a numerical optimization. Its scale depends on the chosen loss normalization, while the conditional minimizers and the shape of the relationship follow from the same two risk minimizations.

Check Your Understanding

Use the conditional-risk minima and the definition of $\psi$ to answer these questions.

For $\eta=0.7$, find $q$, the oriented unrestricted minimizing score, and the minimum conditional risk.
Why is $u=0$ the minimizing score under the constraint $u\leq0$, including when $q=0$?
Calculate $\psi(0.6)$, showing the square-root calculation.
What calibration function results if the loss is $e^{-u}$ instead of $2e^{-u}$?
At $\varepsilon=1$, explain why the unrestricted conditional risk is an infimum rather than a minimum at a finite score.