Common Errors in Margin Exponent Conventions

Why Margin Conventions Need Care

A margin condition describes how much probability lies near the classification threshold. Two common versions measure that distance differently: one uses $|\eta(X)-1/2|$, and the other uses $|2\eta(X)-1|$. The second quantity is exactly twice the first. That simple scaling leaves the exponent unchanged, but it changes the constant in a bound—and can change a calculation if the threshold is not converted too.

This distinction matters when reading or comparing classification results. A constant copied from one convention into the other may be wrong even when the exponent is right. The safest method is to write the two margin probabilities, substitute the rescaled threshold, and only then compare constants.

Two Definitions of the Margin Condition

Let $X$ be a feature vector and let $\eta(x)=P(Y=1\mid X=x)$ be the conditional probability of class 1 in binary classification. Since $0\leq\eta(x)\leq1$, the distance $|\eta(x)-1/2|$ is at most $1/2$. Define the doubled margin score $S(x)=|2\eta(x)-1|$, which is at most 1.

Definition: For $t>0$, define the margin probabilities $$ M_{\eta}(t)=P\bigl(|\eta(X)-1/2|\leq t\bigr), \qquad M_{S}(t)=P\bigl(|2\eta(X)-1|\leq t\bigr). $$ A margin condition with exponent $\alpha\geq0$ and constant $C_{\eta}>0$ in the first convention is $M_{\eta}(t)\leq C_{\eta}t^\alpha$ over the stated range of $t$. In the doubled-score convention it is $M_S(t)\leq C_St^\alpha$ over the stated range.

The range of thresholds is part of the condition. A common choice is $0<t\leq1/2$ for $M_{\eta}$ and $0<t\leq1$ for $M_S$, the full ranges of the respective scores. One may instead state a condition for all $t>0$. When comparing two statements, check both the constant and the threshold range.

The Exact Conversion

The scores satisfy $S(X)=2|\eta(X)-1/2|$ pointwise. Consequently, the event that the doubled score is at most $t$ is exactly the event that the original distance is at most $t/2$. This identity gives the conversion, not an approximation.

Theorem: For every $t>0$, $$ M_S(t)=M_{\eta}(t/2), \qquad M_{\eta}(t)=M_S(2t). $$ Thus, if $M_{\eta}(u)\leq C_{\eta}u^\alpha$ for all $0<u\leq1/2$, then $M_S(t)\leq C_{\eta}2^{-\alpha}t^\alpha$ for all $0<t\leq1$. Conversely, if $M_S(s)\leq C_Ss^\alpha$ for all $0<s\leq1$, then $M_{\eta}(t)\leq C_S2^\alpha t^\alpha$ for all $0<t\leq1/2$.

If $0<t\leq1$, then $0<t/2\leq1/2$. Applying the first-convention bound at $u=t/2$ gives $$ M_S(t)=M_{\eta}(t/2)\leq C_{\eta}(t/2)^\alpha =C_{\eta}2^{-\alpha}t^\alpha. $$ For the converse, if $0<t\leq1/2$, then $0<2t\leq1$, and $$ M_{\eta}(t)=M_S(2t)\leq C_S(2t)^\alpha =C_S2^\alpha t^\alpha. $$ This proves both conversions on the stated ranges. $\square$

The exponents match because multiplying a threshold by a fixed positive number changes the power bound by a constant factor: $(t/2)^\alpha=2^{-\alpha}t^\alpha$. It does not turn $t^\alpha$ into a different power of $t$. The conversion also works for $\alpha=0$; in that case $2^\alpha=1$, so the displayed constants do not change.

Worked Example: Converting a Bound in the Half-Distance Convention

Suppose that for $0<u\leq1/2$, $$ P\bigl(|\eta(X)-1/2|\leq u\bigr)\leq 3u^2. $$ Find a valid constant for the doubled-score condition with the same exponent.

For $0<t\leq1$, the threshold in the first convention must be $u=t/2$, not $u=t$. Therefore, $$ P\bigl(|2\eta(X)-1|\leq t\bigr) =P\bigl(|\eta(X)-1/2|\leq t/2\bigr) \leq 3(t/2)^2 =\frac{3}{4}t^2. $$ Thus the doubled-score condition holds with exponent $2$ and constant $3/4$. The exponent did not change; the constant was multiplied by $2^{-2}=1/4$.

Worked Example: Converting Back to Half-Distance

Suppose that for $0<s\leq1$, $$ P\bigl(|2\eta(X)-1|\leq s\bigr)\leq 5s^{3/2}. $$ To express this in terms of $|\eta(X)-1/2|$, set $s=2t$. For $0<t\leq1/2$, $$ P\bigl(|\eta(X)-1/2|\leq t\bigr) =P\bigl(|2\eta(X)-1|\leq2t\bigr) \leq5(2t)^{3/2} =5\cdot2^{3/2}t^{3/2}. $$ The corresponding constant is $5\cdot2^{3/2}$, approximately $14.14$, with exponent $3/2$. Using $5\cdot2^{-3/2}$ here would reverse the conversion and give an unjustified bound.

What “The Same Condition” Means for Constants

A statement that a distribution satisfies a margin condition often gives a permissible constant rather than the smallest possible one. The exact conversion above says how a given permissible constant transfers. It also identifies the relationship between the smallest constants when the conditions are assessed over their full natural threshold ranges.

Result: Fix $\alpha\geq0$ and define the optimal constants on the natural ranges by $$ C_{\eta}^{*}=\sup_{0<t\leq1/2}\frac{M_{\eta}(t)}{t^\alpha}, \qquad C_S^{*}=\sup_{0<s\leq1}\frac{M_S(s)}{s^\alpha}. $$ These suprema may be infinite. Then $C_S^{*}=2^{-\alpha}C_{\eta}^{*}$.

Proof. Substitute $s=2t$. As $t$ ranges over $(0,1/2]$, $s$ ranges over $(0,1]$, and the event identity gives $M_S(s)=M_{\eta}(s/2)$. Hence $$ \frac{M_S(s)}{s^\alpha} =\frac{M_{\eta}(s/2)}{s^\alpha} =2^{-\alpha}\frac{M_{\eta}(s/2)}{(s/2)^\alpha}. $$ Taking the supremum over $0<s\leq1$ is the same as taking the supremum over $0<t\leq1/2$ after setting $t=s/2$. The fixed factor $2^{-\alpha}$ can be taken outside the supremum, proving the result. $\square$

The identity concerns optimal constants for exactly corresponding ranges. If a paper uses a smaller threshold interval, a constant that works on that interval may not be optimal on the full range. The pointwise probability identity still holds, but a comparison of constants must respect the ranges on which their bounds were established.

Worked Example: Checking the Conversion on a Specific Distribution

Let $X$ be uniform on $(0,1)$, and set $\eta(X)=(1+X)/2$. Then $|\eta(X)-1/2|=X/2$, while $|2\eta(X)-1|=X$. For $0<t\leq1/2$, $$ M_{\eta}(t)=P(X/2\leq t)=P(X\leq2t)=2t. $$ For $0<s\leq1$, $$ M_S(s)=P(X\leq s)=s. $$ Both conditions have exponent $1$. On their natural ranges the optimal constants are $C_{\eta}^{*}=2$ and $C_S^{*}=1$, which satisfy $C_S^{*}=2^{-1}C_{\eta}^{*}$. For a direct threshold check, $M_S(0.6)=0.6=M_{\eta}(0.3)$.

Common Errors and Their Corrections

Keeping the threshold fixed. The incorrect step is to write $M_S(t)=M_{\eta}(t)$. The scores differ by a factor of two, so the correct identity is $M_S(t)=M_{\eta}(t/2)$.
Changing the exponent. A factor of two in the score does not change $\alpha$. It changes the multiplier through $2^\alpha$ or $2^{-\alpha}$.
Using the conversion in the wrong direction. From a half-distance bound to a doubled-score bound, the constant is multiplied by $2^{-\alpha}$. Going back, it is multiplied by $2^\alpha$.
Ignoring the stated threshold range. To convert the bound for $0<t\leq1/2$, substitute only thresholds that remain in its valid range. The natural ranges correspond under $s=2t$.
Assuming the constants are numerically identical. They are identical under this conversion only when $\alpha=0$, or when one is quoting loose bounds rather than converting a particular bound.

One useful check is to test the conversion at a single threshold: a doubled-score threshold $s$ corresponds to a half-distance threshold $s/2$. A second check is dimensional in spirit: the probability is unchanged, while the threshold raised to the exponent acquires the factor $2^\alpha$. These checks catch many sign and direction errors.

Key takeaway: The two margin conventions describe the same near-threshold events after rescaling the threshold. With exponent $\alpha$, the conversion is $C_S=C_{\eta}2^{-\alpha}$ from half-distance to doubled score, and $C_{\eta}=C_S2^\alpha$ in the reverse direction. Always convert the threshold as well as the constant.

Check Your Understanding

For each question, keep the threshold range and direction of conversion explicit.

If $M_{\eta}(u)\leq4u^3$ on $0<u\leq1/2$, what constant gives a doubled-score bound on $0<t\leq1$?
If $M_S(s)\leq2s^{1/2}$ on $0<s\leq1$, what constant gives the corresponding half-distance bound?
Why does changing from $|\eta(X)-1/2|$ to $|2\eta(X)-1|$ leave the exponent unchanged?
What threshold in the half-distance convention corresponds to a doubled-score threshold of $0.8$?
For which values of $\alpha$ does the numerical constant remain unchanged under conversion?