Score-Based Decisions and Decoder Maps

From Scores to Decisions

A multiclass prediction system often produces a vector of numbers before it produces a label or action. Those numbers are not yet a decision: a rule must say how to turn them into one. This separation matters because the same score vector can lead to different actions under different error costs. Choosing the class with the largest score is a natural rule, but it is not universally appropriate.

Let the possible true labels be $1,\ldots,K$, and let $\mathcal{A}$ be a finite set of available actions. The actions may be the labels themselves, but they need not be. For example, an action could be to request an additional test rather than assign a class. A loss function $L:\mathcal{A}\times\{1,\ldots,K\}\to\mathbb{R}$ assigns a loss $L(a,y)$ to taking action $a$ when the true label is $y$.

Definition: A score vector is a vector $s\in\mathbb{R}^K$ whose coordinates are indexed by classes. A decoder map is a specified function $D:\mathbb{R}^K\to\mathcal{A}$ that converts a score vector into an action. If a prediction procedure produces a score vector $f(x)$ for input $x$, its resulting decision rule is $h(x)=D(f(x))$. The meaning of the coordinates and the decoder are both part of the decision rule.

A common decoder is the argmax rule. When the actions are the classes, it chooses a class with the largest coordinate:

D_{\mathrm{argmax}}(s)\in\operatorname*{arg\,max}_{y\in\{1,\ldots,K\}}s_y.

If the maximum is attained by more than one class, the set-valued argmax does not specify a unique prediction. A deterministic decoder must include a tie convention, such as choosing the smallest-indexed maximizer. Unless stated otherwise, numerical examples below use the unique minimizer or maximizer.

Loss-Sensitive Decoding

Suppose the score vector $q=(q_1,\ldots,q_K)$ is a probability vector: $q_y\geq 0$ for every class and $\sum_y q_y=1$. It can then be used to calculate the expected loss of each action under those class probabilities. The loss-sensitive decoder chooses an action with the smallest such expected loss.

D_L(q)\in\operatorname*{arg\,min}_{a\in\mathcal{A}}\sum_{y=1}^K L(a,y)q_y.

This rule uses the whole vector, not just its largest coordinate. Two actions can have different consequences across the possible true labels, so the action associated with the most probable class need not have the lowest expected loss. This definition specifies how to decode probability-valued scores; it does not say how those scores were obtained.

Worked Example: Asymmetric Costs Change the Decision

There are three labels, $A,B,C$, and the available actions are to predict one of those labels. Suppose the probability-valued score vector is $q=(0.46,0.35,0.19)$. The loss matrix below has actions in rows and true labels in columns.

Action	True $A$	True $B$	True $C$
Predict $A$	0	1	2
Predict $B$	1	0	0.2
Predict $C$	1	1	0

Argmax decoding selects $A$, since $0.46>0.35>0.19$. For loss-sensitive decoding, calculate the expected loss of each action:

Predicting $A$ has expected loss $0(0.46)+1(0.35)+2(0.19)=0.35+0.38=0.73$.

Predicting $B$ has expected loss $1(0.46)+0(0.35)+0.2(0.19)=0.46+0.038=0.498$.

Predicting $C$ has expected loss $1(0.46)+1(0.35)+0(0.19)=0.46+0.35=0.81$.

The smallest value is $0.498$, so the loss-sensitive decoder selects $B$. The difference is not a calculation error: predicting $A$ is most likely to be correct, but the stated costs make predicting $B$ the lower-loss action.

Theorem: For any finite action set and loss function $L$, define the action score $T_a(q)=-\sum_{y=1}^K L(a,y)q_y$ for each action $a$. Then a loss-sensitive decoder is equivalently an argmax decoder over these action scores: $$ \operatorname*{arg\,min}_{a\in\mathcal{A}}\sum_{y=1}^K L(a,y)q_y = \operatorname*{arg\,max}_{a\in\mathcal{A}}T_a(q). $$

Proof. For each action $a$, $T_a(q)$ is exactly the negative of its expected loss. For any two actions $a,b$, \[ \sum_y L(a,y)q_y\leq \sum_y L(b,y)q_y \quad\Longleftrightarrow\quad -\sum_y L(a,y)q_y\geq-\sum_y L(b,y)q_y. \] Thus the actions attaining the minimum expected loss are precisely those attaining the maximum action score. This is an equivalence of optimizing the same finite list of numbers with opposite signs; ties are preserved. $\square$

This result also clarifies a possible ambiguity in the phrase “argmax decoding.” One may take the largest class score, or one may first form an action score for each available action and then take the largest action score. They are the same operation only when the score coordinates and the decoder make them so.

When Class Argmax Is Appropriate

For zero-one loss, the loss is $L(a,y)=0$ when the predicted class $a$ equals the true class $y$, and $L(a,y)=1$ otherwise. When the scores are class probabilities, the expected loss of predicting class $a$ is the probability of not being in class $a$. This gives the exact condition under which class argmax is loss-sensitive decoding.

Theorem: Under zero-one loss and probability-valued scores $q$, the actions minimizing expected loss are exactly the classes maximizing $q_a$. Consequently, argmax decoding is Bayes-optimal for this loss when $q$ gives the relevant class probabilities.

Proof. Fix a class $a$. Its expected zero-one loss is \[ \sum_{y=1}^K L(a,y)q_y =\sum_{y\ne a}q_y =1-q_a, \] where the final equality uses $\sum_y q_y=1$. For any classes $a,b$, $1-q_a\leq 1-q_b$ if and only if $q_a\geq q_b$. Therefore the expected-loss minimizers are precisely the probability maximizers, including all ties. $\square$

Worked Example: Zero-One Loss Recovers Argmax

Take three classes $A,B,C$, probability scores $q=(0.52,0.31,0.17)$, and zero-one loss. The three possible prediction risks are:

Predict $A$: $0(0.52)+1(0.31)+1(0.17)=0.48$.

Predict $B$: $1(0.52)+0(0.31)+1(0.17)=0.69$.

Predict $C$: $1(0.52)+1(0.31)+0(0.17)=0.83$.

The smallest risk is $0.48$, attained by $A$, which also has the largest score $0.52$. If two classes instead tied for the largest probability, the theorem says they would also tie for minimum zero-one risk; a unique prediction would then require a tie-breaking convention.

Scores Need an Interpretation

A vector in $\mathbb{R}^K$ is not automatically a probability vector. It may contain probabilities, margins, log-odds, or values designed to rank classes. These interpretations support different decoding operations. In particular, inserting arbitrary real-valued scores into $\sum_y L(a,y)s_y$ does not generally produce an expected loss: negative coordinates or coordinates that do not sum to one invalidate the probability interpretation.

Sometimes a system directly supplies one score per action, with larger values intended to mean more desirable actions. Then its decoder can be the argmax of those action scores. This is distinct from taking class scores and treating them as probabilities. In the first case the action-score meaning is specified directly; in the second, a loss matrix can convert probability-valued class scores into action scores.

Worked Example: The Same Numbers Can Have Different Roles

Suppose three class-indexed outputs are $s=(2.4,1.8,0.5)$. If these are declared to be action utilities for actions $A,B,C$, then maximizing utility selects $A$, since $2.4$ is largest. If instead the numbers are merely class-ranking scores, the same argmax still selects $A$, but their magnitudes do not by themselves define probabilities or expected losses. For example, they sum to $4.7$, not $1$, and using them as probability weights would not be justified by the score vector alone.

The choice of decoder is therefore part of the model specification. “Take the largest score” is well-defined after the score coordinates have been assigned a meaning, but it does not establish that the resulting action minimizes a particular loss.

Why the Distinction Matters

The decoder can change the practical behavior of a classifier even when its scores are held fixed. Under zero-one loss, only the most probable class matters for the final decision. Under unequal costs, the other class probabilities can matter as well, because they contribute to each action’s expected loss. A rare class with a particularly costly error can therefore affect the preferred action.

This distinction is useful whenever the objective is not simply to maximize the probability of a correct label. It also makes evaluation more coherent: the loss used to choose an action should reflect the consequences that matter in the application. If the application permits actions beyond the labels, the decoder can map scores to those actions too, provided their losses are defined.

Common Mistake

Wrong reasoning: “The first score is largest, so predicting class 1 must be optimal.”

Correction: The conclusion follows for minimizing zero-one loss only when the scores represent class probabilities (or otherwise have an appropriate ordering), as the zero-one result shows. With general costs, calculate or otherwise compare the expected loss of each action. With arbitrary scores, first establish what the coordinates mean and what decoder they are intended to support. A score vector alone does not determine an optimal decision.

Key takeaway: A prediction system consists of both a score-producing rule and a decoder. Class argmax is the right decoder for probability scores under zero-one loss; other losses generally call for comparing action-specific scores or expected losses.

Check Your Understanding

For each question, state what the scores mean before deciding what decoder is justified.

Under zero-one loss, why does predicting the class with the largest probability minimize expected loss?
A score vector has a negative coordinate and its coordinates do not sum to one. Why is it not automatically valid to use it as a probability vector in a loss calculation?
Can two decoders choose different actions from the same class-probability vector? Describe a condition under which they can.
What tie convention is needed to turn a set-valued argmax into a deterministic decoder?
Explain the difference between maximizing a class score and maximizing an action score derived from a loss matrix.

Action	True \(A\)	True \(B\)	True \(C\)
Predict \(A\)	0	1	2
Predict \(B\)	1	0	0.2
Predict \(C\)	1	1	0