Loss Matrices and Conditional Risk for Multiclass Decisions

Introduction

In multiclass classification, an incorrect prediction need not have the same consequence for every class. Confusing two similar categories may be relatively inexpensive, while missing a serious condition may be much more costly. A loss matrix records these differences explicitly, and conditional risk summarizes the expected cost of an action given the available information.

The central calculation is simple but consequential: weight each possible outcome’s cost by its conditional probability. This tutorial defines that calculation, proves its basic formula, and works through finite-label examples. It treats risk calculations for specified actions; the further question of how to choose an action from these risks is separate.

Loss Matrices and Conditional Risk

Let the true label \(Y\) take values in the finite set \(\{1,\ldots,K\}\), and let the available actions be \(a\in\mathcal{A}=\{1,\ldots,M\}\). An action might be a predicted class, but the action set need not equal the set of true labels. For example, actions could be treatment choices while labels represent disease states.

Because both sets are finite, a loss matrix contains finitely many finite entries, so the loss for any fixed action is integrable. Losses are often nonnegative, but the conditional-risk formula below only needs finite entries. No symmetry is required: the cost of taking action \(a\) when the truth is \(j\) need not equal the cost of taking action \(j\) when the truth is \(a\). Nor must correct classifications have zero loss, although that is a common convention.

For each label \(j\), write \(\eta_j(X)=P(Y=j\mid X)\) for a version of the conditional class probability. These quantities are nonnegative and sum to one almost surely. When we write \(\eta_j(x)\), we mean values from a chosen version of the conditional distribution at \(x\). If \(X\) is continuous, this notation does not mean a ratio involving \(P(X=x)\), which may be zero.

Proof. For each \(j\), let \(\mathbf{1}\{Y=j\}\) equal one if \(Y=j\), and zero otherwise. Exactly one of these indicators equals one, so

Take the conditional expectation given \(X\). Finite sums can be moved through conditional expectation, and \(L(a,j)\) is a constant for fixed \(a,j\). Also, \(E[\mathbf{1}\{Y=j\}\mid X]=P(Y=j\mid X)=\eta_j(X)\). Therefore

This proves the formula. \(\square\)

In vector notation, if \(\eta(X)=(\eta_1(X),\ldots,\eta_K(X))^\top\), the conditional risk of action \(a\) is the inner product of row \(a\) of \(C\) with \(\eta(X)\). The calculation uses probabilities conditional on the observed information, not the unconditional class proportions.

Unequal Costs in Binary and Nonstandard Actions

Even with two labels, a loss matrix need not treat the two kinds of error equally. In a matrix, the location of a cost matters: the row identifies what was done, and the column identifies what was true. Reading those indices carefully prevents a common reversal of false-positive and false-negative costs.

An action does not have to name a class. When actions have a different meaning from labels, the matrix still works in exactly the same way: each row lists the costs of that action across possible truths. This makes the definition useful whenever the consequences of a decision are specified separately from the outcomes being predicted.

From Conditional Risk to Overall Expected Loss

Conditional risk describes the expected loss at a particular level of observed information. For a rule \(d\) that assigns an action to each observed \(X\), the overall risk averages the loss across the distribution of \(X\) as well. Formally, take \(d(X)\) to be measurable and valued in the finite action set, and define its overall risk as \(R(d)=E[L(d(X),Y)]\).

Proof. For each action \(a\), define the event \(A_a=\{d(X)=a\}\). These events are measurable with respect to \(X\), are disjoint, and together cover the sample space. The loss can be written as

Taking conditional expectation given \(X\), the indicator of \(A_a\) can be taken outside that conditional expectation because it is determined by \(X\). Applying the fixed-action formula to each term gives

Taking expectations of both sides and using the tower property of conditional expectation yields the stated expression for \(R(d)\). All sums are finite, and the losses are bounded because the matrix has finitely many finite entries, so these expectations exist. \(\square\)

Interpretation and a Common Pitfall

A loss matrix converts a conditional distribution over labels into one conditional risk per action. The conditional probabilities describe uncertainty about the truth; the matrix describes the consequences of each action under each truth. Keeping these roles separate is useful in applications: probability estimation and cost specification are different ingredients in the calculation.

A common mistake is to treat the loss matrix as if its rows were interchangeable or to weight costs by the wrong class probabilities. For example, in the binary example, reporting 1 has cost 4 under truth 0, not under truth 1. Its risk is therefore \(4P(Y=0\mid X)\), not \(4P(Y=1\mid X)\). The correction is to label rows as actions and columns as truths before doing any arithmetic, then multiply each entry by the conditional probability for its column.

Another mistake is to call a conditional risk an overall risk. The quantity \(r(a\mid X)\) varies with the observed information because the conditional class probabilities vary. Overall risk averages the relevant conditional risks over the distribution of \(X\). In practice, the matrix, probability vector, and conditioning information should all be stated so that a risk calculation can be checked.