Introduction
A conditional-risk calculation can look simple and still lead to a wrong decision. A familiar summary, such as a mean, may not minimize the loss being used; more than one action may minimize risk; and a risk that can be made arbitrarily small need not have an action attaining its infimum. A further source of error is treating a pointwise regret as though it were already an average over the data.
These are not merely technical distinctions. A decision rule is judged by its loss, its available actions, and the information on which it can depend. The checks below make those ingredients explicit and show how they fit together.
Conditional Risk and the Meaning of a Minimizer
Let \(Y\) be an outcome, \(X\) observed information, \(\mathcal A\) an action space, and \(L(a,Y)\) the loss from taking action \(a\in\mathcal A\). For an action chosen after observing \(X=x\), its conditional risk is the conditional expected loss. When the action set is finite, write
A decision rule \(\delta\) assigns an action \(\delta(x)\) to each observed value \(x\). Its conditional risk at \(x\) is \(r(\delta(x),x)\), and its total risk is \(R(\delta)=E[L(\delta(X),Y)]\), when this expectation is finite. A conditional minimizer at \(x\) is an action in
The set notation matters: \(M(x)\) can contain one action, several actions, or no action. If it is empty, the infimum of conditional risk may still be finite, but no action attains it.
Worked Example: The Mean Need Not Minimize Absolute Error
Worked Example: The Mean Need Not Minimize Absolute Error
Suppose \(X\) is fixed and the conditional distribution of \(Y\) assigns probability \(0.6\) to \(0\) and probability \(0.4\) to \(10\). Compare the conditional mean with actions under absolute-error loss \(L(a,Y)=|Y-a|\).
The conditional mean is \(E[Y\mid X]=0.6(0)+0.4(10)=4\). But the absolute-error risks at three relevant actions are
In fact, \(0\) is the unique minimizer: moving from \(0\) toward \(10\) initially increases risk because the probability mass at \(0\) exceeds the mass at \(10\). The mean \(4\) does not minimize absolute-error risk. Under squared-error loss, by contrast, the conditional mean is the unique minimizer when the conditional second moment is finite. The action must be derived from the loss, not selected from a familiar summary by habit.
Why Ties Must Be Kept
A minimizer is not necessarily unique. Under absolute-error loss, any median of a distribution minimizes expected absolute error. In a discrete distribution, there can be an interval of medians; reporting one action as the only optimum would then be incorrect.
Worked Example: An Interval of Absolute-Error Minimizers
Let \(Y\) equal \(0\) or \(2\), each with probability \(1/2\), and allow any real action \(a\). The absolute-error risk is
If \(0\leq a\leq2\), then \(|a|=a\) and \(|a-2|=2-a\), so \(r(a)=(a+2-a)/2=1\). If \(a<0\), then \(r(a)=(-a+2-a)/2=1-a>1\). If \(a>2\), then \(r(a)=(a+a-2)/2=a-1>1\). Therefore the minimum risk is \(1\), attained by every \(a\in[0,2]\), not just by \(a=1\).
For squared-error loss the minimizing action is instead \(E[Y]=1\). The set of absolute-error minimizers and the squared-error minimizer happen to share an element here, but they are not the same optimization problem.
Conditional Regret Is Not Total Risk
Conditional regret compares an action with the best action at the same observed value \(x\). It is therefore a function of \(x\). Total risk instead averages loss over the joint distribution of the observed information and outcome. The expected conditional regret is the total excess risk over the best achievable conditional risk; the regret at one particular \(x\) is not that average.
Proof. Because the action set is finite, \(m(X)\) is measurable. By conditional expectation and the fact that \(\delta(X)\) is determined by \(X\),
Subtracting \(E[m(X)]\) gives the stated identity. By definition of \(m(X)\), \(r(\delta(X),X)-m(X)\geq0\) almost surely. A nonnegative integrable random variable has expectation zero if and only if it is zero almost surely. It is zero exactly when \(\delta(X)\) attains the conditional minimum, which is equivalent to \(\delta(X)\in M(X)\). \(\square\)
Worked Example: A Large Regret at a Rare Value of X
Let \(X\) take values \(H\) and \(L\), with \(P(X=H)=0.9\) and \(P(X=L)=0.1\). There are two actions, \(a\) and \(b\), with conditional risks as follows.
| Observed value | \(r(a,x)\) | \(r(b,x)\) | Minimum risk |
|---|---|---|---|
| \(H\) | 0 | 2 | 0 |
| \(L\) | 10 | 0 | 0 |
The rule choosing \(a\) at both values has conditional regret \(0\) at \(H\) and \(10\) at \(L\). Its total risk is \(0.9(0)+0.1(10)=1\), and its expected conditional regret is also \(0.9(0)+0.1(10)=1\). The value \(10\) is the regret at \(L\), not the rule’s total excess risk.
The rule choosing \(b\) at both values has conditional regret \(2\) at \(H\) and \(0\) at \(L\), so its total risk is \(0.9(2)+0.1(0)=1.8\). Although its largest pointwise regret is smaller than \(10\), its expected regret is larger. A pointwise comparison alone does not determine total risk; the probabilities of the observed values matter.
An Infimum May Not Be a Minimum
The distinction between minimum and infimum is especially important when the action space excludes a boundary value. The log-loss boundary case provides a direct example. Suppose \(Y=0\) with probability one, and predict a probability \(q\in(0,1)\) for the event \(Y=1\). The log loss for observing \(Y=0\) is \(-\log(1-q)\), so the conditional risk is
This risk is strictly positive for every allowed \(q\), but \(r(q)\to0\) as \(q\downarrow0\). Thus its infimum is \(0\), and no allowed action attains it. Saying “the minimizing probability is zero” would be wrong for this action space: zero is not an available action. If the action space is enlarged to include \(q=0\), with the usual boundary loss for this outcome, the risk is attained there. The optimization conclusion depends on the domain as well as the formula.
Common Mistakes and a Reliable Check
The most common errors can be caught by asking four questions before declaring an action optimal:
- Was a mean substituted automatically? Check which loss is being minimized. The conditional mean is tied to squared-error loss, not to every loss.
- Was a tie discarded? Evaluate the full minimizing set, and establish uniqueness before naming a single optimal action.
- Was pointwise regret reported as total excess risk? Average conditional regret over the distribution of \(X\) to obtain total excess risk.
- Was an unattained infimum called a minimum? Check whether the limiting action belongs to the action space and has the stated risk.
There is also a practical issue with conditional statements: conditional risks are defined up to sets of \(X\)-probability zero. Changing their values on such a set does not change total risk or the almost-sure optimality conclusion in the theorem. An action assigned at an impossible or probability-zero observation should not be used to claim a different overall risk.
These determine what is being optimized and which actions are allowed.
Use the conditional distribution of the outcome given the observed information.
Check both ties and whether the infimum is attained.
Conditional regret is pointwise; expected conditional regret is total excess risk.
Check Your Understanding
For each question, distinguish the conditional calculation from the conclusion about an optimal action or total risk.
- Under what loss is the conditional mean the standard minimizer, and why is it unsafe to use it for every loss?
- For \(Y\) equally likely to be \(0\) or \(2\), what is the full set of absolute-error minimizers? Verify the risk on that set.
- If a rule has conditional regret \(12\) on an event of probability \(0.05\) and zero elsewhere, what is its expected conditional regret?
- What must be checked before calling an infimum of conditional risk a minimum?
- In the two-value example, why does the rule with pointwise regret \(10\) at \(L\) have total excess risk \(1\), rather than \(10\)?