Mixtures That Improve Maximum Risk Without Dominating

Why Maximum Risk and Pointwise Dominance Differ

A decision rule is often compared with another by asking whether it has no greater risk at every parameter value. But a different question is whether a rule has a smaller maximum risk: its worst risk over the parameter values under consideration. These comparisons need not agree. A rule can reduce the worst risk by trading a little risk at one parameter value for a larger reduction at another, without having lower risk everywhere than either rule used to construct it.

With two parameter values, the trade-off is visible in a two-entry risk vector. Randomizing between two rules places the resulting risk vector on the line segment joining their vectors. When one rule has lower risk at the first parameter value and higher risk at the second, a suitable mixture can balance those risks. The mixture can then have lower maximum risk than either endpoint of the segment, even though its risk is higher than each component rule at one of the two parameter values.

To mix two rules, choose one rule at random, independently of the data, before applying it. If \(\delta_A\) is chosen with probability \(p\) and \(\delta_B\) with probability \(1-p\), then the expected loss at each parameter value is the corresponding probability-weighted average. This follows by conditioning on which rule was selected. Write \(\delta_p\) for this randomized rule; its risk vector lies on the line segment between those of \(\delta_A\) and \(\delta_B\).

A Two-Action Loss Table

Consider a problem with no informative observation: the observation has the same distribution at \(\theta_1\) and \(\theta_2\). There are two available actions, \(A\) and \(B\), and the loss table is as follows. Because the observation provides no information, choosing an action gives the risk shown in its row.

When Does the Mixture Equalize and Improve?

The calculation above has a general form. Suppose the risk vectors of two rules are \((a,b)\) and \((c,d)\), where the first entry refers to \(\theta_1\) and the second to \(\theta_2\). Consider the useful crossed-risk case in which the first rule is better at \(\theta_1\), the second is better at \(\theta_2\), and the endpoint maximum risks are respectively \(b\) and \(c\). These conditions can be written \(a<c\), \(b>d\), \(c>d\), and \(b>a\).

Proof. By linearity of risk under randomization, the mixture’s two risks are \[ r_1(p)=pa+(1-p)c=c-p(c-a),\qquad r_2(p)=pb+(1-p)d=d+p(b-d). \] The first is strictly decreasing in \(p\), and the second is strictly increasing. Equating them gives \[ c-p(c-a)=d+p(b-d), \] so \[ c-d=p\big((c-a)+(b-d)\big). \] The denominator is positive because \(c-a>0\) and \(b-d>0\); hence the unique solution is the stated \(p^*\). Its numerator is positive because \(c>d\). Also, \[ (c-a)+(b-d)-(c-d)=b-a>0, \] so \(p^*<1\). Thus the equalizing mixture is strictly interior.

At \(p=0\), the risks are \((c,d)\), and \(c>d\), so the maximum is \(c\). At \(p=1\), they are \((a,b)\), and \(b>a\), so the maximum is \(b\). For \(0\leq p\leq p^*\), the decreasing risk \(r_1(p)\) is at least the increasing risk \(r_2(p)\), so the maximum is \(r_1(p)\); it decreases strictly toward \(p^*\). For \(p^*\leq p\leq1\), the maximum is \(r_2(p)\), which increases strictly away from \(p^*\). Therefore \(p^*\) uniquely minimizes the maximum risk on this line segment. Since \(p^*\) is interior, its common risk is strictly below both endpoint maxima \(c\) and \(b\).

Finally, at an interior mixture, \(r_1(p^*)\) lies strictly between \(a\) and \(c\), while \(r_2(p^*)\) lies strictly between \(d\) and \(b\). In particular, the mixture’s risk at \(\theta_1\) is greater than \(a\), so it does not dominate the first rule; its risk at \(\theta_2\) is greater than \(d\), so it does not dominate the second. The other coordinate is lower than the corresponding component’s risk in each comparison. Thus neither component is dominated by the mixture. \(\square\)

The theorem is a statement about the best mixture between these two rules. It does not establish that this mixture minimizes maximum risk over every rule in a larger decision problem. Other rules may have better risk vectors.

Worked Risk-Table Calculations

What the Calculation Does—and Does Not—Show

The useful geometric picture is a line segment between two risk vectors. In the crossed-risk case, one coordinate falls as the mixture probability increases and the other rises. The maximum is the larger coordinate. Before the crossing, the falling coordinate determines that maximum; after the crossing, the rising coordinate does. Their intersection is therefore the minimum along the segment. The algebra in the theorem makes the conditions for this picture precise.

A common mistake is to argue: “The mixture has lower maximum risk, so it must dominate each rule in the mixture.” This confuses a maximum comparison with coordinatewise comparison. A smaller maximum only says that the largest coordinate is smaller; it does not say that both coordinates are smaller than those of another vector. For example, \((4,4)\) has maximum \(4\), less than the maxima \(7\) and \(6\) of \((1,7)\) and \((6,2)\), but it is worse than the first vector in its first coordinate and worse than the second vector in its second coordinate.

Another important limit is that equalizing two risks is not automatically a global minimax argument. The proof establishes the best point on one particular line segment. To conclude that no other rule does better would require comparison with the rest of the attainable risk set, which is not supplied by a two-rule table alone.